I seem to remember that Edith was the only person to believe that Patrick was the Patrick Gordon, thought to have died on the Titanic. Luckily he survived the disaster, was picked up and taken to Canada under a different name, having lost his memory. He came back to the hospital to recover after being badly burned in WWll. Lady Edith realised his true identity one day as they walked past the folly (actually called Jackdaws Castle) in the grounds of Downton. He mentioned an old German governess from their childhood. Patrick was Lord Granthams heir and was, before the disaster, to become, or actually was, engaged to Mary. He disappeared but I cannot remember the circumstances of that.
Edith believed him when he told the stories of their childhood at DA, but then when someone (possibly Mary?) said that everyone had a bad governess and could tell such stories, and Edith understood that there was a risk that he was not the real Patrick. When Edith told him about Gordon, he understood that he was caught (or close to be), and disappeared. He would rather have lived as poor than being prosecuted as a forger.
What about the funny thing the supposed Patrick Gordon did by marking a cross on his lips - Lord Grantham looked surprised and asked, 'where did you learn to do that?' There was never any explanation of that.
It was explained a Mr Gordon had worked with the real Patrick in the same office, so he likely learned how to do the gesture there. That was also how Mr Gordon (the faux Patrick) knew about the Crawley family and some childhood memories.