In Episode 4 of Series 1, Robert receives a letter from Lady Rosamund, that word was going round London that Evelyn Napier had given up any thought of Mary, and that it somehow reflected badly on Mary. That she had been found wanting. At the end of Episode 5, Lady Edith sent the letter to the Turkish Ambassador. In Episode 6, the Dowager shows Cora a letter from from Lady Flintshire that says that her husband has heard a vile story about Mary. Possibly Rosamund and Susan spoke about it, but more likely Lady Flintshire's maid would have heard it directly from her. These ladies say anything and everything in front of their maids. Susan is a nasty piece of work. Even the Dowager said, when Cora was reading the letter, "Sorry? She's thrilled."
Also in episode 5, Carson approaches Cora about a letter he received from Lord Flintshire's valet about the scandal. He explains that Lord Flintshire (a minister in the Foreign Office) had dealings with the Turkish ambassador. It wasthe ambassador who told Lord Flintshire.