Supposedly there was no way to keep the Abbey in the inmediate family other than for it to be passed to a male relative - an heir. However when they discovered Mathew's will-in-intention Mary inherited his interest in the Abbey. Why couldn't Robert Crawley have just willed it to Mary (and the other daughters for that matter) and simply cut out any of the distant male cousins?
Robert's father created an entail which decreed the title, estate, and fortune would all go to the next male heir - his dad apparently never expected Robert to not have a son. In season 1 members of the family were trying to destroy the entail so that Mary could inherit, but Robert's father had made sure that the entail could not be smashed, which is why Matthew was heir, like James and Patrick before him.
In season 3, when Robert had bankrupted the estate, Matthew had inherited the vast fortune of Reggie Swire, and offered to give it to Robert (it had taken Matthew a while to accept the money) to save Downton. Robert however insisted he would only accept if Matthew instead made an investment in the estate, so that they would be co-owners. It seems Matthew's investment was not part of that entail, and so his place as shareholder was inherted by Mary, though she still cannot hold the family title.
At the beginning of the series Cora and Violet were trying to get Cora's fortune passed to Mary, there was no hope for the title, women would not get to inherit titles from their fathers for a while yet, despite having a female monarch for 60 years. It's weird that the rules were bent for monarchs during an heir shortage but not the peerage. Parliament even selected Mary I I over her father and brothers to be queen, of course it was over religion but still.
Fortunes, estates, and titles weren't always intertwined like that. Women could still inherit all their father's money, just not the titles or the lands. If a rich male cousin was made heir, the estate would survive but if the heir was poor it would have to be sold. The female heirs could buy it but it would be much easier to take the money and marry someone else instead. Robert's father feared that if Cora's money was taken from Downton, the estate would fall and Downton would have to be sold. He knew full well that Cora was the only reason Downton wasn't ruined, if Robert died before becoming the earl and Cora remarried that fortune could be claimed by the new husband. Robert's father made a deal with the Isidore Levinson (Cora's father) to have the money attached permanently to the estate upon the marriage. So it means that unless Mary married well on her own or married the heir, she would be at the mercy of small yearly allowance left by her father. During the Edwardian era, most aristocrats were out of money due to years of over spending, without money or a title Mary stood little chance of a good marriage. This was Cora's main concern. Violet came on board despite knowing it could ruin Dowtown, she valued blood over tradition, which was an unusual attitude. Robert meanwhile, was reluctant try to remove the entail because he felt that Downton was too important. He also strongly believed that Mary would just suck it up and marry the heir. Which was the plan. Then James and Patrick die and an unknown is thrown in. This is too much, so they scheme to get the money back. Cora's feelings of urgency are increased when Mary comes to her about the dead Turkishman. She wants Mary married fast and if she won't marry Matthew, she'll marry someone else but her financial prospects don't draw many in. That's why Carlisle was candidate, he was rich, but the marriage was less than ideal, given his low birth.
If Matthew had died before conceiving George, another random would have to be found and it would begin again. But since Matthew owned half the estate independently because of the Swire money, the new heir would get from Robert the title, half the estate, and no money because Cora's fortune was lost. Mary was made Matthew's heir, she would get half the estate and whatever remained of the Swine fund but still no title. Also she's responsible for Matthew's death taxes but the new earl would be required to pay Robert's. So it was good for everyone that George was born.
You can actually thank death taxes, heir complications, and overspending on for the fall of the aristocrats. Many remain but they aren't nearly as all powerful or as wealthy as they were. The era of big estates, for the most part, has ended for the peerage.
I think there was some act in 1925 which allowed entails to be overturned.
A lot of aristocrats continued to leave the bulk of their estates to their oldest son and heir/whoever inherited the title. E.g. the current Duke of Devonshire has spoken on TV about how he will leave his estate all to his son because the keeps the whole estate intact.
What do you think?