Not always. But he ignored Cora's opinions, and thinks his bad management wasn't that bad. Also when it was turned out that Mary was heiress of Matthew and became Robert's co-owner of Downton, he was really mean to his own daughter. And often yelling at women like Lady Sybil, Lady Rosamund, and Cora.