Catherine Cookson's The Man Who Cried

Set in the 1930's, a man must leave home after an affair with a married woman.