I forgot about his other child. I don't remember if he or the OP insisted in the older child coming over until he/she began to make threats against their younger children.Why do you think he will be available to his children after the divorce? He already has a child who he ignores, as if that child doesn't exist. Once these children no longer live with him, they will never see him either, so their relationship with him will be even less than it already is.
I think it's important to recognize this going forward in order to prepare for what the future holds. You (op) have already been raising the children alone. It may improve when you don't have his chores to do as well as your own, but you won't get any help with the children from him.
He might very well not pursue a 50/50 custody plan, however if he does, he will have to spend some time with them on his days.