I'll leave this right here:Yes, you're correct, op is the one three weeks into this. Keep in mind, we do not know what was happening before that. Everyone can assume....
But maybe depression, or self esteem issues, possibly something that OP is doing is upsetting his spouse, maybe the age/life stage differences are causing a concern, maybe they haven't fostered their affections... there are so many things that can go wrong in relationships. Throwing every other possibility out the window to the point of absolutely no other conversation can happen here will not help.
Ive been open to her fantasies of seeing other men etc