As long as you can say 'I really don't want to lose her' (meaning the OW), then nothing will get better with your wife. You will be constantly be comparing her to the OW and she will always come up short, because that's what you're hoping for.
Until you get the OW out of your mind completely, you cannot fully give yourself to your wife and your marriage. Plain and simple.
Until you get the OW out of your mind completely, you cannot fully give yourself to your wife and your marriage. Plain and simple.