When I was severely mentally ill with Borderline Personality disorder, I took my marriage out of context. Every time my husband tried to be good friends with another woman I ended up accusing him of having an affair. I ended up checking his phone, timing him when he got home from work, questioning him and even checking up on him to see where his car was. I took the whole thing out of context and reacted in absurd ways! I somehow managed to convince myself that what I was thinking was true and because I was thinking it, it was happening. I even ended up looking for evidence which simply did not exist because it wasn't happening. What I learned was to 'turn my mind.' Your brain makes you think things and it can convince you it is true even if it is not! This is 'distorted thinking' which is not easy to deal with when you've been through the nightmare of relationships where it has happened, its happened with your parents and you never healed from it. A past full of violence can cause this thinking too.
The key I have learned is to learn to trust your partner and your self that he is telling the truth, to be mindful that your thinking it and figure out why your thinking it and what doesn't add up.
My husband has a preference for female company and he works with females, this sometimes makes me anxious but I trust he knows his boundaries so when I'm 'obsessed' with the idea he could be having an affair, I do something to boost my confidence and I look at it through his eyes by understanding he could be thinking the same thing, since we don't get much time together now because of work.