Question about letting go

I have a question about letting go.  I am new here, and I have not read any books or anything yet.

I keep hearing about not trying to be logical in a completely illogical situation.... just not to engage in the debate.

I do feel like I have done this in the past, not as a tool, but because I really believed him when it was turned around on me.... but it did still happen.  I thought everything was my fault.

He really did become delusional about a lot of things I feel because of this..... and he has even used this word when apologizing about the past (after reaching a boiling over point, plus affair)

I really feel like I need to defend my point of view to avoid him believing his irrational thoughts.  I mean at some points he blames the children and includes them with me as having emotional problems when he cant handle things.  And 'doesn't want to put up with that crap'  Storms off, or out of the house.

I mean after his affair, he even believed I had given him the okay at some point that we were in an open relationship!  To try to justify it.  I was like okay so then why were you sneaking around lying, if we were in an open relationship..... you wouldnt need to.... thats the point!  and why were you accusing me of cheating the whole time?  He just couldnt even wrap his head around it he believed it so much.  

So is the common thought that he just knows right and wrong deep down and to leave it at that?  Or just that defending myself just leads to an even bigger illogical story in his head?  I don't know, but I have a have a hard time not defending myself or my kids.