Trying vs. doing

I have been reading this forum, as well as books until my eyeballs literally spin!  Here's the nutshell history:

Married almost 20 years to ADHD husband and we have 4 kids ranging from 11 - 19 years.  Two kids have ADHD as well.  He was diagnosed 7 years ago and  he was on meds with moderate "buy in" to the diagnosis until last year when he had an emotional affair with a co-worker and wanted to leave me stating, "I've tried and tried and tried and I'm done trying."  I think at that point he went through a phase of feeling like if he was with someone else, he wouldn't have issues.

Before last year, our main problems focused around (the biggest) not enough sex for me and getting adequate help around the house.  What started out as an incredible love affair when we met turned into me feeling completely lonely, worthless and ugly because of his lack of desire in the bedroom. For years, he would literally take my hands off his body when I would try to be intimate with him.  I have developed such low sexual self esteem I can't even describe it.  Out of the bedroom, we were each others best friends, made each other laugh and had a great marriage.  He is sweet and kind and fun and I truly love and loved him with all my heart.  He's a great dad and we have an awesome family.

Ironically, I had worked really hard to get to a place of acceptance about our sex life and how much he could contribute to the household.  I worked on my own with a therapist to get clear about my own goals, boundaries and make sure I was taking care of myself.  Just about the time I felt like I got to this place, he was done.

Long story short, he decided to stay and work on things.  He went on Wellbutrin for depression and is working with a new doctor to get his ADD meds figured out and is diligent about taking them!  He also works with a therapist and that seems to be going well.

Here's the problem - He says he loves me and is committed to us.  He says I'm attractive.  He says he likes sex.  I don't nag him, I don't parent him. I haven't had anger issues in over a year.  We have talked about creating connection etc. but just this morning he literally turned away from me when I tried to cuddle him in bed.  I felt "the sail go up" and then he rolled away from me. 

I am trying my hardest not to read into it, take it personally and all that but at some point it just breaks my heart! I know people don't like to talk about sex but PLEASE is there anyone out there who feels like a lot of the problems in their marriage stem from not enough sexual connection?  I know the theory is that if you have problems in the bedroom that they started outside of it.  What about me who feels like if we just had a quality sex life, I wouldn't feel so bad outside the bedroom?

I called this "Trying vs. doing"  because this entire last year, I keep hearing that he's "trying"  Yes, he sees doctors and takes meds but so far that's it.  I called him to the mat by telling him if there is no measurable progress that isn't doing.  The month he decided to stay, that was progress.  Seeing doctors - progress.  Taking meds diligently - progress.  That's it...there has been nothing more beyond those things.  How about doing exercises in any of the books we have, or exercises his therapist gives him?  No.  So I am at a point of not knowing what to do.

I am a happy, optimistic person with a great life.  I have wonderful friends, hobbies, activities etc.  I enjoy my kids, exercise regularly and eat well.  Point being - I have a life!  My boundaries are good and I am not waiting around for him to complete me.  I just want an intimate life with my husband and it feels impossible right now.

HELP!