It has always been my experience that playing the rhythm is often harder to play than the head or lead of a lot of music. Especially when you play jazz. Ive found that once you get your playing of a particular song up to speed it sounds a lot better and makes more sense. Of course it is getting it up to speed that is the problem. Eric Clapton gave me a few problems similar to yours until I got parts of it up to speed. Even if you make mistakes you will hear your playing click in.
Bluegrass is really quite hard as well sometimes and is very reliant on playing at the right speed.
There are an endless supply of backing tracks on youtube, I don't know if you have tried this or not. Go to Youtube and just punch in "backing track to ( the name of the song)." Some of them are good and some not so good, but you can play along with them and listen to how the rhythm is played.
