The books that have defeated me are a short list: Ulysses, Gravity's Rainbow, and Skinny Dipping in the Lake of the Dead. I've abandoned many books because they were bad or boring, but these are the three (four including IJ) that I abandoned because I wasn't capable of finishing them. And, maybe no surprise that all three are experimental in their way, simultaneously messing with expectations of narrative, form, and language. Which is a challenge for my logical, (cough, rigid, cough) concrete brain.
I justified my abandonment by scoffingly dismissing the white-boy-blogger-love of IJ and DFW... but that's not really honest, or accurate. I was just defeated. But I will persevere, and will also refrain from making footnoted commentary for the next several weeks (months?) while I'm in process.
I keep a twitter list of other books that I've read/am reading here.