House of Leaves. It's mainly the footnotes upon footnotes that make it a little hard to re-focus your brain every time one ends or begins. Thing is, it's deliberately written that way to add to the level of mindscrew, so I can't bring myself to be annoyed with it. I'm currently making another attempt, as I genuinely enjoyed what little I managed the first time, and got at least three goods scares out of it, despite having barely scraped the surface.
I also got halfway through Twilight, and I'll admit to enjoying it despite all its flaws, but I got distracted from it for some reason and I just couldn't overlook those flaws a second time when I made my reattempt. That and I'd realized that the real plot hadn't started by the halfway point, which is just bad.