I’ve worked on big projects, small projects, in huge teams and by myself, in fossilized federal agencies and cool Silicon Valley companies. I have learned and used at least twenty programming languages. I’ve lived through waterfall/BDUF (big design up front), structured programming, top-down, bottom-up, modular design, components, agile, Scrum, extreme, TDD, OOP, rapid prototyping, RAD, and probably others I’ve forgotten about. I’m not convinced any of these things work.
*****
Whether a methodology works or not depends on the criteria: team productivity, happiness, retention, conformity, predictability, accountability, communication, lines per day, man-months, code quality, artifacts produced, etc. Every methodology works if you measure the right thing. But in terms of the only measurement that really matters—satisfying requirement on time and within budget—I haven’t seen any methodology deliver consistent results.
My own experiences are anecdotal, but they are shared by almost every programmer I know. It turns out that anecdotes are all that anyone has: rigorous studies of software development methodologies haven’t been done because it’s impossible to control for all of the variables.
Try this thought experiment: Imagine two teams of programmers, working with identical requirements, schedules, and budgets, in the same environment, with the same language and development tools. One team uses waterfall/BDUF, the other uses agile techniques. It’s obvious this isn’t a good experiment: The individual skills and personalities of the team members, and how they communicate with each other, will have a much bigger effect than the methodology.

Thought-provoking. The author concludes:

I think programmers should pay much more attention to listening to and working with their peers than to rituals and tools, and that we should be skeptical of too much process or methodologies that promise to magically make everyone more productive. Maybe social skills come harder to programmers than to other people (I’m not convinced that’s true), but developing those skills will certainly pay off a lot more than trying yet another development methodology.

Maybe software development methodologies are like diets, to use an analogy my coworker Eric brought up. Endlessly appealing, rarely successful. Or perhaps they're like workouts. What's needed is variety, and sometimes you just need a new routine to keep things fresh, regardless of what routine you choose.