Steve McConnell XP 2021 keynote

Dennis Mancl
MSWX Software Experts
Bridgewater NJ
dmancl@acm.org

Steve McConnell keynote

Steve McConnell delivered the Tuesday keynote talk (June 15, 2021) at XP 2021. His talk title was “20 Years is Enough! It’s Time to Update the Agile Principles and Values.” Steve is definitely an advocate of agile methods (and has published the management-oriented book More Effective Agile in 2020), but this talk made the argument that the Agile Manifesto can be an obstacle to effective agile adoption today. He believes that the Manifesto is just too old-fashioned, and it needs to change with the times.

Steve’s talk was pretty good. He is an effective advocate for opening up the Agile Manifesto to revision... His critique is based on his view that even the Agile Manifesto should be subject to the same “inspect and adapt” process that many agile methods use. He had several concrete suggestions for how to update the Manifesto to incorporate what the software industry has learned in the last twenty years. Most of his suggested changes are on target, although I think there would be a lot of resistance to making even minimal changes to a document that has such a religious following.

Steve points out that in 2001 (the year the Agile Manifesto was written), many software organizations were still hopelessly backward in their approach to software engineering. Instead of structured development, teams were still using the same “code and fix” model of development from the 1970s. But if you worked for a large software organization in a big company, there was a push throughout the 1990s to introduce a more formal and bureaucratic software development process – especially software methods that followed the SW-CMM (software capability maturity model).

The Agile Manifesto was written in the context of a software industry with these two forces at work: undisciplined chaos on one side and huge overhead on the other. Steve’s claim is that the Manifesto, with its four values statements that denigrate complex processes and tools, comprehensive documentation, contracts, and big detailed plans, is a reaction against the SW-CMM and similar overweight processes. If agile’s enemy was the SW-CMM in 2001, its enemy today is just variations of agile.

The Agile Manifesto and the Scrum Guide could also be seen as too rigid. Steve thinks it is an “undue fixation on 20-year-old artifacts.” I seem to remember that there were similar arguments over Extreme Programming in the late 1990s and early 2000s – did teams need to follow all twelve practices in the original Kent Beck book? Should we follow the updated practices in the second edition? In the 2010s, there are also questions about whether a Scrum team is following all of the rules in the Scrum Guide. (In other words, we seem capable of many different fixations.)

Steve’s suggestions about the four values in the Agile Manifesto:

The first value in the Agile Manifesto is “individuals and interactions over processes and tools,” but Steve points out that agile practice today is full of “process diagrams” – including the Scrum double-cycle diagram that describes the daily cycle and the Sprint cycle. Steve thinks that “getting rid of burdensome processes” may have been an aspiration of the Manifesto writers, but agile still needs some process and tools.
The second value is “working software over comprehensive documentation.” Steve believes that this value is mistargeted. He thinks that “big documentation” is a side-effect of building software in big batches. So this value should be about batch size, not documentation. The “just-in-time practices” of modern agile methods will limit the growth of unneeded and expensive documentation, in Steve’s view. Documentation isn’t the core issue, the issue is large batch size versus small batch size.
The agile community has found customer collaboration to a good thing in general. But Steve has noticed that many teams fail to talk with customers at all: the development team thinks they know what the customer wants. Of course, they are often wrong. Steve suggests for the third value doesn’t need the comparative “over”: Instead of “we value customer collaboration over contract negotiation,” he would say “we value customer collaboration.”
Steve points out that every agile method spends a fair amount of effort on planning. The major difference with non-agile, non-iterative approaches is that the detailed planning is done “just-in-time” rather than way-up-front. So he wants the Manifesto to be more specific than just saying “responding to change.” Instead of “responding to change over following a plan,” he would say “deferring detailed planning to the last responsible moment over creating detailed plans up-front.”

Steve also spent some time critiquing the 12 Agile Principles, and these principles make an easier target. Steve found some concepts that were too specific: early and continuous delivery, business and developers work together daily, and constant pace. It may also be unclear whether a preference for “face-to-face interaction” is still valid after our pandemic experiences.

Could we rewrite the agile values and principles? I think that Steve’s comments on the Agile Manifesto’s four values are well-aimed. But I am not sure that we can succeed in creating a satisfactory Manifesto update. Any attempt to rewrite the four values will be difficult, because the poetic and rhetorical simplicity of the Manifesto’s four statements is quite powerful. Can anyone build a new formulation that is equally compelling?

Steve’s comments about the 12 Agile Principles is a bigger stretch. Many people have given more in-depth feedback on the 12 Agile Principles – I particularly like the comments by Bertrand Meyer in his book Agile: The Good, the Hype, and the Ugly. Bertrand points out that some principles are fairly empty statements, others have a lot of overlap, and they don’t have the coherence of the four values. Steve’s critique isn’t broad enough to suggest a way to make a more balanced list of principles.

The historical critique that Steve proposes is pretty good, but incomplete. The early agile community was pretty impatient with other developments in software development, not just the SW-CMM. In some large companies, there was a big investment in tools and training to support UML models, spearheaded by Rational Software Corporation and others. New language and networking technology (Java, C#, Corba, SOAP) were a big pull to adopt formal modeling in some parts of the software process. But I think many of these initiatives lost steam, mostly due to economic issues, industry reorganizations, and the emergence of new web-oriented development environments such as PHP, Python, and Ruby.

[One thought about rewriting the Agile Manifesto. It won’t be easy, and I think a better goal is to completely replace the core principles with something equally compelling. And... instead of a Manifesto with four “we value X over Y” statements, it will need to have a different rhetorical model, in order to make a clean break with the old.]

[My recurring nightmare is a new process model that emphasizes “power and control” (which will appeal to developers and managers), with the model structured as a small cluster of three or four “power areas”, and a master layer that ties them all together (“One ring to rule them all!”). Yow, this sounds pretty grim.]

Some discussion about the Agile Manifesto

I also took notes at one of the XP2024 panel sessions ("The Stories We Tell: Experience, Research, or Patterns"). The members of that panel were invited to comment on how the Agile Manifesto fits into the stories we tell.

Steve Fraser [panel chair] explained that many people have called for major updates to the Manifesto after 20 years – pointing out that Steve McConnell’s XP 2021 keynote talk would be a critique calling for changes to the Agile Manifesto. There was a wide range of opinions:

Rebecca Wirfs-Brock: “The authors aren’t going to change it.” The Manifesto captured the spirit of what agile was trying to do 20 years ago, but it is not a set of unchanging principles for today.
Ken Power: “It is a wonderful historical artifact.” Twenty years ago, we were pushing back against big, inhuman, non-human-centered ways of working. But many of today’s are either ignorant of the Manifesto, or they treat it as fixed and unchanging. Ken thinks the most important part of the Manifesto is the first line: “We are uncovering new and better ways of doing things...” That is the spirit we need to carry forward.
Ademar Aguiar: “Maybe the generality [of the Manifesto] is too general today, we may need to be more specific.” Ademar related this to writing patterns – you do not want patterns to be too generic (they don’t give useful advice) or too specific (they can go out of date quickly).
Mary Lynn Manns: “The important thing is that people just write.” Mary Lynn gives the Manifesto authors credit for saying what they thought. We should do the same – put out our opinions and let people discuss them.
Casper Lassenius: “It should not be a test of your faith, or whether you are doing the right thing.”
Steve: “Lightning could strike you down...”