Agile Minds: James Coplien Interview Part II

Jan-18-2008

Post written by Chris Spagnuolo. Follow Chris on Twitter Add comment

Jim Coplien is a leading voice in the agile community, especially in the area of organizational development. He is considered the father of Organizational Patterns, and is one of the founders of the Software Pattern discipline. He is currently Second Mate (a curious and only indirectly meaningful title) at Gertrud & Cope in Denmark. He is also the co-author of the widely acclaimed book Organizational Patterns of Agile Software Development. Cope was also a founding member of the Hillside Group along with such visionaries as Kent Beck, Grady Brooch, Ward Cunningham, Ralph Johnson, Ken Auer, and Hal Hildebrand. Here is the conclusion to my interview with Cope.

Q. You spend a great deal of time talking about organizational patterns in agile software development. Can you briefly explain what organizational patterns are?

A. An organizational pattern is a recurring configuration of organizational structure (more precisely, organizational form) that bodes for quality of life. Organizational patterns are the foundations of culture. Anthropology has long understood this, and at least one classic of modern anthropology — Kroeber’s Patterns of Culture (1948) — takes this stance. Patterns fit together into something called a pattern language: a collection of patterns designed to fit together to give rise to a certain culture or organizational style. Any organization aspiring to the cultural trappings and mores adopts the patterns of that culture.

Neil Harrison, Brendan Cain and I collected good organizational patterns of software development over a period of about ten years through a program of empirical research with real-world teams. We incorporated other patterns written by Ward Cunningham, Alistair Cockburn, Steve Berczuk, and others. Kent Beck has often pointed to those patterns as the foundation of XP, and Jeff Sutherland points to that work as the origin of daily Scrums. In a broad sense, our current collection of patterns (Organizational Patterns of Agile Software Development) is the foundation of contemporary Agile values and practice.

Q. What patterns would say are the most important in successful agile teams and organizations?

A. Neil and I have our favorite Top Ten list of patterns. But here, let me instead go back to your earlier question. Whereas Jeff talks about failure modes, Scrum more directly talks about helpful structures that reflect the deeper patterns beneath them. If we turn around Jeff’s pitfalls, we come up with patterns like:

Function Owner and Component Owner

Unity of Purpose

Small Teams

Firewall

Domain Expertise in Roles

Engage Customers

Team Pride

But, what makes a pattern important is that it speaks to the team. Each team will find a different set of patterns that apply to its needs, and any given team would choose a different set at different points in time. They’re about inspiration rather than prescription.

Q. Are there specific techniques organizations can use to assess their current patterns?

A. The foundation of the organizational patterns in any organization lie within its communication network. I like to use a facilitated role-playing exercise to unearth the communication pathways in an organization. You can use that information to visualize the organization’s structure. That structure often makes many patterns obvious, and the exercise itself makes yet other patterns obvious. It’s very illuminating for organizations that do this! I offer a service to facilitate such an exercise and to do follow-up analysis (see Getrud and Cope for more information) , as does Neil Harrison in the U.S.

This technique is one form of retrospective. More broadly, retrospectives are the keystone to process improvement. I don’t mean the quickie two-hour “check-ups” at the end of a sprint, though those are O.K. I mean the kind of retrospective that Norm Kerth talks about in his book: an off-site, three-day head-holding and soul-searching session. We use the role-playing technique as one tool in such retrospectives, but there are many other tools and many resources to help you come to terms with your current situation.

Q. How difficult is it for the average organization to identify deficiencies in their patterns and adopt or create new patterns to reach their desired state?

A. As difficult as it is to do a retrospective. That may sound easy or hard depending on who you are, and both answers are right. If an organization has the courage and will to change, then the organization can easily assimilate to the tools of introspection and deal adeptly with its shortcomings. If an organization is rife with fear, is insecure, or is stubborn, then it may never come to grips with change.

The “difficulty” here isn’t so much about technique as it is about will. An external facilitator can help organizations see themselves in a new kind of mirror. That mirror helps them see the patterns they have as well as the patterns that they need. Having seen themselves as they are, they can effect changes. I can’t change organizations. Only organizations can change themselves. But I can facilitate by bringing tools to bear, and by asking hard questions that keep organizations honest about themselves. Sometimes, facing truth is the hardest part of change.

Q. In general terms, what do you think the future holds for agile software development?

A. That’s a pretty open question… Niels Bohr said to be careful about predicting things, particularly if they involve the future. I’ve thought a lot about this from an unusual perspective.

Most movements in CS have the same distinctive phases. It starts with a brief period of birth pangs in times of broad skepticism, and then takes hold with early adopters. Some of them report success. Then there is a time of rapid, blind growth. Some time during this growth, the backlash starts, and it continues to build until another movement overtakes it. The backlash never quite comes to closure but casts enough aspersions on the movement to make room for the new one. The structured analysis movement followed this, as did ISO 9001, as did component-oriented software and, to some degree, object-oriented software; patterns certainly suffered this fate.

Agile is a bit different because it looks like something new but is in fact almost completely a conscious re-packaging of known ideas. Its early phases reflected the same reactions as any new movement: skepticism, followed by a few good successes, followed by phenomenal growth. Some of the earlier movements were the same way, in the sense of lacking novelty, but were less conscious about it. I think the prime movers behind Agile embraced the security of a more proven position, fueled by a moral code that precipitated largely from the pattern movement (Brian Foote’s “aggressive disregard for originality”).

I’m not sure that Agile could have been born without the sober context of the pattern movement that provided the empirically grounded dialectic from which it sprang. XP pretended that it had found a magical potion that created something new, a gestalt out of just the right mixture of techniques, but few people believe that (I certainly don’t) and the premise has never been established. Scrum swings the sabre of common sense. That’s a humble, believable — and powerful — sabre.

To me, this means that the Agile backlash will look different. Instead of there being aspersions cast broadly on the notion of Agile development, doubts will arise about the less mature or mis-applied and in any case more specific practices. This is happening with TDD now ( a good procedural design technique mis-applied to OO) and I’m happy to be on the leading edge of that effort, together with some good researchers who can back up their claims with real studies. There’s a good group of people out there actually trying and measuring things, like Angela Martin and Pekka Abrahamsson and Maria Siniaalto. I don’t remember anyone inside the community taking the devil’s advocate chair during components, OO, or patterns. That smells of something different and wonderful.