The main question: What are the documented financial returns of agile? Here are the 3 main financial impacts that your executives will understand:

IMPACT #1: Higher, faster ROI

It is rarely possible for traditional projects to return investments within a year. Executives are unwilling to fund projects with long ROI. Agile shows immediate ROI. Based on data from their study, Richard showed that traditional projects show about a 33% ROI after a year. The big risk in traditional projects: We never really know how we’re doing because we haven’t delivered anything at the beginning. The big question: When are we going to deliver? These projects self-fund only at the end of the project when they deliver…maybe.

Show how agile is different: Agile releases value immediately and incrementally. This has a dramatic effect on the ROI case. With agile, self-funding hits very early in the project and profitability hits very quickly. Overall, agile shows a 12X ROI versus only 33% on traditional projects. So, agile has a much higher ROI than traditional projects. Agile uses less income and hits profitability almost immediately. Your execs should like that!

IMPACT #2: Build less, deliver more

Waterfall projects cost 2X what they need to because we’re building wasted functionality. We’ve all seen the Chaos Report and we know the numbers very well by now. The main Chaos Report number to know: 64% of features developed in traditional projects are rarely or ever used. Additionally, there is are 35%higher maintenance costs for life, as well as lower performance. This leads to wasted opportunity costs and key value is delayed.

To demonstrate the increases in the productivity of agile teams using Scrum, Richard showed details from Jeff Sutherland’s IEEE Paper Magic Potion for Code Warriors. The study showed that the company Systematic Software Engineering went from CMMI Level 1 to CMMI Level 5 and showed a 31% decrease in effort by the efficiencies they gained. Next, the team moved to Scrum. By going to Scrum, they reduced effort by 65% compared to their effort at CMMI Level 1. The effort figure includes total work, rework and project focus. Overall, Scrum showed a 50% cut in rework, 80% cut in cost, and a 40% cut in defects. This translated into a 100% increase in productivity. As a result, they now double their price for clients who require a phased approach.

IMPACT #3:

Agile productivity gains let you do more with less. Large companies and their execs want specific metrics, not survey results to prove the impact of a process. QSMA and Michael Mah have the metrics. They have a benchmarked comparison of 29 Agile projects from very large companies like CNET, Accuro Healthcare, HomeAway, Moody’s, and BMC Software. They compared these agile projects to a database of historic data from traditional projects that QSM had studied.

To provide the best index of productivity, QSMA uses this equation: Productivity Index = Size/(Time*Effort), where Size= the number stories, lines of code, and defects, Time=Calendar months, and Effort=Person-months.

To illustrate the benefits, Michael showed stats from BMC Software. BMC delivered in 5.25 months what comparable traditional teams and projects delivered in 1 year with 1/4 of the defects of traditional project teams.

Overall, the 29 projects QSMA studied showed that compared to traditional teams, agile teams have 37% faster time to market. Short iterations and feedback loops in agile are the source of this benefit. Agile teams also showed 16% productivity gains over traditional teams. That means that agile teams are doing more with less. And with 1/4 of the expected defects. Despite shortening schedules by more than 50%, defects remained steady, about 1/4 of what was expected compared to trad projects.

So, if your execs are putting your agile team under pressure, give them a few of these facts and help them understand the value of agile.

OK, so what, right? I’m riding faster now…good for me. But I was thinking about how we manage projects and develop software during my morning ride today. And I realized that my experience doing these things sort of parallels my cycling experience. I learned about project management and software development very early in my career (I was older than five). I had a good mentor and he’s someone I still hold in high regard and respect. However, what I learned back then was to keep my balance, steer the project, and keep pedaling forward. But as I matured professionally, I realized that there was more to project management and software development than this. In fact, if I stopped steering the project and let our clients and developers steer, we’d achieve better results. If I stopped always pedaling forward and took a look back once in a while, we’d improve how we worked and what we delivered. As for my role as a “project manager”…I found that if I stopped trying to ride the bike and just held the seat and gave a push to those on it, I was really doing my job and enabling the riders (our teams) to ride farther and faster.
Sometimes it hurt a little to do these things, but after a while, our teams got used to it. With time, we weren’t sore when practiced these “new” ways of working and we saw great improvements in efficiency and effectiveness. So remember, when you start implementing agile project management and development practices, it going to hurt a little, but it’ll get better over time. And when you’re starting out and going through agile adoption curve, keep this in mind: If it hurts, you’re probably doing it right.

I’m not sure I see much useful data here.  What I think is so valuable about Scrum and tracking our progress in Rally is that we have actual metrics being reported every day directly by our development team.  We have iteration and release burndown charts that show our progress on a daily basis from real reported metrics.  We have a prioritized project backlog that shows what’s been completed and what we have left to do.  Our user stories in the backlog all have story points associated with them, and based on our team velocity, we can estimate how long it will take to complete the entire backlog or any portion of it.  Within our iteration backlogs, we have fine grained tasks with estimated and actual hours for each task.  In addition, we can add hourly rates to each developer, and based on the actuals reported through Rally, we can derive our budget status, which is especially important if we are working on a fixed-price contract.

I think that the metrics provided by using Scrum and Rally together provides far more information and visibility into the state of a project than a simple line item status report.  And, because the metrics are based on actuals being provided in near-real time by project team members, executives and customers can “peek” into the project at any given moment and know exactly what the situation is.  They don’t need to wait for the weekly or monthly status reports.  Rally is especially helpful in this case because it provides project, program, and workspace dashboard widgets which can provide burndown charts for all currently active projects.  With agile project reporting, there is nowhere to hide poor performance with generic terms that don’t deliver much information.  Everything is visible and accessible to everyone.

He also hit on one of the topics that I really think is an issue in software management today and that is the difference between effectiveness and efficiency in software development.  I have written about this before and feel very passionate about it, and it’s worth mentioning again.  Cooper provided a quote from Peter Drucker that I think really summarizes this argument very well:

Effectiveness is more important that efficiency.  Business can decline and even fail at the same time that process reform is dramatically improving efficiency by saving money for the company.

I would agree and would argue along with Cooper that ignoring effectiveness to focus on cost reduction is a primary cause of software construction failures.

Cooper also made some great analogies to describe the way software is developed.  He doesn’t see software development as an industrial activity.  He believes it’s more like a pre-industrial craft in that things are made one at a time, they’re not formulaic, and they are complicated and nuanced.  But, he also see’s it as a post-industrial craft with more pieces, built by disparate teams, with abstracted notions and massive interaction between the parts.  He described the post-industrial craft of software development as existing in a brittle environment with invisible or intangible products, and working in a world of rapidly evolving tools.  I agree with most of that, except I’d like to believe that the products we produce aren’t invisible to our end users.

I really liked the points discussed above.  However, I found the remainder of his talk to be a little off-base and very biased to a waterfall or modified waterfall approach to software development.  In addition to promoting a waterfall approach, he made several misstatements about how agile teams work and plan.  His first comment was about planning.  He advocated that to successfully manage software projects, you must have detailed written plans.  He also claimed that contrary to engineers’ claims, requirements don’t change.  He went on to slam agile practices by saying that agile practitioners say that “We shouldn’t plan because things change so rapidly”.  Anyone who has managed a software project using agile practices knows this is a huge waterfallacy (a fallacy about agile spread around by waterfall advocates).  Agile teams do plan at many different levels.  In fact, studies show that over the life of a project, agile teams do more planning that traditional development teams…they just do it iteratively.

Where Cooper really began to diverge from an agile approach to software development was in his description of what he called The Software Construction Blueprint.  Here are the required elements of Cooper’s “blueprint”:

1. A narrative description of user personas and scenarios
2. A behavioral overview in narrative form
3. Detailed form and behavior specifications document
4. Detailed code examples showing how software will work
5. Detailed schedule and test plan

According to Cooper, by definition, blueprints are buildable and biddable.  If you build the blueprint described above, Cooper wholeheartedly believes that you can give a fixed price bid that you are confident in.  Now, I don’t know about you, but I’ve worked on many projects that produced blueprints or whatever you want to call it, but it was heavy up-front requirements analyses.  They were not successful!  On top of that, our customers are not paying us for all of these narratives and specifications…they’re paying us for working software.  Now, according to Cooper, an interaction designer builds this entire blueprint and distills everything anyone needs to know about building the software and hands off the design to a production team that actually builds the software.  Because the interaction designer does his job and divine’s every requirement exactly, there should be no questions or problems negotiating the software development minefield on the part of the production team.

I have several issues with this line of thinking.  First and foremost, I believe this sets up serious silos within the development organization.  It says that the production team has no input on design and design has no real input on production beyond initial design.  I really believe that teams shouldn’t have different titles and divided responsibilities.  The team is the team, period. Break down these silos and strive for more cross-functional teams.  It builds a better understanding of the various project roles and makes for more effective teams.

A side issue I have to throw out there is the fact that Cooper is an interaction designer and really came across as selling his services in the keynote (something I found rather distasteful).  Afterward, someone asked Cooper if he or his organization ever builds what they’ve designed.  He answered “No, we don’t”.  I won’t elaborate here, but I’m pretty sure you’re thinking what I’m thinking.

The bigger issue with his line of thinking is that all requirements can be defined up front and that they don’t change over time.  We’ve all worked software projects before and we all know this isn’t true.  Customers essentially have some idea of what they want, but as the software is developed and demonstrated, they begin to understand what they really want.  In agile, we can address those new ideas or changes without any worry that our big upfront design and planning would have been wasted.  But, after asking Mr. Cooper about customer involvement in the production phase of his development model, I completely understood why he believes that requirements don’t change.  My question was simple, “Where in the production process do you share completed functionality with the customer and how do you manage the change that is inherent when a client begins looking at what has been developed?”  Aside from the relative hostility with which he answered this question, it was evident that in his model, there is no communication or collaboration with the customer in the production phase.  Apparently, it is possible for an interaction designer to not only define every last requirement up front, but they also have the innate capability to anticipate every change the client would make and addresses them in the blueprint. I guess it would stand to reason that if you ignore your customers and not allow their voice to be heard in the production phase, you’d think requirements don’t change too!

What really perturbed me was that Cooper said he doesn’t believe that the customer even knows what they want, and that the interaction designer knows better than anyone what they need.  I take real umbrage with this assertion.  Essentially, his opinion was ignore your customers, you know what they need.  This to me is a real problem in the software industry.  This kind of thinking is exactly what leads to the 65% of features in most software that are rarely or never used (Check out the Standish Group’s CHAOS Report).  Having someone of Cooper’s stature stand up and advocate this is not going to do anything good for the software development industry.  And, considering the already low rate of agile adoption in the GIS development world, the last thing that we needed to hear in a keynote session were these toxic, old-school notions of software development.  I’ve been searching for a term to describe this way of thinking and thanks to the keynote speech at the Dev Summit this week, I think I have one now…The Cooper Syndrome.  Maybe at next year’s Dev Summit, Dave Bouwman, myself, or someone else can give the agile counterpoint to the Cooper Syndrome.

If you want some more insight on the keynote, check out Jeff Certain’s blog post on the keynote.

Our other option is to do iteration based bonuses.  This idea came to me from my friend Bruce Scott who works at a Silicon Valley company called ParAccel.  Bruce ran a few “experiments” with agile teams and iteration based bonuses.  Before I just launch into the results of Bruce’s experiments you should know who Bruce is.

Bruce has more than 25 years of experience building successful and profitable IT enterprises.  He has been responsible for product management and engineering, research, services and support at SenSage. As VP of Engineering, he improved the product strategy and engineering process, leading to strategic partnerships with EMC, IBM, HP and Tokyo Electron. He also founded and served as CEO of PointBase, a Java-based embedded database provider. Building the company from the ground up, his licensing savvy and strategic partnerships led to its successful sale to DataMirror.  Back in 1984, Bruce co-founded Gupta Corp. As VP of Database and Connectivity R&D, he developed its first database product, SQLBase, the first database server to run on a PC LAN and support the client-server architecture. Prior to Gupta, Bruce was one of four co-founders of Oracle Corp., serving as the principle engineer and architect of Oracle database’s first three versions. Hundreds of thousands of Oracle users know Bruce by the product’s default username and password “Scott/Tiger.” So, Bruce has credibility if know what I mean.

Bruce’s experiment was simple.  In the middle of an iteration that appeared stagnant or not very likely to conclude successfully, he announced that if his development team burned down the iteration to zero by then end of the iteration, each team member would receive a $1000 bonus.  Here’s what the burndown chart looked like for that iteration:

>

What looked to be a failing iteration was rescued by providing an incentive to burn down the iteration successfully.  The burndown rate following the bonus announcement is staggering.

The second experiment Bruce ran was to announce a $1000 bonus at the start of the next iteration.  Here’s what the burndown chart looked like for that iteration:

According to Bruce, the team stayed below the ideal burndown line until 1/30 where they added another significant story. The story wasn’t completed and was removed at the end of the sprint. This was a 31 point sprint, which was the highest this team had ever done. The sprint velocity before this was 23.5!

So, judging from Bruce’s experiments, I’d say iteration based bonuses look like pretty good incentives to keep agile teams going.  And, when it comes to project based bonuses, here’s Bruce’s take on the subject:

On the topic of bonuses per sprint or per project, I had the identical discussion with my CEO. I convinced him to do it per sprint for the following reasons:

1) A project bonus is un-Agile like. What if the scope of the project changes? This is easier and more realistic to manage on a sprint than a project basis.

2) There is a backpackers saying in regards to minimizing the weight of a backpack that says “watch the ounces and the pounds will take care of themselves.”

3) Per sprint bonuses provide more immediate rewards and have a more likely impact on velocity than a long term reward.

I’m not sure which we’ll finally implement at DTS, but I’m leaning heavily toward iteration based bonuses based on Bruce’s results.  If you have any opinions on the topic, or have any metrics you’d like share based on bonuses or incentives, I’d love to hear them.

The backlog I assembled is for the rollout of agile practices throughout our company’s enterprise.  So, I have some things that require immediate attention and some that I know we won’t be considering for a few months.  What I decided was to create a backlog with several planning horizons embedded in it.  The near term horizon has stories that are well defined.  The medium range horizon (2-3 months out) has stories that are defined but are still kind of fuzzy.  And the long range horizon (3-6 months or more out) has stories that are really just headlines.  As we move ahead in time, I’ll work to increase the granularity of the medium and long range stories as they get closer to implementation.  We do this on all of our agile projects.  What is does is effectively reduce the waste of spending too much upfront time defining the details of a story that may or may not ever be implemented.

The next thing I had to do was start prioritizing the stories.  Instead of spending too much time prioritizing the entire list, I actually went through the list and did a top ten list of stories.  I know that we aren’t going to work through more than 10 stories in the next 4 weeks, so I think that prioritizing beyond that can be wasteful as well.  As we complete stories on the top ten list, we’ll move new stories into the top ten to keep it constantly stocked.

I think that the use of both of these ideas keeps the backlog in alignment with value production and reduce waste.

Short of a unifying statement of sorts, how else can you keep your team synchronized with the goal’s of the project, release or iteration?    That’s where the beauty of agile practices comes in.  There are several practices which foster a cohesive, shared understanding of the goals: the iteration planning meetings, the iteration review, the retrospectives…but none more than the daily stand-up.  I think the daily stand-up helps anchor agile teams to their goals more than anything else.  On a daily basis, drift is kept in check by synchronizing the work underway.  If teams treat their daily stand-ups more as a synchronization conversation rather than a status report, I think the daily stand up can go a long way towards preventing drift from happening.

So, we’ve decided that the best of course of action is to strike a good balance between architecture/framework and functionality development.  In earlier sprints, we’ve accepted that architecture/framework will be a big part of what we’re working on.  But we’ve also become cognizant of the fact that we need to show something to the client in terms of functionality.  So, we usually comb through the backlog and pick the highest prioritized backlog item that we can complete early in the first sprints, while still doing the architecture/framework tasks we need to get done.  In this way, we ensure that the client sees forward progress that they can touch and understand right out of the gate, and we build the architecture and framework to support future development efforts on the project.  As we progress through our development, architecture/framework tasks become fewer and our delivery of functionality increases steadily.  In graphic form, it looks something like this:

One of the issues this has raised for us is how valid our current user story estimates are.  Right now, everything seems more complex because we are researching just about everything we’re doing.  Will the story points we assign to a user story today be valid in 6 months when our tasks become commonplace?  Can we use our current velocity based on our learning curve to provide estimates for new work? We have lots of questions running through our heads right now and we’re coming to grips with some of the answers.

Earlier today I was speaking with Ryan Martens, the CTO and Founder of Rally Software Development, and I asked him what he would do in this situation.  Ryan presented two solutions, both of which I think are very useful and effective for resolving our estimating issues.

The 3-Month Sliding Window Approach

One way to address the learning curve is to use a 3-month sliding window approach.  In this approach, recent history is more important than long-term history.  Go ahead and give the user stories the story points you think they deserve based on your current level of knowledge.  But, as you go forward use the past 3-month’s worth of estimates to derive your team’s velocity.  Keep the 3-months of estimates and velocities sliding forward so that as you move forward and learn more, your estimates and velocity are based on your most recent metrics, not on those gathered during your earlier stages in the learning curve.  The further along in your technology transition you progress, the more stable your estimates will become.  Once your transition is complete, your estimates and velocity should be far more valid than they were at the beginning of your transition.

Factor Out the Learning Curve

Another way to deal with the learning curve is to just factor it out of the equation.  Split the new complex user stories into two pieces: the research component and the actual development component.  Add research spikes to your iterations to learn how to do whatever it is you’re getting up to speed on.  Once you figure out how to do it, then estimate the user story in another iteration.  The isolated user story estimate should be closer to a valid value than if you considered it together with your research task.

I have worked with product owners in the past that considered everything a priority. That’s not reality and it certainly doesn’t help the development team focus on delivering valuable functionality quickly. So, how do we deal with a product owner that can’t make these distinctions? How do we help them understand their own priorities? It’s a tricky proposition, especially in the consulting world. It’s hard to tell a client that something they want isn’t important or isn’t of high value. But, you’ll run the risk of having a dissatisfied customer if you simply go along with the “everything is a priority” mode of thinking.

Here’s some practical advice to help you help your product owners prioritize. First and foremost, understand that there is no exact science to prioritizing. With that caveat, one of the techniques we have used that seems to be effective is an adaptation of Karl Weigers’ benefit-penalty prioritization method.  We combine this with a version of the planning poker game and call it “prioritization poker”.  Using this technique, we ask our product owner and other stakeholders to walk through a list of established user stories. We then ask them to estimate the benefit that the user story will provide on a scale of 1 to 9 (1 being of no real benefit and 9 being the highest benefit). We then do the same thing again, except we have the product owner and stakeholders assign an estimate of the penalty for NOT implementing the user story, again on a scale of 1-9 (1 being of no penalty and 9 being the largest penalty).

We then add the numbers together to get a total value for the user story. Lower total value numbers are great indicators of what Wiegers calls “gold plating”…you know, those 65% of requirements that are rarely or never used when an application is complete. To understand how the user stories relate to each other in a real prioritization scheme, we add all of the total values for the user stories to come up with a total value for the entire application. We then divide the user story value by the total value to arrive at a value percent. We then sort the user stories by the value percent to identify the most valuable user stories to be developed. In table form it looks something like this:

Agile teams should also base their work on regularly scheduled releases. These releases should be a culmination of a series of functions and features developed over the course of several iterations. These product releases will be full releases of useable, valuable features to the product owner and their organization. A release review similar to the iteration review described above can be conducted to review release functionality. The product owner will have the ability to accept the functionality within a specified time period after the release date. This provides a second level of acceptance for the product owner.

The final release of the product should follow the same pattern as the release cycle described above. In this way, the final release should really have no other fanfare than any other iteration or release (except for your team’s joy at being done with a product). Because your team has been releasing and gaining client acceptance on a bi-weekly basis, there should be no surprises in this final release. Final client acceptance should just be a formalization of the collective acceptance that has been built up over the life of the project.

However, on the drive home to Ft. Collins, Dave Bouwman and I had a discussion about change management in the hybrid world of fixed-price agile contracting. The main point was about contract negotiation over the lifespan of an agile project. Now, I know…one of the tenets of the Agile Manifesto is that we value client communication over contract negotiation.  While this may be true, there is also an element of reality that requires us as consulting-ware developers to have to deal with contract negotiations in the fixed-price world.

In the consulting world, we are frequently bound by contracts that, at some level, define the requirements we need to deliver.  As we proceed through a project using agile practices, we frequently rearrange the priorities of these requirements (or user stories).  In addition to rearranging the priority of user stories, we often find it necessary during a project’s lifespan to remove some stories from the backlog and add others.  Whether we like it or not…whether we consider it agile or something else, these changes to the product/project backlog may require change orders. Most of the time, this is not really ideal for either the consultant or the client. We’d rather just embrace the change, and move on to continue developing valuable software.

However, at most government agencies and even some private companies, there is a contracting officer, whose job it is to check the “boxes” to confirm that you delivered functionality X, Y, and Z at the end of the project. If we make a noticeable change to the requirements (especially removing one), we may need to change those “boxes” so we don’t get dinged for non-performance. Not quite agile…but a very realistic and common scenario.

So, back to the manifesto…client communication over contract negotiation, right? Well, I’m not sure that these two need to be mutually exclusive (and I don’t believe that the manifesto implies this either). I believe that they are both very relevant in the consulting-ware world. I think that if you establish an open, collaborative relationship with your client that thrives on honest communication, then contract negotiation (especially change management) can be relatively painless. If your client truly understands the agile process and feels well informed and confident about scope changes, they can become a champion for these changes with their contracting agent. They will be able to communicate in clear terms why the changes need to be affected and how these changes provide increased value and quality to their end-product. This could create an easier path to official change orders for you and your client.

I also believe that over time, your client may come to really appreciate the success of agile practices and possibly “sell” their contracting office on the virtues of fully agile contracts. If you and your team work to consistently deliver value and quality using agile practices, maybe you can start to help organizations change the way the way they view software development contracts in the future. Nothing sells better than success!

Rachel Weston

We’ll also be organizing an Agile Contracting discussion at the Agile Development Practices Conference in Orlando, FL, December 3-6.  If you’re interested, please contact Rachel or myself as we are trying to organize this discussion as one of the Open Space sessions on Thursday, December 6.

How to Sell Agile to the Customer A discussion thread on Facebook in the Agile and Lean Development Community group.   Also, I highly recommend joining this group if you’re interested in agile practices.

Changing the Game: Making Agile the “Default Methodology” A short post by my colleague Dave Bouwman describing a company’s view on presenting agile practices at a short list interview as their default methodology.  And it worked.

Winning a fixed-price RFP with an agile proposal and There is no such thing as fixed scope by Eduard Liebenberger

Fixed bids, agile projects by Oren Eini

Optional scope contracts by Kent Beck

Since this appears to be a topic of concern for most agile practitioners, I’ll keep posting any new resources about this issue as I find them.  Thanks to everyone who has contributed to this growing list.

The Dire Consequences of Fixed-Price Contracts on Dr. Dobb’s by Scott Ambler

Fixed Price, Fixed Date Contracts at Engage by Ken Schwaber

Agile Fixed Price Contracting on InfoQ by Deborah Hartman

Fixed Price Contract & Agile Software Development: an “experience in process” report by Christine Moore of Caribou Lake Software

Why Fixed Bids are Bad for Clients on Scrum Alliance by Andy Brandt

Agile Fixed Price Projects Part 1: The Price is Right by Pascal Van Cauwenberghe of Nayima

Agile Fixed Price Projects Part 2: Do you want agility with that? by Pascal Van Cauwenberghe of Nayima

The Illusion of Fixed-Price Contracts by Simon Baker on Agile in Action

More on Fixed Price Agile on The Webfarm Blogs by Paul Webb

I’m sure there are plenty more out there as well.  If you have some good links about this topic, please feel free to post a comment with the link.

Scott Ambler had a good post on this topic and he states that:

“Fixed-price projects are a common practice in IT, either as the result of a fixed-cost competitive bid or as the result of budgeting pressures on internal software development projects.  Organizations often prefer fixed-price projects in an attempt to decrease their financial risk. Unfortunately, the exact opposite seems to happen in practice. Everyone seems to know this, but we somehow can’t find a way to get out of this rut.”

This is something I’ve been wrestling with for quite some time.  In fact, one of my office’s main projects is a severely under-budgeted fixed-price fixed-date project and we have been struggling to manage it using Scrum.  At this point, we have forgone prioritizing the backlog with our client, because according to the contract (and our client), we have to deliver everything…therefore everything is a priority.  We also have to deliver by specific dates and with specific budgets for each release.  Essentially, we’re using Scrum to manage our bi-weekly tasking, but that’s about it.

What I’m wondering now is, can Scrum really work for fixed-price contracts?  I’m not sure.  In a fixed-price scenario, we’d really have no other way to work than to do heavy up front requirements analysis and design so that we can “accurately” bid the contract.  That would mean reverting to Scrum within a waterfall model (if we were to even try to do the project in an agile manner).  But then there’s a Catch-22.  Every developer knows deep down that at the start of a project, there’s no way to accurately estimate the duration of any development task.  The true complexity of any requirement is never really known until you start working on it.  No matter how in-depth the requirements are, there is always some ambiguity.  When a contract is fixed-price fixed-date, we go against all better judgement and provide cost estimates and delivery dates.  We revert to waterfall thinking.  And in the end, we usually go over budget, and our customers probably don’t get what they really want.

So my fellow scrummers, I have three questions for you:

1. Given a project that requires you to work in a fixed-price fixed-date environment, would you take the project or not?
2. If you took the project, how would you effectively use scrum or agile practices to manage it?
3. How do we, as Scrum and agile practitioners, educate potential clients about the perils of fixed-price fixed-date contracts?

The other night, I was at a get together of some regional ArcGIS developers. They asked about Scrum and we started talking about the requirement that every iteration must produce some potentially shippable product increment. Being GIS folks, someone asked how you would apply that “rule” to non-development GIS projects. It made me sit back and think for a bit. So, out of curiosity, and to find out if anyone out there used an Agile approach to these types of projects, I searched Google using several appropriate search phrases. To my astonishment, I couldn’t find a single reference to anyone using Agile methods for non-development GIS Projects. In fact, most led me to very rigid, stepwise, waterfall approaches to GIS project management.

Before I got heavily involved in application development and enterprise GIS architecture, I was a GIS analyst and project manager on lots of non-development GIS projects. How would I have applied Scrum to those projects? I thought about that for a while. First I thought, “Maybe it doesn’t work outside of our little software development fiefdom?” I went to sleep that night without a clear answer in my head. But the next day, I thought some more and decided that Agile practices, and Scrum, could be used to manage non-development GIS projects without any alteration. For one thing, Scrum is not really a set of “rules”, but more a set of guidelines. Ken Schwaber in his book The Enterprise and Scrum states that projects are “too complex for a single set of rules to suffice in all circumstances”. My second thought was that the phrase “potentially shippable product increment” was confusing to some. To me, what it boils down to is that at the end of an iteration, a Team must produce something of value as defined by the Product Owner.

Applying my “new” definition, I started to think about how this would apply to non-development GIS projects. What’s valuable to a GIS Product Owner? Data, definitely data. Analyses, map products, metadata (always of questionable value), models, and probably others I’m not thinking of. Now, in the development world, to deliver a potentially shippable increment (or “something of value“) every iteration, we decompose user stories into their simplest forms and provide targeted functionality to address that story. In non-development projects, we can do the same thing.

Let’s consider an example involving a mutli-layer statewide data request from your Product Owner. We don’t have to deliver all of the data in a single iteration. Have your Product Owner prioritize the data layers. Maybe the hydrologic data layer is a higher priority than the land-use layer. If the hydrologic layer will take longer to deliver than one iteration, break it into smaller prioritized pieces. Maybe your Product Owner is really only interested in second order streams. Produce all of the second order otream data during your next iteration. If you cannot complete that data layer in a single iteration, break it down again. Maybe your Product Owner is currently interested in second order streams in Kitsap County. You probably get the point by now. It works for data production and will work for analyses, map products, models…and yes, even the dreaded metadata! Work can always be decomposed and re-prioritized by you and your Product Owner.

The point is to constantly provide your Product Owner with something of value that s/he can use at the end of every iteration. In our statewide data example above, maybe your Product Owner can only view second order streams in Kitsap County this week, but s/he can at least do that. The next iteration may include second order streams in King County, and so on. This can even be a benefit to your Team. You may find that you do not have to deliver a hydrologic data set for the entire state of Washington. As you deliver valuable increments of data, your Product Owner may decide that the Puget Sound area was really all that was needed now that s/he has been using the data you already delivered. You’re done with that request, on to the next “something of value“!

So, keep delivering. Keep getting feedback. Keep adapting your process to deliver the most value possible to your Product Owner. It doesn’t matter if it’s a GIS application, analysis, map, model, or metadata…you’ll find your Team delivering higher quality, higher value GIS products faster and more efficiently using Agile practices.