Over the last several months, I’ve been thinking more and more that THE underlying principle in all of management and leadership is “balance”. From Ken Blanchard’s “situational leadership” to Greenleaf’s “Servant Leadership”, I’m seeing a pattern that all-or-nothing positions are at the root of the most dangerous and impactful decisions made at work. Instead, the more sustainable position is one that strikes a balance between a spectrum of opposing positions.

I know it sounds rather nebulous, but it sets the tone for why I believe the idea of “balance” helps us navigate the awkwardness around “self-organized estimation”. Whether we are asked the question of “when will this work be done” or “how much work can we get done by the deadline”, your preferred approach to estimation  will be driven by how much influence you give to the team, versus how much you give to the customer or sponsor.

First, we define the spectrum of two opposites we’re working with. For self-organization, the spectrum moves from the extreme left of anarchy to the extreme right of fascism.

High Anarchy (“Estimation is Waste”): Believe it or not, there are experts out there who say you shouldn’t have to estimate your work at all. “The work will get done, when it gets done”. Let’s say the business comes to ask you, “We have shareholders to manage and analysts to satisfy. Our Marketing department needs to know when to have their campaign done. So, how much will this new upgrade cost and when will it be done?”. The anarchist approach is to say “no sir, you are asking the wrong question. You should only focus on the product, and none of the business considerations that surround the product.”  As you can imagine, this is simply not defensible, and most likely will be a career limiting conversation.

High Fascism (“Do exactly this by exactly this time”): Stop me if you’ve heard this one before: the boss comes into the office and says, “you 5 people need to build the entire product exactly as defined in the giant binder of specifications by the end of the month. We can’t leave out any features, we can’t slip the date, and you go no extra help.”  This is the textbook definition of a death march; there is no chance of success.

The anarchist wants to avoid the death march. The fascist wants to avoid randomness and variability. Both are willing to take the extreme position to get what they want and avoid what they don’t want.

Now, let’s get to some of the problematic techniques we see in large scale project estimation.

(Mild Anarchy) Every team does it their own way: Here, we place the expectation on our project teams that they have to have some kind of answer to the business question. However, we want to preserve some degree of self-organization, so we allow them to self-organize in opposite directions. Which gets in the way of the getting the answer we need.

(Mild Fascism) Every team must do it the same way: Here, we simplify the problem by demanding consistency across the teams. If everyone uses Ideal Days, we can add all the data together rather easily, but we sacrifice a lot of the self-organizing dynamic and the goodness that comes with it. Not as bad as before, but also not ideal.

You can see where this is going. The further you move away from an extreme position, the more defensible you position becomes. Or more to the point of this specific conversation

The more you rely only on self-organization or only on expert judgement, the less reliable your projects will be

Business who issue top-down mandates on projects and deadlines lose the benefit of the reality-on-the-ground. On the other hand, those who won’t even attempt to forecast their outputs risk becoming irrelevant to the business world.  So I’m looking for a holistic approach to estimating large projects that balances these two forces. My next post will propose a general framework for doing exactly that.

Meanwhile, what do you think? What does a balanced approach look like? How can we get multiple teams on a large program to answer the question, in a way that is both reliable AND self-organizing?

Sure, all the self-organized team estimation techniques work fine when your planning work for a single team. However, I’m finding more and more that single-team techniques (story points, ideal days, T-shirts, etc) break down when applied to large programs and portfolios.  I know this, because I learned it the hard way.

THE PROBLEM

Let’s say we’re working at a company, where several product lines each need access to several agile teams at the same time. Some of these were component teams and some were feature teams. Pretty standard deal.

One day the lead PM comes to me and says, “Jesse, we have a new project that touches 4 agile teams. Each of them uses a different estimation method. I’m going to the portfolio review next week to get this project authorized, where the executive team needs to know much it will cost, and when it will be done. How to I merge their different estimates into a single answer?”

THE CLICHE’ REPSONSES

1. [Calculate the Average].

Me: “Just average all the estimates together and divide by the average velocity to get the number of iterations needed to finish the work. Once you have the number of iterations, you can easily calculate the schedule and the budget. The mean is a least-squares-sum statistic, and corrects for the intrinsic variation across the teams.”

Greg: “Okay, but here’s my problem. Every two weeks, one team delivers an average of 11 story points. Another averages 55 ideal days. The third team uses T-shirt sizes, and the fourth team averages 97  ’agile function points’. If I mash that all together, I’ll just get soup.”

Apparently, they had learned all these techniques in agile training, MY agile training. In every class I teach, I train people to use any one of a variety of popular estimation techniques, you know, just to give them options. It never occurred to me that so many teams within THE SAME organization would use EVERY technique I taught, and then self-organize themselves into a gordian knot. So I offered a different suggestion:

2. [Implement a universal estimation standard].

Me: “Let’s quickly teach all 4 teams to use ideal days. No story points. No T-shirt sizes. One size fits all.”

Him: “Okay Jesse, but here’s my problem. Our flagship agile pilot team has been using story points, for over a year. A newer team is using T-shirt sizes, and in 1 month they’re already at a 100% hit rate on their commitments. That’s great momentum on the norming-performing scale. If I go back and tell those guys they were wrong about how they estimate their work, they’ll point to their results and then ignore me. I lose a ton of credibility on the whole self-organizing topic we’ve been coaching.”

Ug. He had a point. I was in a coaching quagmire. Credibility with the teams or with the CTO? I knew this was a false choice. So, I put together another option:

3.[The convoluted spreadsheet].

Me: “Go ask each team to use their own estimation technique for their piece of each feature. Then, divide by their respective velocities to get a forecasted number of iterations per team per feature. Then add it all up. You’ll end up with a spreadsheet that looks like this:

PM: “Okay Jesse, I can do that this time for this small project, but here’s my problem. We have dozens of projects that need to be coordinated across dozens of teams.  If we have to generate one of these spreadsheets for every project, then the teams will get sucked up into a TON of estimation and planning. How do I get all the productivity gains if my delivery teams are spending half their time in planning meetings? Doesn’t that amount of big-up-front-planning violate the rule of progressive elaboration?”

THE QUESTION

You can imagine this was a painful situation. But the real tragedy is that this happens over, and over, and over again. Something is wrong here, and I’m going to be spending the next few posts discussing the underlying forces and how they impact the techniques we’ve been teaching and using.

So, here is my question to you: What are we doing wrong? How to we keep true to seemingly contradictory forces of “self-organizing teams” and “value vs. cost prioritization” and “progressive elaboration”? 

courtesy a n i. Y.

I’ve had it.

The other week I heard the umpteenth person make some misguided, mistaken declaration about the rules of Scrum. I hear it way too often. “Scrum says this” or “Scrum says that”. There is a ton of misunderstanding out there about what Scrum is and what it isn’t, and it’s driving me crazy. Using the Scrum.org Scrum Guide and the ScrumAlliance’s definition of Core Scrum in the Agile Atlas as our official definitions of Scrum, let’s see if you’ve heard any of these biggest offenders:

5) “Scrum says we have to estimate with story points”. Um, no it doesn’t. The Guide and the Atlas give very specific guidelines for when to estimate Product Backlog Items. So, you may be surprised to learn Story Points are decidedly NOT included in those rules. In fact, Scrum co-founder Ken Schwaber has been an ardent promoter of Ideal Days. Which estimation technique you use DOES NOT MATTER, only that you use the technique consistently.

4) “Scrum says the product backlog may only include User Stories”. Oh, and while we’re at it, user stories are NOT mandatory.  YES, user stories are a great technique, and YES they facilitate a more robust understanding of the work that needs to be done. But the Scrum Guide does not mention them at all, and the Agile Atlas lists them only as a Generally Accepted Scrum Practice (GASP) and not a part of Core Scrum . If you have a thousand requirements formatted as “the system shall provide…”, and they consistently meet the INVEST principle, then please don’t go spending 2 months rewriting them into the “as a, I want, so that” format. Also, don’t forget infrastructure work and spikes should be well groomed Product Backlog Items, but force-fitting them into User Stories can often be clumsy. The goal of any Sprint is to produce a product increment of running, tested features; however you do that is up to you.

3) “Scrum says the Product Owner is not allowed to attend our meetings.” Okay, I see why you might misinterpret this from the Scrum Guide’s rule that ”only Development Team members participate in the Daily Scrum.”  However, the Atlas asserts a clarifying counterpoint that “only the Scrum Team members, including ScrumMaster and Product Owner, speak during this meeting.” Specifically, it is totally possible (and even desirable) for a ScrumMaster to facilitate a positive dynamic for the meeting, without actively participating in the conversation. Likewise, as a full and equal member of the whole Scrum Team, the Product Owner has every vested interest to observe the meeting, and offer some assistance to the Dev Team Members. The key here is that we protect the self-organizing dynamic of both the internal Development Team and the broader Scrum Team.

2) “Scrum says the Product Owner must be the product business sponsor”. No, Scrum doesn’t say that. Scrum is a team-level method. The Atlas says “The Product Owner is typically the individual [on the Scrum team] closest to the ‘business side’ of the project.” Each and every Scrum team has a team-level business point of contact “who is expected to do the best possible job of satisfying all the stakeholders.” That includes the product business sponsor, the executives, and even the Product Owners of the product’s other Scrum Teams. Often, the business sponsor doesn’t have enough people to staff a team-facing Product Owner for each and every Scrum team. So, we will recruit a Development Team (a business analyst or a tester, maybe) to step up and play the part of the team-level Product Owner. Then, that person will be the one “responsible for maximizing the value the product and the work of [that specific] Development Team.”

…and coming up at Number 1…

1) “Scrum means we don’t do any up front planning”. Really? Where does is say that? If you believe that, then let me ask you this: where does the Product Backlog come from? Can you start Sprint 1 without one? You have to do some minimal amount of Product planning BEFORE you begin formal Sprint Planning. In fact, both standards require Grooming or Refinement as a means ”to prepare for the upcoming Sprints” [fromt the Atlas]. Then, after the first Sprint begins, the Guide explains that Grooming continues “as a part-time activity during a Sprint between the Product Owner and the Development Team…[consuming] no more than 10% of the capacity of the Development Team.”  Furthermore, the larger the product initiative, the more Grooming and Refinement will be needed in advance. Just don’t go planning all the details of all the features for the whole product.

What about you? Have you heard these in your workplace? What other have you heard that are annoying?

I often get asked “What is the role of a BA in an agile environment.” Much like a project manager, a business analyst carries a set of expectations and is legitimately interested in how that changes when we change our approach to work. If you’re a BA, product manager, or someone that works with them, come join the conversation.

Title:
Agile Paralysis: How Agile impacts your role and the BABOK

When: Tuesday Jan 8th @ 7pm

Location:
UMBC Technology Center
1450 South Rolling Road
Baltimore, MD 21227

Description: 
It’s official: Agile is all the rage. In businesses and projects all across the world, we’re being told to use Agile methods to be more successful. But how is that possible, if it means we don’t do documentation any more? What is the role of a BA on an Agile team? And with the upcoming release of the Agile Extension To The BABOK, what does the IIBA think of all of this?

Link: http://baltimore.iiba.org/index.php/chapter-events/event-listing/details/74-Jan13-chapter-meeting

This post was originally published (p28) as part of my recurring “Agile Project Manager” column in PM Network, PMI’s membership magazine. Other installments can be found here.

It drives me crazy. I heard an agile consultant this week say, “We’re trying to force the client to track progress they way we want her to”. Really? Good luck with that.

People tell me over and over the things self-appointed agile experts have said, and when you think about it, not only do they violate the spirit of the agile movement, they just sound silly. Tell me if you’ve heard one of these before.

“Our project will succeed only with a full and immediate implementation of an agile methodology, and its associated project tracking software”.

 The official definition of the agile movement can be found at agilemanifesto.org, which explicitly values Individuals and Interactions over Processes and Tools.Granted, a shrink-wrapped set of people-friendly policies can be a good foundation for transforming a losing team into a winning team. Indeed, I actively promote both the Scrum methodology and the PMI-ACP certification as good things. But that does not mean just doing those things will automatically improve team performance. Instead, a true agile practitioner will ask simple breakthrough questions: Are we all on the same page? What agreements can the team install and self-enforce, to address those gaps? How can we customize our current methodology to support those agreements?

As I’ve mentioned here before, the methodology doesn’t matter. What matters are the principles you embrace towards delivering your project.

“We’re agile now, we don’t do any project documentation”.

 This common myth is seeded in the manifesto’s favoring of Working Software over Comprehensive Documentation. Granted, many projects waste money on piles and piles of reports that nobody reads. But, that does not mean we throw everything out the window.

Instead, a true agile practitioner will seek to understand why those documents are mandated. Sometimes, a better specification can often improve the quality or accuracy of the final deliverable. Sometimes a risk register or Gantt chart can help increase credibility and stakeholder support for a project. The document doesn’t matter; what matters is that we address the underlying need.

“Fixed-price contracts are immoral. All our projects will now be time-and-materials.”

 This is a fanciful interpretation of the agile manifesto stating its preference for customer collaboration over contract negotiation. Granted, many projects are launched as the result of zero-sum negotiations leaving a project team underfunded and understaffed. But that does not mean we refuse to have the conversation.

Instead, a true agile practitioner will strive to forge a positive working relationship with her sponsor with deeper questions: What are the client’s concrete business goals for this project? If push comes to shove, can we achieve the goal with only half the features to stay on schedule? Is it possible more budget would be approved for extra safety reviews?

Fixed-price doesn’t matter. Several people have offered agile contracting structures that support what does matter: a positive working relationship around project goals (for more info, see my PMI webinar on the topic of Agile Contracts).

“We’re agile now, so we don’t have to estimate. The project will be done when it is done”

This delightful gem is inspired the value the manifesto places on responding to change over following a plan. Granted, many projects are mindlessly judged on budget and schedule, regardless of how bad or how wrong the deliverable is. But that doesn’t mean we stop planning altogether.

Instead, a true agile practitioner will generate as much meaningful data as possible about our health of our project: Will we run out of money before we achieve our goals? Does our sponsor have enough data to make hard trade-off decisions? As General Eisenhower said “plans are worthless, but planning is everything.”

“That’s not agile.”

Guest what. I don’t care what you think is agile. I care whether we’re going to deliver. In the end, a project is judged not on the mechanics used, but on the objectives met. The agile movement was launched as a reaction against a focus on bad mechanics. If you’re falling in love with your agile mechanics as the silver bullet, then you’re dangerously close to losing your agility.

What about you? What silly things have you heard agile people say, often in direct contradiction to the Agile Manifesto?

I will be presenting at the AgileDC conference next month on October 23rd.

Sprint Backlog Considered Harmful: Advanced Patterns and Paradoxes of Scrum

I’m really excited to be a part of the event this year. AgileDC has established itself as THE place to be for Washington DC area agile practitioners to trade tips, tricks, and war stories about getting work done better. Last year’s event sold out, so if you’re interested you should sign up ASAP.

Also, the event is being co-promoted by the DC chapter of the Agile Leadership Network. They’re offering a 15% discount when you register using the “ALNDC” registration code.

My “Fixed Price Agile Projects” talk has been getting some traction. Originally broadcast by the PMI Agile Community of Practice, it’s recently been picked up by PMI as part of their On-Demand Webinar series. It’s a free download for PMI members, but in case you’re not a member, I’ve posted the slides below. Meanwhile, I’ll be presenting the talk in person at the PMI Washington DC Symposium Next Week. So if you’re free, stop by and check it out.

Here’s the abstract:

Agile experts tell us fixed price projects are immoral and declare that agility can only be delivered on a slippery schedule and budget. But what about the real world? What about fixed deadlines and fixed budgets? What about projects that are selected based on schedule and cost? How do you agile that? In this session, you will learn key principles for achieving agility in a fixed-price environment. Come discover how to achieve what agile experts tell you is impossible.

• Original Webinar Broadcast Sept 15, 2011 (PMI Members Only) – This version includes live video, but the first 10 minutes are missing.
• Official PMI Webinar (PMI Members Only) – This is a much cleaner re-record, but has only audio and slides.
• Downloadable Slides (Everyone)

This post was originally published (p23) as part of my recurring “Agile Project Manager” column in PM Network, PMI’s membership magazine. Other installments can be found here.

Stop me if you’ve heard this one before: You’ve been handed a project charter, a detailed requirements document, an unrealistic plan — and a mandate to be successful. Then, as a dutiful project manager, you simply accept the unreasonable, and march forward to inevitable disaster.

An agile approach calls for a different idea: replan. Find a new path that involves the shortest distance between today’s problem and tomorrow’s victory.

FIRST, DEFINE BUSINESS SUCCESS. Each project is funded to achieve a better future for your sponsor, so you need to know his or her vision. For example, a new payroll system might be developed to lower the cost of back-office operations. Or it may be that the organization plans to dominate a niche market with an innovative product. Agile frameworks emphasize project charters for good reason. They should have this desired future documented as a high-level scope statement, maybe with some measurable business capabilities. But the important part is to ignore the detailed requirements in the giant 3,000-page contract appendix.

SECOND, DEFINE YOUR BUSINESS CONSTRAINTS. Do not simply assume that scope, schedule, cost and quality are all fixed. Instead, ask what are the most important constraints to your sponsor. Because each case is different, you must have a discussion as to which constraints are fixed and which are flexible. Then you have more options.

THIRD, DELIVER THE MINIMUM NECESSARY TO MEET THE BUSINESS WITHIN THE BUSINESS CONSTRAINTS. This is where you have to unlearn everything you’ve been taught. You’ve been told that project success is implementing all the detailed requirements with the best possible implementation, on time, on budget. But real success is not in the requirements, nor the implementation; it is achieving the high-level business goal within the business constraints. You are moving from a scope-driven project to a constraint-driven one. Agile practitioners call this “flipping the iron triangle upside-down.” Why?

1. Most of the requirements do not contribute to the desired future state: Studies reveal that more than half of all technology features are rarely or never used. That means sponsors are merely guessing which requirements will achieve more revenue, market share or customer satisfaction. Agile frameworks encourage sponsors to regularly reprioritize requirements. What if you were able to deliver the project with half the budget and schedule?
2. 2. Most implementations are over-engineered. Recently, Research in Motion announced the BlackBerry 10 will not be released for another year; as a result, the company’s stock price fell almost 40 percent. Could it be that the project team created an unacceptable schedule in the quest for cutting-edge technology, when in fact it wasn’t really needed? You don’t have to gut a building’s entire internal structure to lower its carbon footprint. Agile finds the minimum solution possible to achieve a business goal. Challenge your project teams to propose the simplest solution. Prototype that solution, and then see if it meets the business objective.

The most dramatic cause of project overruns is that we are doing more work than is absolutely necessary to achieve the high-level scope statement. We dig into the weeds and lose sight of the garden we’re trying to create. But you can be the exception. Do not simply accept the project plan handed to you. Facilitate true innovation: the most business results from the least amount of work.

Today, the Scrum Alliance has announced  its “intent to transition the Certified ScrumMaster post-course test from pass-only to pass/fail on September 1, 2012.”

The rest of the announcement follows below:

After monitoring the results of the current pass-only test over the past several weeks, we are confident in our decision to transition to the pass/fail test. Students who pass the test will receive a list of all missed questions and a list of possible answers with the correct answer highlighted.

Students who do not pass the test on the first attempt will also receive a list of missed questions and possible answers but the correct answer will not be highlighted. These students may re-take the test once more at no additional charge. Students who do not pass the exam a second time will be required to pay $25 USD for each additional attempt until they achieve a passing score of 24 out of 35 questions.

For those interested in the certification, you will now likely see a spike in interest in completing the training and exam in the next month or so. After that point, there will be the added new stress of having to pass an exam to earn the certification.

However, there will be an equal stress for scrum trainers. In the past, a Scrum trainer’s primary concern was to educate students how to use the Scrum framework effectively. Now, there will be added expectation that the class will also equip students to pass the exam, in order to earn the certification.

What are your thoughts? Does this training-based certification mean more if both the Trainer AND the Student have been evaluated? 

One of the stereotypes for agile approaches is that they only work for small projects. Ten or 15 years ago that might have been the case, but things are vastly different today. Agile techniques now are used as part of the day-to-day project operations of major organizations around the world. Over the last two years, e-commerce giant Tesco.com has implemented agile approaches across all its technology initiatives, from internationalization to mobile. A Chinese telecom recently used a blended approach of agile and enterprise methods to execute a project involving 40,000 team members. Salesforce.com transformed the whole 6,000-plus-person company to agile approaches as the official company standard. How do they do it? Here are the most popular techniques for scaling agile approaches beyond a single-team project.

Get it right at at the team Level. For the last several decades, management science has told us that the ideal size of a single team is seven people, plus or minus two. Teams of four or fewer don’t need the overhead of a dedicated team leader; anything more than nine, and the complexity curve flies off the chart. With that in mind, it would be ideal to decompose a project of 700 people into a collection of seven-person teams. Each team needs to implement an agile approach with discipline and intention in order to consistently produce a high- quality deliverable every few weeks.

Firm the big picture, and Flex the details. The larger the project, the more moving parts you have—and the more critical it becomes to get every- one aligned to the big picture. From product vision to solution architecture to work culture, the program manager needs to over-communicate the constraints of each team’s work. If we can prevent each team from going too far off course, they can be given more discretion on the details specific to that team. For example, a weekly cross-team technical committee would feature a rep- resentative from each team and be chaired by the chief project scientist or chief technical officer. The committee chairperson would broadcast the high-level con- straints and direction for the project. Then each team would implement its own detailed approach for building the deliverables within those technical and func- tional constraints. Sponsors often use this kind of committee to convey business decisions across teams. It also can be critical for the project manager to address team coordination issues.

Encourage a culture of organic coordination. Formal artifacts and ceremonies are not enough—nobody can design an organizational system for every little detail that comes up. Instead, larger programs will achieve agility only if there is a culture of on-the-ground collaboration. To achieve this, many organizations implement monthly communities of practice (CoPs) that discuss role-specific topics. The Scrum master CoP, the business analyst CoP or the engineering CoP can provide knowledge sharing and mentoring support to colleagues from other teams or even other projects. Finally, the project manager should make it clear that every team member is expected to inform someone, collaborate with a cross-team counterpart, and track down a solution.

Most agile mindsets provide a well-structured approach for small individual teams. However, if you supplement that approach with some for- mal and informal enterprise techniques, you can grow from small agile to big agile.