What makes a successful waterfall project?

I recently posted a question on linked in wanting to hear from people who have either led or been a part of a successful waterfall project.

Here's the link. There a some awesome answers. What's interesting is that what is required for a successful waterfall project is clearly defined scope, engaged sponsors, and a great, empowered team. Sadly, I rarely see all of these occur in the wild. And, these are the primary elements of what's necessary for an agile project, explaining why there is a higher success rate with agile projects.

Scrum Documentation

If you are on an agile team, and you are telling your team that you don't need to document because your "agile", stop it. Stop it right now!! You're making my life difficult!

There is a strong stereotype out there, and its that with agile, you don't document. Arggghhh!!! I'm sure some of you have experienced it.

Here is what I tell people when they ask about the documentation requirements on an agile project..."If the team needs it, then they will create it". Now, I'm not talking about documents that are delivered to the customer, like help documents. I'm talking about documents such as architecture diagrams, "requirements" documents (eww), process flow diagrams, diagrams, and more diagrams.

IF these kinds of documents will make the team more productive, and produce higher quality software then document! Now, the next thing I tell people is to be LOW TECH. I mean, c'mon, do you really have to transfer that diagram on the whiteboard to visio? I've seen people take 2 weeks to do that, as the rest of the team is "waiting", i.e WASTE. Why not take a picture? It's going to change anyway, right? And, we "want" it to change, if we ever want to improve.

Remember, the end customer does not value these kinds of documents. So, DON'T SPEND A LOT OF TIME ON THEM. If your company "requires" them, then take the lowest road approach possible. Ask questions. Challenge the "process". Processes were meant to change and improve, you should not be a victim to them.

Okay, to review....

• Documentation can be good, if it makes the team more productive and the customer happier because of higher quality. But, BE CAREFUL
  
• Low tech = Good
  
• The customer DOESN'T CARE about those documents, so just take it easy there partner. Only do the bare minimum that you NEED.

The Curse of Traditional Software Development Methodologies

I've realized that very few teams follow the waterfall methodology by the book. However, I think that most shops try...really, really hard. Of course we WANT to define every requirement and scenario that could possibly happen. Once that's complete, THEN we can start building. And, if there is a new scenario, or "missed" requirement (heaven forbid), then the developers can point at the requirement gatherers and say "AH - HA!!! You screwed up, and now you must suffer the consequences!".

Well, this is just silly, we all know that change happens, but yet we try to fight it every step of the way. That's what requirements change management is, right? Its to prevent change, i.e. just say NO! To remain competitive in today's marketplace, we just can't think like this anymore.

Below is a list of beliefs that we (including myself) really wish was true. If they were true, our jobs would be so easy! Of course, it wouldn't be as fun :-)

• The customer knows exactly what they want.
  
• We understand what the customer wants.
  
• We understand who the customer is.
  
• We can define everything up front.
  
• The technical team knows how to build what the customer wants.
  
• We can accurately predict how long it will take to develop what the customer wants.
  
• There won't be any problems (or very few).
  
• Documentation is a true measure of progress.
  
• Nothing should change. If it does, it's [fill in the blank] fault.
  
• Hours worked is infinitely directly proportional to value delivered.
  

Okay, so none of these things are really true. Since we all like to pretend, we have failed over, and over, and over again. I believe that this failure in traditional projects has caused stereotypes to evolve, including:

• The technical team is arrogant and lazy.
  
• The technical team doesn't care about the customer.
  
• The technical team is lying. It can't be as hard as they say.
  
• The customer doesn't respect the amount of work necessary to deliver what they want.
  
• The customer is stupid.
  

This failure has also caused some pretty weird behaviors to spring up.

• A tendency to over-specialize and divide ourselves into silos.
  
• The desire to control and direct every move made by every team member.
• Forgetting that development is creative and innovative and can't be treated as a commodity (this causes some real bad implementation).
• A culture of command and control.
• The desire to create a process for EVERYTHING in an attempt to control.
• A fear of sharing "too much" information with other silos...I mean other teams. If we share "too much" information, then the other teams may want "change".
• A fear of sharing too much information with the customer. Again, for a fear of change.

Please, stop the madness. Let's stop pretending that traditional development works and start working together.

Hard Questions for Agile

I am seeing a lot of the same questions come up over and over again recently. Now that larger organizations are starting to see the benefits of agile, more and more questions are coming up.

Everyone (including myself) has a desire to have a prescribed method, process, checklist, etc. that they can use to make sure they are doing things the "right way". In Scrum, there is a pretty clear list of activities, artifacts, and roles. People struggle because these lists provide the "whats" but not the "hows". For example, when I tell teams that they need to have one product owner for that project, they are sometimes at a loss; "How do we find one product owner? Our organization won't support that!".

Below is a list of questions that I have seen come up over and over again. I'm going to address them over time (as I have more time).

1. How can agile work on large projects?
• I know this is being addressed slowly, but there are just not enough case studies out there. The standard "Scrum of Scrums" just doesn't cut it anymore, meaning that there are serious complexities that aren't addressed in the standard definition.
2. How do I work with a vendor that does not want to take and agile approach?
3. How does interaction design work in an agile environment?
   
• I have not met any interaction designers (HCI type people) who really buy into the concept of iterative, incremental development.
4. Our company is committed to PMBOK, how can agile fit in this environment?
5. Our company needs to make financial projections that are 5 years out. How can we do this using an agile approach without trying to plan everything up front?
   
• We in agile know that it is a myth that you can accurately plan everything up front.
6. Our company needs to be CMMI compliant to be able to work with government entities. How can we be CMMI level 1-5 compliant and still follow the agile values?
7. We have a project that's in crisis. How can agile rescue our project?
   

I'll add more as I hear them, but these are the biggies. Feel free to provide input or links that addresses any of these questions.

Agile won't work on [fill in the blank]

I have four kids ranging from 15 to 3. For some reason, people with no kids really like to give my wife and I advice on how to raise kids. Usually they will provide analogies using their favorite pet. Or, they will tell me about their second cousin twice removed who "just had a baby", and here's what they PLAN on doing. I just politely nod and say "thank you".

So, what's the point? Well, I tend to get a lot of advice, or "words of wisdom" regarding agile development and how it won't work on certain projects. I've heard it won't work on complex projects, big projects, geographically disperse projects, mission critical projects, non-software projects...you name it. Everyone who tells me this has either a) never worked on an agile project or b) worked with a poor implementation of agile.

Instead of politely nodding and saying "thank you for your advice" as when I'm given child-rearing advice, I've recently decided to challenge these ideas. Of course not in a confrontational way, but just in an inquisitive way. "What experiences have led you to that belief?". "Why can't an offshore project implement agile methodologies?".

Below is a post that I saw on LinkedIn:

"As with any project management tool, it is situational. If, like in many software development environments, there are one or two key people involved in each and every task, Agile tends to break down. In this case, there are better methodologies to employ. Like any concept, I think the rub is that no technique fits all situations."

This is clearly someone providing advice on agile development that clearly has never worked in an agile environment, or has suffered a poor implementation. I couldn't resist responding. Below is my post.

"There is a very broad spectrum of agile methodologies out there that will cover any situation. Being agile also means adapting the process to fit your needs as it's an empirical, "inspect & adapt" approach to managing projects. For instance, typically Scrum & XP are implemented together. Scrum only addresses project management issues, while XP addresses development issues.

The failed agile implementations I have seen were caused by one or more of the following:

1. The team is not empowered and self-organizing.
2. Team members are working on too many projects, resulting in excessive context switching.
3. Management is committed to command and control (related to #1)
4. The customer or customer representative is not involved.
5. Processes are not allowed to evolve.
6. There is no dedicated product owner, i.e. someone responsible for the ROI that the team can go to (getting rid of the multiple boss syndrome).
7. The ones leading the agile implementation have not been properly trained in agile.

So, agile breaks down when the organization is not ready to accept its principles. If the organization is not willing to change from command & control to servant leadership, empower the team, assign ONE product owner, and get the customer involved, then it will fail."

Hopefully I didn't come off as a jerk on this. I just couldn't resist.

If you feel that agile just won't work on _____, I encourage you to find out for yourself. There are plenty of cases where it has worked on huge, complex, mission critical projects.