[This post was inspired by a question in Quora - How can the waterfall method be incorporated into an agile software development environment? Is it possible to combine both methods or is it better to stick with one?]

Can I mix water with oil?

Sure, I can put them in the same container, but do they mix well?

[This blog was inspired by a question in Quora]

V model is a conceptual model between design/requirements/needs and confirmation of those needs in the system. In waterfall, it is interpreted, unfortunately, as a staged model, where each stage is completed before the next step. For example, we want to consider user needs and business requirements as a whole before going into smaller details. On the other side of the V we have the verification/validation steps to confirm that the system meets the identified capabilities. First we go down the left arm of the V, and then climb back up the right arm.

This is a logical model - think of the bigger picture before smaller details, and consider how to confirm the system works accordingly at the different stages. But in the way the waterfall approach uses it, there is a huuuuge assumption behind the staged model - that the identified requirements at each level DO NOT CHANGE during the deeper V stages. A lot of people using waterfall approaches do not seem to understand how critical that assumption is, and how huge the impact is to the successful use of the model.

Automation. If we look at the “economics” of testing, the value of testing is in the number of test cycles that can be run and the speed of how quickly they can be run after a modification. If we can run the tests very often and really fast, essentially after every modification, the cost of creating the tests (and test automation) becomes largely irrelevant.

The reverse is true in “traditional testing”. Because that testing, unfortunately, relies on significant amounts of human labor, it is very expensive to run. Any test comprehensive test run will take countless of hours of human work, and will take significant amounts of calendar time. As a consequence, it makes economic sense to collect a larger set of modifications to test at the same time. Because of time constraints, the actual tests executed will be optimised to most likely areas of failure, resulting in increased risk of regression errors going undetected in areas of lower scrutiny. It also, obviously, means that for many features the testing comes much after the actual development, and therefore decreases the value gained. And, I won’t even go into how bad humans are in repetitive testing. I was a test manager at one point doing exactly this, because I didn’t know anything better.

So, because of high costs of test execution, the focus is on the cost of testing, and that consideration trumps most other concerns. But the focus should instead be in the value seek to deliver through testing, and optimising that (as is the Agile credo in general).

[Context: This post is a quick copy/paste from a post in LinkedIn, but I wanted to save it. There may be some quality issues in this text, too. I hope to expand on this idea, with some images to help the understanding, in the near future.]

Most people don't seem to understand the true difference between Agile and traditional software development.

Waterfall is based on the idea that we take something we want to do, and break it down into subitems - actions or sub-components - and then deliver them separately - often by separate people who are experts in the activity or subdomain - and handing over those items to each other as needed. E.g. someone first needs to spec the requirements, then another person designs the UI, third person looks at architecture and technical design, before passing it to a fourth (UI programmer) and fifth persons (BE programmer) for coding. Then yet another person runs (often manual) tests on it and complains that it doesn't work. Yet more people might be involved in deployment, documentation, etc.

In this approach, it truly is difficult to break things down small enough.

Now, Scrum, being properly of Agile mindset, challenges this approach, by expecting people to complete items to a "potentially shippable" quality within each Sprint. "Coded but not tested" doesn't cut it, nor does "works on my machine". Each PBI is an independent capability that doesn't have to be returned to in the future, unless we specifically want to extend or modify that capability.

Enter "vertical slicing". Each slice contains all the activities needed to take an idea/need/desire from 0% to 100%. A little bit of spec, a little bit of architecture, a little bit of coding, a little bit of testing, a little bit of this and that, and at the end of the slice, that small things works in the product.

It can therefore be shown to someone, if we desire to hear feedback. Or it can be given for use if it's useful for some external purpose (not all vertical slices result in end-user valuable feature level). Or, we can simply see that we were able to make something work and we can inspect how well it works or how long it actually took us to do it.

When it comes to progress, there are only two points where we can objectively (well, with high enough confidence) know where we are - 0% and 100%. Anything in between is just guesswork. And people have made very elaborate guesswork tools and systems over the years. Big tools, great methods. But none of that has taken away the fundamental problem - it’s still just guesswork.

The problem with most guesswork is that it’s based only on what we know or can think of. The most significant risks are typically the ones that catch us by surprise. So while some amount of guessing is useful, going too far is sometimes even dangerous (but at least usually futile and waste of time). And trust me, I’ve done my share of those guessworks :). I used to love MS Project.

But, since we can only know 0% and 100%, we should focus on our attention to those numbers. 0% is easy - we haven’t done anything yet. But in waterfall, we start everything (with requirements) very early. We invest some into everything, then invest some more (planning), and invest even more (desing, then coding), before getting to the point where we can evaluate whether it all works or makes sense to our customers. So if we want to know as much of 0% as possible, we should not any work on most of the stuff.

The list already says more than 54 words. There's 55. But they imply much more than their measly number. Let's put a little depth into the 5 critical pitfalls.

Lacking Definition of Done

Scrum is silent on technical practices, simply because it's a work management framework and agnostic to the context. But that doesn't mean the the technical level doesn't matter. Imagine putting Kimi Räikkönen to drive a Lada, will he perform well in a race?

For a Scrum to really take off and work, the team has to at least be able to complete a working version of the product at the end of the Sprint. That's the absolute minimum. It should be better than that, really. Ideally, at the end of every single small modification. The working increment is a confirmation that whatever the team tried to do is working and enables us to actually eliminate the technical risk associated with that change. The smaller the timeframe, the smaller the risk.

In the previous blogs we looked at what teams really are and what the choice of team vs. workgroup means. Here we’re getting to the very foundations of team formation as a process.

In 1936, Kurt Lewin coined his famous formula:

B = f(P, E)

In the first blog I compared two primary approaches for group coordination – workgroup and team. I also tried to emphasize that the two are both valid approaches (I’m assuming everyone understands that a bad workgroup is not a good approach). So when might we choose one or the other?

We can see the most important difference in the graphs below. On the X-axis we have time. On the Y-axis we have “value generation potential”, which is really a terrible way to say how awesome they can get. 

Pros and Cons of Workgroup

In the first graph, to the right, I have the workgroup. The great thing about a workgroup is that it’s easy to set up and get started. As long as we know who the people are and what they can do, the leader can quickly assign them to tasks and things start getting done. It’s also easy to delegate things further to other people. The main drawback is that the potential of the team is limited to the sum of the members individual abilities (including the leadership ability of the leader).