Lean from the Trenches: Managing Large Scale Projects with Kanban

I've recently read the book Lean from the Trenches by Henrik Kniberg. Henrik is a famous agile speaker, teacher, and coach. His book is a reference for agile practitioners.

The book is split in two parts. On the first one, he talks about his experience as a facilitator for the Swedish national police and the PUST project, a soon-to-be largely scaled project. The second part is a collection of techniques he has used along the road.

This blog post particularly focuses on what I learned from his experience on scaling. I will first explain what's the PUST project and why he had so many scaling challenges. And then try to outline how he solved the most important ones: organisation, communication, feedback and vision.

Case Study: The PUST Project

The basic idea of the PUST project is to allow officers to handle all the investigation work quickly and from anywhere, using a laptop.

As Henrik says himself...

PUST is a complicated system because it:

• ...has to integrate with a huge number of legacy systems.
• ...has to be very user friendly since police will be using the system in realtime while doing interrogations.
• ...has to be highly secure.
• ...has to comply with a whole bunch of complicated laws and regulations.

As it is a very ambitious project, they needed a massive workforce and wanted to reach a team of 60. The scaling of this project didn't happen out of the blank. They went from 10 to 30 people in the first year, then over 60 in the next 6 months.

But there are many projects already having a 60 persons team, so what's the difference?

They wanted the project to exactly fit the need of the officers using it. And to reach that goal, they chose to use agile methodologies.

As a reminder, agile methodologies favor:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Plan, processes, tools, documentation are what commonly makes a big team able to work together. On his book, Henrik explains how he got those 60 persons working together smoothly, using agile methodologies.

Divide and Conquer

What he set up seems like basic methodologies with a lot of common sense. It's actually quite hard to make it appear simple and to make it work out smoothly, read on...

He split the team into 3 different and independent feature teams. Each of the teams had their own product owner (he calls them Requirement Analysts), QA engineer (System Tester) and Scrum Master (Development Leader).

The advantage of feature teams is that they're all able to handle any proposed feature from end to end. It drops any bottleneck that could happen between interdependant component teams waiting eachother.

Also, by working into reasonably sized teams, "basic" agile methodologies can be applied and the scaling problems goes from making individual interactions smooth to building team interactions that work.

And to make sure they were keeping the big picture on right tracks, he also set up a PO team and a QA team sharing knowledge between each independant feature team.

Communication Mess

With a small Scrum team, a basic daily stand-up handles the job of sharing information efficiently. Imagine with 60 people! With so many teams, all independent, it's usually a mess to synchronize.

Henrik actually set up as many daily meetings as needed:

• Feature team daily stand-ups
• Speciality synchronization meetings (PO, QA, Dev)
• Project synchronization meetings (1 PO, 1 QA, Scrum Master of each of the feature teams)

The meetings happened in that very order, so that they start from the most detailed view to the broadest view. And since some people actually did the whole series of meetings, it helped synchronize everyone together.

He called that the cocktail party: small groups standing together and moving around every 15 minutes.

Feedback Mess

The communication mess was handled. But in all agile methodologies, it's important to have a clear view of what's going on for everyone.

Can all that fit in a Kanban board? Well, actually no, it can't. But it could fit in many boards!

Henrik set up a project board with the global workflow of features being shipped. Feature teams needed a more detailed way to handle their own workflow, and it wouldn't fit inside the main project board. So they each created their own customised boards and cloned the feature cards of the project board there.

Without the burden of having a huge wall full of post-it that nobody would understand, you can zoom on a particular box of the main kanban board and see what actually happens there.

It seems simple but is very smart and has a big asset: it's still physical. Without that physical board, they would have lost all advantages of the visual management kanban proposes.

Vision Mess

When so many people are working on the same project within independent teams, they can feel kind of clustered. And when you work only on a small part of the whole, you risk loosing the global vision.

Henrik advises to make that vision clear for everyone:

People are more likely to focus on the high-level goal if they know what it is.

And to make it clear for everyone, he simply wrote the high-level project goal and some milestones on top of the board.

To make sure the goal kept being reachable and understood by everyone as time passes, he organised reality check meetings every week or two.

The reality check took the form of asking a simple question to everyone: "Do you believe we'll reach this goal?". And the answer were as simple, a number from 1 to 5 ; from Forget it to Definitely!

Forget it

Probably not

Barely

Probably

Definitely

If there were too many 1s or 2s, an action needed to be taken:

• Remove an impediment
• Help a bottleneck
• Reduce scope
• Adjust goal
• Work harder

That reality check is pure agility applied to the vision. By assessing iteratively if the goal was still reachable, PUST teams were sure that if they failed, they failed fast. And by failing fast, they improved their chances of success.

Also, as the goal is not really set in a top-down approach but collectively, I think that the teams actually own their goal more than when it's set in stone. It provides them more chances to actually reach it, which benefits both the team and the end users!

Back to a Lower Scale

Henrik overcame the scaling problems by dividing it into smaller chunks. He also handled all issues brought by making many small teams work together with the smart tricks mentioned above.

All that's left is to apply the basics of Kanban on each of the smaller teams:

• Limiting the work in progress everywhere (yes, even in the backlog!)
• Defining what done means for every step of the workflow
• Capturing velocity and cycle time to have free approximate estimations
• Releasing often

He also describes how important it is to give enough time for tech stories and fixing bugs, so that the product doesn't become bloated.

Continuous Improvement

But how do you do a starfish retrospective with 60 persons? Hint: you don't.

Just like for the daily meetings, they split the process into many smaller retrospectives.

Each of the feature teams had their own weekly retrospective to improve what concerned them directly. If any impediments seemed to have a wider impact, it needed to be escalated to the process improvement workshop.

What's the process improvement workshop? As Henrik says, it's the "scrum of scrums type of retrospective". Just like for the project synchronization meetings, one person of each speciality and feature team gathered for a retrospective.

At first, and because they had a high rate of growth, they needed to invest 1h30 weekly into process improvement. The rate of process changes was almost unsustainable as many experiments needed to be ran to make sure the new propositions were making sense.

But even in that context, Henrik insists on the need to have follow-ups on previous retrospective actions, and to have a 100% consensus on every decision. As this is a higher view, retrospectives could not include everyone, there were less resistance risk if all the representatives agreed on the solution.

To make sure the processes didn't change too often, and to keep it sustainable, they tried to only choose the most impactful improvements for every retrospective. WIP limits everywhere!

Conclusion

At first, I found it quite intriguing to have a book with two parts so different from each other. It actually works well. I love when books teach you things with a story, like They Told Me It Was Impossible from Jean-Baptiste Rudelle.

As for the second part, I didn't find it as useful as most of it is a plain definition of what agile, lean, scrum, kanban and XP are. It's most likely very valuable if you're new to agile and want a fast overview, though.

My biggest take away is that the team overcame all the challenges they faced incrementally. They did not follow a single methodology, but cherry picked what they thought could be useful to them in all of the existing agile methodologies. True agility is the capability to adapt to the context, not to strictly follow a defined methodology.

Even though we don't face such scaling issues at marmelab, we use this kind of approach. Our processes are mid-way between kanban and scrum, and we don't hesitate to adapt them depending on the context of our projects.

My word, you should read Lean from the Trenches if you haven't. Wether you are an agile amateur trying to get a grasp on what agility is or a guru willing to scale a huge team, or improve what seems already perfect; you will find the answers you are looking for.

Henrik explains his experiences in an astoundingly easy way, making this book a very easy read. He goes quite deep into the explanations through real examples without forgetting the basics: what agility is about.