Agile Implementation and Coaching

From Waterfall, To Agile

Implementing an Agile methodology is a difficult activity for any company. In this case study we'll look at how we approached this problem for an existing company with a development team of about 40 people.

Observation

As it often happens, there's a big difference between theory and practice. Merely asking the development teams about their methodology will invariably end with receiving answers describing how they should work, not how they do. Moreover, answers will be as accurate as the teams' actual understanding of their methodology. Asking a Scrum team if they do daily meetings and attending one produce very different impressions of the level of Scrum adoption!

For this reason, we decided to schedule some of our time attending all of their methodology meetings (planning sessions, status reports and whatnot) and we noted down what kinds of things were discussed and how the hierarchical system worked.

We were looking for three kinds of information. Firstly, it's important to map four main responsibilities of managers onto the actual people performing them:

  • Technical leadership and architecture
  • Project management and team organization
  • Line management and one-to-one support
  • Product management

In the case of this company the first three responsibilities were in the hand of the team leads, and product management was neglected as request were passed without filter from customer to developer through sales and team leads.

The second kind of information that we looked for to understand which topics were discussed and how useful they were. Most of the meetings were focused on status reporting and deadline planning and verification. Since the company had always worked in a waterfall-like way, this was expected.

Last, but absolutely not least, was to understand team composition and the purpose of each team. Was a team able to ship software at all? In the comapny's case: no. There were design teams, software development teams, QA teams, system administration teams, support teams and DBA teams. Each new feature requested by a customer had to navigate through the complicated schedules of most of these teams in order to go from idea to deployment. A quick chat with the CTO confirmed this was a major pain point.

First improvements

The next step was to report on the findings: although everybody knew there were inefficiencies, the simple act of putting them together in a document can improve internal communication and understanding. The report also contained some novel ideas for the companies, like the idea that separate people could take on the different management responsibilities that at the time were mapped on a single, often overworked, role.

There was one change we all agreed we could implement straight away and that was appointing line managers for each person in the development department. This had the immediate effect of relieving some of the workload of team leads and giving everyone a more effective reference point to help guide the department through the changes that we were going to implement.

For the rest, it was obvious to all of us that we were in a situation in which we would need to create understanding before we could implement Agile and Scrum. We thus created a model team, lead by one of our senior consultants jointly with the client CTO, where we could implement the first round of improvements. The consultant would be an overall Agile coach, scrum master and technical lead (the latter for different, unrelated reasons). The CTO was to be the product owner. We created this as a cross functional agile team where there were all people needed to actually ship software. We took on a smaller project and we started working.

We introduced Scrum in a natural way: not by teaching rules but by taking the team through the steps practicing a show-dont-tell approach. This allowed us to bring in the ceremonies without attrition, and allowed the teams to learn on the job and with their own time.

This allowed us to also bring in actual stories, not requirements and starting to make the team understand the difference between the previous fixed-scope, variable-time approach and the new fixed-time, variable-scope agile method.

Extending Agile to the whole department

The first, complex, step was to restructure the teams so they would be cross functional - with a specific "topic", like performance or integrations, but with all the people necessary to deliver continuously.

In particular, we needed to identify scrum masters and product owners. The choice for the latter was crucial and it was very important they be coming from a development background so they could be trusted the team implicitly and not be seen as business imposition. We chose two of the original leads as POs and assigned them two teams each. Not an ideal ratio, but we felt this was the best compromise given the skills and resources we had at our disposal.

In order to extend agile to everyone in the department, we needed to make a lot of changes in the teams too. Moving one team at a time would take too long. Moving all teams at the same time with a scrum coach for each was too expensive for the customer. In the end we decided to have a compromise approach where we professionally developed a custom training video series teaching Agile and Scrum and then support the two POs teams with our senior consultant as personal Agile coach.

Mass training and the overall approach worked, albeit with a few bumps due to the resource constraints we had: some people had more trouble than others adapting to the new methodology but eventually the large majority of develpers reported increased satisfaction and accepted the changes.

Moving to scaled agile

The logical conclusion of the intervention was to scale agile in order to have quarterly planning and help sales and development interact in a structured way. We implemented a simple form of scaled agile through initiatives (macro-epics) very roughly estimated in order to assign sprints, and a quarterly spreadsheet with time blocks.

The initial difficulties were multiple, because it was not obvious to most of the people involved that teams should work on one thing at a time. Also it came as a surprise that deadlines were unmovable, in the sense that we would drop scope if push came to shove but not delay all later activities and all teams were expected to do so. Furthermore sales did not initially know how the customers would react to such a change as the company started to plan features and the product, being proactive instead of reactive.

As it turned out, most of these changes were an easy sell to the customers. Having clarity on the plans, costs and outcomes is a big boon. We helped sales understanding how we work. We also got better at single-threading teams. As the quarters passed, scaled agile is now an accepted practice by all management. I think everyone agrees there's been an improvement and the customers now expect an agile approach from the company.

What is left to be done

Not all that glitters is gold, and one of the big reasons of our success was the ability to choose pragmatically the fights we could win and that were valuable to the overall objective. That said, many future changes should be implemented. The company needs more coaching, more product owners, continuously trained scrum masters and better tools in terms of software practices and DevOps. There's still a divide between sysops and developers that needs to be bridged. The overall skill set is not a 100% match to the skills the company needs. We are and will be supporting them as long as they needs us.

(In writing this article I've also found this reference useful: Scrum Master vs. Agile Coach: Why Successful Transformations Need Both.

We'd love to work with you

Whether you are a battle-proven company, a scale up in hypergrowth or a world-changing startup, we got you covered.