Can You Have an Agile Transformation Without Changing Your Organization?

One of my long-time clients recently informed me that they were undertaking an agile transformation. Which is good.

It would have been great if they included me in that transformation. After all, I have been helping them with agile training for the last 5 years. They hired another agile consultant to lead their transformation.

That experience prompted me to retrospect on the services I had provided them. It seems I missed the boat by giving them what they asked for (agile training) and not what they needed (agile transformation). How did this happen and how did I fail to deliver the services and value that the client really needed? Let’s explore.

Background – A Hierarchical Structure of Silos

The client is a relatively small organization with about 400 employees that has been around for over 160 years. I was first engaged with them to help them kick off some new projects as agile projects. I provided team training for the cross-functional teams the establish to deliver a dozen different projects.

One thing I noticed about those cross-functional development teams is that they always included the same two members – one from information technology and one from marketing. I called them the Kim and Jim team. Every one of the new teams included Kim and Jim.

An obvious problem here is that Kim and Jim couldn’t perform as part of every new team that was being spun up. No worries I was told, Kim and Jim were the best people they had from technology and marketing. So they kept re-assigning them to the new teams with understandable impacts to the previous teams.

Another thing that I noted about the organization was that they were extremely siloed and hierarchical. There were lots of bosses, with narrow but deep spans of authority. Decisions were made by those at the top and employees lower in the food chain were expected to execute them.

At one point I suggested that they consider re-organizing around products, services, or customers rather than around those functional silos. I was politely told that it was out of bounds. And not my concern.

Adding Complexity By Matrixing

Within a year of our engagement, I was excited to learn from the client that they were setting up cross-functional teams to take on 5 strategic objectives. Each team would have a backlog, product owner and Scrum Master and I was proud to see that they were using the concepts I had been teaching in my training courses.

Unfortunately, those teams were staffed by leaders from their various functional silos. On the surface this seems like a good idea, however:

• There were temporary teams, not a change in the functional organization structure
• The leaders were not workers; though they had a backlog and priorities, those on the teams did not do the work
• There was a power differential between the leaders who were playing the “developer” role on the team and the Scrum Master and Product Owner. Often the meetings devolved into a speech by the highest-paid person in the room.
• Most of the leaders on those teams were on multiple teams. The CTO was the lead for one team and a team member on 3 other teams

In other words, these teams did not represent effective cross-functional agile teams. These were only a new layer of complexity created by using a matrix approach.

And since they did not abolish their old org structure, this set of leader-led teams represented another layer of collaboration, communications, and priorities. Leaders were spread among several teams creating scheduling and logistical nightmares. And as you might imagine, they didn’t deliver on their results. The teams met and had vigorous discussions, but little or no real work was being done.

I’ve Noticed This Happens Everywhere

While the case with this particular client was extreme, the challenge is not unique to this particular client. MOST of my clients are organized into functional silos. In the best of cases, the technology teams are organized by solutions and include cross-functional team members. But just as frequently, even the technology organization is organized into functional silos and team members are pulled from the department and assigned to multiple delivery teams. Here is an example of a functional organization compared to a cross-functional team.

Assigning members of the functional silos to multiple development teams is incredibly inefficient; particularly when you consider the overhead of the Scrum Framework. Each team member may be attending Scrum Events for 2, 3, 4 or even 5 different teams. It really doesn’t work.

What should organizations do instead?

Align Along Products, Solutions, or Value Streams

While I don’t usually have positive things to say about the Scaled Agile Framework (SAFe), the one thing that I think it does well is to align the organization to value streams.

It starts with identifying the solutions or products that provide value to the customers of the firm. Then, the value stream or the steps or processes that the organization follows to deliver those solutions is mapped out in detail. Third, the people involved in delivering those activities are organized around the value stream in an Agile Release Train. Without this reorganization, using SAFe principles and practices would not make any sense.

You can see an example of this in the diagram below from the Scaled Agile Framework site:

Another popular scaling framework, Large Scale Scrum (LeSS), also advocates for organizational structure change. The first purpose of LeSS is actually de-scaling through organizational change.

The first purpose of LeSS (Large-Scale Scrum) is actually descaling through organizational change. Descaling the number of roles, organizational structures, dependencies, architectural complexity, management positions, sites, and number of people. LeSS is not about scaling one team into multiple teams. LeSS is about scaling up Scrum itself in order to achieve organizational descaling.

– DeScaling with LeSS, LeSS Website

LeSS recognizes that the complexity in most organizations is created by the organizational structure and not the work itself. LeSS co-founder Craig Larman has established a set of laws of organizational behavior that describes the relationship between organizational structure and culture.

In Larman’s Laws of Organizational Behavior, the organizational structure is called out in law #5 below:

1. Organizations are implicitly optimized to avoid changing the status quo middle- and first-level manager and “specialist” positions & power structures.
2. As a corollary to (1), any change initiative will be reduced to redefining or overloading the new terminology to mean basically the same as status quo.
3. As a corollary to (1), any change initiative will be derided as “purist”, “theoretical”, “revolutionary”, “religion”, and “needing pragmatic customization for local concerns” — which deflects from addressing weaknesses and manager/specialist status quo.
4. As a corollary to (1), if after changing the change some managers and single-specialists are still displaced, they become “coaches/trainers” for the change, frequently reinforcing (2) and (3), and creating the false impression ‘the change has been done’, deluding senior management and future change attempts, after which they become industry consultants.
5. (in large established orgs) Culture follows structure. And in tiny young orgs, structure follows culture.

Law #5 means that the culture will follow your org structure.

Can We Have an Agile Transformation without a Change to the Organization?

The thing is, change is difficult. And proposing change to the structure of an organization is very likely to run into stiff opposition. Those individuals who have achieved a position of power in the organization don’t want to give that up. They have earned their coveted spot and will defend it like a mother bear protects her cubs.

That’s the rub. The organization determines the culture and directly impacts the level of agility you can achieve. You cannot have an agile transformation without an organization structure that is aligned to customers and products.

Bottom Line – Agile Transformation and Organizational Change

In the example I started with above, I can see that I made a mistake with my client. I did not advocate strongly enough for true organizational change. I did not provide a compelling case for why that was necessary. So no wonder it took another consulting firm to make the case to them.

Instead, I provided what was requested – training. By providing training to teams and leaders without encouraging organizational change, we were only nibbling around the edges. We may have helped with agile adoption, but there was little chance of any type of agile transformation. There was likely some benefit, however, it was offset by the added complexity of the matrix organization and the additional meetings and Scrum overhead.

One thing I have learned through this process is that you can’t have an agile transformation with the wrong organizational structure. Since few organizations are truly organized by solutions, products or customers, this means that most attempts at transformation will fail.

Those who continue to use an organizational structure with functional silos will get little or no benefit of adopting agile ways of working.