Don’t Get Stuck in the “Hybrid Agile” Trap
I was recently asked to explain hybrid agile at a dinner meeting for the local Project Management Institute (PMI) chapter. Chapter members had expressed a desire to learn more about how to succeed with hybrid agile.
Specifically, they wanted to know how to use agile and traditional development methods together. Most of their organizations are stuck in the messy middle ground between traditional ways of working and agile.
This article explores the reasons people are pursuing agile and how the current hybrid development approaches will fail to help them achieve their goals. I’ve also provided some recommendations for what they should do instead.
How did we get here with “Hybrid Agile”?
We are here because agile frameworks have proven over the last 20 years to be a more effective way of developing software. They are a huge improvement over the traditional, stage-gated or waterfall approaches.
The waterfall approach requires that you gather requirements and plan everything up front. That works great unless something changes along the way. Which. Is. Certain.
Changes are extremely disruptive; they cause rework which is costly and time-consuming. Success with waterfall meant sticking to the plan and resisting change at all costs.
Obviously, the waterfall or predictive approach is going to fail when customer needs are changing or when we are trying to deliver on unexpressed customer needs. And they will fail when the technology to deliver is changing.
Unfortunately, many organizations are struggling to move from waterfall to agile ways of working and as a result, they are missing out on the benefits of agile ways of working.
For those organizations, traditional or waterfall ways of work remain the standard. The organization’s processes for funding and project selection, stage gates, management and status reporting, and risk and compliance are all built around waterfall ways of working. They’d love to be agile but unfortunately, their existing processes just don’t support it. These organizations are stuck in the messy middle between waterfall and agile.
As a form of end around their processes, they are looking to hybrid ways of working. They hope that they can blend agile and waterfall. They want the benefits of agile ways of working while operating from a waterfall mindset.
What Are Those Agile Benefits?
Most organizations want to be agile. They think it will help them deliver fast. Or they are getting their asses kicked by more nimble competitors who don’t have a heavy waterfall legacy pulling them down like a boat anchor.
Or they are tired of their waterfall projects failing. Industry studies have shown that waterfall projects are 3X more likely to fail than agile projects.
They’ve also read the industry reports, like the one below from CollabNet, which indicates that those that adopt agile are able to address change, get better project visibility and alignment of business and technology. Those benefits are very attractive.
There are 2 Key Drivers for Agile Approaches
Those benefits are important but I believe there are two key drivers or market forces that pushed agile ways of working to the center stage. How companies address these market forces determines their competitiveness and ultimately their survival.
Force #1 — Increased Customer Expectations
The first of those powerful market forces is increased customer expectations. These days, customers are relatively impatient and fickle. They want more all the time and they want it now.
An organization that has listened closely to customers is Amazon. Amazon started as an online bookstore. They disrupted traditional book stores and were able to kill off Borders who could not make the transition to online selling
Amazon has since expanded (and disrupted) a variety of different industries. They have done that with a strong focus on meeting and exceeding customer expectations, even when those expectations were unreasonable.
A recent example is speed of delivery. With Amazon Prime and Prime Now, Amazon will provide you not only overnight delivery but delivery in 1 or 2 hours.
Force #2 — Disruptive Technologies
The second key driver for agile is disruptive technologies. Advances in technology happen quickly and with it, comes the ability to satisfy customer needs in more creative ways.
Consider the Netflix vs. Blockbuster battle. Netflix won by moving boldly into the internet subscription business. Technology enabled Netflix to stream content directly to customers. This eliminated the need for customers to drive to the local store and enabled everyone to be able to watch the same content at the same time which wasn’t possible in a DVD world.
Meanwhile, market leader Blockbuster held on to their brick and mortar stores. They were not able to make the shift and take advantage of the new technology.
Netflix has continued to innovate and has changed the way people consume content on TVs, PCs and mobile devices. By providing the ability to watch entire seasons of TV shows at the same sitting, they met a need that customers had not even expressed — the ability to binge watch entire series of TV shows rather than seeing them as a weekly drip. This created a loyal following and likely impacted the grades of more than one college student in the process.
What Amazon and Netflix know is that the customer is king. Or queen. I seriously doubt that any customer told Amazon “I need you to respond to my impulse purchases within one hour” or Netflix “I want to spend the next 14 hours in my pajamas on the couch”. Rather, both of these companies spent a great deal of energy unearthing and anticipating needs that customers have not yet expressed. And found the technology to deliver on that.
A current example of disruptive technologies is the electric scooter. The most popular brand of these is the Bird scooter but there are competing brands from Lime, Spin and even Lyft.
Providers of these scooters bring in a truck and just dump these things on the street. They are not confined to a station or spot and they don’t need to be returned to any designated area. Riders can download the app, sign up and ride away in less than 5 minutes.
This simple example of the electronic scooter is interesting because it shows both disruptive technology and increased customer demands. The electric scooter by itself is not disruptive; they have been around for years. The key innovation was coupling the rechargeable scooter with GPS tracking and online. And a network of people who charge and look after the scooters.
I am pretty sure that no customer said they wanted a scooter that was available all around town, that they could hop on wherever they found it, and then they could dump on the sidewalk anywhere when they were done. What Bird and others found was that there was an unexpressed and unmet need for local transportation. And they moved quickly to fill that need.
Traditional Methods Won’t Get You There
As mentioned above, most organizations have legacy processes that are built around waterfall thinking and control. Are those waterfall ways of working going to enable them to deliver on the increased customer expectations and rapidly changing technology? It doesn’t seem likely.
Asking traditional organizations to change those entrenched ways of working is likely to elicit a blank stare. They want agile but don’t want to (or can’t) eliminate waterfall ways of working. And that is a fast track to being the next Borders or Blockbuster.
Would a Hybrid of Agile and Waterfall be the Perfect Solution?
I know what you are thinking, what if there was a middle ground, where we could get the best of both worlds — agile and waterfall?
In fact, the most recent PMI Pulse of the Profession showed 23% of respondents said they were using a hybrid of agile and waterfall. And 47% said they were using straight up waterfall/predictive!
PMI members aren’t the only ones pursuing the hybrid approach. In a Forrester Research survey in 2015, 27% of respondents indicated they were mixing agile and non-agile techniques.
The Popular Hybrid Method — Water-Scrum-Fall
One of the most popular hybrid development approaches is referred to as water-scrum-fall. With this approach, organizations use different frameworks for different parts of the project lifecycle.
They use traditional/waterfall approach for vetting new projects. This typical involves business cases, vetting committees, and annual budgeting cycles.
Once the project is vetted and approved, it gets added to a queue for a development team to become available and staffed. Once started, the development team uses Scrum or another agile approach to complete the development. Then the completed work goes on the queue for integration, testing and release into production.
All told, these steps can easily take 2–3 years to complete. I know, I know, 2–3 years doesn’t seem very agile. It isn’t.
For a good view of water-scrum-fall, check out this video from Jonathan Smart of Barclays.
Does Water-Scrum-Fall deliver those Agile Benefits organizations are seeking?
The thing is, water-fall-scrum and other blended approaches don’t really deliver on those two drivers for agility — addressing the changing customer expectations and disruptive technologies. They only make it seem as if they do.
As Jonathan Smart points out in the video above, or as we have discovered in discussions with clients, the water-scrum-fall approach can take anywhere from 16–30 months to deliver new features into production. Is that going to deliver those agile benefits?
Water-scrum-fall gives the appearance of change, but does it represent a true change? Does it deliver the benefits? I don’t think so.
Anti-patterns With Blended Methods
As an Agile Coach, I see organizations using these blended methods like water-scum-fall all the time. They are attempting to achieve those benefits of agility without truly changing the way they work. They give the appearance of change, but the true measure of success is in their ability to deliver results quickly, get feedback, and pivot.
There are 3 anti-patterns that I see organizations using.
- Relabeling or Rebranding
- Big Planning / Fixed Dates and Scope
- Pragmatic Approaches
1. Relabeling / Rebranding
Relabeling or rebranding is an approach that appears to embrace agile but in fact only reinforces the status quo ways of working. Organizations change the name of things, but now what they are or how they work.
Calling something by a different name doesn’t change what it is.
Here are some popular rebranding approaches:
- Project Phases become Sprints — A popular rebranding approach is to use the sequential phases that are a key aspect of gated, predictive approaches but to label those phases as sprints. So there is an analysis sprint, a design sprint, a development sprint, testing sprint, etc. A sprint is a time-boxed development cycle that is part of the Scrum Framework. Sprints aren’t sequential phases.
- Status Meetings become Daily Standups — Another common example of rebranding is to begin to call status meetings a daily standup or a Scrum. In the Scrum Framework, the Scrum is a short daily meeting that team members use to coordinate their work and align to complete the sprint goals. It is not a meeting for the project manager to question each team member to collect their status.
- Project Managers become Scrum Masters — Though anyone in the Scrum community knows there is a gap in skills and behaviors between project managers and Scrum Masters, most people outside the Scrum community continue to behave as if they are the same job. Just renaming your project manager a Scrum Master doesn’t change anything.Actually, that is not true, renaming a person does change the organization's ability to understand and appreciate the role of Scrum Master. [See: Project Managers make Lousy Scrum Masters]
- (And My personal favorite) The Scrum of Scrums — There is a technique used to align multiple Scrum teams and resolve dependencies called the Scrum of Scrums. The technique is a scaling up of the daily meetings held at the team level.
These days when someone says ‘Scrum of Scrums’, I can almost guarantee they are rebranding and misusing the term to mean a status meeting of project managers.
Rebranding gives the appearance of change. “Hey, we are using user stories now so make sure you attach the requirements document to the user story in Jira.” But it is not actually change.
Rebranding is like adding pixie dust or spraying with Febreze
The rebranding efforts are not even wishful thinking. They simply don’t work. You can’t sprinkle agile and scrum terms like pixie dust on your existing waterfall approaches and expect improvements. Don’t treat the Scrum Framework like some sort of Febreze that is going to improve the odor of your legacy processes.
2. Fixed Deadline, Scope, and Budget
The second anti-pattern that I see is organizations that say they are using agile but have a fixed deadline, scope and budget.
Software development is not as predictable as most people would like to believe. Especially for large development efforts. And even more so when we expect customer requirements to change and disruptive technologies are at play.
So why do people insist on creating a big plan up front and “locking in” dates, scope, and budget? Doesn’t that seem like a plan for certain failure?
Amazingly, it still happens. All the time.
It doesn’t work like that in Agile. In agile frameworks, team members work at a sustainable pace and provide their own estimates of how long things will take.
By contrast, in most traditional approaches, a project manager comes up with the schedule. It may or may not be realistic, and it is likely that management will bully him or her into reducing it by 25%. Then the project is handed to the team to execute according to the fixed schedule, scope and budget.
What happens when the project gets underway and things change? This is where the rubber hits the road and most organizations revert to form and what they know best. They implement strict change control processes to limit the “scope creep”. They ask people to work overtime including weekends to hit the commitment date. Usually, agile ways of working are tossed out the window and everyone is told to do whatever it takes to hit the dates.
3. The Need for “Pragmatic” Approaches
This anti-pattern is actually quite similar to the rebranding approach. In many organizations, when I describe how to use an Agile or Scrum Framework, they accuse me of being too much of a purist. And they say that they need to tailor agile and adopt a more pragmatic approach to make it work at the XYZ company.
What they really mean is that we don’t want to give up our traditional ways of working. After all, they’ve worked for us for the last 50 or 100 years. Past success is disabling and can be considered a key risk factor for organizations.
Language is important. Using labels like ‘pragmatic’ implies that the ‘pure’ alternative is rigid & dogmatic.
These organizations ‘tailor’ agile and Scrum to make it work in their environment. Which means that they gut the useful parts that they don’t really want or understand, and they continue to use their pre-existing and ineffective waterfall approaches.
Craig Larman expressed this well in his laws of organizational behavior:
“…any change initiative will be derided as “purist”, “theoretical”, and “needing pragmatic customization for local concerns” — which deflects from addressing weaknesses and manager/specialist status quo.”
Source: LeSS Works Website
Even PMI warns that tailoring agile is an advanced topic and not to be taken likely.
“Tailoring is an advanced topic that should be undertaken by experienced practitioners who have been successful using agile approaches as originally described in multiple environments before they consider tailoring them.”
Source: PMI PMBOK® Guide 6th Edition
Why these attempts at Hybrid Fail
These blended approaches fail for a number of reasons, but the primary reason is that they are not agile. They don’t provide true agility (respond to change, meet customer needs, leverage disruptive technologies).
Additionally, they suffer from the following:
- Hybrid approaches are more heavy and costly. Adding an agile framework on top of what you are already doing is more costly since you are using two methods, not one. Blended is more work/heavyweight than either predictive or agile alone.
- Hybrid approaches don’t leave a path to agility. With hybrid, everyone in your org thinks they are already agile. There has been no real change and resistance to change has not been addressed. There is no path to agility since the organization believes they are already agile. You can’t get there from here.
4. Hybrid approaches confuse and discourage the people who are most important to deliver the results you need.
Recommendations — Do This Instead
Rather than pursue a path of hybrid development methods, I would recommend the following instead.
- Be clear about your goals
- Learn all you can
- Understand how work gets done
- Don’t mix and match (except bimodal)
- Shift your focus to business outcomes
1. Be Clear About Your Goals
As noted above, most people who are using some sort of blended approach want the appearance of agile without true change. So why not just say that?
Don’t say agile but mean status quo.
Don’t do agile because everyone else is doing it.
Being clear will be particularly helpful to the rank and file, those team members who are delivering the work.
2. Learn All That You Can
I am shocked that a lot of the managers and leader implementing agile in organizations have no hands-on experience with agile and most have not taken any training on the topic.
So let me get this straight, you want to introduce a fundamental change to the way your organization develops solutions, but you don’t really understand it yourself?
BTW, Agile doesn’t mean “do more with less” or “go faster”.
A recent article about agility at Roche stated that their leadership team went through 4 days of training on the agile mindset. Most leaders I’ve met don’t want to dedicate even 4 hours to agile training.
In the aforementioned survey of members, the PMI Chicagoland chapter showed a disappointing level of investment in agile and scrum certification. While 70% had their PMP certification, only half that number had any kind of agile certification.
Training is widely available. When introducing agile ways of working, everyone needs it. Team members. Scrum masters. Project managers, especially project managers. And managers and leaders. Everyone.
Agile Leader training and certification is growing. If you are a leader who wants to introduce agile ways of working, lead by example and go first with training and understanding.
3. Understand How Work Gets Done in Your Organization
When considering adopting agile, or especially if you are already partly pregnant with agile, it is helpful to understand how work gets done in your organization and exactly where agile is going to help.
Value Stream Mapping is a great technique for this. You don’t need to be a Lean Black Belt to use this powerful technique. You can start by mapping out how work gets done, literally from an idea or customer request to the delivery of that request into production. You can use sticky notes and the wall of a conference room to engage people in this process to do the following:
- What are the steps in your process?
- How long does each step typically take?
- Which steps add value in the eyes of the customer?
- How can the overall cycle time be reduced?
Having done this with a few organizations, we have found that solutions take from 16 to 30 months to go from idea to delivery. Putting it on the wall in a highly visible way can help with acceptance and lead to a desire to make changes.
The end-to-end cycle time can be a useful proxy for business agility. Each small improvement step can be measured against what it will do to reduce delivery times.
4. Don’t Mix And Match — Use One or the Other
Mixing and matching Agile and Waterfall is foolhardy. This alchemy is likely to result in something worse than either one alone. So don’t do it.
If how you measure success and failure in your organization cannot be met by Agile, don’t do it. Stick with waterfall ways of working. Sometimes, the organizational processes, HR policies, project budgeting, oversight, and compliance is all firmly rooted in waterfall and cannot be changed.
If you are truly interested in agile, run an experiment with a small pilot project. Engage coaches and trainers with agile expertise and choose a team that is willing to make a good faith attempt at it.
Don’t hobble the agile project by forcing it to comply with existing waterfall ways of working. If necessary, isolate the team in a protective bubble away from the agile antibodies in your organization. Run it like a true experiment and collect data about what works and what doesn’t work. Then you can make changes in your organization to accommodate agile ways of working.
5. Leverage Bi-modal
There is one situation where I think you can succeed using both waterfall and agile. Some call this bi-modal. If you have succeeded with agile in pilots, you can begin to introduce agile as a second and separate way of delivering. The key is, that it is a completely separate way of working
Bi-modal doesn’t mean water-scrum-fall. You don’t mix and match with the same team on the same project. Rather, you can use either agile or waterfall on one project.
At a program level, I’ve used bi-modal with great success. Some teams in the program use Scrum for application development, and others like infrastructure use waterfall to complete their work.
An individual is not asked to do both. Each person is either working in an agile or waterfall context.
For bi-modal to work, it is helpful to have a set of decision criteria or fit.
6. Focus on the Outcomes or Why We are Doing this Work
Finally, no matter what approach is being used, the organization and team should focus on the project outcomes and not on the plan.
This flies in the face of what I see in practice. Most organizations need to report RAG or stoplight reports on all projects. These reports are based on traditional project success criteria like on budget, on time and on scope. They are more focused on project activities and ignore whether or not the project is delivering the business value!
I was consulted at one client about the Project Portfolio Management (PPM) tool and any changes to use it for agile projects. I asked if the PPM tool tracked the achievement of benefits and got a blank stare. Then I was flat out told that the leadership team only cared about whether it was on plan, and not whether it was delivering the benefits. This is a major problem.
Organizations should shift their focus to the key outcomes they are trying to produce and view the plan to get there as a series of experiments. This is especially important when the customer requirements are hard to determine or changing frequently, or when technology and how to deliver to customers can change. Delivering perfectly to a plan that delivers the wrong solution too late is a huge fail.
I hope this has been helpful to those of you contemplating or using hybrid agile or blended approaches.
⭐️Thank you for reading. If you enjoyed this article, feel free to hit that clap button 👏 to help others find it.
Let’s connect on Twitter or find me professionally in Linkedin.
Leave a comment below if you have any questions, and subscribe to receive the latest updates in Agile and Scrum.
Anthony Mersino is the founder of Vitality Chicago, an Agile Training and Coaching firm devoted to helping Teams THRIVE and Organizations TRANSFORM. He is also the author of two books, Agile Project Management, and Emotional Intelligence for Project Managers.