Enough Already, There is No “Value” in Earned Value Management

Enough Already, There is No “Value” in Earned Value Management

My colleague Bill Baxter informed me that PMI had just released the Earned Value Management (EVM) guide in December 2019 and that I should take a look. My first thought was, “Geez I hope they aren’t going to force EVM on to agile projects”. Which of course they did.

Those of you born after 1990 have probably never heard of EVM. Unless you are working in the government sector where it may be a requirement.

Earned Value is an old-school way of viewing project costs, schedule and scope together to determine your performance. EVM focuses on delivery against those “triple constraints” which unfortunately tends to be the primary objective of traditional project managers who strive to deliver OTOBOS. While it may be required for some government projects, EVM should never be used on Agile initiatives. Yet that is exactly what PMI is trying to do.

My complaints about Earned Value Management include:

  1. EVM calculations are based on detailed and accurate upfront planning that doesn’t exist in agile project.
  2. EVM ignores the delivery of business benefits.
  3. EVM misuses the term “value”
  4. In reality, no value is delivered until the customer receives the working solution.

Let’s take a closer look, starting with a little bit of background.

Some Background on Earned Value Management

New EVM Standard Published in December 2019

pmi standard for earned value management 2019

This most recent standard for Earned Value Management was published in December 2019. Up till now, I don’t think many people were trying to force-fit EVM on Agile Initiatives.

The most previous version of the PMI Earned Value Management Standard was the second edition, published in 2011. That 2011 version doesn’t mention agile projects at all.

True to the PMI tradition, the 2019 EVM standard is a bloated 182 pages of detailed explanation for a topic that could be explained in 18 pages. Thankfully, the two sections of the document that describe how EVM can be applied to Agile and Hybrid environments is a relatively sparse 9 pages; it probably could have been one simple statement: “Don’t Do It!

Calculating Earned Value Management

I don’t profess to be an expert on Earned Value Management. I learned just enough about EVM back in the 1990s to teach it in various IT Project Management courses. I never worked in an environment where I was required to use it.

In short, EVM is a way to track whether or not you are spending and completing work at the speed that you planned it. Nothing else. It does this by comparing your planned cost and schedule per item to the actual. Rather than just provide a % complete spend, it can reveal problems by showing how much of the planned “work” you spent got done.

Calculating EVM is straightforward. EVM looks at the relationship between 3 project constraints: time, cost and scope. We use the plan and actual for these triple constraints to calculate Planned Value, Earned Value and Actual Cost:

  • Planned Value (PV) – This is the cost of the work you plan to get done at a specific point in time. Though it is labeled as value, it is truly the cost of the work.
  • Earned Value (EV) – Earned value represents the cost of the work that you actually completed at a point in time. value of the work delivered.
  • Actual Cost (AC) – Actual cost is the cost in dollars that you actually spent to complete the work at a specific point in time.

EVM is Based on Detailed Planning Upfront

One of the biggest problems with EVM is that it is all based on having detailed plans upfront. And not having too much change which doesn’t fit with agile initiatives.

Mike Griffiths has been straddling the agile and PMI schools of thought for over a decade now. I recommend his PMI-ACP prep book to everyone who is pursuing that certification. Here is what Mike wrote in 2007 about applying EVM to agile projects:

There are fundamental problems using EVA on agile projects relating to baseline plan quality. Also there are better alternatives available for agile projects.

Earned Value analysis and reporting measures conformance and performance to a baselined plan. So, given that on agile projects we know that our initial plans are likely to change, why track progress against a weak plan?

— Mike Griffiths, Using Earned Value on Agile Projects

You can also see the problem in the statements that PMI makes in the Agile Practice Guide. You may recall that PMI jointly produced the Agile Practice Guide with the Agile Alliance back in 2017. I went back to look at my previous post on the Agile Practice Guide and saw that I had criticized the inclusion of Earned Value in my review.  I called it a “weird fascination”.

The statement below from the Practice Guide reflects a key challenge with applying EVM to agile:

Traditional EVM metrics like schedule performance index (SPI) and cost performance index (CPI) can be easily translated into agile terms. For example, if the team planned to complete 30 story points in an iteration, but only completed 25 then the SPI is 25/30 or 0.83 (the team is working at only 83% of the rate planned). Likewise, CPI is the earned value (completed features value) to date divided by the actual costs to date or, as shown in Figure 5-6, $2.2M / $2.8M = 0.79. This means a result of only 79 cents on the dollar compared to plan (but of course this assumes that the prediction is still correct.)

–Project Management Institute, Agile Practice Guide (2017)

The problem is that our calculations are from a baseline that is a best guess and is expected to change. We then calculate performance indexes against those guestimates and come up with conclusions like the following:

  • the team is working at only 83% of the planned rate
  • a result of only 79 cents on the dollar compared to plan

First, it sounds like the team is not working as hard as they should. And second, it is a measure from someone outside the team about the best guesses (story points) that the team could complete in a sprint. Never mind that a focus on measures like this will have the undesirable impact of the team gaming the measure so that they always hit (or exceed) their estimates in the future.

The problem here is that EVM is based on a detailed and accurate “plan” at the start of the initiative. And Agile avoids that detailed upfront planning because it usually represents waste and throwaway work.

In an agile project, the backlog and release plan may be a best guess based on high-level estimates of the work using imprecise measures like story points. So your actual results are highly likely to differ from the baseline you created. Remember the Agile Value statement, Responding to Change vs. Following a Plan?

If your results fall short of the plan, how can you possibly know if it is because the team did not perform well or that your original estimates were wrong? This comparison is back to reliance on plan-driven methods and assumes or is predicated on the ability to accurately forecast the future. Focusing on the variance is navel-gazing at best.

This effort to compare plan to actual serves no purpose except to encourage gaming and other dysfunctions, and to create a lot of work for the person calculating EVM. In fact, PMs who apply EVM to agile initiatives will find themselves chasing their tails trying to determine why their actuals are not matching the plan (they almost never do!). If you love to hunker down at your desk, endlessly manipulating MS Project and exporting it to MS Excel for hours on end, this is a great tool to help you achieve that.

One other problem is that the PMI proposal for EVM would take agile projects back in time to the “% complete” mentality and style of progress reporting. Which is exactly the wrong direction. According to the standard:

The key assumption in agile environments is that the ratio of story points completed to the total story points planned for a release is a good measure of the actual percent complete”.

— Project Management Institute, The Standard for Earned Value Management (2019)

Aha, we are back to using % complete to report progress. Awesome. What happened to value-driven delivery?

EVM Ignores the Delivery of Business Benefits

Section 3.5 of the 2019 EVM Standard (Applying EVM in an Agile/Hybrid Environment) starts off with a statement about how both plan-driven and agile approaches need to be aligned to achieve business objectives and we need to measure progress.

“All project teams should ensure that work is aligned to key business objectives. It is important that management at all levels can see that the overall progress is measured, risks are managed, and the wider view across the project is maintained”.

— Project Management Institute, The Standard for Earned Value Management (2019)

And that is exactly the rub – EVM doesn’t do anything to ensure the work is aligned to key business objectives, nor does it provide visibility to the progress toward those objectives. EVM’s magical properties only extend to tracking the relationships between planned and actual cost, schedule and scope. If your business objectives are to achieve some sort of business benefit or value, you won’t get there using Earned Value.

I previously ranted about how PMI focuses too much on the triple constraints in Time to Retire the OTOBOS and how they ignore business value in Why Scope Creep is Complete Bullshit so I won’t repeat it here. The point is, PMI focuses only on ticking off all the tasks in the WBS, and completely ignores the achievement of business benefits. It assumes the achievement of the plan is the goal, not the delivery of business benefit.

Earned Value Management Misuses the word “Value”

When PMI tries to apply EVM to agile initiatives, it mis-uses basic terms and sows confusion. Let’s look at the description of each of these terms from the PMI EVM Standard (see section 4.4.3.2 Calculating PV, EV, and AC within Agile).

The comparison of EV with PV lies at the core of EVM. PV is the value of the work planned for a certain date. It is the entire budget for work to be completed at the planned date. In agile terminology: It is the sum of the estimated feature sizes for all the features up until the planned date.

— Project Management Institute, The Standard for Earned Value Management (2019)

Notice that the PV or planned value, is stated as the value of the work, but then in the next sentence, it becomes clear that it is the budget for that work or the sum of the feature sizes. What PMI is saying is that PV is the cost or what you planned to spend, and that is equal to your estimate in story points. To that I say, cost and story points are not “value”.

Let’s look at Earned Value (EV):

For agile projects or control accounts, EV represents the value of work completed (i.e., total number of story points for all completed user stories times number of completed iterations) at the same date as used for PV. EV is not synonymous with actual cost, nor does the term refer to business value. EV refers to technical performance (work) earned or completed against the baseline or work planned. In agile terminology, it is the value of the completed user stories up until the calculation/status date.

— Project Management Institute, The Standard for Earned Value Management (2019)

In this section, it states the EV is the value of work completed or the number of story points completed. Then the kicker, “EV is not synonymous with actual cost, nor does the term refer to business value“. It goes on to point out that EV is a comparison of actual work compared to a baseline. So the earned value is the spent money and it doesn’t relate to business value.

And then we have another misleading statement “In agile terminology, it is the value of the completed user stories up until the calculation/status date.” No, it is not the value, it is either the cost or the number of story points.

Finally, let’s look at Actual Cost:

Actual cost is what the name implies: the cost in dollars to complete a set of user stories by the team per iteration and is usually derived from work hours recorded by the organization’s time reporting/tracking system.

— Project Management Institute, The Standard for Earned Value Management (2019)

Finally, a cost correctly identified as a cost. In reality, PV, EV, and AC all represent costs. None of them represent value.

Unfortunately, the PMI Standard ignores this and keeps on using the term value when it means cost. How about this statement highlighted below where “story points are mapped into financial value based on a value-per-point conversion rate of $3,500 per point (i.e. the burn rate).”

Agile EVM works by comparing the current release plan (taking into consideration changes to the scope in the product backlog) against actual work completed as captured in the release burnup chart. For example, progress can be shown on a burnup chart as depicted in Figure 4-7, where story points are mapped into financial value based on a value-per-point conversion rate of $3,500 per point (i.e., the burn rate).

Oh my, more misusing story points. The PMI Standard probably also contains a conversion chart of story points to hours as well.

In case it is not clear from the above how EVM works, let us take a very simple example.

A Simple EVM Example

Let’s assume that our initiative is to buy Christmas presents for my wife. I have a budget of $200 and I am going to get her 4 presents, one per week for the weeks leading up to Christmas. Each present will cost $50.

  1. Sweater ($50)
  2. Yoga Mat ($50)
  3. Gift Card ($50)
  4. Framed photo ($50)

While the first three are items I can buy off Amazon, the fourth is a combination of a nice picture frame, a printed photo, and my effort to come up with some kind words to add as a statement on the photo.

In terms of schedule, I am going to buy one present per week in December. However, though the deliverables may all be bought each week, they won’t be delivered to the customer until Christmas. That is when the customer (my wife) is going to open them up.

Here is what the EVM charts look like at the end of each week:

Week 1: I successfully purchased the sweater for $65. Planned Value was 50, Earned value was $50 and actual cost was $65.

EVM Week 1

Week 2: I got busy and did not do any shopping. Cumulative Planned Value was 100, Earned value was $50 and actual cost was $65.

EVM Week 2

Week 3: Whew, I carved out some time and was able to get out and purchase the yoga mat for $25 and the gift card for $50. Cumulative Planned Value was 150, Earned value was $150 and actual cost was $140.

EVM Week 3

Week 4: In week 4, I purchased the photo frame, and then I spent some time editing the photo to add a cool caption. Cumulative Planned Value was $200, Earned value was $200 and actual cost was $190.

EVM Week 4

The EVM Charts Don’t Tell the Story

This simple example illustrates two key points:

  1. The “value” perceived by the customer is not the same as the cost
  2. The customer doesn’t get any “value” until the solution is delivered

Let’s explain.

The Perceived Value is NOT the same as the Cost 

The cost of the four example presents was $190 in total. However, that is not the Value of those presents to the customer no matter what the PV and EV say.

When my wife opens all the presents on Christmas, the “value” she places on the first 3 is almost nothing. The first one (sweater) actually had to be returned since it was the wrong size (my bad). The other two are just meh. But the 4th gift, the framed photo of her and I, this gift makes her cry, and cry. She hugs me. She takes the photo and puts it on a prominent place on her dresser and she looks at it every day.

The EVM calculations are about costs and waste, and they have absolutely nothing to do with value. Value is measured by the customer, and it is detached from the cost. The least costly of the 3 gifts above was the most prized.

There is No Value Until Delivered to the Customer

True value is not delivered until the customer is able to use the solution. In the simple example, the presents are purchased each week but they are not opened until Christmas Day. So there is no value in the customer’s eyes until Christmas. Getting those first presents bought and wrapped incurred cost but there is no value until the day the customer gets it into their hands.

In Lean thinking, all interim deliverables represent work in progress and are therefore waste. So any gifts bought that are not delivered to the customer represent waste.

At the end of the third week, EVM says you have earned value of $150 but the reality is all that you have created is waste at that point. And since we don’t get feedback until the end of the project, we also create risk.

What is the Point? Stop Applying EVM to Agile!

By trying to force-fit traditional approaches like EVM to agile initiatives, PMI continues to miss the boat. While PMI is trying to make some strides in the agile space (See: PMI Goes all in with Disciplined Agile and Flex), this EVM effort just shows that they don’t fully appreciate agile ways of work.

I know that I have probably have readers who love EVM. Don’t send me your hate mail to explain that it is required for your government agile program and how valuable you think it is and all the important project status charts you can get, yada yada yada. That may all be true, and good for you if it is helpful.

Like Scope Creep, the focus on EVM in agile environments is misplaced. Stop.

PMI should start focusing on value delivery, a term they use in the PMI-ACP certification. Not “Earned Value”, but the delivery of true customer value.

Agile PM book

This Post Has 11 Comments

  1. Keith Collyer

    When I was first taught about “earned value” some thirty years ago, I made exactly the same point about there being no value until something was delivered. The lecturer simply could not understand that point at all. The real problem is the term “value”. It’s an attempt to turn expenditure into some sort of progress measure and fails hopelessly at it.

    1. Anthony Mersino

      Hi Keith, thanks for weighing in. Yes the term “value” is one of the key problems. Too much focus on the triple constraint and not enough focus on delivering what the customer needs and getting feedback.

      I hope this post was of value to you!
      Anthony

  2. Bill Baxter

    It seems to me that the very idea of trying to apply EVM discipline to an Agile work effort is like an oil-slimed albatross with one wing missing and the other badly broken. It will never fly.

    As Anthony indicated, EVM measures progress relative to detailed cost estimates, and such estimates simply will not exist in a true Agile work effort. They cannot be created up front because the scope cannot be defined up front in sufficient detail. That was why you needed an Agile mode in the first place!

    Still, one has to wonder how to integrate an Agile work effort into a larger, generally predictive DOD project where use of EVM reporting is contractually required. Perhaps just keep it simple and use a ‘level of effort’ EVM ‘value recognition rule’ for the small Agile portion of the project.

    1. Anthony Mersino

      Bill, thanks for your comments and for sparking the topic in the first place. I LOVE your oil-slimed albatross metaphor.

      Thanks!
      Anthony

  3. Maxim Avina

    If not EVM, what method(s) do you suggest to measure the project performance at any time?

    1. Anthony Mersino

      Hi Maxim, is your goal to determine how the team is doing vs. a planned delivery schedule and scope? Or are you trying to determine if the project is a success?

      If the latter, I would meet with the team, the Product Owner and users or stakeholders. I would ask about number of releases of working solutions over the last 90 days. I would evaluate whether customers are adopting the product, measure new feature usage or determine the ROI for new features. Project performance is the delivery of real value to real customers, IMHO.

      If the former, I would ask a PM to pick a color – red, amber or green. And they will say green.

      Anthony

      1. Jeff

        I find this to be an unsatisfying answer. There has to be a leading indicator that project isn’t going to be successful as opposed to waiting until the end or asking the Scrummaster his or her opinion. Your response implies that Agile shouldn’t be applied to large, complicated project … which of course is true in some cases.

        1. Kevin

          The issue here is that you are thinking in waterfall terms. There’s nothing innately wrong with that, but people have got to stop believing they can effectively serve both masters at the same time. You can go through the exercise of adapting EVM for agile projects (I’ve even done it myself!) but ultimately it doesn’t show anything of actual value and just sets up bad expectations or things to shove under a microscope.

  4. David Morris

    Breath of fresh air. I was studying for the PMI-ACP, and just couldn’t really understand when / why I would use EVM for anything in an Agile environment (unless I had time on my hands). I feel like it is a clever practice, but ultimately I don’t see the value it brings to (most) businesses, as if I started talking in EVM terms at most customers I’ve worked with, I just don’t think they would understand it anymore or less than if I was simply talking about burnups, points and velocity.

    Thanks!

    1. Anthony Mersino

      Hi David, thanks for your comments!

  5. Yves Derks

    Hi Anthony,
    Thanks for sharing this perspective!
    I would like to put a perspective against it; as I do see value in EVM. Only we need to upgrade and make it more flexible.
    For example, in your example with the present for your wife. EVM does not fail, but your project setup. You do not involve your wife (business) in the creation of the outcomes. So where a surprise is nice for your wife, for customer it isn’t. They want to get involved in the creation and see value throughout the project; core principle of agile projects.
    EVM helps show value created along the way based on deliverables as outcomes. These deliverables are jointly defined as “value”.
    Curious about your feedback, happy to discuss further!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.