In PM and in Sports, the Metrics Beget the Tactics

In 2004, Michael Lewis’s bestselling book Moneyball: The Art of Winning an Unfair Game revealed how the Oakland Athletics general manager Billy Beane was consistently able to field one of the best teams in baseball by applying new metrics that better model the essence and goals of the sport. In 2011 the book was made into the Oscar-nominated movie Moneyball starring Brad Pitt as Beane.

Although the book and movie deal with baseball, the same principles apply to any endeavor: cricket, football, investing… and yes, project management. The metrics we use must reflect what we are trying to achieve.

Moneyball

Beane got his methodology from the work of sabermetrician Bill James. I had started reading James’s work in 1982 with his 1982 Baseball Abstract (now $177 for a used copy). In the fourteen hours it took me to read all the statistics and explanations in that book, I came to realize that I had never previously understood baseball, despite my almost-fanatical devotion to following the sport and its statistics.

What James pointed out is that the essence of major league baseball is making money, and making money is almost identical with winning games – you win more games, you sell more tickets, you charge more for advertising billboards, and you generate more television revenue. Therefore every tactic, every player, every decision should be measured in terms of its impact on a team’s win/loss percentage.

Baseball statistics in the early 1980s were hopelessly outdated. Batting average was still considered the prime metric of a hitter’s performance (the player who leads the league in that statistic is still called the “Batting Champion”!). But batting average ignores things that contribute greatly to winning games, such as a batter’s walks and extra base hits like home runs.

The result was that, prior to James’s new approach, players with high averages but little power who never walked, like Bill Buckner, were regarded as stars, while much better players like Fiore Gino Tennaci, whose low batting average was far more than offset by his walks and home runs, were often dismissed as mediocre.

Once Billy Beane recognized the importance of James’s approach, he used it to far better assess talent. He stocked up on players who were undervalued by traditional statistics but who actually contributed to winning games. And the result was that, with a much smaller player salary budget than the New York Yankees, Beane was able to build teams that could compete with the big money teams on an even basis. That is, until the big money teams also started using the same techniques and the wealthy Boston Red Sox, with James as an adviser, won the championship for the first time in 86 years in 2004. And they’ve won it twice more since.

What has all this to do with project management? Like batting average in baseball, the metrics we use to measure project performance and to categorize a project as a “success” “or “failure” are deeply flawed.

Baseball is a sport, and its metrics need to be based in the winning or losing of games. Projects are investments, and their metrics need to be, like all other investments, based in creating value above cost, i.e., ROI or profit. The batting average statistic tells us something about a baseball hitter – but disconnected from data about wins and losses, it suffers from distortion. So too traditional PM metrics, such as on time and within budget, tell us something about project performance – but disconnected from the expected monetary value (EMV) of a project, and its expected project profit, they too are subject to distortion. Which would you rather have – a project that creates an adequate product on time and on budget or one that is even more fine-tuned to the needs its intended to address, that is deployed earlier and costs less?

This discussion in the LinkedIn Managing Benefits group led to me mounting my Moneyproject: Metrics, Baseball and Project Management Powerpoint slides under the PRESENTATION tab on my website home page. There also is an article titled Moneyproject that I published in 2006 at ProjectsatWork.com as part of a six-article series on the basics of the Total Project Control methodology. I hope you will check these out when you get the chance.

But meanwhile, in the LinkedIn discussion Jed Simms points out some crucial factors. He writes:

“It is a poor manager who only measures ‘project success’ in terms of ‘profit’.”

He’s correct. Projects frequently aren’t intended directly to generate profit – but they must always be designed to generate benefits. These can then, through the rest of a program and/or other projects, generate further value that eventually becomes revenue, savings or non-monetary benefit. That benefit is the core of the effort and must be quantified in terms of its “expected value”. And I call the expected value of a project minus its expected cost its “expected project profit” (EPP). Someone may call it something else, but it seems to me that the term I use is an appropriate one. And its “expected” because, as with all investments, the future is unknown.

Jed continues:

“The existing project mgt tools are good for delivering projects (outputs/deliverables/products) but NOT for delivering a project investment.:

That’s the crux of it, isn’t it? Why would they be? A project is not defined as an investment – even though every project is one! Not only is the old adage about getting what we measure an accurate one, but the tactics the “performers” use are designed to reflect well on them through those metrics – and who cares about those externalities that we don’t measure?

Jed also writes:

“Use of project mgt tools alone is very wasteful in lost opportunities and returns… It is a poor manager who only measures ‘project success’ in terms of ‘profit’.”

And again he is correct. The metrics are the way we judge performance – and inadequate metrics beget inadequate tools.

But new and better metrics will generate better tactics. In baseball, techniques like bunting and hit-and run plays have decreased in frequency because better metrics show that they usually reduce the number of expected wins for a team. Instead, hitters who take lots of pitches, drive up pitch counts and get lots of walks are now highly valued.

Similarly, better project metrics that assess the value/cost of time and determine the drag of critical path activities can show where spending an extra $40,000 for a resource will reduce an activity’s drag cost by $100,000. A value breakdown structure (VBS) will quantify the value of optional work and show where the drag cost of an activity that has migrated to the critical path makes it no longer worth its inclusion. The tools will always follow the measured and demonstrated need. And once the overall approach is accepted, practitioners will add more and more tools to the toolbox.

It took almost thirty years for Bill James’s approach to become standard operating procedure in major league baseball. The first edition of Total Project Control was published just 16 years ago. But slowly, its techniques are beginning to have an impact.

Have a great weekend!

Fraternally in project management,

Steve the Bajan

2 thoughts on “In PM and in Sports, the Metrics Beget the Tactics

  1. In addition to what you mention in your article about metrics being in align with core business goals, I think we also need to recognize that metrics can drive crazy behavior. As we think about new and better metrics, we need to think about how people can misinterpret or do things inconsistent with what you want the behaviors to be in the end.

    Here is just one simple example. We had a new system coming on board. We wanted people to use the system. We put some interesting content on the system. And then we measured how often people logged in. We figured that measuring logins and putting some good content on the system would be a good tactic. What actually happened, is that people logged in and logged off and logged in again repeatedly to artificially inflate their metrics. I kid you not one person spent 30 mintues doing this. You could argue that perhaps the project was not well scoped (true statement) but the point is really that the metrics were equally as poorly designed. I find that people think metrics through in one direction — how does it align to the core business but forget to think about and how can this lead to crazy or unwanted behavior.

    For me better metrics is beyond aligning to core business goals (i.e. making money) but also about driving individual behaviors that deliver those goals.

    Like

    • Adrien, an excellent point and a slightly amusing story. As you suggest, it makes it all the more crucial to have metrics that are measuring the right things. That’s why Bill James focused on the value of a player’s batting stats in terms of the number of wins they add (either wins above replacement player (WARP) or Win Shares). On projects, simply tracking performance to schedule and budget is insufficient.

      Expected project profit (EPP) and the DIPP measure project performance integrated across the three sides of the monetized Golden Triangle to reflect what the sponsor/customer wants most: maximum expected project profit as driven by their own valuation of the scope, of the cost of time, and of the cost of resources.

      Fraternally in project management,

      Steve the Bajan

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s