California Management Review
California Management Review is a premier academic management journal published at UC Berkeley
by Andy Neely and Shaun West
Image Credit | Luke Chesser
Manufacturers often face customers who say, “we don’t want to buy your products, we want to buy the capability they deliver.”1 Meanwhile, outcome-based contracts are often delivered by multiple organizations working in partnership.2 How can performance measurement systems result in a harmonious working relationship when the service and solution are delivered by organizations working together?
“Changing the Work of Innovation: A Systems Approach” by George S. Day & Gregory Shea (Vol. 63/1) 2020.
We studied five separate organizations, covering a range of different industries and sectors. All were recognized as being at the leading edge of outcome-based contracting, and all were providing service and support for large-scale, complex assets.
The company supports the maintenance and upgrade of naval warships and manages the overall infrastructure on the naval base.3 There was a tendency to measure what was easy to measure, rather than what was needed. The metrics measured activity, rather than improvement or efficiency, they often measured operational details rather than strategic issues.
To address these issues, the company, customer and both sub-contractors viewed themselves as a single entity, agreeing enterprise-wide KPIs to align activities.
The Original Equipment Manufacturer (OEM) was contracted to construct and maintain the wind turbines on a new offshore wind farm. As part of the contract, the OEM was asked to guarantee wind turbine availability.
By not considering the impact of wind seasonality on the earnings potential of the owner, the OEM showed a lack of customer understanding. Technical availability is not the same as commercial availability.
Over a five-year period, a major oil and gas company and its service partners moved from delivering poor operational outcomes with an excessive focus on cost, to an improved outcome-based relationship with good operational performance that focused on value.
This case highlights the way that metrics can evolve as trust and confidence builds between different parties in the outcome-based contract. In this case the parties shifted from an internal efficiency mindset with associated metrics, to a much more explicit outcome-based effectiveness approach.
This case focuses on an industrial and power systems equipment manufacturer, their intermediary dealer - who sells and services the equipment - and the end customer, a quarry. The dealer now offers services that include fleet or quarry optimization. With remote monitoring, they can now track the health of individual machines, and also to monitor working efficiency.
As the relationship between the OEM, the dealer and the customer grows and matures, the dealer is privy to confidential and sensitive production data. They can then advise the quarry operator on lost production time. These data can help their customer - the quarry operator - achieve the outcome they want, i.e., a minimum cost per ton of minerals extracted.
A power generator had focused solely on the wholesale generation of electricity. However, they recognized an opportunity to provide steam and electricity for process manufacturing plants, e.g., paper mills. This required smaller, less capital-intensive power stations, so the firm started producing these through its combined heat and power business.
The power generator entered into an alliance agreement with a constructor of power plants. Using a number of different metrics they measured important outcomes for the construction of a new power plant. All of these metrics had a target plus cap and floor applied. A key lesson from the case was the way aligning economic metrics helped the organizations develop a shared vision. This was made real by mechanisms to share pains and gains.
Table 1 Overview of the case studies
The five case studies each provide important insights into the question of how metrics might be aligned in outcome-based contracts, where multiple parties (including the customer) are directly involved in service delivery.
From the five case studies, we found that there are two clear phases: contracting and delivery/execution. The split is common in practice: one team deals with the ‘sales process’ and a second team deals with ‘project execution’ with a hand over between them. The contract initially agreed was what was initially required, and over time it needed to adapt as situations changed and evolved. The metrics used during the delivery phase drive the behaviors of all parties’ teams involved in delivering the project. Strong governance is necessary to provide guidelines for both performance metrics and contact renegotiation.
It may seem trite, but understanding who the customer is, and particularly what outcome they truly value is fundamental. Tensions can arise when the desired outcomes are not clearly defined and understood.
In case 1, the provider was contracted to maximize uptime and delivered this. Unfortunately, uptime in July is worth less than uptime in November, simply because there is less wind to produce energy, which ultimately is the commodity the customer wants. By focusing on uptime, rather than energy generated, the misaligned KPIs created a tension that ultimately caused the contract to fail.
The same is true in case 4. By focusing on the outcome the customer really values - minimum cost per ton of mineral extracted - the OEM and their dealer have been able to innovate their business model. Rather than simply optimizing the performance of individual machines they have developed approaches to optimize the performance of the system - the entire quarry.
Understanding the pains the customer encounters and the gains they accrue adds a layer of texture to the discussions about the outcomes customers value. Based on this you have a better opportunity to enrich your understanding of the customer and build a stronger relationship with them.
In case 5, by identifying new customers and then understanding their need for electricity and steam, the power company was able to evolve its business model. Working collaboratively, the company was able to quote faster and deliver power plants more cheaply for new industrial customers. Understanding the pain they faced - delayed schedules - and the gain they could benefit from - cheap and convenient energy - meant that the power company and its partners were able to build successful new business.
A crucial step is to understand the customer’s role in the process, as case 1 illustrates. While the pain for the customer is delayed repair of ships and lack of availability, one of the inputs the customer has is control over spare parts. If the Ministry of Defence didn’t make the right spare parts available, the contractor could not complete the repair work. Understanding and making explicit these inter-dependencies is crucial when discussing and defining pain and gain sharing mechanisms.
Finding the external metrics that show your customer’s success may sound trivial, however, time and again neglecting it has led to assumptions that are not confirmed. If the external metrics are set by those negotiating the contract, rather than those who have to operate the contract it can lead to the wrong metrics being put in place. This in turn leads to the wrong outcomes being delivered.
One of the first tasks for the service company once the contract is operational is to reconfirm the metrics. Are they the right ones? Are they understandable? Are they easy to measure? Without this check the risk is that the answers will be ‘no’, which results in infective or detrimental metrics. The wind power case (case 2) was a clear example of this, where technical availability was the measured metric, however, the measure for effectiveness was commercial availability, which was out of alignment.
It goes without saying that project efficiency should be measured. For a firm this provides some important feedback, particularly in terms of the sales and margin generation. Without contract performance measurement, a comparison with the original contract cannot be undertaken, or assessment made of the validity (or otherwise) of the assumptions in the contract.
The project dashboard should present the internal metrics in some form or other and will be seen by the customer at some point in time. There will always be concerns that commercial data is being shared with the customer, however more value can be obtained from joint discussions rather than hiding information.
Limit the number of metrics that are measured and shared with the team to 5. But make them the 5 metrics that really count. Group and team the metrics so that only the important ones are presented. When a measure is not performing, it is important to understand the hierarchy between metrics and to drill down to find the contribution to the problem and then track the metric as part of an improvement action.
All too often the metrics are just tracked and presented; this is insufficient for effective management. The data should be translated into actionable information that local employees can understand. For this reason, the metrics must have a short commentary associated with them so that the performance can be put into context.
The initial targets for metrics are common issues in contract negotiations. Complications occur where some metrics are contractually binding with others being used for guidance only. The wind power example (case 2) set up initial metrics and targets but missed the implications of their choice.
Wherever possible, agree realistic targets based on real data. Where there is no data for target setting, while it is being collected, a desired target will show an expected level of performance.
Without regular meetings misunderstandings can occur, such as the problems with the wind power case (case 2). Initial meetings should be used to confirm that the metrics are relevant and that how they are measured is well understood by all parties. Institutional arrangements, such as ESCROW bank accounts, smart contracts, automated reporting and external facilitation, can help build trust with the alignment of the metrics.
Governance can also provide an important forum where changes to metrics can be discussed. This often requires changes to the contract – some may even say that the contact is being re-opened to negotiation. Over a long-term agreement a contract can be expected to change as markets change, as data differs from expectations, and as technologies develop. In all, proactive re-alignment between partners ensures that the service contract continues to deliver good performance and the outcomes the customer requires.
Performance metrics should be posted and tracked for people to see, as in case 5, so performance can be understood. A range of different metrics gives a balanced view of performance.
With the right customer-facing metrics jointly agreed, the whole team (customer and supplier) can understand where performance is good and where improvements are needed. Design a dashboard to present the service effectiveness from the customer’s point of view. As well as reporting to your customer, a performance dashboard is needed internally. This should be built up from the customer’s performance measures and additional internal metrics – often more financially based – to measure the efficiency of the service delivery.
Case 3 shows clearly how operational metrics change over time, which is why the governance of metrics is important. The customer’s business environment, process or outcomes may change, meaning that new metrics are needed. The customer’s expectations will in any case increase year-on-year. This is particularly the case where performance metrics are routinely achieved. The service provider must be flexible to manage these changes.
Not all metrics are independent. There are often relationships between different metrics, which is very much the case with quality and cost. A payoff chart of the different metrics may help here. Both the customer and the supplier can understand the payoffs, allowing both parties to learn from their joint performance.
Before starting to create a set of outcome-based metrics, it is important to ensure all parties share a common understanding of the desired outcomes. This is often not the case, with assumptions being used incorrectly. Creating measures for contractor efficiency can be relatively straightforward. Standard measures, such as on-time delivery, are often sufficient.
Moving to outcome-based metrics is much harder, as it requires all parties to consider the effectiveness of the contract, and to compare that with the desired outcomes. It also starts to create a mix of ‘soft’ and ‘hard’ metrics, as not everything that should be measured can easily be measured.
It is at this point in the contract negotiations that aspects that the contractor cannot control will have a significant impact on the success of the outcome. An example is case 2, where technical availability was contracted but the commercial success of the wind park was based on commercial availability – a function of wind speeds. In a mature negotiation, the parties will consider the impact of both wind seasonality and technical availability to create suitable metrics to drive the behaviors.
Reporting performance needs to be considered during the initial design of the metrics. For some metrics trend lines may be sufficient, however for others it becomes important also to provide access to metrics one level lower, to give the customer the required level of confidence in service delivery.
The importance of good governance for the metrics following their initial use cannot be overemphasized in a long-term agreement. Both the metrics and the targets will need to be updated due to environmental changes. An inability to identify the need for changes, negotiate new metrics and targets and then to agree them will create tensions between the contracting parties.
A. Neely, “Exploring the Financial Consequences of the Servitization of Manufacturing,” Operations Management Research, 1/2 (December 2008): 103-118; S. Vandermerwe and J. Rada, “Servitization of Business: Adding Value by Adding Services,” European Management Journal, 6/4 (Winter 1988): 314-324.
Grönroos, C. Value co-creation in service logic: A critical analysis. Marketing Theory, (2011). https://doi.org/10.1177/1470593111408177
Sampson, S. and Froehle, C. “Foundations and Implications of a Proposed Unified Services Theory”, Production and Operations Management, (2006): 15, 2, pg. 329–343.
