Getting Advantage of OKRs: Outcome vs. Output

If you're reading this article, Objectives and Key Results (OKR) is probably familiar to you. If you're having success with OKR, congratulations! I would love to hear about your learnings. 

On the other hand, if you still need to realize the benefits promised by OKR or OKR is a frustrating initiative for your organization, I hope that what I have to share will provide value. (And that you didn't just click through because it was a random selection of one of the 30 million options listed when searching "OKR" on Google).

Where Did OKR Come From?

In the book Measure What Matters, the iconic John Doerr shares the history of OKR. This is a must-read if you want to know how OKR came about.

I want to highlight an interesting passage from the book. In one of the meetings Doerr had with Larry Page and Sergey Brin (founders of Google) in the early stages of the company, he explained what OKR was. He provided the following definition for Objectives and Key Results:

"An OBJECTIVE is simply WHAT is to be achieved, no more and no less. By definition, objectives are significant, concrete, action-oriented, and (ideally) inspirational."

"KEY RESULTS benchmark and monitor HOW we get to the objective. Effective KRs are specific and time-bound, aggressive yet realistic. Most of all, they are measurable and verifiable."

The OKR formula

John Doerr provides the example below:

OBJECTIVE:

Build a planning model for their company

KEY RESULTS (AS MEASURED BY):

KR #1: Finish the presentation on time.

KR #2: Create a sample set of quarterly Google OKRs.

KR #3: Gain management agreement for a three-month OKR trial.

How Does OKR Measure What Matters?

When reading this example, I realized that it conflicted with the Objective and Key Result definitions mentioned above. The first question I asked myself when reading this OKR was:

"Is this OKR really measuring what matters?"

At that time at Google, with so many product ideas and alternative experiments on the table, maybe prioritizing activities was all they needed. That's exactly what this OKR is trying to do.

Along with this example, countless OKR examples on millions of blogs (especially activity and project management software blogs) follow the same pattern. Key Results are action-oriented (tasks, initiatives, projects, etc.) or are milestones (one of the main elements of traditional project management).

Here is another example created in the days of Andy Grove (considered the father of OKR) at Intel:

OBJECTIVE: Demonstrate the 8080's superior performance as compared to the Motorola 6800

KR1: Deliver five benchmarks

KR2: Develop a demo

KR3: Develop sales training materials for the field force

KR4: Call three customers to prove the material works

Again, the example isn't measuring what matters. In this case, OKR is being used as an activity management tool. However, Doerr cites the former CEO of Yahoo! Marissa Mayer's rule in the book, a rule that, in my experience, is the main one to start creating good Key Results: “It's not a Key Result unless it has a number”.

Andy Grove and John Doerr are among the greatest business leaders of all time. So who am I to criticize the OKR posted by them?

It turns out that OKR, like other disciplines, has evolved over the years. When it was created at Intel in the 70s (with the name of iMBOs "” Intel's Management by Objectives), there was no clarity around frameworks and methods for managing activities like today. Think Scrum, Kanban, or even PMBOK and PRINCE2.

Moreover, many of the examples in Doerr's book were created before the age of the internet and mobile. With the rapid evolution of technology and the commoditization of information, it has become crucial to have mechanisms to measure more complex elements of the business, such as customer satisfaction and behavior in our products and services, in real-time.

Along with the modernization of the digital product management discipline, we now have a more modern way of working with OKR. One which is less focused on measuring effort (i.e, output) and more on outcome-based planning with measurable benefits for the business, customers, and employees.

Even with the widespread adoption of Agile methods, we still see leaders obsessed with measuring deliveries and increasing the speed of teams. Many leaders are unaware that speed in the wrong direction is wasted. Some are aware, but they measure deliveries by volume, often leading to harmful behaviors in the work environment.

Sadly, it is not unusual to find Key Results like the ones below:

• Deliver 90% of story points in the next release
• Increase the number of items delivered per week (throughput) from X to Y

Yes, that's sad. As Eli Goldratt said: "Tell me how you will measure me, and I will tell you how I will behave."

OKR can be perceived as a big waste of time for many organizations due to two simple facts:

1. Not recognizing the difference between activities, outputs, and outcomes
2. Not managing these three elements differently.

OKR is about focusing on getting better results (outcomes, not outputs). But wait! What is the outcome? 

Why is this word used increasingly in managing products and services?

Enter Third Sector: Outcomes vs. Outputs vs. Activities vs. Inputs

Lately, the term "outcome" has been used more and more, especially in the management of digital products. However, one of the main origins of this term comes from the Third Sector.

According to the writer and anthropologist Rubem César Fernandes,

"...the third sector is formed by private organizations of civil society that have public social purposes, thus distinguishing themselves both from the organizations of the state sector (the so-called first sector whose purpose is to the management of public policies), and private companies (the second sector, whose main purpose is to generate profit for their owners and shareholders).

According to Fábio Ribas, founder of Prattein (link opens information in Portuguese) and a specialist in policies, programs, and studies in "‹"‹social development (and my father),

"...third sector organizations are not-for-profit and can significantly contribute to the development of society. They are private organizations, but they have a public purpose. They question the rigidity of the division of society between State and Market and focus on the need to build more just, balanced and sustainable societies."

Therefore, this long-time sector (the Third Sector) of the economy has already made an important distinction in its management between three distinct elements:

• Inputs
• Activities and Outputs
• Outcomes

The following is a summary of how they describe each of the elements (source: www.oecd.org):

Inputs

Inputs are the resources used to carry out activities, whether financial, organizational, human, or material. Examples:

• Budget
• Social workers
• Computers, servers, classrooms
• Medicines

Activities

Activities is an "umbrella" term used here to represent processes, actions, or events performed through which inputs are mobilized to produce specific outputs. Examples:

• Create the layout of a new page for the project website
• Printing of teaching material for students
• Replacement of defective equipment
• Development of a mentoring plan for new graduates

Outputs

Outputs are the products created or services provided directly from the activities performed. Outputs generally represent the volume of delivery.

• Creation of ten digital marketing campaigns
• Training for five new surgeons
• The entire team trained in the Kanban method

Inputs, activities, and outputs are elements over which we have control. The same does not happen with outcomes.

Outcomes

Outcomes are the expected changes as a result of the activities performed. They are the desired effect or consequence.
Outcomes represent desired changes in people's or customers' behavior, relationships, actions, skills, or knowledge. We can influence outcomes (like outputs), but we have no direct control over them (unlike outputs). Examples:

• 10% increase in visits to health clinics
• 20% decrease in no-shows for medical appointments
• 30% increase in people declaring that they have a better quality of life
• 25% increase in the number of students who obtained a job in their field of training

Thus, in the language of Third Sector social projects, outcome-based planning is a planning, monitoring, and evaluation approach that puts people at the center (social impact), defines results as changes in behavior, and helps to measure contribution to complex change processes.

W.K. Kellogg Foundation suggests leaders ask the following questions to assess their outcomes in social projects:

• What are the critical results you are trying to achieve?
• What impact is the project having on people, your team, your organization, and your community?
• What unexpected impact did the project have?

Do you see how the same questions apply in our world of technology and digital products?

This "result chain" can be illustrated in the image below (image Adapted from Outcome Mapping Learning Community):

Outcomes vs. Outputs in OKR: How to Ask Good Questions to Define Outcome-Based OKRs?

In the modern language of OKR, Key Results represent outcomes, that is, measurable benefits for the business, customers, and employees.

Following this definition dramatically increases the likelihood of OKR becoming a useful tool, generating a sense of purpose at work, focus, alignment, and agility.

Often, the most effective way to identify good outcomes is to get out of our obsession with activities and ask good questions. 

Let me share a real case with you"¦

Once, a client of mine (children's fashion e-commerce) asked me to evaluate one of their OKRs, as they believed it was not well written. One of the aspirational OKRs they showed me was the following:

OBJECTIVE:

• Make our product the best in the segment

KEY RESULTS:

• Launch 15 new features
• Perform 12 usability improvements to the product

When I saw this OKR, I had two options to try to help.

Option 1: Demonstrate my expertise by judging the OKR:

"This is not an OKR! Launching a Website is clearly an activity, and these Key Results are outputs (delivery volume)".

Option 2: Engage with them in a creative conversation to help them reflect and maximize the quality of their OKRs.

Being aware that it was important for them to learn and think for themselves, I chose option 2. 

Below is a snippet of the conversation I had with them:

Me: It's interesting to see that you guys have a lot of new ideas for the product. At the same time, I'm curious to know the most important thing that needs to happen by the end of the next quarter.

Client (after some silence): Well, we really need to make the buying process flow more fluid for the customer.

Me: By the end of the quarter, how would you know if this fluidity has improved?

Client: Looking at the data, we could say that we need to increase the conversion of our shopping cart.

Me: What would be a challenging and realistic raise?

Client: I believe a 15% increase.

Me: OK, how can you measure that?

Client  (after some discussion among team members): Actually, a 20% increase in conversion of products for 0-3-year-olds would be a good idea. So we could focus on improvements for this segment and gather feedback earlier.

The above dialogue (simplified excerpt) was not about who will deliver which tasks by when. It was a business conversation with the client in mind.

That conversation helped the group develop a measurable, specific, and challenging Key Result that made sense for all team members.

We continued the conversation, and instead of having 27 outputs in the OKR (15 features + 12 improvements), they managed to define another 2 Key Results types (outcomes) for the quarter, in addition to re-prioritizing their backlog and product roadmap according to the OKR.

By doing so, they were able to improve their focus and productivity, consequently reducing the number of simultaneous activities (Work In Progress) and the overload.

Conclusion

OKR is a powerful tool to increase an organization's agility, focus, and alignment. However, despite being simple, it is not easy. Three things are important to get the real benefits of these organizational superpowers:

First, avoid using OKR as an activity management tool. Leave that to activity management methods and frameworks.

Second, always seek to identify outcomes instead of outputs while crafting your OKRs. It won't always be easy. The so-called "Milestones Key Results" is tempting but a rather lazy way to define Key Results. 

Finally, in the period dedicated to defining the next OKRs cycle, have some time with product/service people to talk and ask good questions during the process. Great OKRs are not built in 15 minutes and require time, analysis, and leadership.

I wish you success on your OKR journey.

>> You can read more from Thomaz at the Thomaz Ribas Blog