The ROI of L&D

With some frequency, clients ask me to give them a list of measures that will tell them if L&D is delivering value to the business. They ask for best practices or better yet, names of exemplary firms who have demonstrated that L&D is worth the investment that the business is making in them.

It’s not the best practices they're after; their objective is to ‘lift and shift’ the measures and practices to their own environment.

While there are no “magic measures,” several organizations and thought leaders have defined a suite of common measures that enable L&D to evaluate its worth. They fall into three distinct categories:

• Efficiency measures that track L&D activity/throughput, reach, effort, and cost. L&D has a plethora of efficiency measures. The Center for Talent Reporting (CTR) has a library of nearly 150 efficiency measures segmented by sub-type of measures. The Association for Talent Development (ATD) benchmarks many of these measures.
  
  
• Effectiveness measures that track satisfaction, value, application, and results. Organizations typically collect this data via surveys to assess learners’ perceptions of the quality and value of L&D solutions.
  
  The best practices for these measures are well established based on the Kirkpatrick four levels or the Phillips five levels. CTR has nearly 40 effectiveness organizations and Explorance, through its Metrics that Matter platform, has robust benchmarks on many of these.
  
  
• Outcome measures that track the impact of learning on key business measures. This is where most L&D organizations struggle because there isn’t a suite of common and standard outcome measures they can leverage. Rather, they need to choose the outcomes based on the business priorities that L&D plans to support.
  
  These measures may include financial measures (e.g. advance sales), operational measures (e.g. improving employee productivity), client measures (e.g. growing client loyalty), or people measures (e.g. retaining our high performers).
  
  Learning programs could support all these measures through a portfolio of programs aligned to strategic and non-strategic business priorities. There are no standard measures in this category because they are entirely context-dependent.

Because there are so many choices for measures, many L&D practitioners quickly become overwhelmed. 

Rather than randomly choosing measures from a lengthy list of possible success indicators, consider adopting these approaches:

Overall

• • Identify success measures for all learning programs as well as L&D department initiatives to improve processes or systems. (If you're going to measure, don’t cherry-pick which programs or initiatives you should measure, include all of them.)
    
    
  • Consider the purpose of the program when selecting measures. Strategic programs that develop differentiated skills will require more robust measures of effectiveness such as knowledge and the amount of learning applied on the job.
    
    Programs that support the operations and build industry-standard skills may need only a few measures and a ‘lite’ measurement plan.

Efficiency and Effectiveness Measures

• • Include both efficiency and effectiveness measures for all programs and initiatives to mitigate unintended consequences. For example, if you focus solely on throughput (efficiency) measures, you run the risk that quality will suffer. If you focus only on quality (effectiveness), you may spend more to design and develop than the outputs of the program warrant. By including both efficiency and effectiveness measures, you lower the risk of creating ‘perverse incentives.’
    
    
  • For efficiency measures, identify the types of measures you should include such as activity/throughput, cost, utilization, reach, cycle time, headcount, or effort. Then identify a limited set of measures within the appropriate types. Repeat this exercise with effectiveness measures. Reference best practices to help you decide.
    
    
  • Limit the list. More is not always better, particularly when you are first starting your measurement journey.

Outcome Measures

• • For strategic programs that support business priorities, always include an outcome measure. If the program is truly intended to advance a business priority, L&D should commit to impacting the relevant measure for that priority.
    
    
  • Select outcomes for strategic learning programs at the outset prior to the design phase, not after the fact. Ask, “What is the purpose of this program and what outcome do we expect to impact?” Gain agreement with the goal owner on the desired impact to ensure that L&D and the business have aligned expectations.

For more details on selecting measures, read Chapter 7 of “Measurement Demystified” co-authored by myself and David Vance.

Practices of Exemplary Organizations

Although these steps sound simple, few organizations consistently follow them. They may intend to adopt best practices and expert guidelines but often stall when they get resistance from their own staff or business leaders who don’t understand why L&D is asking so many questions.

These organizations focus on the tactics of the task, not the broader processes, practices and behaviors that build a measurement-oriented organization. 

Organizations that are successful over the long term find the magic by shifting what, how and with whom they work. These ‘measurement exemplars’ have six common characteristics:

  1. Broad, deep, and trusted relationships with business partners. L&D speaks the language of the business, they allocate resources to programs and initiatives that will advance the business strategy and make good on their promises by delivering effective and efficient programs that have an impact. These organizations were not born this way. They made it happen with sustained effort and a commitment to continuous improvement

  2. Skilled L&D professionals who have a deep understanding of business requirements and can adeptly translate those requirements into meaningful learning experiences. They start with the ultimate outcome and design their program to achieve it. The universe did not magically drop these skilled professionals into their laps. Rather, these organizations built the skills with their existing workforce using formal and informal methods with sustained effort and a commitment to continuous improvement.

3. Strong stewardship for the discipline of measurement. Every exemplary organization has a strong leader at the heart of its success. The leader provides resources, funding, and direction for measurement.  The leader recognizes that the journey to build a measurement-oriented function will take time, often two years or more. The leader recognizes that success will take sustained effort and a commitment to continuous improvement.

4. Standard methods and tools to drive consistent methods and processes. Exemplars recognize that a myriad of inconsistent measures, disparate collection methods, and no central repository makes it extremely difficult to tell a meaningful story of L&D’s impact. Best-in-class organizations recognize that standards create consistency and enable capabilities that are otherwise out of reach. They are also prepared for sustained effort and a commitment to continuous improvement to keep these standards in place and relevant.

5. Accountability for results to move the organization toward managing its business rather than simply monitoring it. Exemplars understand that without clear accountabilities for results they will never move the needle on measurement (except by accident). Organizations that clarify who owns what and the expectations for performance are positioned to be successful. But again, it takes sustained effort and a commitment to continuous improvement to make this work.

6. Technology that enables the organization to scale. Exemplars understand that measurement is not a one-and-done exercise. It is continuous, repeatable, and must adapt to varied solutions, timeframes, and audiences. It requires a platform for gathering evaluation data and providing role-based reporting that enables users to manage their operations. Technology isn’t static. To make it work and adapt requires, you guessed it, sustained effort, and a commitment to continuous improvement.

The message of course is that exemplary organizations become exemplary through disciplined and sustained execution of a robust strategy and a dedication to continuous reflection, learning, and improvement. The magic is in the dedication to becoming exemplary, not in the measures themselves.