Looking Under the Hood: ORION Technology Adoption at UPS

WEDNESDAY MAR 30, 2016

This case study, published by BSR’s Center for Technology and Sustainability, examines UPS®’s deployment of ORION, a route-optimization software program for its drivers, and focuses primarily on the challenges related to technology adoption, from development to deployment.

Across industries, information and communications technology (ICT) has great potential to address some of the world’s most complex global environmental and social challenges. Even where good technology exists, however, there remains a key challenge related to adoption: selecting and installing a solution, integrating it within existing systems, and ensuring successful uptake by users to verify that the technology delivers on its promise.

New technologies often require new business processes, and even if those processes promise greater efficiency, cost savings, and/or other improvements, they also require behavior change—and change is hard. 

Delivering on Sustainability Solutions: From Idea to Action 
A Center for Technology and Sustainability Framework

To develop this case study, BSR conducted interviews with UPS staff along with background research on the objectives and attributes of the ORION system. We hope that this case study will help other companies successfully implement technology-based sustainability solutions through a change-management framework that takes into consideration design, process, and people.

The Opportunity

Developed by UPS, ORION—or On-Road Integrated Optimization and Navigation—is a route-optimization system that analyzes a collection of data points including the day’s package deliveries, pickup times, and past route performance to create the most efficient daily route for drivers.

UPS created ORION as part of a broader effort to use data and predictive models to increase efficiency, thereby reducing both costs and environmental impacts. By charting more efficient routes, UPS was able to maximize the utilization of delivery vehicles and drivers, resulting in significant fuel savings—one of UPS’s largest costs.

ORION is expected to reduce operating costs by US$300 million to US$400 million a year once it is fully implemented in the U.S. in 2017. More than 70 percent of the company’s 55,000 U.S. routes are now using the software, with an average daily driving reduction of between six and eight miles. To put this into perspective, UPS can save US$50 million a year by reducing by one mile the average aggregated daily travel of its drivers. 

Specific Change-Management Challenges for Orion

To achieve maximum cost and emissions reductions, UPS had to ensure uptake at several different project stages, each with its own unique change-management challenges:

• Development: R&D leads were asked to design a technology solution that worked better than existing practice and to prove to business leaders that the approach had potential.
• Demonstration: Prototype testing and business-case validation. Prototypes developed in the lab were then tested in the field, first for smaller and then for larger groups of UPS drivers.
• Adoption: Operationalizing and rolling out. Convincing thousands of UPS employees to embrace integration of ORION into their day-to-day work.

UPS's Approach

Implementing ORION took place in three major phases:

1. Development: The Idea
   UPS started developing ORION in 2003, with the aim of layering predictive algorithms on top of UPS’s existing package and vehicle tracking systems. The development took place within the Operations Research and Advanced Analytics groups, starting with a small, diverse team: a PhD in operations research, an industrial engineer, a UPS business manager, and several software engineers. 
   
   UPS combined well known software algorithms for optimizing routes with existing business rules (such as package delivery order), but initial results were frustratingly inconsistent and not easily implementable by UPS drivers. The ORION team challenged conventional wisdom and developed a set of new rules to guide their routes. Once ORION started to produce efficient results, UPS business leaders were invited to view ORION in action so they could witness first-hand the differences between a typical driver’s route and an ORION route.
    
2. Demonstration: Testing and Validation
   Next, UPS wanted to demonstrate consistent results. Once it was clear that ORION would work in a lab setting, the team tested the program for an entire operational location and showed that efficiency significantly improved. This was then repeated in two more operational locations, and then in another eight. By then the team had expanded from five to approximately 30 people. The testing and validation at each stage enabled the buy-in and resources for further expansion. The key focus of the analysis shifted from “is there a benefit?” to “how do we scale?”
    
3. Adoption: Operationalization and Deployment
   In the final stage of adoption, UPS developed a scalable software tool and then deployed this tool across the organization. The key challenge here was to inform and motivate UPS staff across the company to adopt this new system. The ORION team has grown from 50 people at the software-development stage to more than 700 people today—the majority of whom are in the field working on technology deployment.
   
   Because UPS knew that the technology development would take significant time, the IT team developed a “hardened prototype” to start deploying even while full software development was still underway. This enabled UPS processes to integrate ORION into daily staff workflows.
   
   At this stage UPS also faced its most significant change-management challenge: Many people didn’t believe that a computer-generated algorithm could be an improvement over decades of driver experience. Making this case involved education, communication, and follow-up, but most importantly a change in how drivers and their managers measured success. 

Results: Seeing the Impact

The successes UPS has seen as a result of implementing ORION support the many years of effort that have gone into its development. The company has already seen an average daily reduction of six to eight miles per route for drivers who are using ORION routes. This reduction has already made significant reductions in fuel use and vehicle emissions.

And when ORION is fully implemented throughout the U.S. in 2016, UPS expects to see annual reductions of 100 million miles driven and fuel savings of 10 million gallons per year. These add up to 100,000 metric tons of greenhouse gas emissions avoided every year.

The Evolution of Analytics at UPS

Over the past several decades, UPS has expanded its decision-making capabilities using data and modeling tools, starting first with descriptive analytics and continuing to predictive and then prescriptive analytics.

Descriptive analytics help you understand what is happening now and what has happened in the past. For UPS, this includes identifying and tracking packages using sophisticated labeling called Package Level Detail. This allows UPS and its customers to track packages as they flow through the delivery system. Descriptive analytics at UPS also include collecting detailed truck telematics data that is analyzed to further support timely preventive maintenance of its fleet.

Predictive analytics allow you to use past data to identify where you will be in the future. For example, using historical package data, UPS can predict future package volumes. This allows UPS to match capacity with demand and better automate the package-delivery process. These Package Flow Technologies save UPS more than 85 million miles driven per year.

Prescriptive analytics go beyond making forecasts to recommending specific courses of action. ORION, with its daily recommendations for driver routes, is an example of prescriptive analytics. Because prescriptive analytics can make different (and sometimes counterintuitive) recommendations for action, change management is particularly important to ensure people trust the system.

Lessons Learned

New Technology Can Mean New Ways of Running the Business

In order to capture the full value of a new technology solution, one must think differently about the business. Because UPS had a long tradition of working in a particular way, it was challenging at first to implement a technology tool that would require a fundamentally different approach to drivers’ routes.

Indeed, when ORION was first being developed, R&D staff struggled to make it work. Developers tried to stick to the prescribed rules that the business had developed over time, but this resulted in an unworkable software algorithm that drivers couldn’t follow. When the developers took a broader perspective and re-examined driver best practices, they were able to develop a version that produced “drivable” results.

The benefits of technology-based sustainability solutions often seem straightforward—reducing miles traveled through more efficient route planning, for example. But realizing that potential requires that we re-think many other supporting systems and activities—for example, how drivers are trained and how they are measured and rewarded—in ways that can be counter-intuitive.

"With technologies that are transformational—like ORION—you have to be willing to let go of your exisiting business paradigms. You have to start with an open mind about how the technology can change the business." — Jack Levies, Senior Director of Process Management, UPS

Plan For and Invest in Implementation

With ORION, UPS found that testing and deployment represented more than 75 percent of the total cost of the project. Successful companies plan for this from the start, investing the funding, staff time, and expertise to scale. In addition, part of UPS’s vetting process for ORION was assessing whether they could easily train staff on the new system and see consistent improvements over time. For large technology systems, companies should include iteration and testing not only in product development, but also in implementation and rollout.

Even with Obviously “Sustainability-Driven” Technologies, Lead with the Business Case

With ORION, UPS’s business outcomes and sustainability outcomes were inextricably linked. Every mile driven correlated not only to reductions in miles driven, but also to reductions in greenhouse gas emissions. What ensured internal support, however, was the financial case, demonstrated through pilot testing. Sustainability advocates should always lead with a strong business rationale for any new technology, even for approaches that create obvious social and environmental impact. Without this, it will be difficult to attract the internal resources to scale.

The Technology Must Be Easy to Explain

Because ORION is based on a new kind of predictive model, it may recommend route scenarios that—while more efficient—are highly counter-intuitive for a driver. Especially at the beginning of ORION’s deployment, the UPS team paid special attention to having every aspect of the system be easily understandable by drivers. This ensured that the operations teams understood how the system worked and why it did what it did, which enabled them to see when and how it provided more efficient solutions. As drivers have become more familiar with ORION and have learned to trust its recommendations, the team has been integrating even more sophisticated and efficient optimizations that are even less intuitive.

Assessment Metrics Must Align

One challenge that UPS experienced early on in the deployment of ORION was significant drops in performance once the deployment team left a site. The core issue was that the operations teams were using old approaches to measure performance. Before ORION, drivers compared themselves against past performance on each route; if they performed better than previous months, then that was a success. With ORION, however, it became possible to compare performance against an ideal route and achieve significant new efficiencies by comparing performance against this best practice. This required adopting new ways to measure performance at each ORION site.

UPS also needed to develop short-term metrics to determine whether an ORION deployment had been successful at a particular site. Actual performance is difficult to measure over the course of just a few weeks, so UPS developed leading indicators—like the percentage of time a driver follows an ORION route—to enable success measurement during deployment. Through iteration, UPS selected short-term metrics tied to longer-term indicators of success, such as cost reductions and performance over time.

Prototypes Make the Opportunity Real

To demonstrate the technology’s potential, R&D staff took decision makers on what they called an ORION ride. Using a rental car, they took a test drive of the UPS route, guided by prototype ORION software. This demonstrated the potential of the ORION technology by allowing operations teams and drivers to witness how it worked and how it could improve on existing approaches. In particular, these rides helped UPS employees see how software algorithms, even if occasionally counterintuitive—like passing a house on an initial visit to the neighborhood only to revisit it to deliver a package later—could result in better efficiency overall.

More Sophisticated Software Systems Require Better Data

With software systems, the output is only as good as the data input. UPS found that available maps weren’t accurate enough to serve as a basis for ORION. As a result, part of any ORION deployment involves validating maps and other supporting data. UPS also conducts ongoing monitoring of data quality and has processes for addressing data inaccuracies. Any company transitioning to predictive algorithms should expect to invest significantly in the underlying data.

Deploy Slowly at the Start to Achieve Greater Scale

UPS started its deployment slowly at the beginning to ensure that the concepts were being trained effectively and that the deployment approach was successful. The company found several early ways to improve—for example, in how they measured the short-term success of the deployment and linked this to longer-term success—which they were able to incorporate. This ensured success at a critical time, when skepticism of the new approach was highest and before results had been demonstrated on a large scale.

Integrate With Existing Work Processes

To be successful, technology tools need to fit inside the processes that people already follow. This requires on-the-ground work, interviews with users, and lots of testing. Indeed, site managers and delivery drivers were integral parts of the ORION development team from the start, since they were able to provide feedback on what actually works in practice. 

Looking Forward

To achieve the business and societal transformations needed for long-term sustainability, new technologies and approaches must be integrated into how we do business. To help companies successfully implement technology-based sustainability solutions, the Center for Technology and Sustainability poses the following questions to help business leaders navigate successful implementation of technology solutions.

#Center for Technology and Sustainability, #Information and Communications Technology