To Pilot or Not to Pilot – That is the Question


To Pilot or Not to Pilot – That is the Question
New technology is often piloted by companies before it’s fully rolled out. Then again, some companies choose not to pilot, and are successful anyway. We review some pros and cons of piloting, and some key factors to examine before jumping in to a technology pilot.

In a recent Berkeley Haas case study, Grameen America (GA), described its successful journey to implementing new technology for both its organization and for the people it serves. During that journey, they conducted not one but two preliminary pilot tests. This helped Grameen to build even more support and commitment to the new technology, and to ensure there were no “gotchas” waiting when the system went live. While many things must go right to successfully roll out new tech, the folks at Grameen highlighted their use of pilots as a crucial factor.

The idea of piloting a new technology is not new, but it begs the question, is it always a good idea to pilot new technology before you roll it out? There are real differences of opinion out there.

The advantages of piloting new tech before fully rolling it out include:

  • Lower cost. Organizations can “test-drive” the technology with a lower up-front investment. If the technology proves not to provide the benefits touted, then a potentially costly mistake can be avoided.

  • Learning and adjusting. New tech often has to interact with existing tech. An organization can learn a great deal about the technology’s features, response times, and ease of integration by piloting it in-house first.

  • Identifying process and organizational change implications. Smart organizations know that technology doesn’t stand alone. It’s embedded in processes and usually used by people. A well-designed pilot not only shakes out a lot of information about how the technology works, but also can highlight needed organizational changes.

Some cons of piloting new tech include:

  • Lost time. A pilot, pretty much by definition, slows things down. If your company is behind the curve of its industry or being disrupted by new competitors, then a pilot—rather than building momentum—may actually get in the way. The former CEO of GE, Jeffrey Immelt1, describes transforming that company into a “125-year-old startup.”[1] Part of his success in making that happen was making sure every employee knew that he–and they–needed to be “all in” on changes to the business. He shared that GE spends less time piloting new technology and more time communicating the business reasons for it and providing training on the tech.

  • No guarantees. A successful technology pilot does not guarantee its successful rollout to the larger organization for any number of reasons:

    • Sometimes the technology doesn’t scale as advertised, and the pilot with 10 users or 100 users comes to a screeching halt at 100,000 or a million users.

    • Never underestimate the power of organizational resistance. Dozens of companies get the technology right but the people aspects wrong. They don’t communicate why the tech is being introduced, or they don’t train their people adequately.

  • Mixed signals. For all the benefits, conducting a pilot can suggest that the organization is not yet committed to the technology. This can lead to an uncertainty about making the change. Some employees may see a pilot as a chance to block a business change they don’t want.

Look Before You Leap

Since there is no magic formula for determining whether conducting a technology pilot is a good idea or a bad idea for your organization; you have to assess the particulars of your organization first. A successful tech pilot needs to:

  • adequately test the new technology for its features, functions and how it will integrate with existing systems if necessary.

  • assess and address the realities of your organization’s culture. Is this expected change going to be embraced by a gung ho, “just do it” culture, or will it face resistance from a more traditional, steady-as-she-goes environment?

  • identify the best case and worst-case scenarios. Articulate what “success” is up front. If the worst case happens, can the organization handle and recover?

  • communicate to its stakeholders and users. Introducing new technology – like any business change – is almost like running a campaign. That communication needs to be two-way. Crucial feedback can come from unexpected sources.

  • provide results that are robust enough to clarify next steps for the organization. These steps can range from discarding the tech to scaling it up.

In the case of Grameen America, the company’s leadership team decided that “…while both pilots were time consuming, they were critical to ensuring GA designed the right technology, garnered organizational buy-in, and solidified the product ahead of a national roll-out.”

Do technology pilots make sense in your organization?


California Management Review

Berkeley-Haas's Premier Management Journal

Published at Berkeley Haas for more than sixty years, California Management Review seeks to share knowledge that challenges convention and shows a better way of doing business.

Learn more
Follow Us