GenAI adoption is an exercise in labor arbitrage that closely mirrors the adoption of IT outsourcing ... [+]
It’s labor arbitrage all over again!
IT leaders spent the first decade of the 21st century perfecting IT outsourcing models in which U.S. labor hours were replaced by much lower cost labor in second and third world countries. It would appear that IT leaders will spend the next decade perfecting the use of LLMs that will replace U.S. labor hours with processing cycles on GPU chips.
Offshore outsourcing ultimately succeeded in becoming a dependable model for procuring IT talent and supporting IT operations. Offshore firms such as Cognizant and Infosys experienced exponential growth as this model was broadly adopted across multiple U.S. companies and industries during the early 2000s. Cognizant revenues increased 33X from 2000 to 2010. Infosys revenues grew by 24X during the same period.
The success of the outsourcing model was achieved through a painful trial and error process that sometimes terrorized U.S. employees, frustrated business leaders and undermined the credibility of multiple CIOs. It would be a shame to throw away the lessons learned during this previous exercise in labor arbitrage as today’s IT leaders attempt to achieve similar or greater cost efficiencies through the use of LLMs. Here’s a brief reminder of what we all learned.
Start small
There was great uncertainty about the type of tasks offshore IT workers could actually perform at the outset of the offshoring experiment. Consequently, initial engagements were typically small and very narrowly focused. Early successes were achieved in outsourcing Tier 1 service desk operations or routine application maintenance (i.e. bug fixes). As confidence grew, these engagements were cautiously expanded. Offshore service desk teams were asked to serve as backup NOCs (Network Operations Centers) during off duty hours in North America. Offshore application teams were gradually asked to implement enhancements or develop wholly new software modules.
Similar caution should be exercised in incorporating LLMs in routine operations. Tasks that are highly repetitive in nature, draw upon a fixed body of knowledge and follow a relatively predictable set of procedural rules are obvious initial candidates for LLM copiloting. As in the case of outsourcing, early successes will inevitably lead to a second generation of use cases that are more ambiguous in terms of outcomes, knowledge requirements and analysis strategies.
It's not a black box
Many offshore firms initially tried to limit the ability of their customers to gain insight into their internal operations. Their attitude was ‘just tell us what you want us to do, but not how to do it’. This sentiment was echoed by IT leaders in the U.S. who frequently had to remind their staffs that they were no longer responsible for managing the day-to-day activities of their outsourced team members.
However, when outsourcing outcomes failed to match customer expectations for cost, quality or timeliness, it became apparent that some degree of customer insight into an outsourcer’s staffing decisions, quality control procedures and progress measures was needed. This is a delicate balance. Customers wanted sufficient insight into how work was being done to remain reasonably confident about the reliability of expected outcomes without having to micromanage work being performed twelve time zones away.
Similar technical oversight is needed regarding the construction, testing and maintenance of LLMs to ensure that outcomes regarding their utility, reliability and cost effectiveness are realized in practice, with particular emphasis on the sources and nature of data employed in their development.
It always comes back to requirements
Some of the biggest initial outsourcing failures devolved into bitter debates about requirements. When outcomes failed to meet expectations, customers would charge their outsourcing partners with ignorance, incompetence and mismanagement. The outsourcing firm’s defense was inevitably based upon the written requirements they had been given. All too often, these requirements lacked the specificity required to produce the desired results. Furthermore, there were no easy contractual mechanisms for adjusting requirements on the fly during the lifetime of a project or engagement.
Imprecise LLM prompts are the direct equivalent of imprecise requirements. High integrity LLM responses (i.e. outcomes) can only be obtained with high integrity prompts. Outsourcing customers had to learn how to construct meaningful and actionable requirements for their partners. LLM users will need to learn how to construct meaningful and actionable prompts.
Establish business performance metrics sooner rather than later
One of the least memorable experiences of the early outsourcing era was the failure of IT leaders to establish performance metrics that could be used to gauge the success of their offshore experiments. They started with a single performance metric: labor cost reduction. But when outcomes failed to meet business expectations, they learned that business leaders had additional performance measures in mind regarding the relevance, quality and timeliness of outsourced services and deliverables.
IT leaders overcompensated for their initial failure to establish enterprise-wide business metrics for outsourcing success by developing exhaustively detailed Service Level Agreements (SLAs) that prescribed performance levels for every outsourcing outcome they could think of. Large enterprises with expansive outsourcing arrangements established dedicated teams to measure and report SLA performance and adjudicate alleged SLA violations. Over-engineered SLAs predictably became a new source of friction between customers and outsourcing partners that ultimately did more harm than good in realizing the strategic benefits of outsourcing.
Business leaders are likely to have divergent views about what constitutes GenAI success in the same way that their predecessors had divergent opinions about measuring outsourcing success. It’s far better to expose and resolve these divergence views before the proverbial ‘s—t hits the fan’, which it undoubtedly will. The first CEO who has to publicly apologize for offending a particular group of existing customers because of the way they were treated by a customer-facing GenAI application is not going to be happy, no matter how many labor hours were eliminated through the use of that application!
Teach it about your business
After learning how to clearly define what they wanted their outsource partner to do and establishing effective oversight procedures for monitoring how the partner did their work, IT leaders discovered that they achieved better results if the outsourcing partner understood why their services and products were important to the customer’s financial success. Offshore teams consistently produce better results if they have an in-depth understanding of the customer processes they are supporting and the importance of those processes to the financial success of their customers. Conversely, some of the biggest offshoring failures have resulted from an incomplete understanding of a specific process or a failure to comprehend the impact of their work on upstream or downstream business processes.
The implications for LLM customization are obvious. A deep contextual understanding of a company’s operational processes and training that includes historical company data is likely to produce superior LLM results compared with a generalized model trained exclusively on publicly available data.
You may love it, but will your customers?
It’s easy to get swept up in the initial enthusiasm that surrounds almost any new idea, whether it’s offshore outsourcing or LLM adoption. But the most dangerous interface to navigate in trying something new is the willingness of customers to embrace a different way of doing business.
To put it gently, let’s just say that many company employees were less than enthusiastic about their initial dealings with outsourced service desks. Employees found it frustrating to explain their problems to non-native English speakers. They were frequently underwhelmed by the technical competence of the desk agents (e.g. ‘have you tried to reboot your machine to see if that solves the problem?’). And in many instances they felt that the agent did nothing more than transcribe their problem onto a trouble ticket that would be forwarded to someone else for resolution (i.e. there had been no meaningful progress or attempt to actually solve their problem).
Unhappy company employees will likely remain on the job until initial adoption problems can be overcome. Paying customers are another matter entirely. Their tolerance for change may be even lower and they’re far less likely to persevere until kinks in the new way of doing business can be ironed out. Companies exposing their paying customers to LLM-supported applications or bots should do some extensive A/B testing on narrowly focused user groups before exposing such new capabilities to the entire spectrum of paying customers.
Who’s got your back? – probably no one
We’re all deeply immersed in a collective cultural honeymoon with LLM technology. Venture capitalists are throwing money at GenAI startup companies. Leaders of such firms such as OpenAI’s Sam Altman make front page news with their futuristic prognostications. CEOs and COOs are instructing their CIOs to ‘give it a try’, ‘go fast’, ‘don’t be afraid to make mistakes’ and ‘make sure our competitors don’t get a jump on us’.
But what happens when things go wrong? What happens when one of your newly developed chatbots has an expensive hallucination? What happens if an LLM inadvertently displays discriminatory behavior towards a particular cross section of your paying customers? What happens if you’re slapped with a copyright infringement suit that will cost far more to fight than any incremental revenue generated by the LLM that caused the problem? Whose got your back then? Is your CEO ready to say ‘it’s not my CIO’s fault – I made her do it’?
Attention all CIOs: LLMs may not cost you your job, but they might turn you into the personal butt of some company-wide jokes and also undermine the credibility of the entire IT function. Proceed with caution.
Best piece of advice
Those who ignore history are forced to repeat it. Early adopters of GenAI technology can materially de-risk their initial initiatives by enlisting the aid of individuals who led the IT outsourcing crusade 15+ years ago. There are still a few of us around!
