Part I - Forecasting
This paper will focus on risk in compliance where I will be joined by Ben Locwin, Director of Global R&D at BioGen and an operational strategist in pharma and healthcare, to explore risk forecasting, risk assessment and risk-based monitoring for the compliance profession.
The genesis of this paper began when I read a piece by Locwin in Contract Pharma’s Expert’s Opinion column, entitled “Pharma Life After Brexit”, where he posited that “forecasting has once again taken a hit for being less than accurate.” And this was before the US Presidential election. I was intrigued by his thoughts around forecasting and what (apparently) went wrong in the UK over Brexit. I immediately noted the implications for the compliance practitioner. This led to a series of interviews with Locwin leading to several podcasts and this paper.
At its heart, every business tries to plan for its future. It is a critical aspect of any management of any organization, non-profits, privately owned for profits and, of course, publicly traded companies. It is important that management be able to set out what it opines will happen in the next three, six, twelve and twenty-four months. Locwin said this “is really something that the businesses try to wrap their heads around in such a way that they can shunt resources where they think is appropriate in order to meet these future demands. Forecasting really at its heart is an educated guess and really as much as it becomes a reliable model more so and less so a guess, is based on the quality of the input data.” It is a process through which you are attempting to “prognosticate what the future will bring to you”. Unfortunately, forecast models are only as good as the data which are put into them or the GIGO (Garbage In, Garbage Out) Principal.
Locwin said that forecasting “should be broadly defined as a technique to estimate future aspects of any sort of business or operation.” He divided forecasting methods into two major categories; qualitative and quantitative. While both methods use past or historical data, in the quantitative method, “you would use time series analysis, for example, to see how certain trends appear in the data in the past.” Contrasting the qualitative method, which Locwin said is “a little more subjective, and you’re using less collective data which has been, let’s say, put into some sort of time series plot. It could be finances fluctuating over time or maybe it’s various incidences. I guess in the context of your work specifically, various instances of corruption that have been occurring. How would you document these over time? When were there spikes? Those spikes, related to what sorts of things?”
Under either approach whether you are using the qualitative or quantitative method for forecasting, Locwin noted “what you’re really trying to do is say that, “We expect that the trends that we’ve seen will be somewhat predictive of future behavior.” Otherwise, if you don’t consider that past behavior is in some ways indicative of future performance, you would not engage in any forecasting whatsoever.”
Forecasting typically will raise risks (and opportunities) which you might consider going forward. However, it does not assess or monitor these risks. Those are handled by risk assessments and risk monitoring. Locwin cautioned that simply because something is forecast does not mean will occur. He cited to Nobel winning physicist Niels Bohr for the following, “Prediction is difficult, especially about the future.” Locwin went on further to explain, “Whenever you’re trying to say how something will go, really the best you can do is try to look at past data and try to say what’s going to happen with that. In my prior probabilities, my prior knowledge tells me this, and therefore what will that mean for the final outcome?”
This last point led Locwin to write in his Brexit piece about the need for other tools. He wrote, “Similarly, people in the industry will tend to overly worry, or not be concerned enough, with the changes coming to the pharmaceutical landscape as a result of Brexit. But what we can all do as an industry in order to insulate ourselves from overly adverse outcomes is to be more agile and adaptable in how we respond to the changes that are coming. Standing immutable behind hardline policies can make the necessary operational changes difficult to absorb and lead to more variance and extended costs in the long run. This concept is known as anti-fragility, where the idea isn’t to become more impervious to change and market forces, but to be more adaptable to these changes. The ancient philosopher Heraclitus of Ephesus said, “Change is the only constant.” This is true in pharma as well. Closing our eyes, covering our ears and hoping the changes will pass us over are not viable strategies. However, expecting the change and being adaptable and resilient to its effects are strategies for success. Just ask Charles Darwin.”
So while the lesson on risk in compliance begins with forecasting you should consider the research by Guy Mayraz, who ran a series of experiments at Oxford University’s Experimental Social Science center. The first lesson is the bias towards predicting what people hope will happen. If you want your business to increase, you have to believe your transaction/investment/deal will always make money. After all, have you have ever seen a business plan that was designed to lose money?
The second lesson derived from Phillip Tetler’s the Good Judgment Project and almost sounds like someone channeled their inner Howard Sklar and his maxim of “Water is Wet”. It is that “self-critical, open-minded forecasters do a better job than narrow-minded overconfident ones.” He goes on to further note that dwelling on our own fallibility is not something people do very well; whether it involves hanging out with our friends or on cable news. The result is that “Confident, eye-catching forecasts are the snack food of analysis”. Unfortunately, this is even more true in the business world.
Finally, forecasters must always remember that more than one outcome is possible. A strong possibility may be a possibility but it is not a certainty. One way to overcome this bias is to develop alternative scenarios. Richard Lummis, host of the podcast 12 O’Clock High-a podcast on business leadership has called this the “devil’s advocate” role at the business planning table and that every scenario-planner should create at least two contradictory alternatives to their rosier, positive scenario. Super forecaster Tetlock has noted that “Superforecasting Requires “Counterfactualizing””.
The ultimate point is that in any forecast there must be preparedness for contra-events. Elizabeth Holmes, founder of Theranos, famously said that if you have a Plan B as a back-up, you have already lost. I find that to be worse than not helpful in any setting, particularly the business setting. No matter what your forecasting or scenario planning model shows, prepare for other results. For any Board of Directors overseeing a compliance program or managing any type of risk, it all begins by asking questions.
Part II - Risk Assessments
One cannot really say enough about the role of risk assessment in compliance. Each time you hear a regulator talk about compliance programs, it starts along the lines of you cannot manage your Foreign Corrupt Practices Act (FCPA) risk without first determining what that risk is and that process comes through a risk assessment. While I have written extensively on risk assessments in the past, I want to take different approach in this series as I continue to explore risk forecasting, risk assessment and risk-based monitoring for the compliance profession with Ben Locwin.
The differences between forecasting and risk assessment is that risk assessment attempts to consider things which forecasting either did not reliably predict for, or those things which the forecasting models have raised as potential outcomes which could be troubling, critical themes and issues. As Locwin explained, “What you’re trying to do then is decide on how you would address these. Risk assessments will percolate to the top of the list, your risk registry. Those items which are most consequential for your organization, whatever it happens to be. Again, just like forecasting, risk assessments apply to every organization.”
Within the context of an anti-corruption compliance program, you are trying to make adjustments based on the risks of violation of the law, out in the marketplace. For instance, in a compliance forecast, third-party risk should be considered at the top of your ordinal list of risk and you should consider a multitude of factors such as the operating procedures, processes and systems and training. Of course, the execution of that process is a critical component as well.
All these things, to some degree, should appear in a risk assessment for the organization. Meaning, at the corporate level, what happens if you change products or sell into a new geographic area which is perceived to be more high-risk? There should be a risk assessment node which has a component that notes these changes so that you can adapt as necessary. Locwin stated, “The risk assessment itself is designed to be able to elevate these, and if something does happen, the next step would be to take appropriate course of action to address any of those risks.”
We discussed an example which illustrates the differences between forecasting and a risk assessment, yet how the two are complimentary. This winter when I began purchasing hot coffee products from Starbuck, as opposed to the cold drinks I buy during the hotter parts of the year, I discovered that baristas’ no longer put sleeves on coffee cups but now require you to ask for one. The second time I had to ask for a sleeve, I inquired from the barista why I had to do so. She replied that corporate had changed the policy for environmental reasons and that she could only provide a sleeve at the specific request of the customer. When I pointed out that it slowed the line down and was much less efficient in the delivery of Starbuck’s coffee, she replied, “You're absolutely right. I hate it. Would you please email Starbucks and tell them of your dissatisfaction?”
I will let Locwin pick it up from here, “what you’ve put your finger on is the crux of the balance of forecasting versus risk assessment. They’re two very different things, but at the same time, as they weave through time, they interchange. For example, Starbucks would potentially say, “We forecast that consumers are going to be more concerned about paper use, sleeves, the economic costs to the world, of extra paper waste and things. We’re going to, in certain locations, let’s say across Texas, we’re going to pilot that we don’t give out sleeves unless they’re asked for.” In their risk assessment, which I can tell you didn’t change from that forecast, what they then should have had was a commensurate line item which said, “If consumers start to have a problem with what’s being done at these locations, our immediate contingency plan is to do the following, to strip it away immediately, full stop, so that every cup gets a sleeve, so that they’re not slowing down lines, consumers say you heard us immediately, and then the organization is back on track.”
Their forecast plans something, the risk assessment should have had countermeasures to address, and instead if they didn’t have this in place, they’re going to have to wait until they start to have a Twitter feed that blows up… The risk assessment model should say, “Then we will do the following.” Really they don’t have the capability in a lot of cases to measure the effect of this and immediately course correct. It’s probably going to be a month, two months, four months before they start to get wind of this in a consistent way to say, “Texas was dissatisfied by this change and same in our pilot in Wisconsin. Let’s stop not giving out sleeves… Then eventually that starts to dissipate and they get rid of this whole new silly paradigm.”
Locwin’s point was that your risk assessment can help to inform your response to FCPA violation, corporate crisis or even (in my opinion) the misstep of requiring Starbucks customers to ask for sleeves for their coffee purchases. In another article by Locwin, entitled “Quality Risk Assessment and Management Strategies for Biopharmaceutical Companies”, he noted, “knowledge is power”. He went on to add, “Once we have assessed risks and determined a process that includes options to resolve and manage those risks whenever appropriate, then we can decide the level of resources with which to prioritize them. There always will be latent risks: those that we understand are there but that we cannot chase forever. But we need to make sure we’ve classified them correctly. With a good understanding of each of these, we’re in a much better position to speak about the quality of our businesses.”
This is certainly true about a risk assessment in the anti-corruption compliance space.
Part III – Risk Based Monitoring
On the subject of risk-based monitoring Locwin said, “Risk-based monitoring is really about continuous, ongoing monitoring for those things which provide the most potential future risk to you. In other words, instead of a static risk registry that may come in part with forecasting, where you would say, “We’re trying to anticipate these risks.” By using risk-based monitoring to review issues on an ongoing basis, and the models that are behind the risk-based modeling, risk-based monitoring models, they’re continuously refined based on incoming data.”
The problem for many companies is they are siloed in not only their data but also in the systems. Locwin explained that because of the disparity of data systems, “They may not be tracking rigorous, quantified information all the time.” He cited to an example from the pharmaceutical world where a company could well have 50 worldwide sites where a drug product is being tested. Some patients receive a placebo and some patients receive the medication being tested. As data comes in you begin to note patterns in certain patients and groups, which might actually point towards a variety of testing errors by physicians administering the test.
Through the use of risk-based monitoring, you can begin to see things in “almost real-time, time-based trends of real data that you can then jump on and try to make adjustments before things get really wacky.” The implications to the compliance practitioner? Having access to information around sales, the sales process and corporate largess in things from Corporate Social Responsibility (CSR) work to gifts, travel and entertainment to conferences for customers and end users. Through the use of such risked-based monitoring a compliance professional would have the opportunity see trends developing which could allow an intervention for a prescriptive solution which could prevent an issue from becoming a Foreign Corrupt Practices Act (FCPA) violation.
Yet Locwin cautioned that compliance professionals should guard against bias. In an article by Locwin, entitled “Be Careful When Appraising Industry Trends”, he stated, “Social media has rapidly accelerated the agility with which the public can change allegiance and direction. It used to be that when information dissemination was slower and more compartmentalized within regions and market segments, that the market resistance to fluctuation was more robust. Now well-placed advertising, social commentary, or public response to corporate missteps can swirl into a maelstrom of market changes within hours that is agnostic to region or market segment.”
In today’s world, the speed at which reputational damage reigns out can overwhelm a corporation’s ability to respond. Here one might consider Wells Fargo and how fast the situation spun out of control for them after its $185MM fine was announced. It is through the use of risk-based monitoring, which allows for this almost real-time input, that a response to a forecasted, assessed or even unassessed risk can be developed. In the compliance world, such tools could be brought to bear when considering not only the expense side of such areas as gifts, travel and entertainment but also sales side data. This could be internal company data on its own salesforce and also information developed from or concerning your third-party sales team.
In Locwin’s primary world of pharmaceutical testing and product development, the need for such real-time information can be more critical. Yet through the development of these techniques as compliance tools, the compliance profession can add value to an organization through the use of risk-based monitoring. With the plethora of data on where and how corruption is likely to occur, coupled with meaningful sales and expense data, the compliance professional should be able to move from detect to prevent to prescriptive compliance solutions to prevent legal violations.
Finally, the beauty of all these techniques articulated by Locwin is that they are tools that can make companies more efficient and, at the end of the day, more profitable. They also move compliance into the fabric and DNA of an organization or in the words of Hui Chen, the Department of Justice (DOJ) Compliance Counsel, operationalize compliance. It will not be long before the DOJ mandates such techniques for any company under FCPA investigation. You need to develop your response now.
Part IV – White Noise and Interpreting Data
Next I will consider the issue of white noise in risk-based monitoring and how it can impact your data interpretation. White noise is generally defined as information which is not meaningful. The compliance practitioner often struggles if they can get the underlying data but they do not know what is meaningful. Just as often, the compliance professional will not know how to interpret it. Matt Kelly, writing in Radical Compliance, provided the following example, “you don’t want a metric about whistleblower allegations that only tells you how many complaints you have; you want a metric that categorizes them by nature of complaint, or division of the company that’s complaining. Likewise, a metric that monitors new vendors with incomplete due diligence should also track which business units are on-boarding these laggard third parties.”
Locwin called this the “signal to noise ratio” and said it is “the mean of the data or the standard deviation, that’s basically just telling you, often used in radio frequency, how much of what’s coming through the airwaves is the actual content you care about versus what’s the static, the background noise, and everything else. Every time we measure anything - it could be these 50 trial locations and what’s good or bad, it could be how many good meals did we serve at Chipotle versus contaminating meals that inoculated a consumer - what we always have when we’re measuring all this stuff is a lot of the [white] noise, which is measurement error.”
Locwin cited to an article he wrote, entitled “Better risk-based thinking will help produce better risk-based monitoring”, for this problem of white noise and data interpretation on the prognostication of future problems. In the article he pointed to information and data in the criminal justice system where a new technology is available “called the Beware system. The system is in use in Fresno, California and other police departments. It’s an electronic database which takes into account GPS coordinates, spatial distributions of localized criminal activity, as well as past track record of individuals involved in 9-1-1 calls. The system “searches, sorts and scores billions of commercial records in a matter of seconds-alerting responders to potentially deadly and dangerous situations while en route to or at the location of a call.” Based on the software’s calculation of the factors, it assigns a ‘threat rating’ and a red, yellow, or green indicator for the officer.”
“All of these data are critically important when appraising a police response situation. When using it as a guide, it’s important to understand the underlying risk assessment principles calculated by the Beware system: Not every address or location has an equivalent level of risk, and the system allows responding officers the opportunity to be prepared. Criminal recidivism refers to an individual’s preponderance to recommit crime after he or she has been involved in criminal acts in the past. So to say that past performance is an indicator of future behavior is an understatement.” For the compliance officer the issue is that “you need to know what to fix first; and this usually goes wrong in the form of companies being unable to differentiate the signal from the noise. To not do it properly leads to a lot of organizations that I’ve seen expending a tremendous amount of resource and capital on trying to fix what actually isn’t the problem.”
I would also note that Cathy O’Neil explored similar issues in the criminal justice system in her recent book Weapons of Math Destruction noting the discriminatory nature of the outcomes. Yet her critique emphasizes Locwin’s need to think about the context of the risk (data) you receive.
Locwin admonishes the compliance professional to separate the wheat from the chaff. Focus on what your company’s highest risk is, focus on what really matters to you. This means if you are using third party agent to sell your products and services, you should focus your data analysis and risk management on this area. Conversely, if you largely use your employee base as your sales channel that would be your highest compliance risk.
Part V – Tying it All Together
I want to tie forecasting, risk assessments and risk-based monitoring together. There are several key lessons to be considered by any Chief Compliance Officer (CCO) or compliance practitioner. The first is the process around risk management. Most compliance practitioners understand the need for a risk assessment as it is articulated as Hallmark No. 4 of the Ten Hallmarks of an Effective Compliance Program. From the FCPA Guidance, the Department of Justice (DOJ) and Securities and Exchange Commission (SEC) said, “Assessment of risk is fundamental to developing a strong compliance program, and is another factor DOJ and SEC evaluate when assessing a company’s compliance program.” In addition to this business case, the FCPA Guidance also specified the enforcement reasons for performing a risk assessment, “DOJ and SEC will give meaningful credit to a company that implements in good faith a comprehensive, risk-based compliance program, even if that program does not prevent an infraction in a low risk area because greater attention and resources had been devoted to a higher risk area.”
