System Integrator Risk Mitigation Processes Often Have the Opposite Effect

I am going to go on a bit of a rant here.  I am getting fed up with all of the IT project horror stories I have been reading and writing about.  How can it be that projects continue to go wildly over budget or result in significant business disruptions but profits by systems integrators continue to rise?  How is it that with fixed-price contracts, modern implementation tools, and supposed sophisticated project management capabilities, clients are still getting burned?  After giving this some serious thought, I have concluded that the risk management processes proposed and implemented by most of the big SI’s are seriously flawed and likely increase (not minimize) the client’s risk profile.

5 Reasons System Integrator’s Risk Management Processes are Flawed

Every RFP I have ever seen for SI implementation services asks a question regarding the vendor’s risk management practices.  Almost every vendor responds in exactly the same fashion, providing a method to collect risks, evaluate them using a risk matrix, and then plan mitigation strategies based upon the placement on this matrix.

As I took a step back and looked at these processes in the context of vendor selection and contracting, it became apparent that there are some major flaws.  Further, during the execution of these processes, the flaws are enhanced by the pressures created during the project.  These fail points, in all likelihood, increase rather than decrease the potential for project failures.

Here are five proof points of this hypothesis:

1. Risks are not considered upfront as part of the contracting and RFP process. For companies that have not undertaken a major transformational effort in their recent history, there is a tendency to underestimate the complexity and risks involved.  As a result, vendors who come in with well-designed processes specifically to mitigate these risks are considered “heavy” in their proposals.  Vendors understand these biases and as a result, will counter with proposals that minimize the possibility of these risks.  This generates an inventory of bids for a project that are artificially low, instantly creating massive risk for the project.
2. Fixed-price contracts create biases to focus on cost and only schedule risk. There is a tendency of many organizations to mistakenly assume that fixed price contracts mitigate all risks to the client.  On the contrary, these types of contracts can have an impact of amplifying the risks associated with operational continuity and the ability to obtain benefits.  With a focus on cost and schedule, the SI may be inclined to ignore operating risks or even worse, take actions to achieve the budget and stay on schedule which results in an increased probability of operating failure.  A simple example would be the decision to combine the last test cycle of integration testing with the UAT process.  There is a reason that these tests are typically not planned in parallel and it is not so they can be combined in the future to make schedule.
3. Failure to mitigate risks is money in the SI’s pocket. While the vendors will never admit to this, all SI’s understand that failure on behalf of the client to adequately address their own execution risks will likely lead to profitable change orders.  Want proof?  Check the RACI’s on the vendor’s RFP response and see how complete a job is done in the identification of actions required to mitigate client risks.  Leaving the mitigation activities out of the RACI means the activities will not take high priority to get planned.  This will result in late discovery of risks and then a change order will be required to support a client-induced delay.  Cha-ching.
4. Vendor’s capacity to assess and mitigate risks suppressed by budget. To properly execute a well-managed risk management process takes time and resources.  When project timelines get tight and budgets become squeezed, the risk management process is viewed as simply overhead with capacity that can be repurposed to project delivery.  It is like not paying for insurance because you can’t afford it.  Those that need it the most don’t have it and, therefore, disaster has a high probability of following.
5. Vendor’s recommended approach can produce the wrong results. Almost all vendors propose to use the standard risk matrix as an integral part of the risk management process.  These methods and matrixes are inherently flawed from three aspects:

• Risks that are identified with high potential consequences and low probabilities are typically given a lower priority to mitigate, thereby raising the possibilities of catastrophic failures.
• The risk management processes focus on the execution of the plan itself vs. the attainment of the expected outcome.  This tends to leave significant risks unidentified and ultimately untreated.
• The methods do not readily account for risks or events that can occur more than once nor take into consideration the correlation of various risk factors.

Consider this example:  Suppose there is a 5% chance that a company may not be able to report earnings on a timely basis due to the lack of training received and the potential for numerous transaction processing errors.  With the use of the vendor’s risk management process, there is a low possibility that this risk will make it onto the risk matrix and if it does, it will be placed in the lower right-hand corner due to the low probability, so the risk will be accepted. What CFO would readily accept a 5% chance of not closing the books on time?

Methods to Hold SI’s to Greater Account

So what can be done to hold vendors more to account?   Or in other words, what mitigating actions can be taken to address the risks of a flawed risk management process?  Below are a few that I came up with:

• Start with an inventory of typical and common transformation risks. I can’t stress this one enough.  Do your homework and start by identifying the known unknowns.  There are any numbers of places to obtain risk inventories for major transformations.  Use these risks to score the quality of your SI’s proposals.  Click here for our risk heat map.
• Ask the vendors, “What risks were considered and what contingencies planned?” As a part of the RFP process, ask the vendor to identify what they perceive to be the top risks for your program.  This will allow you to update your initial inventory of risks and it will also put the SI on notice that this is an evaluation criterion.  As a result, you will likely see an increased quality of the initial estimate.
• Implement a risk management process adherence metric. There are any number of process adherence and deliverable metrics that are tracked to validate that SI’s are delivering the value contract.  Rarely do I see a risk management process adherence metrics as part of a PMO metrics package.  Simply by requiring metrics on process adherence, the client will benefit from the Hawthorn effect (simply by measuring processes will improve).

• Balance risks across 3 effect domains (cost, quality, and benefits). Almost without fail, risk management processes get focused on budget and schedule.  Oftentimes, the mitigations that are put in place associated with these risks will result in increased risk levels for operational continuity and business case attainment.  As a practice, I recommend that every mitigating action proposed for budget and schedule be evaluated as a risk to quality and benefits.
• Quantitative modeling. The use of Monte Carlo simulation and other quantitative methods don’t have to be costly.  The ability to efficiently evaluate multiple risk mitigation scenarios or access vendor proposals will likely pay dividends.  Quantitative methods and models have become more popular and understood as a result of common use in the financial planning industry.
• Independent risk assessments. Finally, I recommend the use of independent risk assessments and evaluations.  Getting a view from the balcony associated with a major transformation project can be eye-opening.  Most system integrators are not keen on outsiders examining their work; however, external reviews can be used to identify potential blind spots as well as incent the SI to execute their own processes as designed.

Project failures can occur for many, many reasons, but protecting your organization against risks that may seem obvious is often not thoroughly covered in your contracts.  Relying on your SI to tell you how to mitigate risks is a flawed strategy that often does not have a happy ending.

Comment below, follow me on Twitter @jmbelden98 find my other UpperEdge blogs and follow UpperEdge on Twitter and LinkedIn.

Related Posts

• 5 Mistakes to Avoid When Developing a SOW With Accenture, Deloitte, and IBM
• How to Amplify the Value of Your System Integrator – Wish You Had a Manual?
• Dirty Little Secrets of your System Integrator: Why Companies Still Go Over Budget
• When is it Time to Say Goodbye to Accenture, Deloitte, IBM or KPMG?
• Is Your SAP SI Partner Evaluation Missing the Mark?