Customers experience a considerable amount of stress when they get an audit notification letter from Oracle, which is considered one the industry’s most aggressive when it comes to compliance. This discussion will cover what’s involved in an Oracle audit, and how you can make the best of it when Oracle comes knocking. Step one: As soon as you get a notification letter, locate your Oracle License and Software Agreement (OLSA), a.k.a. the contract, which you would have received with each Oracle product purchase. Check the audit clause in the contract(s), which spells out exactly what you’re legally responsible for, and when: “Upon 45 days written notice, Oracle may audit the use of your programs. You agree to cooperate with Oracle's audit and provide reasonable assistance and access to information. You agree to pay within 30 days of written notification any fees applicable to your use of the programs in excess of your license rights. If you do not pay, Oracle can end your technical support, licenses and/or this agreement. You agree that Oracle shall not be responsible for any of your costs incurred in cooperating with the audit.” Ideally, you already have an ongoing method to track your licensing information, relevant machine specs and virtualization solution data. If not, it’s time to do so because the clock is ticking. Inform all appropriate internal contacts about the audit and designate an individual to be the main point of contact with the Oracle Licensed Managed Service (LMS) consultant who will conduct the actual audit. Your internal group will include the systems admin, IT manager or specialist, and an Oracle Database Admin, if you have one. This team should prepare to conduct an inventory of all your Oracle specific product installs, in all environments, including outside data centers. They will want to gather machine specifications of all systems and devices with Oracle installed. For example, it’s important to know the exact number of physical processors, associated core count, threads, the virtualization solution, and which version, in all settings. Step two: Hold the kick-off call with Oracle, the LMS and your team. Be as cooperative and friendly as possible to ensure a mutually beneficial environment for the audit to progress. This advice cannot be overstated; it will pay dividends down the road. The kick-off is a meet and greet call to introduce the players, discuss the scope of the audit and provide a forum for questions and answers about the process. Oracle will provide an overview of what types of data they need and review the audit timeline. They will say the audit will span eight weeks, but realistically, you should expect it to take from three to seven months. The audit will experience delays for a variety of reasons, such as vacations, sick time, inadequate staffing for data collection on your side, or aligning data generated from the scripts with the customer inventory reports, verification of customer statements and report generation on the auditor’s side. Step three: This is the data collection part of the audit, which takes the most time. Oracle will provide an Oracle Server Worksheet (OSW) which you complete and return to the LMS consultant. You will provide all the deployment information you’ve gathered, as thoroughly as possible. This exchange is followed by a series of back-and-forths to clarify any confusion, correct mistakes and keep everyone on the same page. Next, Oracle will supply their proprietary scripts for you to run on your systems. (These scripts make some customers worry about security. As a former professional Oracle auditor, I can assure you that these scripts have been thoroughly tested and in no way gather any sensitive company data: they only gather Oracle-related information and machine specs are very secure.) The scripts are used to verify the information you provided in the OSW, to ensure everything matches. They take only 2-10 minutes to run. The LMS will follow up to resolve any discrepancies. Step four: Once your OSW data and the Oracle scripts are aligned, the LMS will present its compliance findings for your confirmation in a Deployment Report and a Reconciliation Report. This gives you the chance to review the findings before the formal audit report is drafted. The biggest area of findings in most audits is virtualization. The vast majority of Oracle customers use VMware as their server virtualization software, so I will focus this product. VMware usage is a big revenue generator in Oracle audits because many customers fail to understand the complexity of achieving Oracle compliance policy regarding virtual environments, which is found in Oracle’s Partitioning Document. Oracle consider VMware a “soft partitioning” method according to its official documents, and, “soft partitioning is not permitted as a means to determine or limit the number of software licenses required for any given server or cluster of servers.” In other words, soft partitioning equals noncompliance, and therefore, virtualization with VMware —depending on the version in use — can have a huge impact on an Oracle audit. We urge clients check if the virtualization policy is stipulated clearly in the OLSA contract; if not, this could provide you with leverage to mitigate some of the findings. Other examples of soft partitioning applications include Solaris 9 Resource Containers, Nutanix, HP Process Resource Manager, and AIX workload Manager. Even Oracle’s own virtualization product, OVM, is considered soft partitioning unless it’s configured as Oracle trusted partitions for Oracle engineered systems and includes Oracle Enterprise Manager! “Hard partitioning” methods are considered viable partitioning methods by the vendor, and examples include IBM LPAR and Solaris Zones in capped Solaris containers. Step five: It’s now time for the audit resolution and reconciliation process. Again, you must remain non-confrontational here to achieve the best outcome, so be positive and friendly even if you have to fake it. You can (and should) challenge the findings and double-check the math, but cooperation is your best weapon if you want Oracle to be forgiving and flexible. For example, you might inform the vendor that you need to delay payment beyond the standard 30 days; if the relationship has been adversarial this will not be acceptable. And if you’re very difficult, the sales rep may find ways to punish you down the road, by denying discounts or manufacturing past usage on some of the software.
Begin with the Virtualization Segmentation Option. This is offered by the LMS and allows you to reconfigure the virtualization environment to be compliant.
Make an Oracle product purchase to leverage an existing audit deficit if possible. For example, an Oracle cloud purchase can sometimes make up for the compliance discrepancies or shortfalls, as long as sales approves it with LMS, and the price of the cloud purchase will be in excess of the audit findings.
Create a Certification Letter of Non-use, in which you promise to discontinue future use of certain Oracle products. The LMS will determine whether or not to waive past (noncompliant) use in light of this letter.
And finally, in worst-case-scenarios, there are two risky tactics: some customers threaten to change suppliers or threaten to go to court and dispute the findings, fines and penalties, in hopes that the legal costs would dissuade Oracle from pursuing the matter. These tactics are very rarely viable or successful.
To review, it’s always best to keep a professional and positive attitude with your Oracle sales rep and LMS before, during and after an audit. And if you carefully gather and prepare the information I discussed earlier, remain responsive, and take advantage of the negotiation tactics that apply to your situation, the result will be a quicker, less painful audit with lower compliance costs. For more information on this subject, please view our webinar here, read our blog titled Top Findings in an Oracle Software Audit, or contact a member of our Compliance team.
Moe De Luca joined the ClearEdge Compliance Practice after six years as a senior license consultant at Oracle, where he led numerous customer audits.