Surviving Daubert: Bad Benchmarking by Expert Witnesses Puts Cases at Risk

Courtesy of West LegalEdcenter's Wall Street Lawyer - Securities in the Electronic Age.

To the challenges of managing complex litigation, add one more: Expert witnesses often make critical mistakes that put litigation at risk.

Specifically, experts often use the wrong benchmarks to calculate damages for lost profits, lost enterprise value, or shareholder damages, making assumptions that don't stand up to scrutiny, and causing their testimony to be excluded on Daubert challenge.1

In particular, these experts rely on benchmarks used for valuation analysis when estimating lost profits or enterprise value, and rely on benchmarks appropriate for lost enterprise value when estimating shareholder damages. Benchmarks should be based on the specific type of damages to be estimated. For lost profits and lost enterprise value, benchmarks should be based on what actually happened after the alleged event. For shareholder damages, benchmarks should be based on what could be foreseen as of the date of the alleged event.

The consequences are significant. Benchmarking is critical in the analysis of damages from lost profits, lost enterprise value and shareholder damages. Using inappropriate benchmarks or methodologies can lead to insupportable conclusions--which, in turn, lead to the exclusion of expert testimony, reduction in awards and the reversal of jury verdicts.

Critical Lesson: The Benchmarking Method Must Match the Type of Case

Among the critical lessons to be learned about benchmarking:

  • One benchmarking "size" does not fit all. Both litigators and expert witnesses often fail to understand which benchmarking methodology is the right one for a particular situation. They use the same approach across a range of scenarios. That leads to erroneous conclusions.
  • Benchmarking for estimating shareholder damages are not appropriate for estimating lost profits or lost enterprise value. Valuation experts and investment bankers use metrics that are designed to forecast future performance. They measure past performance, then look ahead in order to create a prediction, using data that might include the company's own forecasts. In estimating damages associated with events that lead to lost profits or business reputation (lost enterprise value), the task is different--the starting point is a given date in the past, the date when the alleged harmful event took place, and the requirement is to try to calculate two different scenarios. One scenario reflects what actually happened, and the other reflects what might have happened if the event had not occurred. There is more and different data available--including information about how the company, the industry and economy actually performed. None of that would apply in projecting a valuation for shareholder damages, but it is essential for calculating lost profits or lost enterprise value. Many experts have had their testimony excluded specifically because they used valuation type metrics in cases involving damages from harmful acts. Benchmarking for damages to shareholders resulting from accounting or financial fraud does, however, need to be based on forward-looking valuation methods. It is critical that experts--and litigators--know the different requirements and apply the right benchmarking methods for the particular type of case.
  • Inaccurate benchmarking creates risks for both the inclusion of expert testimony and the outcome of the case. Expert testimony is aggressively challenged. According to an annual study by PricewaterhouseCoopers, in 2010 more than 50% of accounting and financial expert witnesses were excluded in whole or in part on Daubert challenges.2 Bad benchmarking is often the basis for successful Daubert challenges. In a recent case involving Celebrity Cruises,five of seven experts testifying on behalf of Celebrity and one of three experts testifying on behalf of the defendants were excluded on Daubert , all because of their benchmarking assumptions. And the case as a whole shows that incorrect assumptions, such as using forward looking (or ex-ante ) valuation benchmarks for damages related to lost profits or lost enterprise value, can greatly weaken the case.
  • Benchmarks for lost profits and lost enterprise value need to be grounded in business and economic reality. To be defensible, benchmarks for lost profits or lost enterprise value need to conform as closely as possible to actual business and economic conditions and control for all factors other than the alleged act of harm. In practice that means applying "ex-post" benchmarks--measurements applied after the period under scrutiny that factor in such details as operating costs, fluctuations in business demand, and changes in the economy. Ex-ante benchmarks--the kind of forward-looking assumptions used by valuation experts and investment bankers--are not appropriate for estimating damages in lost profits and lost enterprise value. But again, benchmarks for damages to shareholders as a result of accounting fraud and other fraudulent actions do require the application of ex ante benchmarks.


The Risks of Benchmarking: Small Initial Errors Lead to Major Distortions

Why is benchmarking so inaccurate? The process of benchmarking involves making assumptions--about future business performance, economic conditions and the competitive landscape of the litigant's industry. As we all know from well-worn bromides, making assumptions is a dangerous business. The particular problem here is that any error in assumption will be ramified through years of projected business or financial performance. Even a small error in the assumption can lead to a wildly inaccurate conclusion. When using peer companies to benchmark a particular company, the expert is always subject to the criticism that the peer benchmark is not similar enough to the company in question.

In addition to the general problem of making assumptions, there are specific problems as well. As discussed, one is reliance on a "one size fits all" approach. Many experts rely on a single benchmarking methodology that they apply across a wide range of situations. The methodology may or may not fit. If it does not, a successful Daubert challenge is likely to result.

Case Study --Exclusion of Experts Who Used Valuation Techniques to Estimate Lost Profits and Lost Enterprise Value

The above-mentioned case, Celebrity Cruises, Inc. v. Essef Corp., involved allegations of lost profits and lost enterprise value after a defective water filtration system led to a 1994 outbreak of Legionnaires' Disease aboard a Celebrity cruise ship, the Horizon. The question of enterprise value was particularly critical since three years after the incident, Celebrity was acquired by Royal Caribbean Cruise Lines (RCCL, later Royal Caribbean Cruises Ltd. or RCL). In the course of the case, several different errors, all related to the use of valuation techniques, led to the exclusion of Celebrity's expert witnesses' testimony on Daubert. Among the errors the expert witnesses made were:

  • Projecting lost enterprise value using rosy forecasts instead of peer or industry performance as the basis. Pamela M. O'Neill, a principle of XRoads Solutions Group, projected an expected growth rate for Celebrity's revenue which in turn was based on a proxy consisting of two other cruise operators, RCCL and Carnival Corp. She took projected growth rates for each of the two companies as established by analysts in 1994, then applied them to Celebrity.

The problem is that the market proxy did not in fact display anything close to the projected growth rates. According to analysts, RCCL's revenue was expected to grow at rates ranging from 2.5% to 4.9% between 1994 and 2000; Carnival was expected to grow at rates ranging from 0.6% to 2.1%. As it happened, RCCL's growth in 1995, 1996 and 1997 was 0.21%, -1.08% and -1.25%, respectively. Carnival's growth rate during the same three years was -2.6%, -3.7% and -1.7%. Ms. O'Neill admitted in her deposition that she did not know these actual figures were available, and later acknowledged she would have considered this information had she known about it.

Forward-looking methodology such as this can be appropriate for valuing an enterprise at a single point in time. But it does not adequately measure damages that occur after the point when the projection was made. In this case, all three companies lost ground. But Ms. O'Neill attributes Celebrity's shortfall to the effects of Legionnaire's Disease, while making no attempt to determine why the other two companies declined.

Even worse, Ms. O'Neill's lost enterprise value calculation began with Celebrity's 1993 budget projections, which she then compared to 1997 actuals. As he excluded the evidence offered by Ms. O'Neill, Judge James C. Francis pointed out the main problem with using ex-ante benchmarks as opposed to ex-post benchmarks:

Indeed, to take 1993 as an example, Celebrity budgeted earnings before interest, taxes, depreciation, and amortization (EBITDA) of $61.9 million for the Horizon and its sister ships, the Zenith and the Meridian, but the actual EBITDA for those three vessels in that year was $55.4 million. Using Ms. O'Neill's methodology, this would indicate damages of over $83 million for that year, even though the Legionnaires' outbreak had not yet occurred.4

  • Taking the company's word for growth rate projections. Dr. David B. Lasater, a senior managing director of FTI Consulting Inc., did without industry proxies. Instead, he estimated projected profits based on Celebrity's own five-year plan as formulated by management in January 1994. He compared anticipated profits with actual profits and concluded that Celebrity had lost approximately $101 million He then adjusted his projections in various ways, calculating higher or lower lost-profit figures, but always using the company's own projections as the basis.

Judge Francis rejected this approach, stating:

Dr. Lasater's lost profits analysis is flawed in at least one major respect: the projection of profits based on Celebrity's five-year plan is wholly unreliable. [T]he entrepreneur's `cheerful prognostications' are not enough.5

Indeed, Robert P. Schweihs, another of Celebrity's experts, explicitly rejected use of the five-year plan to project anticipated profits after December 31, 1994.

To calculate the value of the company after the outbreak, Dr. Lasater began with the $1.312 billion purchase price paid by RCCL, then adjusted it for anticipated synergies. But he does not take into account that the purchase price--including the synergies--was negotiated between Celebrity and RCCL. Celebrity might have lost bargaining power because of the Legionnaire's outbreak, or there might have been balanced negotiations.

In either case, the synergy figure is the result of a negotiation--it is too subjective to be the basis for an enterprise value calculation. For his excessive reliance on the company's own projections, Judge Francis noted that "a methodology so sensitive to one highly subjective variable lacks the necessary reliability." Thus, Dr. Lasater's testimony was excluded.

  • Using more sophisticated methodologies that still rely on the company's growth projections. The lost-profits analysis by Allan Pfeiffer, managing director of Standard and Poor's Corporate Value Consulting, also relied on Celebrity's five-year plan. He used a more conservative approach than Dr. Lasater--for example, excluding out-of-pocket and "brand repair" costs--but still used Celebrity's projections as the basis for his calculations, projections that Judge Francis again rejected.


Judge Francis noted that:

"This analysis suffers from the same fatal flaw as Dr. Lasater's methodology: reliance on projections that were not borne out in reality. This defect drives the entire calculation and is not repaired by identifying a lower bound using a methodology which, standing alone, might be more reliable. Mr. Pfeiffer's lost profits analysis is therefore excluded."

This fatal flaw is the basis for the entire calculation, and using lower boundaries for the estimate does not offset the fundamental problem. Mr. Pfeiffer's lost-enterprise value analysis used six different sets of calculations to arrive at an average figure, but still relied either on Celebrity's five-year plan or a 1995 projection prepared for the company by The Blackstone Group LLC that was nearly identical. Mr. Pfeiffer developed a reasonable rate of return analysis, but did not justify it by comparing it to other companies. Mr. Pfeiffer's testimony was thus excluded.


When Damages Arise from Accounting Fraud or Other Fraudulent Actions, the Ex-Ante Valuation Method of Benchmarking Is the Right One to Use

But when harm is caused to shareholders as a result of accounting fraud or other fraudulent actions, the opposite approach is the right one to use. In such cases, the standard of value should be based on valuation methods traditionally used for valuing an entity on an ex-ante basis.

For example: In a case of fraudulent conveyance, an entity is sold and then goes bankrupt shortly after the sale. The new buyers claim the sellers should have known the company was insolvent at the time of the sale and thus was fraudulently conveyed.

A damages expert would determine what the value of the entity should have been at the time of sale, based on what was knowable at the time. In this case other events that occurred in the economy or industry after the acquisition that were not known or knowable would not be relevant to estimating the solvency of the company at the time of sale and thus should not be used as part of the solvency test, unless it is to prove the fact that these events would not have been knowable by the sellers at the time the entity was sold.

For financial fraud, damages should be based on what the company would have been worth but for the fraud. This analysis would involve determining how the stock price reacted at the time the fraud was announced--an approach known as event study benchmarking --or by using Wall Street valuation methods such as earnings multiples based on revenues or earnings corrected for the fraud and comparing them with the value based on misstated earnings or revenues. Or there might be a combination of these two methods.6

In both cases the benchmark would be ex-ante and not ex-post. That is, it would be based on what the impact was at the time of the fraud or when the fraud was revealed or occurred and not what occurred subsequent to the fraud.

Rx: Add a Financial Economist to the Litigation Team

How should litigators manage and supervise the benchmarking process? Expansion of the litigation team may be in order. While econometric and valuation experts are appropriate to cases involving financial fraud, in cases that require lost profits and lost enterprise value, a financial economist who understands the operating realities of a business is an essential partner in arriving at an accurate benchmark. In these instances, litigators need to make sure that they bring industry, institutional and operating experience to the table--either with the financial economist serving as an expert witness, or guiding the benchmarking work of other experts. The result will be expert witnesses and cases that have a much better chance of standing up in court.

End Notes

1. A Daubert challenge is an argument concerning the rules of evidence regarding the admissibility of expert witnesses' testimony during legal proceedings in federal court. The standard takes its name from Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993), one of three cases that articulate the Daubert standard.
2. Daubert Challenges to Financial Experts: An 11-year study of trends and outcomes , PricewaterhouseCoopers, February 2011.
3. Celebrity Cruises, Inc. v. Essef Corp., 434 F. Supp. 2d 169 9 (S.D.N.Y. 2006).
4. Id.
5. Schonfeld v. Hilliard, 218 F.3d 164, 173 (2d Cir. 2000) (quoting Dobbs Law of Remedies section 3.4).
6. May, Donald M., As Traditional Methods Fail in a Flood of Bad News, Courts Should Turn to Techniques Used by Investment Analysts to Calculate Shareholder Damages, Securities Litigation Report, Dec/Jan 2011 (vol. 8, no. 1). The article provides more detail on how to derive benchmarks for such cases.