Harper Litigation Consulting and Research : 214-244-4608

What does the Manual For Complex Litigation say about Surveys?

The Manual For Complex Litigation provides the standards regarding sampling and opinion surveys

Sampling/Opinion Surveys

Statistical methods can often estimate, to specified levels of accuracy, the characteristics of a “population” or “universe” of events, transactions, attitudes, or opinions by observing those characteristics in a relatively small segment, or sample, of the population.

Acceptable sampling techniques, in lieu of discovery and presentation of voluminous data from the entire population, can save substantial time and expense, and in some cases provide the only practicable means to collect and present relevant data. In one case, for example, a statistical expert profiled the compensatory damage claims of the class members to assist the jury in fixing the amount of punitive damages.

The choice of appropriate sampling methods will depend on the objective. There is a difference between sampling to generate data about a population so the data will be verified or declared true and sampling, like polling, to measure opinions, attitudes, and actions by a population. In the case of the former, the reliability and validity of estimates about the population derived from sampling are critical.

The sampling methods used must conform to generally recognized statistical standards. Relevant factors include whether

  • the population was properly chosen and defined;
  • the sample chosen was representative of that population;
  • the data gathered were accurately reported; and
  • the data were analyzed in accordance with accepted statistical principles.

Laying the foundation for such evidence will ordinarily involve expert testimony and, along with disclosure of the underlying data and documentation, should be taken up by the court well in advance of trial. Even if the court finds deficiencies in the proponent’s showing, the court may receive the evidence subject to argument going to its weight and probative value.

By contrast, questioning a sample of individuals by opinion polls or surveys about such matters as their observations, actions, attitudes, beliefs, or motivations provides evidence of public perceptions. The four factors listed above are relevant to assessing the admissibility of a survey, but need to be applied in light of the particular purpose for which the survey is offered. In addition, in assessing the validity of a survey, the judge should take into account the following factors:

  • whether the questions asked were clear and not leading;
  • whether the survey was conducted by qualified persons following proper interview procedures; and
  • whether the process was conducted so as to ensure objectivity (e.g., determine if the survey was conducted in anticipation of litigation and by persons connected with the parties or counsel or by persons aware of its purpose in the litigation).

Parties who propose to offer sampling or survey evidence may want to consider whether to disclose details of the proposed sampling or survey methods to the opposing parties before the work is done (including the specific questions that will be asked, the introductory statements or instructions that will be given, and other controls to be used in the interrogation process). Objections can then be raised promptly and corrective measures taken before the survey is completed. A meeting of the parties’ experts can expedite the resolution of problems affecting admissibility.

Parties sometimes object that an opinion survey, although conducted according to generally accepted statistical methods, involves impermissible hearsay. When the purpose of a survey is to show what people believe—but not the truth of what they believe—the results are not hearsay. In the rare situation where an opinion survey involves inadmissible hearsay, Federal Rule of Evidence 703 nevertheless allows experts to express opinions based on the results of the survey.

Ms. Rhonda Harper, MBA

Retained by more than 125 law firms, Ms. Rhonda Harper is courtroom proven in virtually every circuit along with JAMS and TTAB.  Ms. Harper's 30 year career includes serving as a Fortune 100 chief marketing officer and an adjunct marketing professor. In addition to providing litigation consulting and research in the areas of business, licensing, marketing, advertising, and strategy, Ms. Harper is routinely retained to formulate expert surveys, conduct rebuttal critiques, or construct rebuttal surveys to show the potential difference in results with properly designed and executed surveys. She has extensive experience and a deep understanding of survey design, sampling, question construction, data analysis, and the methodological pitfalls that can introduce bias or systematic error.

Professional Memberships