Questions of Choice in Kleen Products, Technology-Assisted Review Series Part 3

3 / 14

Seven years after it first rose to prominence in eDiscovery, technology-assisted review remains an important, and at times controversial, tool in the eDiscovery practitioner’s toolkit

In “Still Crazy after All These Years,” we discussed the slow but steady growth in the importance of TAR.  In “In the Beginning Was da Silva Moore,” we discussed the first case to address TAR.  In this Part, we review the next prominent case to arise: Kleen Products.

The next major TAR case after da Silva Moore was Kleen Products LLC, et al., v. Packaging Corporation of America, et al. (N.D. Ill. Aug. 21, 2012).  The Kleen Products case was a consolidated action over alleged anticompetitive behavior in the corrugated cardboard industry.  Discovery in the case started at the beginning of 2011, but at the end of the year, several discovery issues remained unresolved, including the search methodology to be used.

The Defendant’s Proposal

The Defendants wanted to use a traditional keyword and key phrase search methodology, and they had already engaged outside consultants and begun work on that approach.  Their work in 2011 included iteratively testing and refining search terms to be used and doing sampling to measure their results.  The Defendants contended that their search process would be tested and validated to ensure accuracy as good as, or better than, any other methodology.

The Plaintiffs’ Objections

The Plaintiffs opposed this plan and wanted the Defendants to utilize a “content-based advanced analytics” methodology instead, a.k.a. predictive coding, a.k.a. TAR.  The Plaintiffs criticized traditional keyword search methodologies as inherently inadequate and flawed and argued that a TAR methodology would likely find a much higher percentage of the relevant material than the Defendants’ searches would at no greater cost.  They also criticized some specifics of the proposed sampling and validation methodologies.

The Judge’s Analysis

Extensive hearings were held on the proposed search tools and methodologies, and during them, the Magistrate Judge emphasized her support for Sedona Principle 6 and the idea that responding parties are best situated to pick their own discovery processes:

I am a believer of principle 6 of Sedona, and I’m not just because it’s Sedona, but I think the people who are producing the records, producing the documents, are in a better position to know, since they have to do the work, spend the money, spend the time, they know their people, they know their material, so as a basic premise, I think that’s a pretty fair premise here.  [emphasis added]

Consistent with this approach, she also emphasized the importance of the results and their validation over the tool or process used to achieve them:  “I assume . . . what you really are interested in is a search, regardless if it’s Boolean or computer-assisted, that is fair and statistically – and that can be validated statistically because that would be a good word search” [emphasis added].  She goes so far as to say that the whole TAR debate was “almost . . . a detour.”

These two ideas – that responding parties choose methods and requesting parties judge results – are important ones that will come up again and again in later TAR cases.

The Ultimate Resolution

Based on those two ideas, and in light of the extensive work already completed by the Defendants, the Magistrate Judge urged the parties to try again to consider whether the Defendants’ search methodology might be tweaked in some way that would satisfy the Plaintiffs.  After an additional five months of meetings and negotiation, the parties reached an agreement regarding search methodology for the first phase of discovery that dropped the demand for TAR.

A Note on the Hearing Transcripts

The parties reaching agreement means there isn’t an explicit approval order for TAR here like the one in da Silva Moore, but it is still significant for its analysis of the method selection question and for its very educational hearing transcripts, which include extensive discussion of, and testimony about, predictive coding, Boolean searching, and validation methodologies:

Upcoming in this Series

In the next Part of this series, we will review the Global Aerospace case, which provided us with our first instance of on-the-record results from a completed technology-assisted review process.

About the Author

Matthew Verga

Director of Education

Matthew Verga is an electronic discovery expert proficient at leveraging his legal experience as an attorney, his technical knowledge as a practitioner, and his skills as a communicator to make complex eDiscovery topics accessible to diverse audiences. A fourteen-year industry veteran, Matthew has worked across every phase of the EDRM and at every level from the project trenches to enterprise program design. He leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.

Whether you prefer email, text or carrier pigeons, we’re always available.

Discovery starts with listening.

(877) 545-XACT / or / Email Us