Over the nine years since it first rose to prominence in eDiscovery, technology-assisted review has expanded to include numerous new tools, more potential workflows, and a variety of legal issues
In “Alphabet Soup: TAR, CAL, and Assisted Review,” we discussed TAR’s rise to prominence and the challenges that it has created for practitioners. In “Key Terms and Concepts,” we discussed the key terms and concepts practitioners need to know. In “Applications, Aptitudes, and Effectiveness,” we discussed use cases, relative merits, and overall efficacy. In “Are You Allowed to Use TAR?,” we discussed case law on authorization to use TAR. In “Can You Be Compelled to Use TAR?,” we discussed case law on being compelling or directed to use TAR. In “When Is the Right Time for TAR Process Objections?,” we discussed case law on objection timing. In “What TAR Process Results Are Objectionable?,” we discussed case law on how good is good enough. In “Is Process Transparency Required?,” we discussed case law on process transparency requirements. In this final Part, we review case law on using keywords with TAR, as well as key takeaways.
The final question for us to address, before we review our key takeaways from across the series, is whether you can use keywords and TAR together. Due to the enormous volume of materials collected in some cases, and due to the challenges associated with discovery from low-prevalence collections (even when using a TAR approach), some parties wish to use keyword searches first followed by a TAR approach applied to just the results of those searches. This is not an approach typically recommended by the developers of TAR tools, and some parties have objected to their opponents taking it.
Several cases have allowed parties to apply search term filtering to a document population before applying a TAR approach to the results, over the objections of the requesting parties (and, in one case, over the stated preference of the judge):
The issue before me today isn’t whether predictive coding is a better way of doing things than keyword searching prior to predictive coding. I must decide whether [the Defendants’] procedure satisfies [their] discovery obligations and, if so, whether [they] must also do what the [Plaintiffs] seek. [emphasis added]
After reviewing the work performed, the Judge concluded that the defendants’ process did, in fact, satisfy their discovery obligations (reasonableness). The Judge also considered the costs that would have been associated with the plaintiffs’ requested approach and concluded that a cost of millions of additional dollars would outweigh the potential benefit gained from finding some more of the remaining relevant documents (proportionality).
In this case, we are talking about millions of documents to be reviewed with costs likewise in the millions. There is no single, simple, correct solution possible under these circumstances. [emphasis added]
If the Responding Party determines it to be reasonable and appropriate, the Responding Party may use search terms and other criteria (the “Culling Criteria”) to reduce the volume of the Document Universe.
Later, in a letter to the Magistrate Judge dated April 6, 2015, the parties documented a number of discovery disputes, including the plaintiff’s complaint that the defendant “unilaterally decided to apply search terms to its Document Universe without first consulting with” the plaintiff, which led the Magistrate Judge to allow the plaintiff an opportunity “to suggest ‘reasonable’ additional search terms.” In a later order, the Magistrate Judge referenced the issue and commented that “The Court itself felt bound by the parties’ protocol, such as to allow keyword culling before running TAR, even though such pre-culling should not occur in a perfect world” [emphasis added].
In at least one case, the court declined to allow the use of keyword searches until after the TAR process was complete:
While the parties agree on many issues relating to discovery and particularly electronic discovery, they unfortunately cannot reach agreement over whether the universe of electronic material subject to TAR review should first be culled by the use of search terms. . . . They have left the matter for the Court to decide. The Court does so rather reluctantly. . . . Be that as it may, having reviewed the letters and proposed orders together with some technical in-house assistance including a read of The Sedona Conference TAR Case Law Primer, 18 Sedona Con. J. ___ (forthcoming 2017), the Court is satisfied that FCA has the better position. Applying TAR to the universe of electronic material before any keyword search reduces the universe of electronic material is the preferred method. The TAR results can then be culled by the use of search terms or other methods. [emphasis added]
Although courts’ preference for allowing producing parties to choose their methods likely means it will continue to be permitted, using keyword searches before applying a TAR approach may draw greater scrutiny to your process and its results.
Over the course of this series, we have taken a look at seventeen cases, plus numerous articles and other sources, and together, these materials have provided us with an effective overview of the technology-assisted review landscape. Here are the eight key takeaways from that overview:
TAR approaches encompass a family of similar-but-distinct workflows that rely upon similar-but-distinct mathematical approaches and software tools, which can be broadly grouped into two categories we have called TAR 1.0 and TAR 2.0. TAR 1.0 approaches are first-wave, predictive coding-style approaches. They take longer to set up but provide more statistical information and are often better for larger or more complex projects. TAR 2.0 approaches are second-wave, continuous active learning-style approaches. They are faster and easier to set up and, typically, cheaper to use, making them suitable for smaller, simpler projects than TAR 1.0 approaches.
Numerous studies have shown that, when used correctly, TAR approaches are at least as effective as older approaches. This is, in part, because TAR approaches are good and, in part, because keyword searches and human review are not as good as many practitioners assume. In reality, untested keyword searches are often fairly ineffective and even the best reviewers make numerous mistakes due to simple human fallibility or due to differing understandings of relevance and privilege.
As clearly shown by da Silva Moore, Global Aerospace, In Re: Actos, Bridgestone, Dynamo, Rio Tinto, and others, courts in the United States generally accept the use of TAR approaches as viable alternatives to older, manual methods. Additionally, we have seen on-the-record acceptance of its use in Ireland, England, and Australia.
Moreover, as emphasized in Dynamo and others, parties are not required to seek prior judicial approval for the use of TAR approach any more than they would be required to do so for an older approach.
Although you are free to use TAR yourself, you cannot compel another party to use TAR if they prefer another methodology. As emphasized in Kleen Products, Hyles, In Re: Viagra Prods., and In Re: Mercedes-Benz, courts adhere to Sedona Principle 6 and the idea that producing parties are in the best position to choose appropriate methods. The requesting party’s interest is in the results rather than the process employed.
While you cannot compel a party to use TAR, judges have directed parties to use TAR to expedite matters in cases like EORHB, Independent Living Center, and Winfield.
As emphasized in da Silva Moore, Dynamo, Hyles, and others, parties are not meant to raise adequacy objections to another party’s discovery processes until after at least some production has actually occurred. There is no mechanism for objecting to a hypothetical efficacy issue with a process that has yet to be carried out; objections must be rooted in identified issues with actual results (as also noted in Entrata).
Whether a party has satisfied their obligations can only be determined as a fact-based, post hoc analysis like the one conducted in Winfield.
There is an oft-repeated refrain in discovery that the expectation is reasonableness not perfection. This is drawn from the requirement in Federal Rule of Civil Procedure 26(g) that a “reasonable inquiry” be made to ensure the completeness of discovery responses, as well as from the case law addressing this issue (e.g., Dynamo). Thus, if a TAR approach in question was a reasonable inquiry, its results will generally be deemed acceptable even if imperfect. The reasonableness of what has been done is evaluated using a post-hoc, fact-based proportionality analysis (e.g., da Silva Moore, Hyles, and Winfield)
For a time, the pattern in early cases (e.g., da Silva Moore, Global Aerospace, In Re: Actos, and Bridgestone) made it seem like a high level of transparency might be a requirement for TAR use, despite the facts that: (a) transparency in those cases was often offered voluntarily to help overcome objections to what was then a novel approach and (b) traditional document review approaches were never that transparent (or expected to be).
In cases where parties have tried to compel such transparency (e.g., In Re: Biomet, Winfield, and Entrata), however, judges have declined to compel most of what’s been requested, citing relevance, privilege, and work product concerns. Although In Re: Biomet didn’t compel the transparency, it did include a warning that a lack of voluntary transparency might negatively affect the application of judicial discretion.
Several cases (e.g., In Re: Biomet, Bridgestone, and Rio Tinto) have allowed parties to apply search term filtering to a document population before applying a TAR approach to the results, over the objections of the requesting parties (and, in one case, over the stated preference of the judge). In at least one case (FCA), the court declined to allow the use of keyword searches until after the TAR process was complete. Although courts’ preference for allowing producing parties to choose their methods likely means it will continue to be permitted, using keyword searches before applying a TAR approach may draw greater scrutiny to your process and its results.
For Assistance or More Information
Xact Data Discovery (XDD) is a leading international provider of eDiscovery, data management and managed review services for law firms and corporations. XDD helps clients optimize their eDiscovery matters by orchestrating precision communication between people, processes, technology and data. XDD services include forensics, eDiscovery processing, Relativity hosting and managed review.