By Chuck Kellner, SVP, Discovery Engineering
Do we come to LegalTech to bury search terms, not to praise them? Are they relegated to the dustbin of history? Are they now a hulk of an old station wagon rusting on blocks in the front yard?
Ok, enough. Search terms have been very good to us. In the decades that we used them while heeding the warnings of Blair and Maron, we put search terms to very good use at very minimal costs reliably and responsibly to reduce the expensive volumes we have had to review. Now we have Predictive Coding and other forms of analytics that may get us there cheaper, faster, and more reliably and more responsibly. Before we put the shovel to search terms, let’s take a look.
In US v. O’Keefe 537 F Supp 2d (DDC 2008), Judge Facciola sounded an industry warning that you can’t just trust the technology or the law degree in fooling around with search terms. And for judges to opine which search terms may work better than others is “truly to go where angels fear to tread”.
In Gross Constr. Assoc. v. Am. Mfrs. Mut. Ins 256 FRD 134 (2009), Judge Peck served “a wake-up call to the Bar…about the need for careful thought, quality control, testing and cooperation…in designing” a search protocol using keywords. “This case is just the latest example of lawyers designing keyword searches in the dark, by the seat of the pants, without adequate (indeed, here, apparently without any) discussion with those who wrote the emails.” He advised making sure you have input from the business clients, and, as above, heeding Blair and Maron and a long history in Information Sciences, you need to do quality assurance and testing.
In Kleen Products v. Packaging Corp.,1:10-cv-05711 (W.D Ill.) (d)uring an iterative process, Defendants and their consultants revised and refined the search terms over the course of several months.” The parties effectively met and conferred, and Defendants delivered a substantial production. But plaintiffs were concerned that Defendants weren’t finding everything that they should, and argued that Defendants should be using Predictive Coding instead of their careful and detailed iterative search protocol with quality assurance. Judge Nolan, however, agreed that Defendants did very reasonably enough for a compliant production, and she reserved judgment on whether the advanced technology in this case was warranted.
In Victor Stanley v. Creative Pipe (250 FRD 251 (D.Md. May 29, 2008) Judge Grimm pointed out “the well-known limitations and risks associated with [the use of keywords], and proper selection and implementation obviously involves technical, if not scientific knowledge.”
This LegalTech panel impressed the idea that to work with search terms in order to filter documents in discovery, the attorney needs knowledge, skill, and good technology to develop and document a responsible plan. Is it better to use Predictive Coding or other advanced technology? Still, the use of those requires knowledge and skill to develop a responsible plan for using that technology. The consensus of the panel is that search terms are still alive and kicking, but the warnings and wake-up calls need to be heeded. The watchwords for keywords are testing, sampling, revision and iteration, and a “test the rest” method of quality assurance. The behavior of litigation teams has to include documentation, cooperation, and reasonable transparency with opposing counsel or the investigating agency.
So no, keywords are not dead. But should they be? How the responsible use of keywords compares to the responsible use of technology assisted review is necessarily a question for another day (or panel or D4 Roundtable). In our experience, the use of keywords for filtering, even when done responsibly and expertly, still yields in the review a substantial number of non-responsive search hits. The thoughtful client and litigator need to evaluate the return on investment using Technology Assisted Review. They can compare the cost of using the technology to the savings in review. And understand that there is still a role for keyword search in the context of a protocol for Technology Assisted Review.
Thanks to former client and EDRM colleague Carolyn Southerland (and thanks for the callout, Carolyn!), and to Judge Peck, David Chaumette, Matt Baker, Rob Kidwell, and Gail Foster for a thoughtful, informative, and lively discussion.
Tags: collection, Computer Forensics, data, e-discovery, ediscovery, electronic discovery, LegalTech, LegalTech NY, LegalTech NY 2013, litigation technologies, predictive analytics, Predictive Coding, review tools, Technology Assisted Review, Test the Rest