It might be overshadowed by the ramp-up to LegalTech, but a big project of the Sedona working group on e-discovery will be kicking off later in February, the 2009 Text Retrieval Conference (TREC) Legal Track.  For going on 18 years, TREC has been a workshop for encouraging research in information retrieval.  The three-year-old legal track is organized through the National Institute of Standards and Technology and co-coordinated by Jason Baron, director of litigation at the National Archives.

Participating teams work with a test case to evaluate the most effective search protocols for finding relevant documents in e-discovery, i.e. what finds the responsive documents best?  Concept search, expert human reviewers, Boolean keyword search using “x and y or z” or other methods?  I spoke with Jason Baron briefly about this year’s TREC, which will switch test collections from the tobacco litigation “Master Settlement Agreement” repository to the Enron collection of email and attachments.  He believes it will increase participation by reducing the need to search OCR’d documents.

Most of the participants in past years have not been vendors, but a few we know of are H5 and Clearwell Systems.  It’s certainly a worthy goal to find the most effective methods and work towards improving standards in how we approach sifting through legal documents for relevancy.  You’d think more vendors would be willing to put their tools to the test for the greater good, “walking the walk” if you will.  Instead their absence is a reminder that e-discovery is still a wild west scenario with no “standards sheriff” in town.

How likely is it that vendors will put their products to the test in an attempt to back up claims and find better methods in review and analysis of Electronically Stored Information for trial?  We hope that more of them will answer the call.  It’s a good time to prove your chops if you’re in e-discovery – anyone going to LegalTech next week will be able to attest to the dizzying number of vendors making the market ripe for consolidation, which we have already partly seen through the high volume of acquisitions in the last few years.  It remains to be seen how many of the smaller fish will still be left swimming at year end given the economic climate and fierce competition.  Proving their mettle and working toward the common good wouldn’t hurt any.

I will be at LegalTech from February 2-4 talking to vendors about their products and their own methods for finding responsive documents.  If you would like to get in touch or schedule a meeting, I can be reached here.