Services

In the US alone, there are over 1100 documented legal decisions where a court or tribunal explicitly found or implied that a party relied on hallucinated content¹. For legal teams worried about the accuracy and reliability of AI tools in legal contexts, here’s how I can help.

AI Governance & Verification Audits

Legal teams, both in-house counsel and at law firms, are using AI tools in patent prosecution and litigation, but the teams don’t know the error rate of their tools. It’s not that the tools are broken — they are just unaudited. Nobody has run them against a known-answer test-set to find out how often they’re wrong, under what conditions, or on what kinds of tasks. For example, a patent drafting assistant writes a specification that cites prior art, but the prior art may fail to disclose what the assistant claims it does. Or a litigation support tool summarizes deposition transcripts, but hallucinated details may propagate into the briefing because there’s no process to spot-check the tool’s output.

A verification audit starts by building a set of known-answer scenarios drawn from the team’s actual cases. The team’s tooling runs on these known-answer scenarios, and I measure where it falls short — where the prior disclosure is different than the tooling thinks, or where hallucinated details propagated from a deposition summary to a briefing. The audit culminates with a written risk assessment and a verification protocol for spot-checking future work products before filing. Typically three to four weeks per tool.

Vendor Evaluation

Legal teams evaluating different tools for assistance in patent prosecution and litigation need independent assessments of their candidates. I work with the team to develop vendor-independent test scenarios based on the team’s actual cases. The candidate systems run the test scenarios, and I measure errors, failures, and hallucinations. The engagement concludes with a written report containing the evaluation results and a recommended candidate to purchase. Typically three to four weeks per tool.

Training & Competence Documentation

ABA Formal Opinion 512 puts a “reasonable understanding” obligation on lawyers using AI tools in their practices. I conduct workshops on AI literacy for legal teams to satisfy this requirement. Workshops are interactive and built around the team’s actual workflow, so participants walk away with practices they can use immediately rather than just sitting through another generic lecture. I also collaborate with teams to produce written policies and procedures to supplement malpractice defense and judicial requirements. Typically two to four weeks.

Custom Internal Verification Tools

Legal teams often have their own internal precedent bank, claim library, office action history, or prior art repository. These teams want to respond to office actions and compose other documents with citations limited to these internal resources, not a worldwide corpus of largely irrelevant literature. I engineer custom internal tools to provide this functionality by combining LLMs with retrieval augmented generation (RAG) on the internal corpus. Typically four to six weeks.

If you want more information about any of these services, or if you’d like to discuss a bespoke engagement, please reach out.