“The real AI advantage for law firms isn’t models; it’s institutional memory,” said the Covington & Burling LLP’s Senior Director of AI. “And that memory is not something that can be licensed or outsourced – it has to be cultivated from within.”
For a small firm, that sounds like bad news, like one more person to hire or system to buy. But getting your knowledge and institutional memory in order is simpler than you may think. You only need one folder, a naming rule with the word “Approved” in it, and a partner willing to sign off. You also need the discipline to keep it up.
Why small firms skip this
Small firms don’t think about knowledge management because they haven’t had to. You’re small, share an office, and see each other often. Institutional knowledge moves around by walking to a colleague’s desk.
So your gold standard templates end up scattered, some in attorneys’ heads, others on personal drives. This was fine when a human did the file retrieval.
But this breaks when an AI does the retrieving. Machines are rigid. They need to be told which templates are approved and where to find them, or else they’ll pull an outdated draft and hand you something that is “polished” but wrong.
It’s worth doing this right because the stakes are high. A wrong precedent exposes your client to risk. It’s the computer science concept of “Garbage in, garbage out”, except now it comes back at machine speed and looks finished.
The good news is there is a simple fix.
Put the judgment in the document
Knowledge Management is about letting the firm reuse its own work product and know-how instead of rebuilding it every time.
The machines need to be explicitly told which documents and precedents to use in a given situation. And the most effective way to do that is to put the judgment about which document to use inside the documents themselves.
The simplest solution for this is one dedicated folder, one naming rule, and one spreadsheet.
Create one folder called Approved Precedents with subfolders by document type: PSA, NDA, Leases.
Then, a naming convention. Something like: Type_Subtype_Jurisdiction_Approved_YYYY-MM.docx. So a mutual NDA for a DC deal approved this past March becomes NDA_Mutual_DC_Approved_2026-03.docx. The word “Approved” is the signal for a usable document. Anything without it is not signed off.
The filename is doing real work here. It’s classifying the document by matter type, clause type, jurisdiction, and year. The filename is the metadata layer because it’s the one a lawyer will maintain without thinking much about it.
Lastly, maintain one index spreadsheet at the top of the folder. One row per template, with columns for filename, type, jurisdiction, last reviewed, and approving partner. This is the thing you point the AI at first, so it reads the index and picks the template instead of guessing from the pile.
Governance
The hard part isn’t the setup thus far. The hard part is the version control and trust over time. Laws change, templates go stale, and someone has to update the current one and archive the old one so the AI never sees it again.
Fortunately, there are only two roles you need assigned to maintain the whole system. First is a lawyer to approve documents. That sign-off is what “Approved” in the filename actually means, and it’s the only way a new document enters the folder. Second is a librarian or admin who keeps the folder and index clean.
It’s worth the discipline
The payoff is concrete: higher-quality work products and more consistency across the practice, and institutional knowledge that doesn’t walk out the door when someone leaves.
None of this is glamorous work, but it’s the work that makes AI deliver real value instead of generic output. And it compounds: every matter feeds back into a base the next one draws on. The messy-folder firm gets the generic outcome forever. The disciplined firm gets a little sharper with every deal it closes.