Source item

Quoted context

These are the quoted author’s words.

Today we're sharing our first research collaboration with @baseten on open-weight legal agents. Using signal from LAB (our Legal Agent Benchmark of 1,200+ tasks across 24 practice areas), we post-trained an open-weight model to match closed-source frontier performance. Training open-weight agents for legal creates three major advantages for Harvey: 1) Cost and latency improvements: The best-performing closed-source frontier models took an average of 22 minutes to complete each LAB task, with $50 per-task average inference cost. Open-weight inference is substantially faster and cheaper. 2) Reasoning visibility: In a high-stakes domain like legal, it's incredibly important to get visibility into agents' internal reasoning states - for audit and governance and also as a lever for Harvey to improve agent performance. Closed-source foundation model providers avoid exposing raw reasoning tokens via API to prevent model distillation. For Harvey, that visibility is a major advantage. 3) Custom training: Owning the model weights lets us customize training and modify architecture. One example: our blog’s final note on training a custom KV cache compactor. More to come on our research collaboration with Baseten.

winstonweinberg ↗