Source item

2/ Step one sounds simple: log expert actions. Every redline, override, or escalation is a datapoint. In theory, you can clone a policy from history. In practice, logs don’t capture why. A clause might be struck for legal risk, client preference, or sheer habit. 3/ Next, train a reward model from feedback. Collect comparisons: “this draft is safer than that one.” Works great in RLHF papers. In high-stakes domains, expert time is scarce and their judgments often conflict. Sparse, costly, and messy signal. 4/ Even when you have outcomes, credit assignment seems kinda brutal? A case outcome depends on years of motions; a patient’s recovery on dozens of interventions. Which action “earned the reward”? RL thrives on short horizons, not institutional timescales. 5/ unlike my fave games, most of human workflows don’t come with clean scores. Explicit, verifiable rewards — a motion granted, a readmission avoided, a customer renewed — are a tiny fraction of what matters. The rest is tacit judgment with no +1/-1 attached. 6/ Hard constraints make it worse. You can’t “explore” by breaching privilege, prescribing contraindicated drugs, or refunding against policy. These aren’t soft penalties, they’re bright lines that aren't easily explored. I think RL doesn’t like absolute walls? 7/ Distribution shift bites in the real world. Judges rotate, formularies update, brand policies change. A policy trained on last year’s data can collapse tomorrow. Real products need to be configurable, not frozen policies. 8/ And even if you got a working policy for a hot minute, institutions need to know why. To defend it in court, satisfy regulators, or keep customer trust. RL doesn’t naturally produce explanations, provenance or rationales.