Measure, then scale
An agent without an eval set is a performance. Define the task distribution and the cost of failure first.
About Eliot Leu
Eliot Leu — 刘俊, also called Vakkie. AI researcher, full-stack engineer, prompt engineer, PMP project manager, product manager, and CMA.
Six roles, one way of working: research sets the boundary, full-stack turns it into a system, prompts make model behavior predictable, PMP holds scope, product designs the consequence of every autonomy notch, and CMA gives every dollar an object. Collaboration usually starts from a task distribution, not from a model demo. Call me Eliot, 刘俊, or Vakkie.
Humans decide, models work
Scope, tradeoffs, release, and accountability stay human. Models draft, explore, and execute.
Prompts are interfaces
They version, test, and roll back. They do not live only inside a chat.
Every dollar has an object
AI cost must trace to a product, a program, or a customer. Otherwise it cannot be managed.