LLM Agents Get Real‑World Tests
Lead‑in: From bottling to bargaining, researchers rolled out a suite of benchmarks to see how agents fare outside the lab. Agent‑in‑a‑Bottle measured cheap, task‑specific solutions, while WorldSolver asked agents to generate physics solvers across 168 tasks. BazaarBench probed safety in decentralized marketplaces, and a new humility metric checked how agents handle contradictory evidence. Together they map a growing ecosystem of agent evaluation.
- Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?Wed 7 Oct · arXiv cs.AI
- WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?Wed 7 Oct · arXiv cs.AI
- BazaarBench: Delegation Safety in Decentralized C2C Marketplaces Run by LLM AgentsTue 6 Oct · arXiv cs.MA
- Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge ConflictFri 9 Oct · arXiv cs.AI