🐍 Newsletters
Open-world evaluations for measuring frontier AI capabilities
Introducing CRUX, a new project for evaluating AI on long, messy tasks
Latest from AI Snake Oil
20 articles
🐍 Newsletters
New Paper: Towards a science of AI agent reliability
Quantifying the capability-reliability gap
🐍 Newsletters
AI Won’t Automatically Make Legal Services Cheaper
Applying the AI as Normal Technology framework to legal services
🐍 Newsletters
AGI is not a milestone
There is no capability threshold that will lead to sudden impacts
🐍 Newsletters
We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.
Technology Isn’t the Problem—or the Solution.
🐍 Newsletters
Does the UK’s liver transplant matching algorithm systematically exclude younger patients?
Seemingly minor technical decisions can have life-or-death effects
🐍 Newsletters
Can AI automate computational reproducibility?
A new benchmark to measure the impact of AI on improving science
🐍 Newsletters
AI companies are pivoting from creating gods to building products. Good.
Turning models into products runs into five challenges
🐍 Newsletters
AI existential risk probabilities are too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
🐍 Newsletters
Scientists should use AI as a tool, not an oracle
How AI hype leads to flawed research that fuels more hype
🐍 Newsletters
AI leaderboards are no longer useful. It's time to switch to Pareto curves.
What spending $2,000 can tell us about evaluating AI agents