Three questions about AI safety

Start with a system that follows a rule but misses the point. Then compare other failures and examine what researchers can learn by looking inside a language model.

What would help us tell whether an AI system is doing what we intend?

DeepMind’s examples show systems meeting a specified objective while missing the designer’s intent. Amodei and colleagues group several kinds of unintended behavior into research problems. Anthropic then examines evidence from inside a language model, including cases where its explanation doesn’t match its computations. These readings won’t settle whether a system is safe, but they help us be more precise about what we’ve observed.

Begin with the specification-gaming examples. For the paper, read the overview and introduction before choosing a section to spend more time on. In the circuit-tracing overview, pick one case study and read its limitations alongside the finding.

A question to carry between readings

For each source, write down the intended outcome, the evidence about actual behavior, and what remains uncertain. What additional observation would change your interpretation? Where does a technical description leave a question about people’s goals unanswered?

What question stayed with you?

Bring a question, an observation, or a disagreement from these readings to the A4 Discord.