The AI company Anthropic released a 244-page "system card" (PDF) this week describing its newest model, Claude Mythos. The model is "our most capable frontier model to date," the company says, and supposedly is so good that Anthropic has decided "not to make it generally available." (The company claims that Mythos is too good at finding unknown cybersecurity bugs, and so the model is only being released to select companies like Microsoft and Apple for now.)
Whatever the truth of this claim, the system card is a fascinating document. Anthropic is well-known as one of the more "AI might be conscious!" companies in the industry, and its new system card claims that as models become more powerful, "It becomes increasingly likely that they have some form of experience, interests, or welfare that matters intrinsically in the way that human experience and interests do."
The company isn't sure about this, it makes clear, but it says that "our concern is growing over time."
