Kaplan indicated that he remains optimistic about aligning AI systems with human values up to the point where they match human intelligence, but expressed concern about what may follow once they surpass that threshold.
digital
S
Storyboard03/12/2025

Anthropic co-founder Jared Kaplan warns AI may design its own successor by 2030, risking control.

  • Anthropic co-founder Jared Kaplan warns AI could design its own successor by 2030.
  • Kaplan believes AI self-improvement could start between 2027 and 2030, posing profound risks.
  • Artificial general intelligence (AGI) remains poorly defined, with labs racing to achieve AGI-level models.
  • Kaplan is optimistic about aligning AI with human values until it surpasses human intelligence.
  • Two major risks: loss of human agency and rapid AI self-improvement potentially outpacing human oversight.

Why it matters: AI self-design could outpace human control, risking unpredictable and irreversible consequences.

Related Articles

Loading more articles...