OpenAI releases MentalHealthBench, an open set of 1,215 mental health conversations graded against clinician-written rubrics

OpenAI introduced MentalHealthBench on September 23. It is an open benchmark for evaluating how AI systems respond in mental health conversations. The release contains 1,215 synthetic conversations, ranging from everyday well-being topics to urgent emergencies. Each conversation is paired with rubric criteria. A cohort of more than 80 licensed psychologists and psychiatrists wrote the criteria, 5,262 in total, according to the accompanying research paper. The cohort spans 22 countries, 19 languages and nearly 20 mental health subspecialties. OpenAI said most earlier evaluations in this area focused mainly on emergency scenarios and measured success with broad, predefined criteria. That left a gap in understanding how models handle the full range of conversations people have with them. The company is releasing the benchmark openly so other researchers can examine its methods, run their own evaluations and extend the work. American Psychological Association CEO Arthur Evans, quoted in the announcement, said mental health exists on a continuum and that AI systems engaging people across that range need grounding in clinical expertise. The benchmark gives developers, clinicians and regulators a shared, inspectable yardstick. The results it produces will depend on how well the synthetic conversations reflect real use.