Study of 72,000 judgments finds people often mistake AI writing for human, but direct questioning helps

A study published in Nature Human Behaviour finds that people often fail to tell machine-generated language from human writing in Turing-style tests, but asking questions directly works better. Mengmi Zhang of Nanyang Technological University in Singapore and colleagues collected more than 72,000 judgments of human and AI responses across three language tasks: image captions, word associations and conversational replies. In all three, participants often mistook the machine output for human work. In a smaller follow-up experiment, participants questioned the system directly instead of just reading its responses. That approach was better at exposing a chatbot, although Tech Xplore notes that it also comes with caveats. The authors' main point is that whether a model can pass as human depends heavily on how the test is run, not only on how convincing a single answer seems. The findings matter for anyone who relies on people to spot AI-generated content, including teachers, moderators and platform users. They suggest that passive reading is an unreliable way to detect it, and that a fair evaluation of human-likeness needs a clearly specified, interactive testing method.