Google-led team builds StreamVibe, a video assistant meant to decide for itself when to offer help
The researchers' benchmark of about 4,100 examples shows that knowing when to speak up remains hard for today's models.
A research team led by Google, with researchers from Stanford, Oxford and several US universities, has introduced StreamVibe, a video agent meant to watch people as they cook, assemble furniture or use software and speak up only when needed. The aim is to handle vague requests such as "Please help me if I run into any difficulties." According to the paper, as summarized by The Decoder, existing video assistants are built for clear-cut commands like "Let me know when the water starts boiling" and rarely pick up on unspoken intentions. They also have trouble keeping track of what has already been done, so they repeat advice or respond too late. When users change their requests partway through, the assistants often stick to the original instructions. StreamVibe separates intentions that are stated outright, left unspoken, or changed during a task. To measure the problem, the team built a benchmark of about 4,100 examples drawn from more than 183 hours of recordings of everyday life and software use. The Decoder says the tests mainly show how difficult this kind of timing and judgment still is for current AI models. The work is relevant to always-on assistants in wearables and on screens, where help that comes too often or too late is a practical problem.