TwelveLabs launches Pegasus 1.6 to turn first-person video into robot training data
In short
Korean video-understanding AI company TwelveLabs said on Oct 7 it released Pegasus 1.6, a vision-language model that structures egocentric — first-person — video into data robots can learn from. The company says the model automates action segmentation, captioning, quality assessment, search and privacy detection, work that has largely been done by hand. A US robotics training-data company is already using it on factory footage, TwelveLabs said.
Read the full story 2 min read
TwelveLabs, a Korean video-understanding AI company, said on Oct 7 that it released Pegasus 1.6, a vision-language model whose main advance is understanding egocentric — first-person — video so that it can be turned into training data for robots and humanoids. The company announced the release in a statement carried by ZDNet Korea, Maeil Business and Yonhap. [ 1 , 2 , 3 ]
Egocentric footage is recorded by workers wearing action cameras or smart glasses while carrying out a task. TwelveLabs says turning that footage into robot training material has required people to pick usable scenes, split human actions into segments and identify how hands interact with tools and objects and what the result was. ZDNet Korea reported that annotating the public Ego-Exo4D dataset — 1,286 hours of video — took more than 200,000 hours of accumulated work, or roughly 155 hours of manual effort for each hour of raw video. [ 1 , 2 , 3 ]
The new model analyses actions, surrounding objects and context together to divide a task into segments and describe them. TwelveLabs gives the example of a worker picking up a part, working on it with a tool and moving the finished object, and says the model can structure each action, the objects involved and the order of actions. It supports five functions: action segmentation and labelling, detailed caption labelling, quality assessment, search and curation, and privacy and compliance detection. A time-based metadata feature lets users define the items and structure they want and organise video content with time information, and the company says object recognition for hands, objects and tools has been improved. [ 1 , 2 , 3 ]
TwelveLabs said a US robotics training-data company is already applying Pegasus to factory footage, evaluating quality, filtering out bad data and adding detailed descriptions to the footage it keeps. According to TwelveLabs, that company refines hundreds of thousands of hours of video a day into a form usable for robot training. Neither TwelveLabs nor the reports name the customer. [ 1 , 3 ]
CEO Lee Jae-sung said people's real-world experience, such as correcting an object that slips, has mostly not been recorded in a form machines can learn from. “Pegasus 1.6 will turn video into structured knowledge so that physical AI and robotics companies can use real human experience for learning,” he said. In Maeil Business he added that TwelveLabs has consistently aimed at helping machines understand how the real world works through video, and that physical AI is a natural next step in that direction. [ 1 , 2 , 3 ]
Why it matters
Robotics and physical AI developers have struggled to turn raw footage of human work into labelled training data; automating that step could lower the cost of teaching robots everyday tasks. The release also positions a Korean video-AI firm in the physical AI supply chain, and the company cites an unnamed US customer as early evidence of adoption.
Key facts
- TwelveLabs said it released the vision-language model Pegasus 1.6 on Oct 7, 2026, with strengthened ability to understand egocentric first-person video. [ 1 , 2 , 3 ]
- Egocentric video is recorded by workers wearing action cameras or smart glasses while performing tasks. [ 1 , 2 , 3 ]
- Preparing such video for robot training requires selecting usable scenes, segmenting human actions and identifying interactions with tools and objects. [ 1 , 2 , 3 ]
- Pegasus 1.6 supports five functions: action segmentation and labelling, detailed caption labelling, quality assessment, search and curation, and privacy and compliance detection. [ 1 , 2 ]
- According to TwelveLabs, a US robotics training-data company uses Pegasus to assess factory footage quality, filter out bad data and add detailed descriptions. [ 1 , 3 ]
- TwelveLabs CEO Lee Jae-sung said Pegasus 1.6 converts video into structured knowledge so physical AI and robotics companies can learn from real human experience. [ 1 , 2 , 3 ]
- ZDNet Korea reported that annotating the public Ego-Exo4D dataset, 1,286 hours of video, took more than 200,000 hours of work, or about 155 hours of manual effort per hour of raw video. [ 1 ]
Confirmed by several sources
- TwelveLabs released Pegasus 1.6, a vision-language model with enhanced understanding of egocentric first-person video. [ 1 , 2 , 3 ]
- The model is intended to convert first-person footage into structured data usable for robot training. [ 1 , 2 , 3 ]
- A US robotics training-data company is using Pegasus on workplace footage to evaluate quality and add detailed descriptions, according to TwelveLabs. [ 1 , 3 ]
- CEO Lee Jae-sung said the model turns video into structured knowledge for physical AI and robotics companies. [ 1 , 3 ]
Still unclear
- The identity of the US robotics training-data company using Pegasus is not given. Documents 1 and 3 refer to it only as a US company, so the claim cannot be checked independently.
- TwelveLabs has not stated pricing, general availability or how customers can access Pegasus 1.6. No document mentions commercial terms or availability beyond the launch announcement.
- The figure that one hour of raw video required about 155 hours of manual processing comes from one outlet only. It appears in ZDNet Korea's report and is not repeated or independently confirmed by the other outlets.
- ZDNet Korea's figure of more than 200,000 hours of annotation work for the 1,286-hour Ego-Exo4D dataset is single-source. Only document 1 provides these numbers.