Google brings real-time video segmentation to YouTube app
Tech giant Google announced the introduction of real-time, on-device mobile video segmentation to the YouTube app, by integrating this technology into the latter’s stories feature, a new lightweight video format, designed specifically for YouTube creators on its beta version.
The new segmentation technology will allow creators to replace and modify the background, effortlessly increasing videos? production value without specialised equipment, Google’s research blog noted. “Video segmentation is a widely used technique that enables movie directors and video content creators to separate the foreground of a scene from the background and treat them as two different visual layers.
By modifying or replacing the background, creators can convey a particular mood, transport themselves to a fun location or enhance the impact of the message. However, this operation has traditionally been performed as a time-consuming manual process or requires a studio environment with a green screen for real-time background removal. In order to enable users to create this effect live in the viewfinder, we designed a new technique that is suitable for mobile phones,” the blog read.
The new technology has been developed using machine learning to solve a semantic segmentation task using convolution neural networks. To provide high-quality data for the machine learning pipeline, …read more