Surface Duo front
Image Courtesy: MKBHD

Microsoft is working on a ‘Depth and multi-spectral’ camera, possibly for future Surface devices including the Surface Duo and Neo. Microsoft’s multi-spectral camera could excel over Apple’s TrueDepth camera in long-range and few other aspects.

Spotted by us, the patent filing is titled ‘OBJECT RECOGNITION USING DEPTH AND MULTI-SPECTRAL CAMERA’. The patent application was filed and published in the second half of 2019, so it’s fairly new and it’s timed well with the Surface Neo/Duo launch.

We don’t know for sure if this patent applies to Surface Neo and Surface Duo, but a similar technology could be used in these new devices as the company wouldn’t want to launch advanced dual display devices and rely on older 2D facial recognition cameras.

The filing discusses a camera configured to output depth-multi-spectral image and each pixel includes at least a depth value and spectral value. It uses object recognition machine combined with a depth+multi-spectral image to understand the structure of the object and produce output.

The recognition machine uses a convolutional neural network to determine the face or object.

Face recognition
Image Courtesy: WIPO / Microsoft

Microsoft says the convolutional layers can be used to recognize 2D visual features including shapes, patterns, and features of faces such as noses, eyes or lips. It can recognize 3D visual features of faces such as forehead and chin.

Some layers of convolutional are designed to understand and detect spectral features of faces such as freckles or eye colour.

Back in October, Panos Panay, chief product officer of the Microsoft Devices Group, confirmed that the firm is working on a world-class camera for the Surface Duo.

It’s likely that this technology won’t be limited to a single device and Microsoft could also use it in other products such as HoloLens.

Surface Duo and Surface Neo

Surface Duo and Surface Neo were announced at the company’s annual hardware event in October.

The Duo and Neo are still a year away, with a public release anticipated to roll out when holiday season in 2020 is around.