Sony and the first Artificial Intelligence Image Sensor

Sony announced the first image sensor with integrated artificial intelligence smarts in the world. The new IMX500 sensor incorporates both memory and processing power. Thus, it allows it to perform machine learning-powered computer vision tasks without extra hardware. Sony says that the result will be cheaper, faster, and more secure artificial intelligence cameras.

Devices ranging from surveillance cameras to smartphones, in the past few years, benefited from the integration of artificial intelligence. Machine learning is not only for improving the quality of the pictures we take. Moreover, it also is able to understand videos like a human would, identifying objects and people in the frame. The applications of that technology are worrying (sometimes huge), enabling everything from automated surveillance to self-driving cars.

Nevertheless, many applications are relying on sending videos and images to the cloud for analyzing. That can be an insecure and slow journey, exposing data to hackers. Nevertheless, manufacturers must install specialized processing cores on devices for handling the extra computational demand, in case of other scenarios. That is what new high-end phones from Google, Huawei, and Google do.

Nevertheless, Sony said that its new image sensor is offering a more streamlined solution than either of these approaches.

Mark Hanson is Sony’s vice president of business and innovation. He told The Verge that there are some other ways of implementing those solutions. Thus, he referred to edge computing, which uses dedicated artificial intelligence chips that are not attached directly to the image sensor. Nevertheless, Hanson does not believe they will be anywhere close to as cost-effective as Sony shipping image sensors in the billions.

Sony in the Market

Moreover, the huge presence of Sony in the market of image processing will undoubtedly push that technology at a massive scale to clients. Hanson noted that the company has a more than 60% market share. Moreover, last year, they shipped about 1.6 billion sensors, including for all three cameras in Apple’s iPhone 11 Pro.

However, that first-generation artificial intelligence sensor is unlikely to end up in consumer devices like tablets and smartphones, at least to start with. Nevertheless, Sony will target industries and retailers. Hanson referred to Amazon’s cashier-less Go stores as a potential application.

The retailer is using scores of artificial intelligence-enabled cameras for tracking shoppers in Amazon’s Go stores. Moreover, they use scores to charge them for objects that they grab from the shelves. Hanson said that they placed hundreds of cameras. Daily, in a small convenience store, they are running petabytes of data. Nevertheless, Hanson continues, if we could miniaturize that capability and put it on the backside of a chip, we can do all sorts of exciting things.

Moreover, there are privacy benefits in addition to cost savings. For example, imagine an artificial intelligence chip stuck directly onto the back of the image sensor. In that case, object detection can be done on-device. It will not send off data to be analyzed, either to a nearby processor or the cloud. Nevertheless, the sensor of the image is itself performing whatever artificial intelligence analysis is necessary. Thus, it is merely producing the metadata instead.