There has been a lot of excitement around Amazon Go, the latest announcement from the retail giant which will allow customers to simply walk into a store, pick up items and walk out, with an automatic bill sent to their Amazon account.
This excitement is warranted. From a pure convenience point of view Amazon Go is a step change in the shopping experience, with queues and pesky human interactions (aka jobs) at risk of being eliminated, in an extremely Whole Foods-looking future.
The promotional video for Amazon Go vaguely refers to the underlying technology made up of "computer vision, deep learning algorithms and sensor fusion".
Most of the mainstream media reported the announcement breathlessly, but some are already raising concerns which track back to a 2014 patent Amazon Technologies, Inc. filed. In Amazon's words the patent is for: "A system for automatically transitioning items from a materials handling facility without delaying a user as they exit the materials handling facility."
The patent, titled 'transitioning items from a materials handling facility', shows the underlying technology behind Amazon Go and it consists of, primarily, lots and lots of cameras.
Presumably this is where the 'computer vision' (read: facial recognition) technology comes in, as store cameras will verify you as you walk in by matching your face to an ID held on your Amazon account.
This computer vision tech will also be used to verify the item a customer has picked up. As the patent states, along with the sensors underneath items, Amazon will verify the item through: "An image of the item captured by a camera when the item is picked by the user, a RFID tag detected by a RFID reader when the item is picked by the user, or a change in a weight measured at the inventory location."
The 2009 acquisition of image recognition startup SnapTell may have planted the seed for the sort of product recognition technology that Amazon will need to perfect to make Amazon Go a reality.
Not only could customers be tracked by a host of cameras but, even creepier, microphones. The patent reads: "Microphones may record sounds made by the user and the computing resource(s) may process those sounds to determine a location of the user."
Listen: The UK Tech Weekly Podcast discuss Amazon Go
Depending on what sort of guidelines are set by the company, this opens the door to Amazon listening and logging all conversations had within its walls, and indeed perhaps also our homes thanks to its Alexa voice service.
I hereby resign from The Future. https://t.co/jAdBi1Lyh8— Silkie Carlo (@silkiecarlo) December 6, 2016
Go will allow Amazon to know your whereabouts, spending habits, consumption habits and more. This will allow the retailer to not only have a trove of data on you but also push unprecedented levels of targeted advertising your way.
The patent even states that the cameras will be able to identify a customer's "skin tones", so, combined with the camera feeds, gender and racial demographic columns on its database are a distinct possibility.
The video shows that when a customer picks up and puts down a cupcake, the system recognises these movements. In theory, Amazon could ping the customer a notification with a 10 percent discount for said cupcake.
The patent itself hints at this capability: "Likewise, the inventory management system may also include one or more communication devices, such as wireless antennas that facilitate wireless communication (e.g., Wi-Fi, Near Field Communication (NFC), Bluetooth) between the inventory management system and the portable device [of the customer]."
Techworld recently covered Google Photos and the inherent trade off consumers are increasingly asked to make when engaging with big technology companies - sacrificing personal privacy in favour of convenience.
These services may seem free and convenient, until we remember that tech companies value themselves on the quality and quantity of proprietary data they are able to accrue.
The testing phase will have to confront a host of potential issues, like identifying if the person holding a phone is, in fact, the owner and how to accurately understand which items a customer has picked up.
The issue will come with scaling. With Amazon in control of what is stocked in its own stores, it can more easily overcome these sort of hurdles. If it wanted to sell the technology to other retailers, it would have to take a more bespoke approach to implementation, especially when items are of high value or are priced according to weight.
Amazon is yet to respond to Techworld's questions. The Amazon Go grocery store in Seattle is still in the testing stage.
Find your next job with techworld jobs