Amazon is finally opening the doors to its concept store of the future, to a reaction that blends excitement with concerns over privacy.
Amazon Go, which is being trialled in Seattle, is a cashierless store that uses state of the art cameras and sensors to track customers and their grocery purchases.
The system allows customers to simply walk into a store, pick up items and walk out. They enter the store by scanning an Amazon Go smartphone app as they pass through a gated turnstile. Sensors track items that they take off the shelves. Anything that they purchase is automatically charged to their Amazon accounts when they leave the store.
Amazon Go is a step change in the shopping experience that retail experts predict could transform brick and mortar stores by eliminating queues and pesky human interactions (aka jobs). The added convenience could, however, come at a big cost to privacy, as the system allows Amazon to track troves of data, from cameras, sensors and microphones.
A 2016 promotional video for Amazon Go gave the public an early glimpse into how the futuristic store would work. The narrator described the underlying technology as a combination of "computer vision, deep learning algorithms and sensor fusion".
Most of the mainstream media reported the announcement breathlessly, although some raised concerns over how this vague description would apply in practice. Their unease goes back to a 2014 patent filed by Amazon Technologies, Inc. for what the company describes as "a system for automatically transitioning items from a materials handling facility without delaying a user as they exit the materials handling facility."
The patent, titled "Transitioning items from a materials handling facility", showed the store would track the customer journey through a vast collection of cameras and sensors.
In the public store in Seattle, hundreds of matte black video cameras that hang from the ceiling constantly follow shoppers around the store.
An algorithm analyses the gestures captured by the cameras to identify which items a customer picks ups, while weight sensors assess which ones leave the shelves.
The patent shows that Amazon has considered using further technologies to verify the purchase. The options it applied for an "an image of the item captured by a camera when the item is picked by the user, a RFID tag detected by a RFID reader when the item is picked by the user, or a change in a weight measured at the inventory location."
Early looks at the store suggest that the RFID tag has been shelved. Amazon also says the system does not involve any facial recognition or phone tracking, perhaps partly to quieten the concerns of privacy campaigners.
The current incarnation of Amazon Go is unlikely to silence them. It still tracks a customer's every movement in the store, and has created a new level of personal surveillance.
It also adds provides Amazon with another source of customer data covering location and consumption habits. The company could use this to manipulate your purchasing impulses on a granular level through targeted recommendations and offers.
It could sell this information to marketers or integrate it with other aspects of Amazon's business, such as the anticipatory shipping model that predicts the products a customer is likely to buy.
The 2009 acquisition of image recognition startup SnapTell may have planted the seed for the sort of product recognition technology that Amazon needed to perfect to make Amazon Go a reality.
The current technology may be only the start. Not only could future customers be followed by a host of cameras and sensors, they may also eventually be tracked by microphones, as the explains: "Microphones may record sounds made by the user and the computing resource(s) may process those sounds to determine a location of the user."
This opens the door to Amazon listening and logging all conversations had within its walls, and even our homes thanks to its Alexa voice service.
I hereby resign from The Future. https://t.co/jAdBi1Lyh8— Silkie Carlo (@silkiecarlo) December 6, 2016
The patent even states that the cameras will be able to identify a customer's "skin tones", which would open the possibility of categorising customers into gender and racial demographic columns on its database.
The promotional video showed that when a customer picks up and puts down a cupcake, the system recognises these movements. In theory, this would enable Amazon to ping the customer a notification with a 10 percent discount for said cupcake.
The patent hints at this capability: "Likewise, the inventory management system may also include one or more communication devices, such as wireless antennas that facilitate wireless communication (e.g., Wi-Fi, Near Field Communication (NFC), Bluetooth) between the inventory management system and the portable device [of the customer]."
These services may initially appear to be free and convenient, until we remember that tech companies value themselves on the quality and quantity of proprietary data they are able to accrue.
The first Amazon Go customers were Amazon employees, in a trial that began in December 2016. The retail giant had hoped to open the store to the public in early 2017, but kinks including issues around tracking shoppers and their items reportedly delayed the launch. The flagship store in Amazon's home state of Seattle finally opened its doors on Monday, 22 January.
To support scaling, the algorithm has been fine-tuned to identify items quickly and accurately, a task that was likely made easier because Amazon can control what is stocked in its own stores. If it wanted to sell the technology to other retailers, it would have to take a more bespoke approach to implementation, especially when items are of high value or are priced according to weight.
Amazon made another big move into the high street shopping in 2017, with the $13.7 billion (£9.9 billion) purchase of the Whole Foods chain. The Go store is a separate venture, and Amazon says it has no current plans to bring its technology to Whole Foods stores, but some items stocked at the test store come from Whole Foods.
The company's future plans for the Go store remain unclear for now, but the ambitions are unlikely to end at a single store. The success of the inaugural Seattle facility will determine whether the concept becomes the future of retail, with Amazon potentially selling the concept to other retailers or rolling it out further themselves.