This Is Probably What a Store Will Be Like in the Future

Amazon announces a new type of smart brick-and-mortar shopping with no checkouts.

Anyone finding this post and video 10-15 years from now will think, “Huh? How else would you shop? What’s checking out? Lines, really?” Amazon has announced a new brick-and-mortar “Amazon Go” store in Seattle they plan to open early in 2017. It fundamentally changes the way we shop. And it’s an idea that makes so much sense, if it does work, that other sellers will inevitably follow suit over time. The store is currently — this is a little weird — being beta-tested by Amazon employees.

For some time now, you’ve been able to go into an Apple Store and purchase items yourself from your phone without requiring a sales person. The Amazon Go store and phone app go way beyond that. They let you keep your phone in your pocket, place the items you want in your own bags, and then just leave when you’re done. Here’s the company’s introductory video.

The system works through an Amazon Go app you install on your phone. As a registered Amazon customer, you arrive at the store, swipe yourself in using the app, put your phone away, and hit the shelves, adding items to your “virtual shopping cart” It’s like a shopping cart on a website, not the kind with a bent wheel that you try and push and drag through a physical store.

While Amazon’s not being overly transparent about how they’re doing this, their video does say, “So how does it work? We used computer vision, deep learning algorithms and sensor fusion, much like you’d find in self-driving cars.”

Last year, tech site Recode uncovered a patent filing from Amazon that likely offers some clues about the tech being used.

Normal-looking shelves. (AMAZON GO)

The first challenge a smart store like this would face is knowing with certainty what products are on their shelves and which products are being removed. Amazon’s patent suggests a system of multiple sensors working in tandem to identify the goods:

In some implementations, data from other input devices may be used to assist in determining the identity of items picked and/or placed in inventory locations. For example, if it is determined that an item is placed into an inventory location, in addition to image analysis, a weight of the item may be determined based on data received from a scale, pressure sensor, load cell, etc., located at the inventory location. The image analysis may be able to reduce the list of potentially matching items down to a small list. The weight of the placed item may be compared to a stored weight for each of the potentially matching items to identify the item that was actually placed in the inventory location. By combining multiple inputs, a higher confidence score can be generated increasing the probability that the identified item matches the item actually picked from the inventory location and/or placed at the inventory location.

The deep learning algorithms apparently have to do with using the buyer’s Amazon shopping history as a backup-check to help ensure the system is correctly identifying each item being purchased.

For example, if the inventory management system cannot determine if the picked item is a bottle of ketchup or a bottle of mustard, the inventory management system may consider past purchase history and/or what items the user has already picked from other inventory locations. For example, if the user historically has only picked/purchased ketchup, that information may be used to confirm that the user has likely picked ketchup from the inventory location.

Making a purchase. (AMAZON GO)

Since the system watches everything so closely, should you change your mind and put a product back, it’s automatically removed from your virtual cart.

Finally, you leave the store, shopping done. A receipt appears on the phone for what’s just been charged to your Amazon account:

“[W]hen the customer passes through the exit (transition area) of the retail location, the items picked by the user may be automatically transitioned from the materials handling facility to the user and the user may be charged a fee for the items. … For example, if the user is purchasing items from a retail location, rather than the user having to stop and ‘check out’ with a cashier, teller or automated check station, because the picked items are already known and identified on an item identifier list associated with the user, the user may simply exit the retail location with the items. The exit of the user will be detected and, as the user passes through the exit (transition area), the user, without having to stop or otherwise be delayed, will automatically be charged a fee for the items (the items are transitioned to the user).”

Receipt (AMAZON GO)

Of course, all of this happens invisibly to the customer — except for the receipt and charge — and we can assume Amazon has been refining these systems since the patent was filed in June of 2013. Still, this is probably pretty close to what’s going on, though some suspect RFID tags are also involved now.

There are tantalizing questions for which we don’t yet have the answers. Will stores like this have delis where servings to order can be obtained? Will we have to bring our own bags? (We’d guess not.) Will there be actual carts? (Presumably.) Where will they display tabloid rags and impulse-buy treats? And maybe most important, without checkouts and baggers where will teenagers work over the summer?

How to vaccinate the world’s most vulnerable? Build global partnerships.

Pfizer's partnerships strengthen their ability to deliver vaccines in developing countries.

Susan Silbermann, Global President of Pfizer Vaccines, looks on as a health care worker administers a vaccine in Rwanda. Photo: Courtesy of Pfizer.
  • Community healthcare workers face many challenges in their work, including often traveling far distances to see their clients
  • Pfizer is helping to drive the UN's sustainable development goals through partnerships.
  • Pfizer partnered with AMP and the World Health Organization to develop a training program for healthcare workers.
Keep reading Show less

Scientists claim the Bible is written in code that predicts future events

The controversy around the Torah codes gets a new life.

Michael Drosnin
Surprising Science
  • Mathematicians claim to see a predictive pattern in the ancient Torah texts.
  • The code is revealed by a method found with special computer software.
  • Some events described by reading the code took place after the code was written.
Keep reading Show less

A dark matter hurricane is crashing into Earth

Giving our solar system a "slap in the face."

Surprising Science
  • A stream of galactic debris is hurtling at us, pulling dark matter along with it
  • It's traveling so quickly it's been described as a hurricane of dark matter
  • Scientists are excited to set their particle detectors at the onslffaught
Keep reading Show less

Orangutans exhibit awareness of the past

Orangutans join humans and bees in a very exclusive club

(Eugene Sim/Shutterstock)
Surprising Science
  • Orangutan mothers wait to sound a danger alarm to avoid tipping off predators to their location
  • It took a couple of researchers crawling around the Sumatran jungle to discover the phenomenon
  • This ability may come from a common ancestor
Keep reading Show less