22 Ways Algorithms Know How You'll Behave Before You Do

Prediction is reinventing industries and running the world. More and more, predictive analytics drives commerce, manufacturing, healthcare, government, and law enforcement.

The future is the ultimate unknown. It's everything that hasn't happened yet.


Prediction as a capability is booming. It reinvents industries and runs the world. More and more, predictive analytics drives commerce, manufacturing, healthcare, government, and law enforcement. In these spheres, organizations operate more effectively by way of predicting behavior—i.e., the outcome for each individual customer, employee, patient, voter, and suspect.

Predictive analytics’ expansive deployment has taken hold. Accenture and Forrester both report that predictive analytics' adoption has more than doubled in recent years. Transparency Market Research projects the predictive analytics market will reach $6.5 billion within a few years. Predictive analytics is becoming a standard safeguard for business, and even demand from consumers for its capabilities promises to surge.

New groundbreaking stories of predictive analytics in action are pouring in. A few key ingredients have opened these floodgates:

  • Wildly increasing loads of data
  • Cultural shifts as organizations learn to appreciate, embrace, and integrate predictive technology
  • Improved software solutions to deliver predictive analytics to organizations
  • I've listed below a slew of examples—from the likes of Facebook, the NSA, Hillary for America, Uber, Airbnb, Google, Shell, UPS, Amazon.com, Coned, Yahoo!, and the U.S. government. 

    22 EXAMPLES OF PREDICTIVE ANALYTICS:

    What’s predicted:

    Example:

    Which Facebook posts you will like in order to optimize your news feed

    Facebook: Predicts which of 1,500 candidate posts (on average) will be most interesting to you in order to personalize your news feed. To optimize the order of content items, the News Feed ranking algorithm weights around 100,000 factors such as recency, likes, clicks, shares, comments, time spent on posts, poster popularity, your affinity for the poster and content area, and measures of relevance and trustworthiness. This intensifies the “addictive” engagement, with two-thirds of Facebook’s 1.44 billion monthly users logging in daily.

    Who’s in a photo (aka facial recognition)

    Facebook: Improved the state of the art for identifying people from photos to virtually the same performance level as a human: Given two face images, it can determine whether they’re the same person with 97 percent accuracy. Facial recognition helps users tag photos, which they do more than 100 million times a day. The company has also developed predictive models to identify people even if it can’t see the face, achieving 83 percent accuracy when faces are at least partially obscured half of the time, based on elements such as clothing, hair, and pose.

    Clicks in order to select which to display

    Facebook: In order to increase revenue from its pay-per-click advertisers, predicts ad clicks based on user attributes, device used, and contextual factors.

    Terrorism

    The National Security Agency: Obtained software solutions for and core competency in predictive analytics. It’s clear that the NSA considers predictive analytics a strategic priority as a means to target investigation activities by automatically discovering previously unknown potential suspects.

    Where you are going

    Uber: Can predict the specific destination address of San Francisco riders based on exact drop-off location with 74 percent accuracy, despite, for example, how many businesses there are within 100 meters in a typical city area (just taking the closest candidate address achieves 44 percent accuracy).

    Acceptance of booking request in order to match guests to hosts

    Airbnb: Rank orders accommodations that fulfill a user search in part by the predicted probability each host would accept the user’s booking request. By surfacing likely matches more prominently, the company increased booking conversions by nearly 4 percent—a significant gain considering its estimated annual booking of over 12 million guest nights.

    Accommodation bookings at a given pricefor dynamic pricing

    Airbnb: Suggests each day’s price for an accommodation listing (the “Price Tips” feature) by way of predicting whether the listing will be booked—predicted demand directly informs optimal pricing. Bookings are predicted by day of the week, seasonality, and local events, as well as characteristics of the listing such as the neighborhood, size, amenities, key words like “beach,” number of reviews, and photographs. Hosts who set prices within 5 percent of the suggestions improve their chance of booking by a factor of nearly four.

    Spam to send it to your spam folder

    Google: Decreased Gmail’s prevalence and false positive rate of spam from disruptive (in 2004) down to negligible.

    Oil refinery safety incidents

    Shell: Predicts the number of safety incidents per team of workers at oil refineries, globally. One example discovery: Increased employee engagement predicts fewer incidents; one percentage point increase in team employee engagement is associated with a 4 percent decrease in the number of safety incidents per FTE.

    Maritime incidents  

    RightShip: Predicts dangerous or costly maritime incidents in order to assess vessel risk that informs shipment decisions when selecting between vessels. The 10 percent highest-risk vessels are three times more likely than average to experience an incident in the next 12 months, and are 16 times more likely to incur a casualty than the 10 percent least risky. Risk assessment is based on vessel age, type, carrying capacity, origin, registration, ownership, management, and other factors.

    Deliverieswhich addresses will receive a package

    UPS: Cut 85 million miles from annual delivery vehicle driving with a semiautomatic optimization system that plans vehicle/package assignments, as well as package placement within the vehicle, based upon each day’s analytically predicted delivery destinations.

    Product choices

    Amazon.com: Thirty-five percent of sales come from product recommendations. The company may also develop “anticipatory shipping” that would proactively place packages before they are ordered at hubs or on trucks in order to reduce delays between ordering and receiving purchases, for which it has obtained a patent.

    Product choices

     

    Spotify: Is augmenting its song recommendation algorithm to incorporate musical attributes.

    Voter persuasion

    Hillary for America 2016 Campaign: Given Obama’s success with persuasion modeling in 2012, Hillary Clinton’s 2016 campaign appears to be planning to employ it as well. Analytics job postings reveal they’re going to be “helping the campaign determine which voters to target for persuasion.”

    Restaurant health code violations via Yelp reviews

    City of Boston: Sponsored a competition that generated the ability to predict whether a restaurant will have more violations than normal with 75 percent accuracy, in part by way of discovering clues within Yelp reviews, in order to target city health department inspections. Similar work for Seattle restaurants distinguished severe violators with 82 percent accuracy.

    Lead poisoning from paint

    City of Chicago: Identified 5 percent of homes that are at more than twice the risk for lead poisoning incidents than average based on the age of the house, the history of lead paint exposure at that address, the economic conditions of the neighborhood, and other factors. This serves as an early warning system to proactively flag, as an improvement over the more common reactive steps taken after a positive test for poisoning. The risk scores serve to target homes for inspection and children for testing, and could help people determine safer homes to move to.

    Fire

    City of New York: Targets the fire inspections of its 330,000 inspectable buildings with a predictive model that assesses risk based on about 60 factors.

    Manhole explosions and fires

    Con Edison: Predicts dangerous manhole explosions and fires in New York City, identifying a 2 percent of manholes that have a 5.5 times greater than average risk of an incident.

    Beauty

    Yahoo! Labs: Developed a model to categorize photographic portraits as to the subjective human aesthetic of beauty with 64 percent accuracy based on various image attributes. The study determined “that race, gender, and age are largely uncorrelated with photographic beauty.”

    Overpriced property leases

    U.S. Postal Service Office of Inspector General: Predicted the amount paid over market value for each of their 26,000 leased facilities (e.g., retail unit, plant, warehouse). Targeting facilities in the Northeast Region, USPS auditors projected that 250 of the leases predicted as most overpaid represent a potential savings of $6.6 million by way of renegotiating their next year of lease terms.

    Surgical site infections

    University of Iowa Hospitals and Clinics: Identifies cases greater than four times as likely to develop surgical-site infections. Targeting anti-infection therapy accordingly reduces the cost of each colorectal surgical procedure an average of $1,300 and will provide a projected annual savings of several million dollars once expanded to other forms of surgery.

    Airfares

    Hopper: Predicts airfare changes in order to recommend to consumers whether to buy or wait. Ninety-five percent of these predictions save the consumer money or do no worse than the first price seen, saving users an average 10 percent on ticket price.

    This flood of predictive activity gains its potential simply because prediction boasts an inherent generality—there are just so many conceivable ways to make use of it. Want to come up with your own new innovative use for predictive analytics? You need only two ingredients. Each application of predictive analytics is defined by:

    1. What’s predicted: the kind of behavior—i.e., action, event, or happening—to predict for each individual (e.g., person, Facebook post, photo, ad, trip destination, marine vessel, safety incident, transaction, or other organizational element).

    2. What’s done about it: the decisions driven by prediction; the action taken by the organization in response to or informed by each prediction.

    We can confidently predict more prediction. Every few months, another big story about predictive analytics rolls off the presses. We’re sure to see the opportunities continue to grow and surprise. Come what may, only time will tell what we’ll tell of time to come.

    --

    These examples are new in this year's Revised and Updated edition of my book, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die. With these newly added cases, the book's central compendium of mini-case studies has grown to 182 entries (most were sourced from presentations at Predictive Analytics World, the event series I founded—for more information about each example, access the book's Notes PDF, available at www.PredictiveNotes.com, and search by organization name).

    Related Articles

    Why the worst part about climate change isn't rising temperatures

    The world's getting hotter, and it's getting more volatile. We need to start thinking about how climate change encourages conflict.

    Christopher Furlong/Getty Images
    Politics & Current Affairs
    • Climate change is usually discussed in terms of how it impacts the weather, but this fails to emphasize how climate change is a "threat multiplier."
    • As a threat multiplier, climate change makes already dangerous social and political situations even worse.
    • Not only do we have to work to minimize the impact of climate change on our environment, but we also have to deal with how it affects human issues today.

    Human beings are great at responding to imminent and visible threats. Climate change, while dire, is almost entirely the opposite: it's slow, it's pervasive, it's vague, and it's invisible. Researchers and policymakers have been trying to package climate change in a way that conveys its severity. Usually, they do so by talking about its immediate effects: rising temperature, rising sea levels, and increasingly dangerous weather.

    These things are bad, make no mistake about it. But the thing that makes climate change truly dire isn't that Cape Cod will be underwater next century, that polar bears will go extinct, or that we'll have to invent new categories for future hurricanes. It's the thousands of ancillary effects — the indirect pressure that climate change puts on every person on the planet.

    How a drought in the Middle East contributed to extremism in Europe

    (DANIEL LEAL-OLIVAS/AFP/Getty Images)

    Nigel Farage in front of a billboard that leverages the immigration crisis to support Brexit.

    Because climate change is too big for the mind to grasp, we'll have to use a case study to talk about this. The Syrian civil war is a horrific tangle of senseless violence, but there are some primary causes we can point to. There is the longstanding conflicts between different religious sects in that country. Additionally, the Arab Spring swept Syria up in a wave of resistance against authoritarian leaders in the Middle East — unfortunately, Syrian protests were brutally squashed by Bashar Al-Assad. These, and many other factors, contributed to the start of the Syrian civil war.

    One of these other factors was drought. In fact, the drought in that region — it started in 2006 — has been described as the "worst long-term drought and most severe set of crop failures since agricultural civilization began in the Fertile Crescent many millennia ago." Because of this drought, many rural Syrians could no longer support themselves. Between 2006 and 2009, an estimated 1.5 million Syrians — many of them agricultural workers and farmers — moved into the country's major cities. With this sudden mixing of different social groups in a country where classes and religious sects were already at odds with one another, tensions rose, and the increased economic instability encouraged chaos. Again, the drought didn't cause the civil war — but it sure as hell helped it along.

    The ensuing flood of refugees to Europe is already a well-known story. The immigration crisis was used as a talking point in the Brexit movement to encourage Britain to leave the EU. Authoritarian or extreme-right governments and political parties have sprung up in France, Italy, Greece, Hungary, Slovenia, and other European countries, all of which have capitalized on fears of the immigration crisis.

    Why climate change is a "threat multiplier"

    This is why both NATO and the Pentagon have labeled climate change as a "threat multiplier." On its own, climate change doesn't cause these issues — rather, it exacerbates underlying problems in societies around the world. Think of having a heated discussion inside a slowly heating-up car.

    Climate change is often discussed in terms of its domino effect: for example, higher temperatures around the world melt the icecaps, releasing methane stored in the polar ice that contributes to the rise in temperature, which both reduces available land for agriculture due to drought and makes parts of the ocean uninhabitable for different animal species, wreaking havoc on the food chain, and ultimately making food more scarce.

    Maybe we should start to consider climate change's domino effect in more human and political terms. That is, in terms of the dominoes of sociopolitical events spurred on by climate change and the missing resources it gobbles up.

    What the future may hold

    (NASA via Getty Images)

    Increasingly severe weather events will make it more difficult for nations to avoid conflict.

    Part of why this is difficult to see is because climate change does not affect all countries proportionally — at least, not in a direct sense. Germanwatch, a German NGO, releases a climate change index every year to analyze exactly how badly different countries have been affected by climate change. The top five most at-risk countries are Haiti, Zimbabwe, Fiji, Sri Lanka, and Vietnam. Notice that many of these places are islands, which are at the greatest risk for major storms and rising sea levels. Some island nations are even expected to literally disappear — the leaders of these nations are actively making plans to move their citizens to other countries.

    But Germanwatch's climate change index is based on weather events. It does not account for the political and social instability that will likely result. The U.S. and many parts of Europe are relatively low on the index, but that is precisely why these countries will most likely need to deal with the human cost of climate change. Refugees won't go from the frying pan into the fire: they'll go to the closest, safest place available.

    Many people's instinctive response to floods of immigrants is to simply make borders more restrictive. This makes sense — a nation's first duty is to its own citizens, after all. Unfortunately, people who support stronger immigration policies tend to have right-wing authoritarian tendencies. This isn't always the case, of course, but anecdotally, we can look at the governments in Europe that have stricter immigration policies. Hungary, for example, has extremely strict policies against Muslim immigrants. It's also rapidly turning into a dictatorship. The country has cracked down on media organizations and NGOs, eroded its judicial system's independence, illegalized homelessness, and banned gender studies courses.

    Climate change and its sociopolitical effects, such as refugee migration, aren't some poorer country's problem. It's everyone's problem. Whether it's our food, our homes, or our rights, climate change will exact a toll on every nation on Earth. Stopping climate change, or at least reducing its impact, is vitally important. Equally important is contending with the multifaceted threats its going to throw our way.

    Personal Growth

    Design is all around us in a myriad of forms. From the screen interfaces on your phones and devices to the handles on your shower faucets. We often know instinctively what constitutes great design, there's an almost ephemeral quality to it. Great design offers comfort, ease of use and a feeling of being in the know and in control.

    Bad design on the other hand hits us like an ill-shaped rock – hard to navigate websites, Rube Goldberg machines and a general sense of annoyance and confusion. Design is both a science and an art and everybody is affected by it in some way. Whether you're a designer or just appreciate design and want to know more, here are the 10 best books on design.

    The Design of Everyday Things

    In a clear and concise matter, Don Norman writes about the flaws that plague the design of everyday objects, which makes our lives more trouble than they need to be, more inconvenient and sometimes downright dangerous. This was a book written in the late 1980s, but is still relevant today, as it has been updated a few times.

    The book isn't just an exposé of horrid design, but also a tale of how designers in all industries can become better apt to customers' and end-users' needs. It's a must read for any type of designer, as Norman goes into great detail about design methodologies, ideals and psychology. He has many thoughts about how if you can't figure something out, it's not always your fault but often the designer's. His philosophy of design is proper communication and usability, Norman states:

    "Eliminate the term human error. Instead talk about communication and interaction. When people collaborate with one another the word error is never used to characterize another person's utterance."

    About Face: The Essentials of Interaction

    Let's face it, the majority of design today is within the digital field: software design, websites, applications and other mediums of online & digital expression. Alan Cooper & Co.'s About Face is the premier book for interaction design. It covers project processes, goal directed design and everything you could ever need to know about user feedback, controls and comprehensive overview of interaction.

    The book is sprawling and deep dives into just about any common UI widget in existence. It's considered a pillar of learning material for UI/UX designers. While some may get turned off by its length and pedantic explanations, it also serves as an excellent reference book for UX designers.

    A Designer's Art 

    Paul Rand's book was published in 1985 and was one of the first of its kind. The renowned graphic designer wanted to create a book that would explain the art of a growing discipline, rather than just show it visually. The book is packed with personal views on design, peppered with his expansive portfolio and also cites a number of renowned academics.

    Rand was another designer who felt that communication is absolutely key when it comes to design. He states:

    "Graphic design which evokes the symmetria of Vitruvius, the dynamic symmetry of Hambidge, the asymmetry of Mondrian; which is a good gestalt, generated by intuition or by computer, by invention or by a system of coordinates is not good design if it does not communicate."

    Beauty and symmetrical supremacy doesn't mean a whole lot if it can't communicate its intended message. For students of design, teachers and professionals, this is a book that is great for explaining and expressing the creative communication of ideals.

    A Product Guide to UX Design

    Business and design often coalesce together in an alliance of production. A professional designer is going to be required to interact with other aspects of running a business. Ensuring that a user interaction is running smoothly and the design assets are glowing in perfect fidelity and union with the product are all well and good and the meat of a UX designer's job; but working this into an overall business perspective is also an important skill to have.

    This book by Russ Unger and Carolyn Chandler covers a breadth of topics for those who might have minimal experience in UX design, but are interested in applying their newfound skills in a business setting.

    Elements of User Experience

    Jesse James Garrett exposes in a very clear way the essence of user experience for the web. He breaks down the ux for the web into five different planes going deep into the vocabulary and strategy for designing better experiences for our digital world.

    He sets out some simple rules for consistency and great design:


    "Presenting a style on your Web site that's inconsistent with your style in other media doesn't just affect the audience's impression of that product; it affects their impression of your company as a whole. People respond positively to companies with clearly defined identities. Inconsistent visual styles undermine the clarity of your corporate image and leave the audience with the impression that this is a company that hasn't quite figured out who it is."

    Geometry of Design: Studies in Proportion and Composition

    images-na.ssl-images-amazon.com

    Kimberly Elam's Geometry of Design brings out the mathematical guns in analyzing and postulating about the inherent symmetrical nature of great design. She explores the relationships between visual representations and their foundations in geometry. It's a great book that focuses on the golden ratio and root rectangles.

    Elam utilizes overlays and grids in order to identify designs in different works of design and art. She looks at the underlying geometric structures in architecture, compositions and even furniture. The author has a great ability to distill these high level math concepts and distill them in an understandable and relatable way with insight into the design process.

    Universal Principles of Design 

    This landmark book is the ultimate reference and cross-disciplinary design book. With richly illustrated and fantastic design elements, this book clearly displays a wide range of visual and design concepts. From anthropomorphic form to the Golden ratio, these over 100 design concepts are well-defined and thought out for readers to expand their principal knowledge.

    It's a great book for skimming and also using a reference. There's also a few mind-benders in there as well, for example:

    "The 80/20 rule asserts that approximately 80 percent of the effects generated by any large system are caused by 20 percent of the variables in that system."

    Apply this same concept to an app and you'll find that this is also true. These principles are a great starting off point to delve deeper into the fundamentals of design in all types of mediums.

    Don’t make me Think! 

    Written and first published in 2000, Steve Krug's Don't Make Me Think has served as a bible for a countless number of web designers and businesspeople. With an updated version for mobile usability, Krug presents his ideas in an understandable way for web designers to learn more about navigation and information design.

    It's an excellent introduction to creating websites with some just plain common sense advice. As the title states, a website should be first and foremost functional and something people barely need to think about when using it.

    The Visual Display of Quantitative Information

    This classic book on statistics, graphs, charts and tables puts together both theory and practice in the visualization of data graphics. The text has some 250 plus of some the best and worst graphics for review. The book takes into account a number of highly sophisticated graphical design aspects, including:

    • High resolution displays
    • Editing graphics
    • Data-ink ratio
    • Time Series
    • Relational graphics
    • Data maps
    • Design variations versus data variations
    • & more!

    Many people don't understand the importance of graphical competence as it requires a number of skills, both statistical and even artistic. Edward R. Tufte does a great job pointing out that while graphical representation is usually lacking in media publications, journals and general reading materials – graphical representation and comprehensive is a necessary in many fields for experts.

    The One Device: the Secret History of the iPhone

    While this book doesn't necessarily tout the fundamentals of design, it's an exciting historical view of what some people consider to be one of the greatest designed devices within the past few decades. There is no doubt that the iPhone has revolutionized the world, smartphone industry and changed our modern way of life. A mastery of design and functionality, the iPhone is the holy grail of devices.

    Packed within this slab of computational glass is a story that needed to be told. Brian Merchant's book does just that. The history of the phone, electronics, early start of the secretive project within the Apple headquarters – all of this tells a tale of an exceptionally well-designed product.

    DIY electrical brain stimulation is a worrying new trend

    There's still a lot even doctors don't know about it.

    (Dierk Schaefer/Flickr)
    Surprising Science
    • Scientists are experimenting with applying electrical current to brains as a potential therapy and enhancement.
    • A wave of DIY brain-shocking is worrying experts.
    • Would you ever zap your own brain to see what happens? DIY and direct-to-consumer devices are available, but researchers have called for an open dialog with the DIY community about the risks.
    Keep reading Show less