Understanding Data - Context
Data is an abstraction of real life, and real life can be complicated, but if you gather enough context, you can at least put forth a solid effort to make sense of it.
Look up at the night sky, and the stars look like dots on a flat surface. The lack of visual depth makes the translation from sky to paper fairly straightforward, which makes it easier to imagine constellations. Just connect the dots. However, although you perceive stars to be the same distance away from you, they are actually varying light years away.
If you could fly out beyond the stars, what would the constellations look like? This is what Santiago Ortiz wondered as he visualized stars from a different perspective, as shown in Figure 1-25.
The initial view places the stars in a global layout, the way you see them. You look at Earth beyond the stars, but as if they were an equal distance away from the planet.
Zoom in, and you can see constellations how you would from the ground, bundled in a sleeping bag in the mountains, staring up at a clear sky.
The perceived view is fun to see, but flip the switch to show actual distance, and it gets interesting. Stars transition, and the easy-to-distinguish constellations are practically unrecognizable. The data looks different from this new angle.
This is what context can do. It can completely change your perspective on a dataset, and it can help you decide what the numbers represent and how to interpret them. After you do know what the data is about, your understanding helps you find the fascinating bits, which leads to worthwhile visualization.
Without context, data is useless, and any visualization you create with it will also be useless. Using data without knowing anything about it, other than the values themselves, is like hearing an abridged quote secondhand and then citing it as a main discussion point in an essay. It might be okay, but you risk finding out later that the speaker meant the opposite of what you thought.
You have to know the who, what, when, where, why, and how -- the metadata, or the data about the data -- before you can know what the numbers are actually about.
Who: A quote in a major newspaper carries more weight than one from a celebrity gossip site that has a reputation for stretching the truth. Similarly, data from a reputable source typically implies better accuracy than a random online poll.
For example, Gallup, which has measured public opinion since the 1930s, is more reliable than say, someone (for example, me) experimenting with a small, one-off Twitter sample late at night during a short period of time. Whereas the former works to create samples representative of a region, there are unknowns with the latter.
Speaking of which, in addition to who collected the data, who the data is about is also important. Going back to the gumballs, it's often not financially feasible to collect data about everyone or everything in a population. Most people don't have time to count and categorize a thousand gumballs, much less a million, so they sample. The key is to sample evenly across the population so that it is representative of the whole. Did the data collectors do that?
How: People often skip methodology because it tends to be complex and for a technical audience, but it's worth getting to know the gist of how the data of interest was collected.
If you're the one who collected the data, then you're good to go, but when you grab a dataset online, provided by someone you've never met, how will you know if it's any good? Do you trust it right away, or do you investigate? You don't have to know the exact statistical model behind every dataset, but look out for small samples, high margins of error, and unfit assumptions about the subjects, such as indices or rankings that incorporate spotty or unrelated information.
Sometimes people generate indices to measure the quality of life in countries, and a metric like literacy is used as a factor. However, a country might not have up-to-date information on literacy, so the data gatherer simply uses an estimate from a decade earlier. That's going to cause problems because then the index works only under the assumption that the literacy rate one decade earlier is comparable to the present, which might not be (and probably isn't) the case.
What: Ultimately, you want to know what your data is about, but before you can do that, you should know what surrounds the numbers. Talk to subject experts, read papers, and study accompanying documentation.
In introduction statistics courses, you typically learn about analysis methods, such as hypothesis testing, regression, and modeling, in a vacuum, because the goal is to learn the math and concepts. But when you get to real-world data, the goal shifts to information gathering. You shift from, "What is in the numbers?" to "What does the data represent in the world; does it make sense; and how does this relate to other data?"
A major mistake is to treat every dataset the same and use the same canned methods and tools. Don't do that.
When: Most data is linked to time in some way in that it might be a time series, or it's a snapshot from a specific period. In both cases, you have to know when the data was collected. An estimate made decades ago does not equate to one in the present. This seems obvious, but it's a common mistake to take old data and pass it off as new because it's what's available. Things change, people change, and places change, and so naturally, data changes.
Where: Things can change across cities, states, and countries just as they do over time. For example, it's best to avoid global generalizations when the data comes from only a few countries. The same logic applies to digital locations. Data from websites, such as Twitter or Facebook, encapsulates the behavior of its users and doesn't necessarily translate to the physical world.
Although the gap between digital and physical continues to shrink, the space between is still evident. For example, an animated map that represented the "history of the world" based on geotagged Wikipedia, showed popping dots for each entry, in a geographic space. The end of the video is shown in Figure 1-26.
The result is impressive, and there is a correlation to the real-life timeline for sure, but it's clear that because Wikipedia content is more prominent in English-speaking countries the map shows more in those areas than anywhere else.
Why: Finally, you must know the reason data was collected, mostly as a sanity check for bias. Sometimes data is collected, or even fabricated, to serve an agenda, and you should be wary of these cases. Government and elections might be the first thing that come to mind, but so-called information graphics around the web, filled with keywords and published by sites trying to grab Google juice, have also grown up to be a common culprit. (I fell for these a couple of times in my early days of blogging for FlowingData, but I learned my lesson.)
Learn all you can about your data before anything else, and your analysis and visualization will be better for it. You can then pass what you know on to readers.
However, just because you have data doesn't mean you should make a graphic and share it with the world. Context can help you add a dimension -- a layer of information -- to your data graphics, but sometimes it means it's better to hold back because it's the right thing to do.
In 2010, Gawker Media, which runs large blogs like Lifehacker and Gizmodo, was hacked, and 1.3 million usernames and passwords were leaked. They were downloadable via BitTorrent. The passwords were encrypted, but the hackers cracked about 188,000 of them, which exposed more than 91,000 unique passwords. What would you do with that kind of data?
The mean thing to do would be to highlight usernames with common (read that poor) passwords, or you could go so far as to create an application that guessed passwords, given a username.
A different route might be to highlight just the common passwords, as shown in Figure 1-27. This offers some insight into the data without making it too easy to log in with someone else's account. It might also serve as a warning to others to change their passwords to something less obvious. You know, something with at least two symbols, a digit, and a mix of lowercase and uppercase letters. Password rules are ridiculous these days. But I digress.
With data like the Gawker set, a deep analysis might be interesting, but it could also do more harm than good. In this case, data privacy is more important, so it's better to limit what you show and look at.
Whether you should use data is not always clear-cut though. Sometimes, the split between what's right and wrong can be gray, so it's up to you to make the call. For example, on October 22, 2010, Wikileaks, an online organization that releases private documents and media from anonymous sources, released 391,832 United States Army field reports, now known as the Iraq War Logs. The reports recorded 66,081 civilian deaths out of 109,000 recorded deaths, between 2004 and 2009.
The leak exposed incidents of abuse and erroneous reporting, such as civilian deaths classified as "enemy killed in action." On the other hand, it can seem unjustified to publish findings about classified data obtained through less than savory means.
Maybe there should be a golden rule for data: Treat others' data the way you would want your data treated.
In the end, it comes back to what data represents. Data is an abstraction of real life, and real life can be complicated, but if you gather enough context, you can at least put forth a solid effort to make sense of it.
Excerpted with permission from the publisher, Wiley, from Data Points: Visualization That Means Something by Nathan Yau. Copyright © 2013
Nathan Yau, author of Data Points: Visualization That Means Something, has a PhD in statistics and is a statistical consultant who helps clients make use of their data through visualization. He created the popular site FlowingData.com, and is the author of Visualize This: The FlowingData Guide to Design, Visualization, and Statistics, also published by Wiley.
What would happen if you tripled the US population? Join Matthew Yglesias and Charles Duhigg at 1pm ET on Monday, September 28.
Whether or not women think beards are sexy has to do with "moral disgust"
- A new study found that women perceive men with facial hair to be more attractive as well as physically and socially dominant.
- Women tend to associate more masculine faces with physical strength, social assertiveness, and formidability.
- Women who display higher levels of "moral disgust," or feelings of repugnance toward taboo behaviors, are more likely to prefer hairy faces.
Beards and perceptions of masculinity<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMjU5OTg0MC9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY0NzkxMjM3N30.cH-GqNwP5GVqvstgJWAhBPn1B_lYpVEAI0I7iax7EQw/img.jpg?width=1245&coordinates=0%2C1900%2C0%2C849&height=700" id="caae6" class="rm-shortcode" data-rm-shortcode-id="cb0a355a4e8e1899789bc45f3f7aef56" data-rm-shortcode-name="rebelmouse-image" />
Photo Credit: Wikimedia<p>The study used 919 American (mostly white) women ages 18-70 who rated 30 pictures of men they were shown with various stages of facial hair growth. The photographs depicted men with faces that had been digitally altered to look more feminine or more masculine, with a beard and without a beard. The women rated the men according to perceived attractiveness for long-term and short-term relationships. The study found that the more facial hair the men had, the higher the men were rated on their attractiveness, particularly for their suitability for a long-term relationship.</p><p>Part of this might be attributed to facial masculinity — i.e. protruding brow ridge, wide cheekbones, thick jawline, and deeply set narrow eyes — which conveys information to a woman about a man's underlying health and formidability. Women tend to associate more masculine faces with physical strength and social assertiveness. It can also indicate a man with a superior immune response. The researchers suggested that their findings favoring bearded men could be due to the fact that facial hair enhances the masculine facial features on a man's face, like creating the illusion of a thicker jaw line. This could communicate direct benefits to women like resources and protection that would enhance survival among mothers and their infants. In other words, while a beard doesn't mean superior genetics in and of itself, it might be a primitive, ornamental way of saying, "Hey girl, I'm a testosterone-fueled lean, mean, pathogen fighting machine." <br></p><p>It could also be that a beard becomes its own destiny. The researchers in this study cite prior research that found that by growing a beard, men felt more masculine and had higher levels of serum testosterone, which was linked to a higher level of social dominance. They also tended to subscribe to more old-school beliefs about gender roles in their relationships with women as compared to men with clean-shaven faces.<span></span><br></p>
What does disgust have to do with beard preference?<p>Obviously, not all women dig beards. The researchers were particularly interested in what traits make a women prefer bearded men over clean-shaven faces. They looked into several factors including a woman's disgust levels on various concepts, her desire to become pregnant, and her exposure to facial hair in her personal life. </p><p>According to the study, women who were not into facial hair were turned-off by potential parasites or other critters they imagined could be in the hair or skin. Women ranking high on this "ectoparasite disgust" scale might have viewed beards as a sign of poor grooming habits. However, women who ranked higher in levels of "pathogen" did find the bearded men to be desirable, possibly because they perceived beards as a signal of good health and immune function. An intriguing discovery in the study was links to morality. Women who displayed higher levels of "moral disgust," or feelings of repugnance toward taboo behaviors, were more likely to prefer hairy faces. The authors opined that this could reflect a link between beardedness, politically conservative outlooks, and traditional views regarding performances of masculinity in heterosexual relationships.</p>
Additional findings<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMjU5OTg1My9vcmlnaW4uZ2lmIiwiZXhwaXJlc19hdCI6MTYyNDI1NjUyOX0.P9B8WbmJR0q4nfzYZKbuNSA-2SAigVWJgrQE-_Gxlds/img.gif?width=980" id="49143" class="rm-shortcode" data-rm-shortcode-id="2ed3b1d6f20fc170bf2974646e565e8d" data-rm-shortcode-name="rebelmouse-image" />Giphy<p>The correlations that existed between married and single women's rating on the attractiveness of beards were not particularly clear, although the researchers noted that single and married women who wanted children tended to find beards more attractive than the women who didn't want children. They also found that women with bearded husbands found beards to be more attractive, which might indicate that social exposure to beards influences how desirable they are perceived of as being. Or it could be that men with wives who like beards grow beards.</p><p>It's important to note that culture plays a huge role in how attractive women perceive certain male characteristics as being. This study looked at a small, culturally specific group of American women, so no big, universal claims should be made about masculinity, facial hair, and male desirability to women. However, research like this is important in highlighting how human grooming decisions are driven by much more than fashion trends. Sociobiological, economic, and ecological factors all play a part in the way we choose to present ourselves.</p>
Dominique Crenn, the only female chef in America with three Michelin stars, joins Big Think Live.
Having been exposed to mavericks in the French culinary world at a young age, three-star Michelin chef Dominique Crenn made it her mission to cook in a way that is not only delicious and elegant, but also expressive, memorable, and true to her experience.
New experiments find weird quantum activity in supercold gas.
Quantum Mechanics, Onions, and a Theory of Everything<span style="display:block;position:relative;padding-top:56.25%;" class="rm-shortcode" data-rm-shortcode-id="036ae7b8dd661df2d125a3421a0299ba"><iframe type="lazy-iframe" data-runner-src="https://www.youtube.com/embed/bcVruA0AJ-o?rel=0" width="100%" height="auto" frameborder="0" scrolling="no" style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe></span>
Researchers say that moral self-licensing occurs "because good deeds make people feel secure in their moral self-regard."
Books about race and anti-racism have dominated bestseller lists in the past few months, bringing to prominence authors including Ibram Kendi, Ijeoma Oluo, Reni Eddo-Lodge, and Robin DiAngelo.