from the world's big
Crawling at Top Speed
Josh Cohen is the Senior Business Product Manager at Google News, where he manages global product strategy, marketing, and publisher outreach. He was previously vice president of business development for Reuters Media and director of business development for SmartMoney.com.
Question: What exactly is a search algorithm, and how does Google News’s algorithm differ from Google’s?
Josh Cohen: Sure. Yeah, it does. I mean, I think the mission behind Google News is really the same thing that we're trying to do with Web search overall, for the Google mission statement, of organizing the world's information and make it universally accessible and useful. That's what we're trying to do for news as well. We're just trying to sort of apply that to -- in every single different language, every single country, every single source, and ultimately down to every single story. So the process of putting together Google News, I think the easiest way to think about it is in sort of three easy steps of how you get to Google News. So first is the crawl process, where we go off and actually crawl the Web looking for news information. There are -- technically it's the same process as Web search does. There's a Google bot or crawler that will go off and index information.
There are two key differences, though, in the crawling process between how news works and how Web search works. One is that Google News is a closed index, versus an open index for Web search. And what that means is for Web search, if you put up a personal blog, you start a Web site tomorrow, at some point Web search is going to go and pick up that information and index it. Google News is focused just on news sites. So it's a smaller set of sites. So it's -- we're really just looking for news sites, so it's a smaller set of sources in our index. And the second is the speed with which we'll crawl it. So Web search has hundreds and millions of different URLs that they're trying to crawl. And for Google News, we have a smaller set of that, but it's also the speed with which we crawl; it has to be much, much faster. So Web search, you'll have a wide variety of crawl times, because if it's a personal blog, maybe you update it once a month or so. You don't need to crawl it every single second. For news, since we're crawling these news sites, speed just has to be -- it's a matter of minutes. It really has to be as close to real time as possible. So that's one key difference in sort of the crawl part of it.
The second part of this whole clustering, which is again one of the key innovations that Krishna applied to news, which is taking all these different sources and trying to cluster them into similar types of storylines. So it can be trying to understand, obviously, the same languages, same editions, same topics, and those same story clusters together. And then the ranking part of it is completely different from Web search. And you'll hear things -- when people talk about Google Web search, you'll hear about page rank, and inbound links and outbound links.
And it's really a very different process that we're trying to do for Google News. So some of the things that we're looking at is obviously timeliness -- that matters for a news site; location: is it a local source reporting on a local story, doing original reporting? That's something else we're going to take a look at, as well as the nature of the source as well. So it's not just -- you know, it's not a human coming in and saying, well, this is a good source or a bad source, but rather looking at a number of different signals to try and determine the quality of a given source. Are they producing the original content? You know, how do users respond to the brand? Is there a sort of certain value when they see that link, and sort of -- not just the most clicks, because that becomes somewhat self-fulfilling, but rather trying to discern the user's trust for a given source. So those are all things that are really unique to news as opposed to how Web search works.
An inside look at the Google News search algorithm and how it differs from its more famous parent.
Sallie Krawcheck and Bob Kulhan will be talking money, jobs, and how the pandemic will disproportionally affect women's finances.
Vaccines find more success in development than any other kind of drug, but have been relatively neglected in recent decades.
Vaccines are more likely to get through clinical trials than any other type of drug — but have been given relatively little pharmaceutical industry support during the last two decades, according to a new study by MIT scholars.
An article in Journal of Bioethical Inquiry raises questions about the goal of these advocacy groups.
- Two-thirds of American consumer advocacy groups are funded by pharmaceutical companies.
- The authors of an article in Journal of Bioethical Inquiry say this compromises their advocacy.
- Groups like the National Alliance on Mental Illness act more like lobbyists than patient advocates.
The Corruption That Brought Prozac to Market — Robert Whitaker, Journalist<span style="display:block;position:relative;padding-top:56.25%;" class="rm-shortcode" data-rm-shortcode-id="bea9cff2b25efc18b663a011a679ba16"><iframe type="lazy-iframe" data-runner-src="https://www.youtube.com/embed/UyaJExxFPAE?rel=0" width="100%" height="auto" frameborder="0" scrolling="no" style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe></span><p>Consumer-oriented groups gained steam over the ensuing decades. Their efforts helped inspire the 1938 Food, Drug, and Cosmetic Act after over 100 people (mostly children) died from a sanctioned drug, Sulfanilamide. If not for the hard work of these advocates, this case might have been overlooked.</p><p>Early efforts also focused on the food industry, which was increasingly using chemical preservatives. The origin of Consumer Reports can be found in the consumer advocacy movement. Both the food and drug industries were getting a free pass to experiment on citizens with few repercussions.</p><p>These movements provided a social foundation for important advocacy work in the second half of the century. Female-led groups evolved to focus on women's reproductive rights, AIDS, and mental health. As the authors write, these groups struck a balance between working <em>with</em> and <em>against</em> current trends. Sometimes you need to craft legislation with officials; at other times, you have to rage against the machine with everything you've got. </p><p>Advocacy marked an important turning point in public health (and culture in general). These groups were tired of placating to a medical model that treated the male body as the standard. This wasn't limited to anatomy. As I <a href="https://bigthink.com/coronavirus/pandemic-warnings-rp-eddy" target="_self">wrote about last week</a>, a high-profile 1970s-era conference about the role of women on Wall St featured no women on stage. You can imagine what reproductive health looked like during that time. </p><p>Advocacy groups made real impact in public health. Then the money began pouring in. </p><p style="margin-left: 20px;">"These groups were funded largely by individual donations with some foundation support, but in the late 1980s, newer women's health groups moved to professionalize, effectively splitting the women's health movement."</p><p>A number of groups resist corporate ties to this day, such as the National Women's Heath Network and Breast Cancer Action. Too often, however, groups argue that their existence depends on corporate funding. This can lead to uncomfortable compromises. </p><p>An estimated two-thirds of patient advocacy groups in America accept funds from the pharmaceutical industry. Pharma companies gave <a href="https://link.springer.com/content/pdf/10.1007/s11673-019-09956-8.pdf" target="_blank">at least $116 million</a> to such groups in 2015 alone.</p><p>For example, over a three-year period, the National Alliance on Mental Illness (NAMI), which was founded by two mothers whose sons suffered from schizophrenia, received nearly $12 million from 18 pharmaceutical companies. The largest donor was Prozac manufacturer, Eli Lilly. By 2008, three-quarters of NAMI's budget was funded by the pharmaceutical industry. It gets worse:</p><p style="margin-left: 20px;">"An Eli Lilly executive was even 'on loan' to NAMI, paid by Eli Lilly, while he worked out of the NAMI office on 'strategic planning.'"</p>
A customer waiting for his medication at the Headache Bar in a pharmacy in Sydney, Australia. Among the items on sale are 'Paigees with Chlorophyll' and Alka Seltzer on tap.
Photo by Dennis Rowe/BIPs/Getty Images<p>This influx of cash skews public understanding of drugs. It also influences advocates to overlook real problems caused by pharmaceutical interventions, especially when it comes to mental health.<br></p><p>For a real-world example, consider how Xanax came to market. As journalist Robert Whitaker <a href="https://www.youtube.com/watch?v=2e829xdb4AA" target="_blank">explains</a>, an <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1463502/?page=1" target="_blank">initial study</a> was conducted to determine efficacy in treating panic attacks. After four weeks, Xanax was outperforming placebo, which is common with benzodiazepines over short-term usage. But it wasn't a four-week study; it was a 14-week study.</p><p>At the end of eight weeks, there was no difference in efficacy between Xanax and placebo.</p><p>At the conclusion of the study after 14 weeks, the placebo outperformed Xanax. By a lot.</p><p>Why is Xanax still prescribed for panic attacks? Because the pharmaceutical company, Upjohn, only published the four-week data. The 14-week data was not in its favor. Nearly forty years later, over <a href="https://www.statista.com/statistics/781816/alprazolam-sodium-prescriptions-number-in-the-us/" target="_blank">25 million</a> Americans receive a prescription despite its <a href="https://drugabuse.com/xanax/effects-use/" target="_blank">long list</a> of side effects and addictive profile. </p><p>As the authors note, many consumers are not aware of how advocacy groups are funded.</p><p style="margin-left: 20px;">"An international study of groups in the United States, United Kingdom, Australia, Canada, and South Africa found that the extent of relationships with industry was inadequately disclosed in websites that addressed ten health conditions: cancer, heart disease, diabetes, asthma, cystic fibrosis, epilepsy, depression, Parkinson's disease, osteoporosis, and rheumatoid arthritis."</p><p>That's a tangled web of relationships. Pharmaceutical industry funding negatively impacts the work advocacy groups should be focused on: protecting us. NAMI, for example, claims that as a "natural ally" to the pharmaceutical industry, it helps consumers access "all scientifically proven treatments." When the industry ignores evidence of long-term damage caused by its treatments, you have to wonder what's being advocated. </p><p>Although, as the authors conclude, that question is easy to answer. </p><p style="margin-left: 20px;">"Instead of drawing insights from patient experience to set organizational agendas and challenge industry agendas, today's groups are silent on high prices and drug harms, oppose efforts to regulate these basic rights, and demand access to drugs that challenge the safety and effectiveness."</p><p><span></span>--</p><p><em>Stay in touch with Derek on <a href="http://www.twitter.com/derekberes" target="_blank">Twitter</a>, <a href="https://www.facebook.com/DerekBeresdotcom" target="_blank">Facebook</a> and <a href="https://derekberes.substack.com/" target="_blank">Substack</a>. His next book is</em> "<em>Hero's Dose: The Case For Psychedelics in Ritual and Therapy."</em></p>
Health officials in China reported that a man was infected with bubonic plague, the infectious disease that caused the Black Death.
- The case was reported in the city of Bayannur, which has issued a level-three plague prevention warning.
- Modern antibiotics can effectively treat bubonic plague, which spreads mainly by fleas.
- Chinese health officials are also monitoring a newly discovered type of swine flu that has the potential to develop into a pandemic virus.
Bacteria under microscope
needpix.com<p>Today, bubonic plague can be treated effectively with antibiotics.</p><p style="margin-left: 20px;">"Unlike in the 14th century, we now have an understanding of how this disease is transmitted," Dr. Shanthi Kappagoda, an infectious disease physician at Stanford Health Care, told <a href="https://www.healthline.com/health-news/seriously-dont-worry-about-the-plague#Heres-how-the-plague-spreads" target="_blank">Healthline</a>. "We know how to prevent it — avoid handling sick or dead animals in areas where there is transmission. We are also able to treat patients who are infected with effective antibiotics, and can give antibiotics to people who may have been exposed to the bacteria [and] prevent them [from] getting sick."</p>
This plague patient is displaying a swollen, ruptured inguinal lymph node, or buboe.
Centers for Disease Control and Prevention<p>Still, hundreds of people develop bubonic plague every year. In the U.S., a handful of cases occur annually, particularly in New Mexico, Arizona and Colorado, <a href="https://www.cdc.gov/plague/faq/index.html" target="_blank">where habitats allow the bacteria to spread more easily among wild rodent populations</a>. But these cases are very rare, mainly because you need to be in close contact with rodents in order to get infected. And though plague can spread from human to human, this <a href="https://www.healthline.com/health-news/seriously-dont-worry-about-the-plague#Heres-how-the-plague-spreads" target="_blank">only occurs with pneumonic plague</a>, and transmission is also rare.</p>
A new swine flu in China<p>Last week, researchers in China also reported another public health concern: a new virus that has "all the essential hallmarks" of a pandemic virus.<br></p><p>In a paper published in the <a href="https://www.pnas.org/content/early/2020/06/23/1921186117" target="_blank">Proceedings of the National Academy of Sciences</a>, researchers say the virus was discovered in pigs in China, and it descended from the H1N1 virus, commonly called "swine flu." That virus was able to transmit from human to human, and it killed an estimated 151,700 to 575,400 people worldwide from 2009 to 2010, according to the Centers for Disease Control and Prevention.</p>There's no evidence showing that the new virus can spread from person to person. But the researchers did find that 10 percent of swine workers had been infected by the virus, called G4 reassortant EA H1N1. This level of infectivity raises concerns, because it "greatly enhances the opportunity for virus adaptation in humans and raises concerns for the possible generation of pandemic viruses," the researchers wrote.
Want help raising your kids? Spend more time at church, says new study.
- Religious people tend to have more children than secular people, but why remains unknown.
- A new study suggests that the social circles provided by regular church going make raising kids easier.
- Conversely, having a large secular social group made women less likely to have children.