Collective intelligence out-diagnoses even professionals
The Human Diagnosis Project project is building the world's "open medical intelligence" system.
- The Human Diagnosis Project can develop medical diagnoses with startling accuracy.
- The platform combines the knowledge of medical professionals and artifical intelligence.
- The goal of the project is to provide open, readily available high-level guidance and training to health care professionals across the globe.
The world-class Mayo Clinic is often the place patients go for a second opinion on a medical diagnosis. It's a good thing they do. According to a report issued by the clinic in 2017, 88 percent of them return home with either a completely different diagnosis or a significantly altered one. Only 12 percent receive confirmation of their doctors' original conclusions.
It's hard to overstate the life-and-death importance of medical misdiagnoses, and with all the artificial intelligence and data collection tools out there, you'd think there might be a way to improve on these statistics. This said, the goal of the Human Diagnosis project, or "Human Dx," (a triple pun their site explains) is to create the world's open medical intelligence system, a "collective intelligence" that can produce vastly improved diagnostic accuracy.
In early March, JAMA published the results of an experiment conducted by Human Dx in cooperation with Harvard, and the results were impressive. Where 54 individual human medical specialists correctly diagnosed 156 test cases 66.3 percent of the time, collective intelligence achieved an 85.5 percent accuracy rate. Nine medical professionals contributed to the collective intelligence conclusions.
Human Dx founder Jayanth Komarneni tells Big Think that, "We can get numbers in the 97th, 98th [percentile], and even — if we have sufficiently large numbers of participants — we can get to super intelligent results. That means that it outperforms 100 percent of individual participants."
About Human Dx
The Human Dx project is a partnership between the social, public, and private sectors — in the U.S., it's a 501 (c)(3) not-for-profit/public-benefit corporation. According to Komarneni, Human Dx's business model is as free of cost to users as possible while still generating enough income to be self-sustaining. There are now nearly 20,000 medical professionals in almost 80 countries contributing. Among Human Dx's partners are, as the company states: the American Medical Association, the Association of American Medical Colleges, American Board of Medical Specialties, and the American Board of Internal Medicine. They're also working in collaboration with researchers at Harvard, Johns Hopkins., University of California San Francisco, Berkeley, and MIT.
While diagnoses produced by Human Dx do bring together the opinions of multiple medical professionals, it's far from a simple voting system. It incorporates its own massive data set, machine learning, and artificial intelligence in addition to the input from medical professionals to develop its diagnoses. In designing their collective intelligence, says Komarneni, Human Dx had to first re-think the idea of open intelligence itself.
"We believe that open intelligence is the third form of open knowledge," he explains. The first was open source-protocols such as those on which the internet is based, as well as operating systems such as Linux. These protocols enabled the second form, open content: Wikipedia, data libraries, and so on. Open intelligence combines the first two: "And when you think about A.I. in the context of software," says Komarneni, "it really is code which is smartly delivering content to you based on what you put into the system."
The importance of open intelligence is that without it being available at low cost or free, the cost of A.I. is going to be so prohibitive that it'll "exacerbate, as opposed to close, income, health, and other disparities in society," warns Komarneni. Nowhere will the ramification be more serious than in health care, since "there is nothing we care more about than the well-being of the people we love and ourselves."
How Human Dx collective intelligence works
Collective intelligence in the Human Dx project is not unlike a panel of participants, when are referred to as "agents." Some of these are medical professionals, but they may also include the outputs of other systems. For example, Komarneni mentions that it's entirely possible IBM's Watson could be one of these agents, or even a data set from the National Institutes of Health.
Of course, individual agents, even the human participants, express themselves in their own ways — is a lump "blue" or "blueberry-colored," for example — not to mention that contributions from some agents such as A.I. or datasets may be in the form of raw data. Before any meaningful synthesis of all these opinions can be performed, the first step is to convert them all into a common language of some sort. Human Dx's AI uses natural language processing, text prediction, and medical ontologies to derive these translations as the process's first step.
Human Dx establishes the capability, or CQ ("clinical quotient"), of each agent. To do this they rank agents' skills using test cases with known diagnoses, including "some of the most wickedly complex cases," says Komarneni. This allows Human Dx to determine how accurate agents' diagnoses can be expected to be, and how heavily they should be weighted against other participants' contributions in solving the current case.
A.I. joins the panel
At this point, the agents' inputs are synthesized to derive the most likely diagnosis, and this is combined in an A.I. model with all of the aggregated case data that's ever been captured by Human Dx — interactions in the "tens of millions" — including how "lots of other participants over many other cases have solved these cases." This A.I. model then joins the panel in arriving at the final diagnosis.
"And those [agents] combined," says Komarneni, "are how we can get to results that outperform the vast majority of individual participants."
The Harvard and Johns Hopkins studies
The Harvard study published in JAMA is the first public demonstration of the Human Dx system as a diagnostic tool. Working with an international cohort of medical students and professionals, the results were unquestionably amazing. There were 2069 users working 1572 cases — again, these were cases with known correct answers — from the Human Dx data set. About 60 percent of the participants were residents or fellows, 20 percent were attending physicians, and another 20 percent were medical students. In the study, as more medical professionals were added to the collective intelligence "panel," up to nine individuals, its accuracy consistently rose. Physicians who weren't specialists in their test-case areas achieved just a 62.5 percent accuracy score.
A previous study published in JAMA in January, and done in cooperation with Johns Hopkins, looked at Human Dx as an automatic platform for assessing the diagnostic abilities of health care professionals and students. That the scores of participants looking at 11,023 case simulations were consistent with their training level shows, in Komarneni words, "that we provided a valid, quantitative, scalable measure of medical reasoning." While he admits this doesn't sound like a big deal, it is, since it offers a far more accurate and scalable option to current multiple-choice assessments, which have been shown to correspond poorly to real-world diagnostic skills.
The future of health care and Human Dx
Komarneni says that there are basically only two ways to provide global universal health care, a pressing need since, "Almost half the world has no access to essential health services." One way, he says, would be to create a God-like A.I. system to provide health care to everyone, but, "We know that's not going to happen." God-like AI is just too hard, potentially requiring having to know everything about a patient from the tiniest details — say, the quantum behavior of electrons in mitochondria — to the huge, as in the kind of environment a patient lived in as a child.
In addition, Komarneni says, "In a world where data is locked up in many disparate silos, there isn't going to be a single collective agent. There's going to be a collective of many intelligent agents, both human and machine. The key is how do you integrate intelligence into larger buckets of intelligence than can solve the world's hardest problems."
This is where the Human Dx project, and the second approach, comes in. It actually has two components:
- The first is the expansion of existing medical professionals' diagnostic accuracy skills by providing them access to the Human Dx platform and its collective intelligence as a diagnostic tool.
- The second is helping to train new professionals, and Human Dx Training is already offering this on the Human Dx site.
For those concerned with privacy in a system such as Human Dx, Komarneni says it'll be a non-issue, explaining with an example. When two people converse, "We don't have access to the underlying data of each others' minds. We're agents that are interacting with each other to gain relevant and useful information from each other." Similarly, Human Dx's system of interacting agents doesn't require the exposure of patients' personal data. What's shared with Human Dx are the conclusions agents draw from that data, not the data itself. In the case of a dataset operating as an agent, the data would be anonymized.
Human Dx's interest in all this is developing a platform it hopes others find uses for. "We believe we're just building the enabling technology that many other stakeholders could use." As examples, Komarneni imagines, "The VA could implement their own version of this. Kaiser Permanente could implement their own version. Employers could contract with us or with their own insurers. You could even also have individual and group practices use Human Dx software to serve patients directly."
Human Dx is currently looking at ways to open up as much of the project for non-professionals as possible, and they've already made a start: On their home page is a diagnosis cloud — mouse over the various blue bubbles to see different conditions, and then click for further details. In addition, just beneath the cloud is a search field with which you can look up diseases and symptoms.
Join Radiolab's Latif Nasser at 1pm ET on Monday as he chats with Malcolm Gladwell live on Big Think.
A vertical map might better represent a world dominated by China and determined by shipping routes across the iceless Arctic.
- Europe has dominated cartography for so long that its central place on the world map seems normal.
- However, as the economic centre of gravity shifts east and the climate warms up, tomorrow's map may be very different.
- Focusing on both China and Arctic shipping lanes, this vertical representation could be the world map of the future.
The world, but not as we know it<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMDU1Nzg1NS9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYzNTkwMjIyNn0.qmQfwUdjQka8JX6q4KGANagleiuucpWay5ytMenZxUU/img.jpg?width=980" id="b95e4" class="rm-shortcode" data-rm-shortcode-id="ac088ec55c0585a93a9a310faab9a4c7" data-rm-shortcode-name="rebelmouse-image" />
A Chinese 'vertical world map,' showing the world in a different perspective from the one we're used to.
Image: Prior Probability<p>Europe is tucked away in a corner, an appendage of Asia dwarfed by neighboring Africa. North America is stood on its head, facing the rest of the world from the top of the map — cut off from South America, which cuts a solitary figure at the bottom. Africa is justifiably huge, but equally eccentric. </p><p>The eye scouts elsewhere for a place to land: not the Indian Ocean, which dominates the middle of the map, but some terra firma. Antarctica and Australia are too small, mere stepping stones for the land mass of Asia. Ultimately our gaze is drawn toward China, the lynchpin of this unfamiliar world. </p><p>Managing to leave both poles intact, this "vertical" world map is about as far away as you can get from the classic Mercator projection – which slices up both, giving center stage to a puffed-up Europe. Perhaps this new map will become more familiar soon: It may do more justice to the world of the near future, dominated by China and determined by shipping routes across the iceless Arctic. <br></p>
China's 'ten-dash line'<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMDU1Nzg1Ni9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY1NTI4MzQyNn0.sBe0oFTif4Jef1vWh1kAnUylU_QMPXT5xQjm-5aA3sA/img.jpg?width=980" id="a3b81" class="rm-shortcode" data-rm-shortcode-id="80fc6e4f5c9c1c978f698be2c8de5484" data-rm-shortcode-name="rebelmouse-image" />
'China without any part left out': includes Taiwan and the islands and atolls in the South China Sea, surrounded by a ten-dash line
Image: Global Times<p>While there's no indication that this map represents the Chinese government's "official" worldview, it is no secret that China has a thing with maps – and more specifically, the country's representation on them. </p><p>In China, the country's current economic success is seen as a redress of the unequal treatment meted out by western superpowers in the 19th century. China's world dominance is a return to a more natural state of world affairs, many feel. Cartographic rectifications are a symbolically significant corollary of that sentiment.</p><p><a href="https://www.citylab.com/equity/2015/12/china-cracks-down-on-politcally-incorrect-maps/421032/" target="_blank">Fines are regularly imposed</a> on companies – domestic and foreign – that fail to represent China to the fullest extent of its external borders, disputed though they may be by others (e.g. India, Taiwan and any of the countries with claims overlapping China's in the South China Sea). But the People's Republic's cartographic obsession doesn't end at China's territory itself. It also includes the country's position on the world map. <br></p>
The Kingdom at the Middle of the World<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMDU1Nzg2MS9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYyOTkwODEzMX0.SGrAZBH6iJVggFYSaIahzv9GvfEh17y1SwUNINbVicQ/img.jpg?width=980" id="1774c" class="rm-shortcode" data-rm-shortcode-id="99790d80a909d17a948f7c5d463d7d98" data-rm-shortcode-name="rebelmouse-image" />
Early Japanese color copy of Ricci's world map
Image: public domain<p>China's name for itself is <em>Zhōngguó</em>, which means 'Central State' or 'Middle Kingdom', reflecting its ancient self-image as the civilized center (<em>Huá</em>) of the world, with wild tribes (<em>Yí</em>) at the edge. That view is not unique to China. Vietnam, for example, at certain times also styled itself as the "central state" (<em>Trung Quóc</em>) – considering the Chinese in turn as the uncouth outsiders.</p><p>It may be surprising to recall, but Europeans themselves once considered their own continent a relative backwater, viewing Jerusalem as the true center of the world. That changed with the Age of Discovery, which placed Europe at the center of an ever-expanding world. Maps reflected that worldview, and largely continue to do so. That's why today's standard world map still has Europe at its center – with China off toward the periphery on the map's right-hand side. </p><p>The most notable feature of the very first major modern world map produced in China, the <em>Kunyu Wanguo Quantu</em> (1602), is that it places China firmly at the center of the world. Produced for the Chinese emperor by Jesuit missionary Matteo Ricci, it was the first map ever to combine that perspective with modern western knowledge: it was the first Chinese map to show the Americas, for instance. </p><p>That representation may not have taken off elsewhere, but it will be instantly recognizable to Chinese students, as it's the standard format for world maps in China's schools today.<br></p>
America on its head<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yMDU1Nzg2My9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTYwMzQ5NTc0MH0.EqadI2Yp-2dPwi3VccFZelIDK4V9t0ZOfTfHjdB6wVw/img.jpg?width=980" id="97104" class="rm-shortcode" data-rm-shortcode-id="2b66e8de389b3d736bc28e019e445cd0" data-rm-shortcode-name="rebelmouse-image" />
Upside down you turn me: North America on its head, in Chinese characters
Image: Prior Probability<p>For those used to "classic" Eurocentric world maps, Europe's marginalization may come across as a bit of an upset. America's new position on the horizontal Chinese world map is less jarring: It merely moves from the left- to the right-hand side of the picture. But then there's this vertical world map, which deals a similar blow to the American land mass: divided in two and pushed to the upper and lower edges of the map.</p><p>Unfamiliar? Sure. Shocking? Perhaps. Wrong? Not really. First off, no world map is totally right, since it's mathematically impossible to transfer the surface of a three-dimensional object onto a flat surface without some distortion. And since the world is a globe, where you center that map is a matter of purely subjective choice.<br></p><p>Those choices have historical reasons. Mercator's map was not specifically designed to put an inflated Europe at the center of the world. That was just a side effect; its main purpose was to aid shipping: Straight lines on the map correspond to straight lines sailed on the seas.</p>
By 2050, a completely melted Arctic could enable the Transpolar Passage, shortening trade routes between Asia and Europe and boosting business for Alaskan ports like Nome and Dutch Harbor.
Image: The Maritime Executive<p>The vertical world map, showing the relative proximity of China (and the rest of Asia) to Europe and (even the East Coast of) North America, has a similarly maritime <em>raison d'être</em>, or it will have by mid-century. <a href="https://www.maritime-executive.com/editorials/the-arctic-shipping-route-no-one-s-talking-about" target="_blank" rel="noopener noreferrer">Experts project</a> that by 2050 (if not sooner), the Arctic will be sufficiently ice-free to enable the so-called Transpolar Passage, i.e. shipping straight across the North Pole. </p><p>That would shave more than three weeks off a traditional sea voyage between Europe and Asia, via the Suez Canal – and even be significantly faster than other northern alternatives like the Northwest Passage (via Canada) or the Northern Sea Route (hugging the Siberian coast). Since ships would not need to go through locks or pass over shallow waters, it would also remove current restrictions on tonnage per ship. <br></p><p>The only country seriously preparing for such a future: China. None of the other Arctic powers is giving the Transpolar route any strategic thought. On the other hand, China's Arctic Policy document, released in January 2018, already matter-of-factly refers to the Transpolar route as the 'Central Passage' – one of several 'Polar Silk Roads' that China seems to want to develop. And they already have the world map to go with it.</p>
The Labour Economics study suggests two potential reasons for the increase: corruption and increased capacity.
Cool hand rebuke<img type="lazy-image" data-runner-src="https://assets.rebelmouse.io/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpbWFnZSI6Imh0dHBzOi8vYXNzZXRzLnJibC5tcy8yNDQyMTIyNy9vcmlnaW4uanBnIiwiZXhwaXJlc19hdCI6MTY0NjY1NTYyOH0.0MCPKN3If94mYCNf3mMNrnTvJXjXN_bKLhgk9203EXk/img.jpg?width=917&coordinates=0%2C0%2C0%2C0&height=453" id="1627b" class="rm-shortcode" data-rm-shortcode-id="6d76421ba1ea0de4b09956b97e80c384" data-rm-shortcode-name="rebelmouse-image" />
A chart showing prison population rates (per 100,000 people) in 2018. The United States has the highest rate of incarceration in the world.
Who profits with for-profit prisons?<span style="display:block;position:relative;padding-top:56.25%;" class="rm-shortcode" data-rm-shortcode-id="97ac37e6c7f6f22ec130ea2d56871701"><iframe type="lazy-iframe" data-runner-src="https://www.youtube.com/embed/dB78NV2WpWc?rel=0" width="100%" height="auto" frameborder="0" scrolling="no" style="position:absolute;top:0;left:0;width:100%;height:100%;"></iframe></span><p>The Labour Economics study suggests that privately-run prisons do convicts a few favors at the moment of sentencing. However, proponents of private prisons often point to other benefits when making their case. Specifically, they argue that private prisons reduce operating costs, stimulate innovation in the correctional system, and reduce recidivism—the rate at which released prisoners are rearrested and return to prison.</p><p>In regard to recidivism, the research is mixed. <a href="https://journals.sagepub.com/doi/abs/10.1177/0011128799045001002" target="_blank">One study</a> compared roughly 400 former prisoners from Florida, 200 released from private prisons and 200 from state-run facilities. It found the private-prison cohort maintained lower rates of recidivism. However, <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-9133.2005.00006.x" target="_blank" rel="noopener noreferrer">another Florida study</a> found no significant rate differences. And two other studies—one from <a href="https://journals.sagepub.com/doi/abs/10.1177/0011128799045001002" target="_blank" rel="noopener noreferrer">Oklahoma</a> and another out of <a href="https://journals.sagepub.com/doi/abs/10.1177/0734016813478823" target="_blank" rel="noopener noreferrer">Minnesota</a>, both comparing much larger cohorts than the first Florida study— found that prisoners leaving private prisons had a greater risk of recidivism.</p><p>The research is also inconclusive regarding cost savings. <a href="https://www.hamiltonproject.org/assets/files/economics_of_private_prisons.pdf" target="_blank" rel="noopener noreferrer">A Hamilton Project analysis</a> noted that such comparisons are difficult because private prisons, like all private companies, are not required to release operational details. In comparing what studies were available, the authors estimate the costs to be comparable and that "in practice the primary mechanism for cost saving in private prisons is lower salaries for correctional officers"—about $7,000 less than their public peers. They add that competition-driven innovation is lacking as the three largest firms control nearly the entire market.</p><p>"We aren't saying private prisons are bad," Galinato said. "But states need to be careful with them. If your state has previous and regular issues with corruption, I wouldn't be surprised to see laws being more skewed to give longer sentences, for example. If the goal is to reduce the number of incarcerated individuals, increasing the number of private prisons may not be the way to go."</p>
What exactly does "questions are the new answers" mean?
- Traditionally, intelligence has been viewed as having all the answers. When it comes to being innovative and forward-thinking, it turns out that being able to ask the right questions is an equally valuable skill.
- The difference between the right and wrong questions is not simply in the level of difficulty. In this video, geobiologist Hope Jahren, journalist Warren Berger, experimental philosopher Jonathon Keats, and investor Tim Ferriss discuss the power of creativity and the merit in asking naive and even "dumb" questions.
- "Very often the dumb question that is sitting right there that no one seems to be asking is the smartest question you can ask," Ferriss says, adding that "not only is it the smartest, most incisive, but if you want to ask it and you're reasonably smart, I guarantee you there are other people who want to ask it but are just embarrassed to do so."