What happens when an astrophysicist puts ChatGPT to the test?
- ChatGPT is a remarkable chatbot whose confidence, fluency, and ability to compose its own sentences, paragraphs, poems, and more is wildly impressive.
- But this confidence can be misleading, as it succumbs to many misconceptions that are widespread among the general populace, even though the experts know better.
- Can an astrophysicist get ChatGPT to learn and assimilate new information and give correct answers where it gave confident but erroneous ones previously?
All of us, whenever we seek to understand something more deeply, run into an awkward situation: where we think we understand how something works, only to discover that we ourselves are misinformed. Sometimes it’s only about trivial matters that don’t impact our ability to accomplish what’s important in our lives, but at other times it’s absolutely essential that we put in the time and effort to improve our understanding. This involves not only learning what’s true, but learning why what we thought was true was, in fact, false, and how to catch ourselves from slipping up again in precisely the same fashion.
This is something that every budding and aspiring scientist runs into frequently along their educational journey: discovering our own misconceptions. Those of us who go through the hard work of learning how we led ourselves astray, what the actual truth is instead, and how to get it right from now on often indeed go on to become successful in our careers; those who continue to insist, incorrectly, that they were right all along rarely do.
It’s challenging enough to convince a room full of students (or humans in general) to replace their incorrect misconceptions with actual, correct knowledge, but how good is the world’s most advanced AI chatbot, ChatGPT, at the same task? I gave the chatbot four misconceptions — one trivial, one easy, one intermediate, and one advanced — to try and find out.
Trivial: Edwin Hubble, in his early life, was an exceptional boxer.
Background: Edwin Hubble was a 20th century astronomer famed for discovering the expansion of the Universe, measuring the distances to galaxies beyond the Milky Way, and pioneering the redshift-distance relationship. Decades after his death, the Hubble Space Telescope was launched bearing his name; he remains one of astronomy’s influential figures. In his early life, however, Hubble was a phenomenal athlete, having played high-level basketball, football, track, and boxing. As a young man in Chicago — where, as luck would have it, I was also an undergraduate some 90 years later — he was a “Gold Gloves” boxer, and allegedly even knocked out the German champion at one point. But is this just an urban legend I heard? I decided to ask ChatGPT about this.
ES: How good of a boxer was Edwin Hubble?
ChatGPT: Edwin Hubble was an American astronomer who is known for his contributions to the field of astronomy, including the expansion of the universe. He was not known for being a boxer. Hubble was a talented athlete in his youth, but he is not known to have pursued boxing as a career or to have had any notable achievements as a boxer. Instead, he focused on his studies and went on to become one of the most important astronomers of the 20th century.
ES: What sports did Edwin Hubble play?
ChatGPT: Edwin Hubble was a talented athlete in his youth, and he is known to have played several sports. According to biographies of Hubble, he was particularly skilled at football and boxing. He played football at the University of Chicago and is said to have been an excellent boxer. In addition to football and boxing, he is also known to have enjoyed playing tennis and to have been an avid golfer.
ES: In his youth, what sorts of boxing achievements did Edwin Hubble have? I have read that he was a gold gloves boxer with very impressive punching power, and even knocked out a European heavyweight champion at one point. Do you have any knowledge of that event?
ChatGPT: Edwin Hubble did indeed have a successful amateur boxing career. He was awarded the Illinois State Golden Gloves championship in 1919, and he also won the Central A.A.U. Light Heavyweight Championship in 1920. He was known for his powerful punching, and it is said that he once knocked out a European heavyweight champion in a sparring match. Unfortunately, there is no record of the specific event or details.
Verdict: After an initial surprise — I had not expected ChatGPT to tell me “He is not known to have pursued boxing as a career or to have had any notable achievements as a boxer” — ChatGPT seems to have assimilated this knowledge correctly and was even able to pull up details I myself didn’t know. (And whose truth I cannot corroborate; it seems suspicious that Hubble, who graduated in 1910 and who became head of Mt. Wilson Observatory in California in 1917, would have been fighting as a 30-something year old man in Chicago in 1919 and 1920.) I’m very impressed, but also very suspicious.
Easy: Meteor showers are not caused by comet tails.
Background: Most of us, when we first learn about meteor showers, learn that they’re caused by comets. This is mostly true, as most are created by comets, although some are created by asteroids or other volatile-containing bodies. But we also learn — incorrectly — that they’re caused by the tails of comets, which is so prevalent of a myth that it’s even been repeated by NASA. It’s not the tails; the tails are blown away (and out of the Solar System) by the Sun. Instead, it’s small fragments of the comet’s nucleus that are broken apart by gravitation and heating, and stretched out along the comet’s (or asteroid’s) orbit. When Earth crosses that orbit, and the debris stream that exists along it in particular, that’s where meteor showers appear.
ES: What is the cause of meteor showers?
ChatGPT: Meteor showers are caused by streams of debris from comets or asteroids that collide with Earth’s atmosphere. As these particles enter the atmosphere, they heat up and create a brilliant streak of light, known as a meteor. When many of these meteors occur at the same time, they create a meteor shower.
ES: What creates debris streams from comets and asteroids?
ChatGPT: Debris streams from comets and asteroids are created through a variety of processes. In the case of comets, the debris streams are created as the comet orbits the sun and its icy surface begins to vaporize. This vaporization releases dust and other debris, which forms a long tail behind the comet as it moves through space. In the case of asteroids, debris streams can be created through collisions with other asteroids or the effects of solar radiation on the asteroid’s surface. These collisions and processes can create debris streams that can eventually intersect with Earth’s orbit and create a meteor shower.
ES: What is the difference between the debris streams in a comet’s tail and the debris streams from comets and asteroids that create meteor showers?
ChatGPT: The debris streams from comets and asteroids that create meteor showers are composed of small particles of dust and rock that are released from the comet or asteroid as it orbits the Sun. These particles enter Earth’s atmosphere and burn up, creating a spectacular display of shooting stars. On the other hand, the debris streams from a comet’s tail are composed of larger particles of dust and ice that are ejected from the nucleus of the comet as it approaches the Sun. These particles are pushed away from the Sun by the solar wind, forming a tail-like structure that can be seen as the comet moves away from the Sun.
Verdict: I don’t think it quite understands that the debris stream that gives rise to meteor showers is separate from and independent of the debris particles that give rise to comet (and sometimes asteroid) tails. The debris stream critical to meteor showers gets created when tidal forces and uneven heating cause the nucleus of the parent body to fragment and small particles get stretched out across the entire orbit of the comet or asteroid. Despite its assurances that it does, in fact, understand, ChatGPT’s responses indicate that it does not.
Intermediate: When will Earth’s final total solar eclipse occur?
Background: As the Earth-Moon system orbits the Sun, you’ll notice the Earth and Moon behave differently. While the Earth spins on its axis rapidly, showing its entire surface to the Moon and the Sun roughly once per day, the Moon is locked to the Earth: the same face always points toward us. As the Earth-Moon system orbits the Sun, the effects of tidal friction — gravitational interactions between all three bodies — cause the Earth’s rotation to slow (lengthening the day), the Moon to migrate away, and the orbital distance of Earth from the Sun to slightly increase. Over time, the Moon’s outward migration will cause the Moon to appear smaller in the sky, eventually leading to a situation, ~600 million years from now, when Earth no longer experiences total solar eclipses.
ES: At what point in the future will there no longer be total solar eclipses on Earth?
ChatGPT: [Long, unnecessary explanation of eclipses omitted.] It is not possible to predict exactly when total solar eclipses will occur in the future, as they are influenced by the complex gravitational interactions between the Sun, Moon, and Earth. However, it is likely that total solar eclipses will continue to occur on Earth for many millions of years. The Moon is slowly moving away from Earth at a rate of about 3.8 cm (1.5 inches) per year, but this process is very slow and it will take a very long time for the Moon to move far enough away from Earth to no longer be able to cause total solar eclipses.
In summary, total solar eclipses will continue to occur on Earth for many millions of years, but it is not possible to predict exactly when they will happen.
ES: Will there be any total solar eclipses one billion years from now?
ChatGPT: It is not possible to accurately predict what will happen on Earth one billion years from now, as it is a very long time in the future and many changes could occur over such a vast period of time. However, it is likely that total solar eclipses will continue to occur on Earth for many millions of years, as they are caused by the Moon’s orbit around Earth and the Moon is not moving away from Earth at a particularly rapid pace.
[Long, unnecessary explanation of eclipses omitted.] In summary, total solar eclipses will continue to occur on Earth for many millions of years, but it is not possible to predict exactly when they will happen.
ES: How far away will the Moon eventually get from Earth?
ChatGPT: It is difficult to predict exactly how far away the Moon will eventually get from the Earth, as this will depend on a number of factors such as the exact rate of tidal acceleration and any changes in the gravitational forces acting on the Moon. However, it is generally believed that the Moon will continue to move away from the Earth for at least the next few billion years. Based on current estimates, it is likely that the Moon will eventually move far enough away from the Earth that it will no longer be able to cause total solar eclipses. It is not clear exactly when this will happen, but it is likely to be a very long time in the future.
ES: Assuming there are no additional gravitational perturbations acting on the Earth-Moon-Sun system, the Moon will continue to migrate away from the Earth, making annular solar eclipses on Earth more common and total solar eclipses less common. Approximately 600 million years from now, Earth will experience its final total solar eclipse, as beyond that point, the Moon’s angular diameter will always be smaller than the Sun’s as viewed from Earth’s surface. With this in mind, will there be any total solar eclipses on Earth 1 billion years from now?
ChatGPT: You are correct that the Moon is slowly moving away from Earth at a rate of about 3.8 cm (1.5 inches) per year due to the gravitational interaction between the Earth and the Moon. This process, known as tidal acceleration, is caused by the gravitational attraction between the Earth and the Moon, which causes the Moon’s orbit to become slightly more elliptical over time. As the Moon’s orbit becomes more elliptical, the distance between the Moon and Earth can vary more over the course of its orbit, and the size of the Moon’s shadow can also vary more.
[Long, unnecessary explanation of eclipses omitted.]
It is estimated that total solar eclipses will continue to occur on Earth for at least the next 600 million years. After that time, the Moon’s angular diameter will always be smaller than the Sun’s as viewed from Earth’s surface, and total solar eclipses will no longer be possible. This means that it is unlikely that there will be any total solar eclipses on Earth one billion years from now.
In summary, total solar eclipses will become less common on Earth as the Moon moves further away from Earth, and it is unlikely that there will be any total solar eclipses on Earth one billion years from now. Annular solar eclipses, in which the Sun appears as a bright ring around the Moon’s silhouette, will become more common over time.
Verdict: It knows that the Moon is migrating away from the Earth, and it even knows the current rate, but it doesn’t know enough to put the pieces together (i.e., to either do the math or to find the sources that have done the math) to conclude when the last total solar eclipse will be. But it can repeat what you taught it back to you! There will come a point, approximately 600 million years from now, when the Moon is far away enough that even at lunar perigee, even when the Sun is at aphelion, the Moon’s disc is insufficiently large to completely cover the Sun’s as seen from any location on Earth.
Advanced: How did cosmic inflation set up and give rise to the hot Big Bang?
Background: The Universe is expanding (and cooling) today, and that implies that it was smaller, denser, and hotter in the past. If we extrapolate this back arbitrarily far, you might think to go back to a singularity: to when all the energy in the Universe is compressed into a single point in space and a single moment in time. But that’s no longer a good extrapolation, as our observations don’t match up with that picture. Instead, they match up with extrapolating that back to a hot, dense state, and then having a period of cosmic inflation precede it. During inflation, the Universe empties out and has energy not in matter or radiation, but in a field that’s intrinsic to space itself. It expands exponentially during this period, driving the Universe to a flat, uniform state with seed quantum fluctuations stretched across it. Although we don’t know how or when inflation began, we know that when inflation ends, this field dumps its energy into matter and radiation, changing how the Universe expands and corresponding to what we know as the start of the hot Big Bang.
ES: How did cosmic inflation give rise to the hot Big Bang?
ChatGPT: Cosmic inflation is a theory that explains the observed homogeneity and isotropy of the universe on large scales. It suggests that the universe underwent a rapid period of expansion in the very early stages of its history, during which the size of the universe increased by a factor of at least 10^30 (a 1 followed by 30 zeros) in a fraction of a second. This rapid expansion is thought to have occurred shortly after the Big Bang, which is the term used to describe the beginning of the universe as we know it.
[Long explanation of cosmic inflation, its history, and how it explains certain puzzles.]
In summary, cosmic inflation is a theory that explains the observed homogeneity and isotropy of the universe on large scales, and it suggests that the universe underwent a rapid period of expansion in the very early stages of its history. This rapid expansion is thought to have given rise to the hot Big Bang, which is the term used to describe the beginning of the universe as we know it.
ES: You said that inflation “occurred shortly after the Big Bang” and then later, you said that inflation “is thought to have given rise to the hot Big Bang,” and it cannot be both, unless you are using two different definitions of the term “Big Bang.” Did inflation occur before or after the Big Bang?
ChatGPT: You are correct that my previous statements may have been confusing. I apologize for any misunderstanding.
The Big Bang theory is the prevailing model that describes the origin and evolution of the universe. According to the Big Bang theory, the universe began as a singularity, a point of infinite density and temperature, about 13.8 billion years ago. From this singularity, the universe expanded and cooled, eventually giving rise to the matter and radiation that make up the universe today.
[Repeat of cosmic inflation explanation omitted.]
To clarify, the cosmic inflation theory suggests that the universe underwent a rapid period of expansion shortly after the Big Bang, which is the term used to describe the beginning of the universe as we know it. The cosmic inflation theory does not propose a new model for the origin and evolution of the universe, but rather it builds upon and expands upon the Big Bang theory by explaining the observed homogeneity and isotropy of the universe on large scales.
ES: You must understand that the Big Bang now has two meanings that are not congruent to each other. The hot Big Bang, which describes the early, hot, dense state of our Universe, is a certainty. However, it does not lead directly to a singularity, in fact it cannot do so. Cosmic inflation preceded the hot Big Bang and set it up. Cosmic inflation went on for at least a tiny fraction-of-a-second but could have gone on for much, much longer. We do not know how or under which conditions cosmic inflation began, or whether the Universe ultimately began from a singular state or not. The idea of the Big Bang as a singularity that gave rise to the Universe we know is still possible, but not universally accepted as necessary. Can you now understand the difference between the hot Big Bang, which is definitely correct, and the idea of the Big Bang as a singularity, which may not be relevant to our Universe?
ChatGPT: I apologize if my previous explanations were unclear. You are correct that the Big Bang theory describes the origin and evolution of the universe, and that it does not necessarily lead to a singularity.
The Big Bang theory is the prevailing model that describes the origin and evolution of the universe. According to the Big Bang theory, the universe began as a singularity, a point of infinite density and temperature, about 13.8 billion years ago. From this singularity, the universe expanded and cooled, eventually giving rise to the matter and radiation that make up the universe today.
[Repeat responses about inflation and the Big Bang omitted.]
In summary, the Big Bang theory describes the origin and evolution of the universe, and it suggests that the universe began as a singularity about 13.8 billion years ago. The term “Big Bang” is also often used to refer to the hot, dense state of the early universe, known as the “hot Big Bang,” which is thought to have occurred about 10^-32 seconds after the Big Bang. Cosmic inflation is a theory that explains the observed homogeneity and isotropy of the universe on large scales, and it suggests that the universe underwent a rapid period of expansion in the very early stages of its history.
Verdict: Okay, this was an advanced one for sure: advanced even for many astrophysicists. I can get annoyed that it says about the Big Bang first, “According to the Big Bang theory, the universe began as a singularity,” and then later says, “The term ‘Big Bang’ is also often used to refer to the hot, dense state of the early universe, known as the ‘hot Big Bang,’ which is thought to have occurred about 10^-32 seconds after the Big Bang” without any care to its self-inconsistency. But at least, at the end, it appears to assimilate the knowledge that one (old, now speculative) definition of the Big Bang refers to an initial singularity and that another (modern, thoroughly robust) definition refers to the conditions known as the hot Big Bang. But hey, it’s a chatbot designed to learn; perhaps someday soon, it’ll be ChatGPT explaining this to certain astrophysicists, not me!
