In a tantalizing article in The Conversation, Robert Nichol (University of Surrey) offers a look at where new physics might just be emerging in conjunction with the study of dark energy. Nichol is an astronomer and cosmologist deeply experienced in the kind of huge astronomical surveys that help us study mind-boggling questions like how much of the universe is made up of matter, dark matter or dark energy. We’ve assumed we had a pretty good idea of their proportions but a few issues do arise.

One of them seems particularly intriguing. Nichol’s article asks whether dark energy, regarded as a constant, may not actually vary over time. That’s quite a thought. The consensus over a universe made up of normal matter (5 percent), dark matter (25 percent) and dark energy (70 percent) came together early in our century, with dark energy taking the role of the cosmological constant Einstein once considered. Although he came to reject the idea, Einstein would doubtless take great interest in the work of observational cosmologists like Nichol, who keep refining the numbers to reduce errors.

Addendum: I hate typos, and thankfully a reader pointed out that in the penultimate sentence above, I had accidentally typed “with dark matter taking the role of the cosmological constant,” when of course it should be dark energy. Now corrected. Not enough caffeine in play this morning, evidently.

Image: The University of Surrey’s Nichol. Credit: University of Portsmouth.

At the heart of the investigation is the Dark Energy Survey, an international effort involving some 400 scientists in seven countries. The survey’s latest numbers, Nichol reports, are that 31.5 percent of the universe is either dark or normal matter, with an error on the measurement of a scant 3 percent. The question of how almost 70 percent of the universe could be in the form of something we can’t see, and something that is indeed not associated in any way with matter, is what Nichol calls “the biggest challenge to physics, even after 20 years of intense study.”

Remember that we first learned of the acceleration of the universe by studying Type Ia supernova (SNeIa) explosions. These occur in binary systems when a white dwarf star begins drawing off material from its companion, usually a red giant. Reaching the Chandrasekhar limit (approximately 1.4 times the mass of the Sun), the white dwarf releases vast amounts of energy, forming a ‘standard candle’ for cosmologists because the luminosity of these events is completely predictable. In other words, supernovae like these have an intrinsic brightness that can be compared to what is observed, making their distance measurable. Plug in the observed redshift and astronomers can use supernovae to make measurements on the rate of the universe’s expansion.

Image: The Hubble Ultra Deep Field, a view of nearly 10,000 galaxies, a reminder of the stunning scope of cosmological studies. The snapshot includes galaxies of various ages, sizes, shapes, and colours. The smallest, reddest galaxies, about 100, may be among the most distant known, existing when the universe was just 800 million years old. The nearest galaxies – the larger, brighter, well-defined spirals and ellipticals – thrived about 1 billion years ago, when the cosmos was 13 billion years old. This image required 800 exposures taken over the course of 400 Hubble orbits around Earth. The total amount of exposure time was 11.3 days, taken between Sept. 24, 2003 and Jan. 16, 2004. Credit: NASA, ESA, and S. Beckwith (STScI) and the HUDF Team.

The Dark Energy Survey has now reported results on such supernovae over a decade of study which included thousands of such events. The paper makes for fascinating reading. Titled “The Dark Energy Survey: Cosmology Results With ∼1500 New High-redshift Type Ia Supernovae Using The Full 5-year Dataset” (citation below), it significantly adds to the number of observed supernovae. There is just a hint here of flexibility in the direction of a variable dark energy. Let me quote the paper:

The standard Flat-ΛCDM cosmological model is a good fit to our data. When fitting DES-SN5YR alone and allowing for a time-varying dark energy we do see a slight preference for a dark energy equation of state that becomes greater (closer to zero) with time (w_a < 0) but this is only at the ∼ 2σ level, and Bayesian Evidence ratios do not strongly prefer the Flat-w₀w_aCDM cosmology.

Untangling: The standard Flat-ΛCDM model is the current description of cosmological structure and evolution, using cold dark matter (CDM) and a cosmological constant (Λ). “Flat’ means that the total energy density of the universe equals the critical density (i.e., a flat universe that continues to expand but at ever slower rates). Again, the cosmological constant is what we associate with dark energy and use to explain the accelerating expansion of the universe. And as the paper makes clear, the DES data fit the existing model, but it’s interesting that a dark energy that varies with time is not ruled out, even if the evidence for this is only enough to hint at the possibility.

Now it gets more intriguing. Nichols points out that a second probe looking at Baryon Acoustic Oscillations (BAO), which are “relics of predictable sound waves in the plasma…of the early universe, before the CMB [cosmic microwave background],” likewise hints at the possibility of dark energy that varies with time. This work is being done with the Dark Energy Spectroscopic Instrument (DESI), which has taken position as the successor to the Sloan Digital Sky Survey (SDSS), which had focused on measuring galactic redshifts.

The DESI results are indeed provocative, especially when seen in light of the supernovae results. From the paper on that work (citation below):

…combining any two of the DESI BAO, CMB or SN data sets shows some level of departure from the ΛCDM model. Relaxing the assumption of a spatially flat geometry through varying ΩK [the curvature density parameter] marginally increases the uncertainties but does not change the overall picture. It remains important to thoroughly examine unaccounted-for sources of systematic uncertainties or inconsistencies between the different datasets that might be contributing to these results. Nevertheless, these findings provide a tantalizing suggestion of deviations from the standard cosmological model that motivate continued study and highlight the potential of DESI and other Stage-IV surveys to pin down the nature of dark energy. (italics mine)

As Nichol puts it in his article:

In particular, when DESI analyses the combination of its BAO results with the final DES SNeIa data, the significance of a time-varying dark energy increases to 3.9 sigma (a measure of how unusual a set of data is if a hypothesis is true) – only 0.6% chance of being a statistical fluke.

Most of us would take such odds, but scientists have been hurt before by systematic errors within their data that can mimic such statistical certainty. Particle physicists therefore demand a discovery standard of 5 sigma for any claims of new physics – or less than a one in a million chance of being wrong!

As scientists will say: “Extraordinary claims require extraordinary evidence.”

Indeed. If we do learn that dark energy varies over time, that would mean that there is less of it now than in the past. We would also need to reconsider our notions about the ultimate fate of the universe depending on this new variable. What a time for physics, when the European Southern Observatory is getting ready to start another massive redshift survey and the European Space Agency’s Euclid mission, launched in 2023, is engaged in its own compilation of redshift data. And then there’s the Vera Rubin Observatory in Chile, which will one day soon be adding its own results to the mix. And then there is the quantum question. Adds Nichol:

According to quantum mechanics, empty space isn’t really empty, with particles popping in and out of existence creating something we call “vacuum energy”. Ironically, predictions of this vacuum energy do not agree with our cosmological observations by many orders of magnitude.

So we’re likely to be learning a great deal more in short order, for the Dark Energy Survey continues to compile its own data, and combining these with the above sources should give us a pretty good handle on the question of a variable dark energy. It’s intriguing to think that we may pin down why current dark energy studies are at variance with quantum mechanics. This is new physics of the kind that should make for Nobel Prizes down the road whatever the outcome of the combined data studies. Cosmology is in Nichol’s view likely entering a ‘new era of cosmological discovery.’

The Nichol article is “Dark energy: could the mysterious force seen as constant actually vary over cosmic time?” in The Conversation 10 October 2024 (full text). The DES paper is DES Collaboration, “The Dark Energy Survey: Cosmology Results With ~1500 New High-redshift Type Ia Supernovae Using The Full 5-year Dataset,” Astrophysical Journal Letters Vol. 973, No. 1 (1 October 2024) L14 (full text). The paper on the BAO measurements is DESI Collaboration et al., “DESI 2024 VI: Cosmological Constraints from the Measurements of Baryon Acoustic Oscillations” (abstract) and available in full text as a preprint.