In a tantalizing article in The Conversation, Robert Nichol (University of Surrey) offers a look at where new physics might just be emerging in conjunction with the study of dark energy. Nichol is an astronomer and cosmologist deeply experienced in the kind of huge astronomical surveys that help us study mind-boggling questions like how much of the universe is made up of matter, dark matter or dark energy. We’ve assumed we had a pretty good idea of their proportions but a few issues do arise.
One of them seems particularly intriguing. Nichol’s article asks whether dark energy, regarded as a constant, may not actually vary over time. That’s quite a thought. The consensus over a universe made up of normal matter (5 percent), dark matter (25 percent) and dark energy (70 percent) came together early in our century, with dark energy taking the role of the cosmological constant Einstein once considered. Although he came to reject the idea, Einstein would doubtless take great interest in the work of observational cosmologists like Nichol, who keep refining the numbers to reduce errors.
Addendum: I hate typos, and thankfully a reader pointed out that in the penultimate sentence above, I had accidentally typed “with dark matter taking the role of the cosmological constant,” when of course it should be dark energy. Now corrected. Not enough caffeine in play this morning, evidently.
Image: The University of Surrey’s Nichol. Credit: University of Portsmouth.
At the heart of the investigation is the Dark Energy Survey, an international effort involving some 400 scientists in seven countries. The survey’s latest numbers, Nichol reports, are that 31.5 percent of the universe is either dark or normal matter, with an error on the measurement of a scant 3 percent. The question of how almost 70 percent of the universe could be in the form of something we can’t see, and something that is indeed not associated in any way with matter, is what Nichol calls “the biggest challenge to physics, even after 20 years of intense study.”
Remember that we first learned of the acceleration of the universe by studying Type Ia supernova (SNeIa) explosions. These occur in binary systems when a white dwarf star begins drawing off material from its companion, usually a red giant. Reaching the Chandrasekhar limit (approximately 1.4 times the mass of the Sun), the white dwarf releases vast amounts of energy, forming a ‘standard candle’ for cosmologists because the luminosity of these events is completely predictable. In other words, supernovae like these have an intrinsic brightness that can be compared to what is observed, making their distance measurable. Plug in the observed redshift and astronomers can use supernovae to make measurements on the rate of the universe’s expansion.
Image: The Hubble Ultra Deep Field, a view of nearly 10,000 galaxies, a reminder of the stunning scope of cosmological studies. The snapshot includes galaxies of various ages, sizes, shapes, and colours. The smallest, reddest galaxies, about 100, may be among the most distant known, existing when the universe was just 800 million years old. The nearest galaxies – the larger, brighter, well-defined spirals and ellipticals – thrived about 1 billion years ago, when the cosmos was 13 billion years old. This image required 800 exposures taken over the course of 400 Hubble orbits around Earth. The total amount of exposure time was 11.3 days, taken between Sept. 24, 2003 and Jan. 16, 2004. Credit: NASA, ESA, and S. Beckwith (STScI) and the HUDF Team.
The Dark Energy Survey has now reported results on such supernovae over a decade of study which included thousands of such events. The paper makes for fascinating reading. Titled “The Dark Energy Survey: Cosmology Results With ∼1500 New High-redshift Type Ia Supernovae Using The Full 5-year Dataset” (citation below), it significantly adds to the number of observed supernovae. There is just a hint here of flexibility in the direction of a variable dark energy. Let me quote the paper:
The standard Flat-ΛCDM cosmological model is a good fit to our data. When fitting DES-SN5YR alone and allowing for a time-varying dark energy we do see a slight preference for a dark energy equation of state that becomes greater (closer to zero) with time (wa < 0) but this is only at the ∼ 2σ level, and Bayesian Evidence ratios do not strongly prefer the Flat-w0waCDM cosmology.
Untangling: The standard Flat-ΛCDM model is the current description of cosmological structure and evolution, using cold dark matter (CDM) and a cosmological constant (Λ). “Flat’ means that the total energy density of the universe equals the critical density (i.e., a flat universe that continues to expand but at ever slower rates). Again, the cosmological constant is what we associate with dark energy and use to explain the accelerating expansion of the universe. And as the paper makes clear, the DES data fit the existing model, but it’s interesting that a dark energy that varies with time is not ruled out, even if the evidence for this is only enough to hint at the possibility.
Now it gets more intriguing. Nichols points out that a second probe looking at Baryon Acoustic Oscillations (BAO), which are “relics of predictable sound waves in the plasma…of the early universe, before the CMB [cosmic microwave background],” likewise hints at the possibility of dark energy that varies with time. This work is being done with the Dark Energy Spectroscopic Instrument (DESI), which has taken position as the successor to the Sloan Digital Sky Survey (SDSS), which had focused on measuring galactic redshifts.
The DESI results are indeed provocative, especially when seen in light of the supernovae results. From the paper on that work (citation below):
…combining any two of the DESI BAO, CMB or SN data sets shows some level of departure from the ΛCDM model. Relaxing the assumption of a spatially flat geometry through varying ΩK [the curvature density parameter] marginally increases the uncertainties but does not change the overall picture. It remains important to thoroughly examine unaccounted-for sources of systematic uncertainties or inconsistencies between the different datasets that might be contributing to these results. Nevertheless, these findings provide a tantalizing suggestion of deviations from the standard cosmological model that motivate continued study and highlight the potential of DESI and other Stage-IV surveys to pin down the nature of dark energy. (italics mine)
As Nichol puts it in his article:
In particular, when DESI analyses the combination of its BAO results with the final DES SNeIa data, the significance of a time-varying dark energy increases to 3.9 sigma (a measure of how unusual a set of data is if a hypothesis is true) – only 0.6% chance of being a statistical fluke.
Most of us would take such odds, but scientists have been hurt before by systematic errors within their data that can mimic such statistical certainty. Particle physicists therefore demand a discovery standard of 5 sigma for any claims of new physics – or less than a one in a million chance of being wrong!
As scientists will say: “Extraordinary claims require extraordinary evidence.”
Indeed. If we do learn that dark energy varies over time, that would mean that there is less of it now than in the past. We would also need to reconsider our notions about the ultimate fate of the universe depending on this new variable. What a time for physics, when the European Southern Observatory is getting ready to start another massive redshift survey and the European Space Agency’s Euclid mission, launched in 2023, is engaged in its own compilation of redshift data. And then there’s the Vera Rubin Observatory in Chile, which will one day soon be adding its own results to the mix. And then there is the quantum question. Adds Nichol:
According to quantum mechanics, empty space isn’t really empty, with particles popping in and out of existence creating something we call “vacuum energy”. Ironically, predictions of this vacuum energy do not agree with our cosmological observations by many orders of magnitude.
So we’re likely to be learning a great deal more in short order, for the Dark Energy Survey continues to compile its own data, and combining these with the above sources should give us a pretty good handle on the question of a variable dark energy. It’s intriguing to think that we may pin down why current dark energy studies are at variance with quantum mechanics. This is new physics of the kind that should make for Nobel Prizes down the road whatever the outcome of the combined data studies. Cosmology is in Nichol’s view likely entering a ‘new era of cosmological discovery.’
The Nichol article is “Dark energy: could the mysterious force seen as constant actually vary over cosmic time?” in The Conversation 10 October 2024 (full text). The DES paper is DES Collaboration, “The Dark Energy Survey: Cosmology Results With ~1500 New High-redshift Type Ia Supernovae Using The Full 5-year Dataset,” Astrophysical Journal Letters Vol. 973, No. 1 (1 October 2024) L14 (full text). The paper on the BAO measurements is DESI Collaboration et al., “DESI 2024 VI: Cosmological Constraints from the Measurements of Baryon Acoustic Oscillations” (abstract) and available in full text as a preprint.
It is a rather tall order to hope that cosmological observation alone will bridge the largest discrepancy in physics – a factor of 10^120. But perhaps we will get lucky.
“with dark matter taking the role of the cosmological constant Einstein once considered.”
This is an error in the 2nd paragraph. It should of course be dark energy. I don’t see this error in the remainder of the article.
Thanks for spotting that, Ron. I just fixed it.
“For what it is worth”, when dark energy confronts me in texts or discussions, I come to think of it like something that would be in excess of a coasting or more Newtonian universe, that it is something that pumps it up like a tire. The pumping would be akin to work – and so is energy in terms of units. So, while “dark matter” has us all searching for elusive particles, energy has us looking for something like elusive wiring for distributing “energy”.
On the other hand, if there is a pervasive mechanism for pumping, does that alter the search? And when there is pumping, it is often spoken of as space itself. And this is after the word was spread that space was not made of “aether”, a medium I would not have otherwise known about other than from its mention in pre 1900 science fiction stories.
Then there is the issue of how fundamental is this pumping from the standpoint of structures within the cosmos: Dark energy accelerates structures like galaxies away from us, but how far up and down the hierarchy of matter does it spread its influence? Does it expand ancient ellipticals from their earlier configurations? The separation of atoms in gases or solids? Subatomic particles in the nucleus or their components?
All these stretches I have heard mention of, but yet as discussed, one of the most important candles for detection of dark energy is the Super Nova Ia acting much the same hereabouts as it does giga-parsecs away. If the presumed expansions at the lesser scale are valid, then perhaps there is a geometrical issue similar to those associated with Special or General Relativity. If, however, at some lower scales, there are exceptions to this expansion of “space”, then that might be
an important clue to connect with Dark Energy’s “hide and seek” over time.
If we have a very granular measurement of mass, darker matter, and recession velocities, we might be able to determine is dark energy is intrinsic to space, like a field, or whether it is associated with matter.
If it turns out that the expansion rate is slowing, that might suggest that dark energy is finite and therefore has a reduced effect as the universe expands, and is possibly associated with the galactic structures. We might detect greater expansion rates where the mass/dark matter in the universe is greater and slower in the voids. This is conceptually different from the old steady-state universe where new hydrogen atoms would appear to maintain the average density of matter in the universe.
If dark energy is some sort of expansionary force, is it possible that it could be harnessed for interstellar propulsion? OTOH, what if dark energy and dark matter are just “software variables” to maintain the stability of the simulation of our universe?
Expansion rate of the Universe”>Expansion Rate of the Universe
A.T.,
Both of us are probably looking for something to measure here. But my understanding thus far is that dark energy IS responsible for increasing expansion.
If we had simply a Newtonian expansion due to a blow up such as a Big Bang, then matter receding into the distance would decelerate owing to the action of gravitational attraction. Add Einstein, and beside matter’s attraction there would be the attraction of energy as in e=mc^2. But applying dark energy ( or as I suggest above “dark work”), we have a pumping that is in excess of the former two explanations for expansion. As though – perhaps? – the space within which we reside is spreading like rising bread in an oven. Of course, when that analogy is used, I am not sure how literally it should be taken. We measure the position of say raisins in a bread loaf rising in an oven, but there are some details to address about the flour and other ingredients…
On the propulsion side, vacuum energy associated with particle pairs, created or destroyed cause infinitesimal imbalances of matter and anti-matter and possible net forces directed one way or another; but I don’t recall discussion of this phenomenon’s possible connection with dark energy. But if the extent of space is expanding, are the properties of one cubic meter to remain constant in terms of vacuum energy? Or does that property have to be more widely shared? .. I am running out of fingers to tie strings around for questions to ask authorities on these matters.
I think you stated it better than I did. If dark energy is a thing, is space itself expanding (dragging along the raisins) or is the effect between matter, a repulsive force that is not 1/r^2.
Gravity in Einstein’s formulation is the curvature of space by matter. We harness gravity, even for propulsion, e.g. gravitational slingshots), so I just ask if dark energy might be harnessed in some way.
We assume that the universe is natural and not a simulation in some cosmic computer. But what if it isn’t, and dark energy and dark matter are just software variables with no physical basis? [I surely hope not, as I dislike the idea that the universe is a simulation, or nested simulations.]
A.T.,
As I grew up, I never thought that a certain concept would ever again become that handy, nor was I even reminded of it – until these deliberations started pulling up floorboards and foundations only to suspect we are scratching at another. Universe simulation and mimicked curvature both difficult to take in stride.
i.e., the echo of childhood dialogs of a foundational nature prefaced by:
“Yeah… and before that?”
Parents’ Answer:
….The Little Man Who Wasn’t There.
Beginning to wonder if the above non-entity has a lot of responsibilities these days.
Instead of all this speculation, why not just read the article Robin referenced for the current state of observations, theory and interpretation. The Wikipedia article on dark energy also appears to be reasonably up to date.
There is a constant stream of academic papers on the subject as well, and Paul pointed to one. However, each is typically too narrow in scope and too technical to be useful for those interested in a broad understanding of the subject.
Hello, R.S.
Admittedly, my last comment above was a mixture of jest and wonder, wonder that a concept as enigmatic should have even another wrinkle to unfold. But in addition, A.T. and I were in a dialog looking for ways to describe the phenomenon in as much as it was understood by us in piecemeal – say yesterday.
Initially, there had been little comment on the subject at all, so I thought that “dark work” notion might be an opportune trial balloon.
Dark matter, for example, I find that reading the papers can be quite illuminating, but haven’t reached a similar threshold with dark energy thus far.
Breaking the code likely depends on which principles of cosmology one is already acquainted with or already grasps.
R.S.,
At first I was not sure which of several possible articles you were referring to.
And then found the article that Robin provided.
I think some of the things contained in it were either generally understood or explained prior, but I am not sure it addressed explicitly the issues that
A. T. and I were examining. E.g., “objects separated in space”. Does that mean everything as an object defined or objects down to some magnitude not so much influenced by gravity but fundamental forces such as strong, weak or electromagnetic.
If someone wants to make a call on that and explain, pro or con, I’m
an interested reader and there might be more. I feel as though I could still err either way in interpreting this, much like if I had purchased some dark energy in a package from a store.
“I am not sure it addressed explicitly the issues that A. T. and I were examining. E.g., “objects separated in space”.”
That was addressed in the very first sentence of Robin’s referenced article. You have to understand that gravitation is an exceptionally weak force. Any close binding energy — gravitation, EM (e.g. molecular bonds) — especially the latter are orders of magnitude stronger. Note, for example, there is no cosmological expansion correction for the precession of Mercury’s orbit; it’s absurdly negligible. General relativity suffices.
We can also argue the existence of dark energy, but that also begets the question of where all forms of energy come from, which includes matter and dark energy. It’s just a small pimple on a sperm whale of a mystery. So we measure, study, theorize and, yes, speculate.
Re: “Note, for example, there is no cosmological expansion correction for the precession of Mercury’s orbit; it’s absurdly negligible. General relativity suffices.”
Well, from an orbital mechanics point of view based on gravitational interaction with both the sun and planets , with the celestial sphere as reference, Mercury’s orbit does experience precession, the position of its perihelion rotating about the ecliptic plane. But there is an additional increment due to General Relativity that seems to explain the difference adequately. Unless the rotating sun turns out to be more oblate than generally assumed…
wdk, that’s what I said.
Another take on dark energy, just up on arXiv:
On a model of variable curvature that mimics the observed Universe acceleration
https://arxiv.org/abs/2410.08306
“We present a new model based on General Relativity in which a subtle change of curvature at late times is able to produce the observed Universe acceleration and an oscillating behavior in the effective equation of state, similar to what has been claimed by recent results from the Dark Energy Spectroscopic Instrument and Baryon Acoustic Oscillation observations.”
That is way over my head. I would like to see comments and critiques by experts in the field.
Constants are nice in that they are tightly restricted and easy to work with, but beyond that is there any actual reason it would be constant?
One might be conservation of energy (or maybe any other quantity that we consider to be conserved) in that as the dark energy multiplier changes would the total (energy or whatever quantity) of the universe have to change?
And what does it even mean for an infinite quantity (in the event that the universe is infinite) to change?
You could ask the same questions of the plethora of physical constants: are they truly constant and why those particular values? All we know is that if some (any?) were different we wouldn’t be here to ask questions. But that’s not a satisfying answer.