Q: It seems that skeptics just debunk and naysay, but if they’re so confident in their explanations, why don’t they recreate the mystery and prove it?

—A. J. Kitt

A: This question was directed at me in the context of Florida’s famous Coral Castle (see this column, May/June 2006) but has appeared many other times when skeptics have offered plausible explanations for famous mysteries. In last issue’s column, I addressed the (faulty) premise underlying the common assumption that because the famous 1967 Patterson and Gimlin Bigfoot film hasn’t been replicated, it is not the hoax skeptics claim it is.

The question has broader implications for investigative skepticism. In many cases, using the principle of Occam’s razor, replication should be enough to demonstrate that an extraordinary claim is indeed unlikely to be true. For example, self-described psychic-turned-“entertainment artist” Uri Geller rose to international prominence in the 1970s and 1980s performing various acts that were replicated by professional magicians, most notably the late James “The Amazing” Randi. Of course, this does not provide conclusive proof that Geller was simply a skilled magician instead of a psychic. It does, however, give critical thinkers a logical, rational, science-based reason to doubt the claims.

The issue of replication in science is pretty straightforward: It is essential for establishing the validity of an experiment. Because well-controlled studies are difficult to design and carry out, there is always the chance that a given outcome will be the result of random chance, experimenter bias, or any number of other factors. If a result is true and valid, then other researchers following identical procedures should, in theory, get similar results. Replication is such an issue in science that the inability to replicate results has garnered significant attention. BBC News reported that “Concern over the reliability of the results published in scientific literature has been growing for some time. According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist’s experiments” (Feilden 2017). Skeptical investigation sometimes involves designing experiments and scientific protocols when testing dowsers, psychics, and others (see, for example, “Testing Natasha” by Ray Hyman in the May/June 2005 Skeptical Inquirer and James Underdown’s work with the Los Angeles–based Center for Inquiry Investigations Group).

Depending on what’s being measured, replication can be difficult under the best of circumstances. If you’re trying to replicate a population study, it’s important to look at the statistical methodology to be sure a representative sample was used; slight variations in the underlying populations can introduce confounders and thus create spurious (Type I and Type II) errors, suggesting that an experiment has not been replicated when in fact it has, or vice versa.

Replicating Unique Events

The idea of replicating an event or project to test the validity of explanations for it sounds simple in theory. For example, some people claim that the Egyptian pyramids were made by ancient astronauts. The (ahistorical) assumption is that people at the time didn’t have the intelligence or technology to move the stones and build a pyramid shape. Since the pyramids were built around 2560 BCE, there are no photographs or depictions of them being created, although in 2015 papyrus records were found of pyramid construction tools. Egyptologists have a pretty good idea of where the rocks were quarried and how they were cut and moved, but conspiracy promoters are fond of noting that scientists have never actually replicated the pyramids. They claim that skeptics should build an entire pyramid using only materials and tools of that era to prove how it could have been done.

This might seem like a reasonable challenge until you realize that such an effort would never be done—not because it can’t be done but because it would be impractical. Duplicating the great Giza pyramid would take many years and cost tens (or hundreds) of millions of dollars. Who’s going to pay for it? It would also be pointless, because such a replication would not be valid unless you used tens of thousands of workers (estimates range from 15,000 to 40,000) and spent a decade or more building it (as the original builders did). If some eccentric billionaire wants to fund it, he or she is welcome to, but scientists recognize it as a fool’s errand.

In May 2015, a man named Paul Salo launched a project he claimed would prove once and for all whether the conspiracy theories about the September 11, 2001, attacks are true or not by replicating the World Trade Center collapse: “We will recreate 9-11 to the best of our ability [with] a fully loaded 767 and a similar structure to the WTC. We will crash the fully loaded (with fuel) plane (complete with black box) into the building using autopilot at 500 MPH.” Salo aimed to test the debunked claim that jet fuel can’t burn hot enough to weaken a building’s steel structure to the point of collapse. While Salo’s scheme had populist appeal, actually pulling it off as a valid recreation would be incredibly difficult, if not impossible, and expensive. For a real science experiment, you need to control for variables that could affect the results. In this case, there are many variables, including the size and weight of the plane, the building type, and so on. It also matters where the building is, because building codes vary wildly by country and region. Buildings in earthquake-prone regions are built differently (and able to sustain greater structural damage without collapsing) than those built elsewhere. Variations in construction materials will also complicate comparisons. For the experiment to be valid, Salo would need to build an exact replica of the Twin Towers; just any tall building will not do, because the load-bearing structures vary. (For more details, see “9/11 Conspiracy Crowdfunding Project Rises—and Falls” in the September/October 2016 SI and “Skepticism and Pseudoexperiments” in the September/October 2020 SI.)

Salo planned to “recreate as best as we can” the circumstances of the World Trade Center attacks. The problem is that “as best as we can” leaves an enormous margin of error, one so big as to make any results invalid and pointless. His results, were he to pull it off, would be sensational but hold no evidentiary value at all. The replication would be pointless and inconclusive no matter its outcome: If the building collapsed exactly as happened on September 11, conspiracy theorists would argue (correctly) that the conditions weren’t exactly the same as in the original building collapse. If the building collapsed differently, that wouldn’t prove anything either for the same reason. No one either questioning or defending the “official story” would accept his conclusions.

Photographic Replication

A photographic attempt at replication is seemingly more straightforward. Replication can certainly play an important role in skeptical investigation, though recreating the circumstances surrounding an “unexplained” event is far more important than necessarily duplicating or replicating a given sighting, photograph, or video.

It’s actually quite easy to capture a photographic image that cannot easily be duplicated. A photograph is a two-dimensional representation of a split-second moment in time (depending, of course, on how long the exposure is) in a constantly changing environment. Replicating some photos is easy; a close-up photograph of an egg taken in 1950 might look identical to one taken with the same camera and lighting as one taken a century later. But outdoor photographs, or those of urban settings, are a different matter. Clouds come and go by the hour (sometimes by the minute), leaves change their position relative to each other, cars and people constantly change position, and so on. In most cases, a simple landscape photo will be difficult or impossible to exactly replicate 100 years later.

In the case of Bigfoot, recreating the film, as a practical matter, is enormously difficult under the best of circumstances—regardless of whether it depicts a Bigfoot. We can begin with the terrain, which like all other natural habitats has changed significantly in the past fifty-five years. Consider all the environmental factors at play: Trees die and fall, rivers and streams move, and so on. If someone did spend considerable time and effort to create a convincing Bigfoot costume and setting, that would not serve to silence the critics but bolster them. The response would not be, “Yes, you’re right; it can be credibly duplicated,” but instead, “Yes, but see how much money you had to spend! Patterson and Gimlin didn’t have these resources, so it couldn’t have been faked!”

A full and true replication may require the original people or objects, under the exact same conditions, depending on what part of the image is under scrutiny (a dark manlike patch in a shady wooded area, a face seen in a ghost image, or an odd light in the sky). Sunlight reflecting off a gleaming polished fender of a 1958 Corvette, for example, might potentially help explain a mysterious light or image, or an unnoticed off-camera shadow might create an optical illusion of a face. Substituting a 1984 Honda or a 2012 Ford in a replication photograph may or may not get the same results.

I’ve done replication in several of my investigations, but I’m always careful to include qualifiers and not claim to have duplicated anything exactly. There are also natural occurrences that can serve as a kind of replication of unusual events (for more, see Mick West’s column in this issue). In the end, the suggestion that skeptics must replicate historical events with enough accuracy for a valid comparison is both a bald attempt to shift the burden of proof and a waste of everyone’s time.

Reference

Feilden, Tom. 2017. Most scientists “can’t replicate studies by their peers.” BBC News (February 22). Online at http://www.bbc.com/news/science-environment-39054778.