Extraordinary Claims: Belief, Respect, and the Scientific Method
Presented at the 2015 Meeting of the Middle Atlantic Archaeology Conference, Ocean City, MD
Abstract: This paper will examine different types of extraordinary claims in archaeology coming from the lay public, popular media, and archaeologists themselves. Why do beliefs that are unsupported by facts persist? How can we thoughtfully and respectfully respond to proponents of these ideas while honoring the hard evidence?
“Extraordinary claims require extraordinary evidence.” Carl Sagan said that and I have a feeling you might hear it more than once during this session. It’s a wonderful declaration and it’s saved countless numbers of us from causing harm to others, wasting time and money, and just plain making fools of ourselves. Archaeology is not entirely a “hard science,” but that doesn’t stop this principle from being true for us.
What do I mean by extraordinary claims? Ideas and conclusions that are very difficult (or impossible) to prove. Conclusions that are highly unlikely, given the state of current knowledge. Results that are not easily reproducible.
This session is, at its heart, a delicate one. This paper and others will be calling out “bad science,” faulty assumptions, and emotional arguments. But it’s not personal. We’re all here seeking something that’s closer to the truth.
Archaeology as anthropology and social science does a lot of things. It seeks to broaden and deepen our understanding of human cultures and the world around us. We do this first by using logic and evidence and then by weaving information back into narratives about the meaning of the past (more on that later).
But we can’t address everything. Archaeology (and empirical observation in general) can only speak to the spiritual, the metaphysical, that which is “untestable” in some ways. We can infer imbued power and value based on patterns, placement, and other physical characteristics, but we can’t really measure the substance of the magic.
This paper is not about a battle between processual and post-processual archaeology. You’ll never hear me avow that it’s anywhere near possible to draw completely objective conclusions through the archaeological record. That’s not the point. The point is that we as humans have tendencies to arrange information to suit our needs. We’re always building these unconscious structures and it takes a little work to see them.
 If you’re an archaeologist with a published phone number, you’ve probably gotten calls from a group I’m going to refer to as “passionate non-professionals.” These are members of the lay public who are excited about what they’ve found and are hoping it’s something big. They’re interested in history and invested in being part of the story. Sometimes the calls are pretty pedestrian; a woman walking in the woods finds some lithics on the surface and claims herself a “village.” Others are farther afield. Temples, runes, execution stones, effigies and altars.
The next category of individuals I want to discuss are what I term Legacy Builders. By no means do I intend to imply that building a professional legacy is a bad thing. I have a great reverence for those who have come before me and devoted lives and careers to painstaking research. I admit that I’m not a research archaeologist, so it’s quite easy for me to throw blunt observations around. I’m not speaking about people who are making things up or intentionally lying in order to get famous. More like those of us with a whole lot invested and a whole lot to lose.
Of course, there are the individuals who may be legitimately experiencing delusions. I’m not focusing much on this group, but these people are real and interactions are not uncommon.  I do want to make a point right out front that we must do our best to resist the urge to pathologize. I’m fairly sure none of us are that kind of doctor. And then there are the rest of us. Impervious to any flaws of judgment. Paragons of rational thought and expression.  In all seriousness, there are characteristics that bind us all together; tendencies of the human brain with lots of inertia.
I’m going to put this one right at the top. I know I’m painting with a broad brush here, but as archaeologists, it’s a force whose power we wield to make our work “meaningful” beyond raw numbers, statistical distributions, or clusters on a map. We tell stories. We string our facts together in a particular order to create a picture. Narrative Bias plays into my discussion of extraordinary claims in two ways. We make the story about us. By we, I mean human beings in this case. The lay citizen who drops by the office with a revolutionary find may be already writing her autobiography in her head, complete with the discovery of North America’s earliest known Paleo-Indian settlement. And she’s not some kind of egomaniac; she’s just like the rest of us.
And then there is the storytelling that we do as archaeologists. In my day-to-day life, I’m dealing mostly with run-of-the mill CRM archaeology. Not to disparage the work of any of my colleagues, but there’s just not a lot of time, money, or room for waxing theoretic in most CRM archaeology. But still we tell stories. We create narratives in order to talk about significance of a site. I do not suggest that we stop weaving stories from physical evidence. To the contrary; I believe that the stories are maybe the most accessible and lasting products that we produce. But we have to be mindful that the natural tendency is there to make the evidence fit the narrative. Like bad car alignment pulls you in one direction, narrative has a whole lot of gravitational mass that indiscriminately attracts facts.
 Post Hoc Reasoning and Confirmation Bias
A man comes into our office with a box full of stone objects. “They’re all tools. I found them in the river. See? They fit so nicely in the palm of my hand!” An informal survey of my public-facing archaeologist colleagues tells me this scenario is almost universal. Because the rock fits in a human hand, it MUST be some kind of hand axe. And look! There are dozens! What a great discovery! This is a fantastic example of the post hoc fallacy.
It’s easy to sneer at an absurd archaeological conclusion held by a random member of the public, but what if it’s one of our own? I argue that the hard sciences have it much easier. A microbiologist running a series of experiments can ensure the accuracy of her results by using double-blind methods and running a test over and over again to look for statistically significant patterns of results. We’re pretty much stuck with what we’re able to find in most cases. A historical or cultural context exist and we have to decide whether our evidence fits the pattern or represents something new. We’re already biased to fit our data into a preconceived picture, so we’re fighting an uphill battle from the very beginning. This point speaks directly to another kind of bias, confirmation bias. “Confirmation bias is a filter through which [we] see a reality that matches [our] expectations.”
 Sunk Costs/Escalation of Commitment
My hypothesis is that A = B. I’ve spent 45 years amassing evidence and, slogging away at fieldwork, intensely researching, and formulating arguments to further my conclusion that A = B. Then, a few people come along and assert that after all this time A = C, not B at all. The Nerve! Even though data is stacking up to support the latter conclusion, I find it very difficult to discard my hypothesis. In this case, as in the others, my idea isn’t merely a string of facts tied together, it has become a belief. In the end, the only real antidote to any of these entirely normal and predictable tendencies is critical thinking. I understand that everything I’m about to say falls firmly within the category of “Easier Said than Done,” but bear with me. We must open ourselves to collegial criticism. How much do I want my Brilliant Idea to be true? Am I unwittingly introducing bias because I’m emotionally attached to it (or, perhaps, to the idea of discovering something great)? Hint: the answer is usually yes here. Do my colleagues buy the argument? Why are they skeptical?
Say my Brilliant Idea is actually spot on true. The uncomfortable reality is the more the claim diverges from accepted thought, the harder I’ll have to push for acceptance.  That’s ok; that means science and reason are working. It’s nothing personal. Nobody ever told us shifting a paradigm would be easy.
 Our human brains don’t want to tear down constructed assumptions and start again. That’s something that we must do mindfully. I argue that, as archaeologists, we can do much more to battle our biases. We need to get comfortable with dismantling our own beliefs and hypotheses and coming up with alternative explanations. We should invite close scrutiny of our data from the beginning and make our datasets open and available for others to analyze and reinterpret.
But what about the “passionate non-professional” I mentioned earlier? We can’t control how these well-meaning folks come to their conclusions and we may only have a few minutes with them to hear their perspective and provide our feedback. I believe that we as public-facing archaeologists could use a sort of toolkit to steer us through these sometimes sensitive encounters.
Return the call. I’ll be the first to admit that it’s too easy to dismiss an “out-there” request form a member of the public, especially when I anticipate that I’m going to have to tell a person something that they don’t want to hear. Interacting with these folks takes a great deal of time and patience (and is often not even in our job-descriptions). But, still, call them back.
Listen. Hear them out. Let them tell you their whole story. Write it down if you want. Paraphrase what they’ve told you and explain it back to them so they truly understand that you’ve been listening to them. Depending on how “fringe” the belief is, they’ll probably be expecting dismissal. Their confirmation bias may be already at work painting you as an elite academic (or bureaucrat) out to crush dreams. Let them know that you hear them and you’re just a regular person. If they’re not visiting in person, ask for photos.
Give honest feedback and be clear about the limitations of what we can see through archaeology. This comes back to clarifying that we can gather information based on the physical characteristics of material objects, but we can’t quantify what we can’t measure. Ghosts, magic, and supernatural power are simply out of our realm (literally). Provide alternate explanations. If what they’re seeing appears to be natural geology, send them some links. I have big dreams to create a website of “almost archaeology” in Virginia. It would cover examples of strange looking landscape features,  pareidolia (simply put, the natural human tendency to see faces and images in objects), along with examples of verified human artifacts and some stumpers. 
99% of these claims relate to purported Native American sites or artifacts. Work your contacts and refer folks to relevant experts within indigenous communities. This is a lot to unpack in a different paper, but I argue that many of these ideas come from essentialist beliefs about Indian cultures by non-Native people. While certainly not malicious, perpetuating these myths is harmful to the real, living Native communities by eroding culture and replacing it with something manufactured. Have awareness of this process (even if it’s beyond the scope of a helpful phone conversation with an artifact aficionado),
Understand that you likely won’t change anyone’s mind right away. I can count on one hand the number of times I’ve ended a call or a meeting like this with a conversion experience. People are going to be defensive when you tell them that they’re wrong and why.It may take a while to sink in. It may never sink in.
Encourage exposure to “real” archaeology. In Virginia, we’re exceptionally privileged to have the Archeological Society of Virginia as a resource for avocational archaeology. I recommend that folks attend ASV chapter meetings all the time. An added benefit is that sometimes, individuals who may be skeptical of my authority as a government employee may be more inclined to hear the perspectives of non-professional ASV members. I can’t say enough great things about the ASV.
Reframe the goal to mutual understanding. If you’re setting out to win an argument, I can guarantee that things won’t go well for archaeology. If the end product is mutual understanding, you’re much closer to a positive outcome. We’re not playing for points.
It’s critical to understand that not all opinions, ideas, and conclusions are equally valid, and therefore I feel that it is my ethical and moral obligation as an archaeologist (and as a human being) to always be questioning. We should remain healthily critical of our own ideas as well as those of our colleagues. We are all as human beings engineered to jump to conclusions. Inviting respectful engagement on challenging issues is the only way forward, whether with public audiences or our archaeological peers.