You have done it hundreds of times. A grid of blurry squares appears; you click the ones with traffic lights, and you move on. Nobody has ever asked what was going through your mind while you proved you were not a robot.
In a paper just published in Behavior Research Methods, my colleagues and I used that same task to bring religious ideas to people’s minds. Then we checked whether it actually worked.
An old question with a hidden problem
For about 20 years, psychologists have been asking whether reminding people of God makes them more honest or generous. In his 2013 book Big Gods, Ara Norenzayan argued that belief in a god who watches and punishes helped strangers cooperate once societies grew too large for neighbors to monitor each other. Dominic Johnson had been making a related case since 2004, with more weight on the fear of punishment than on being watched, a point he later expanded in his book God Is Watching You. If either version is right, even a short reminder of God should change behavior in the lab.
The problem is the reminder itself. In implicit priming, participants unscramble sentences containing words like “divine” or “spirit,” in the hope that religious thoughts become active without anyone noticing. These primes are often too weak to demonstrate activation. In explicit priming, participants write a few sentences about God. This works, but many participants can guess what the study is about, and some then behave the way they think a good person should.
The findings have been messy as a result. A 2016 meta-analysis by Azim Shariff and his colleagues concluded that religious priming increases prosocial behavior. But when Michiel van Elk and his colleagues reanalyzed the same data in 2015, two different bias-correction methods yielded different answers: One suggested the entire effect could be explained by publication bias; the other suggested a real effect remained. Their conclusion was that meta-analysis alone cannot settle the question, and that only large, preregistered studies can. A 2024 meta-analysis by Shariff’s research group added another twist: The association between religious belief and prosocial behavior appears mostly in what people say about themselves, and much less in what they do.
And almost none of these primes had ever been tested to see whether they really make people think about religion. We assumed they did.
Building a prime we could trust
Fırat Şeker, a former master’s student of mine who is now pursuing his Ph.D. at UMass Amherst, and Ensar Acem, one of my current Ph.D. students and the manager of our MINT Lab at Kadir Has University in Istanbul, co-led a project to change this.
The idea was to hide the prime inside something people see every day. We built two versions.
In the grid version, participants saw a photo of an open book divided into 16 squares. A short phrase sat in one corner: “God sees,” “God punishes,” “God rewards,” or “God forgives.” The control group saw “He runs” in the same spot. Everyone was told to select the squares with no readable text, just as in an ordinary CAPTCHA.
In the keyword version, participants saw a picture of a Bible and had to pick the word that best described it. The right answer was “sacred,” hidden among options like “train,” “bed,” and “pencil.” The control group saw a photo of a table and had to pick “table” in the same way.
Then came the test. In a preregistered study of 2,770 American Christians, all participants completed a word-completion task. Participants saw fragments such as “f a i _ _” and filled in the blank, which could become “faith” or “faint.” Count the religious words people produce, and you have a behavioral measure of how easily religious thoughts come to mind. After that, participants played a standard dictator game, deciding how much of a small sum to share with a stranger. At the end, we asked them what they thought the study was about.
What worked and what did not
The explicit prime produced the strongest effect, as expected. But 11 percent of those participants correctly guessed the study’s purpose.
The keyword version worked. When people had to find the word “sacred” under a picture of a Bible, they later produced more religious words than the control group did, and the same was true for the versions that pointed to a rewarding or a forgiving God. The version that pointed to a punishing God did not raise religious thoughts in general, but it did raise thoughts about sin and punishment specifically. Only 6 percent of these participants guessed what the study was about. The grid version did not work at all. We told people to select the squares without text, so they barely processed the sentence. A prime can be subtle, but it still needs a few seconds of attention.
Generosity did not change under any prime, including the explicit one. The experimental record here is mixed. In 2018, Joseph Billingsley and his colleagues found that an explicit prime increased giving in a dictator game while an implicit prime did not, and Cristina Gomes and Michael McCullough had found no effect of implicit priming on the same game in 2015. The meta-analytic picture depends on how one corrects for publication bias, as van Elk and his colleagues showed. So a null result is not out of line with what others have found. There is also a design reason. Norenzayan’s point in Big Gods was that religion works when it is in the situation, at the moment a person decides. In our study, the word-completion task came between the prime and the game, so by the time people chose how much to give, the reminder may already have faded. But when a religious prime fails to affect behavior in future studies, researchers can now rule out the most boring explanation: that the prime never reached anyone in the first place.
For people who work in this area
There is now a tested, low-suspicion method for activating religious thoughts in online studies. It has separate primes for punishing, rewarding, and forgiving images of God. That separation matters. Researchers disagree about whether it is the feeling of being watched or the fear of punishment that makes religious people behave better, and until now there was no clean way to pull the two apart. The materials and data are freely available on OSF.
Almost 10 years ago, Hasan G. Bahçekapili and I argued in Religion, Brain and Behavior that the question “Is religion necessary for morality?” has an easy answer: no, since plenty of people are moral without any religious belief. The harder question, and the one that remains unsolved, is how religion transforms morality once it enters the picture, in our evolutionary past, in history, and in the life of a single person. One reason it stayed unsolved is that we could not trust our own manipulations. Some questions that were put aside as unanswerable may only have been waiting for better tools. The same trick should also work outside religion, for anything that fits in the corner of a picture.
The quieter crisis
Everyone has heard of the replication crisis in psychology. Its sibling, the validity crisis, gets much less attention. In 2021, David Chester and Emily Lasko reviewed the experimental manipulations used in social psychology and found that most had never been tested to see whether they changed what they were supposed to change. A result can replicate perfectly and still be about nothing in particular.
Validation studies are not glamorous and rarely make the news. But before we ask whether God makes people good, we need to be sure that we have actually made them think about God. The field needs many more studies like this one, and far fewer assumptions.
