The birth of Jesus Christ is pretty much the most basic element of the Christmas story. From common carols to the imagery of the nativity scene, the idea that Christmas is his birthday is everywhere.

But what’s commonly accepted by those who observe the holiday isn’t exactly true to history, according to Mark Goodacre, a professor in the Duke University Department of Religion.

The Gospel of Matthew and the Gospel of Luke are the only accounts of Jesus’ birth in the New Testament. Even accepting their word as the truth, each tells a different story—Luke’s starting in Nazareth and Matthew’s focused solely on events in Bethlehem—and neither is particularly detailed in terms of calendar dates. “It’s a tough one really because there is just so little in Matthew’s and Luke’s birth narratives. The gospel writers very rarely tell you when things happened and the time of year,” Goodacre says. “You don’t get any official detail so we’re in the dark really on that.”

Both accounts mention that the events of Jesus’ birth took place when Herod the Great, a Roman King, was still around, but they still probably differ from each other by about 10 years. Luke’s account mentions Jesus was around 30 years old when he started preaching, which would take his birth year back to 1 BCE. Today, however, it seems likely that Luke slightly miscalculated the death of Herod, meaning that Jesus would have been born in 4 BCE, about 2,019 years ago. And, Goodacre cautions, even that knowledge is imprecise. “You might say you’re in your 30s when you’re really 38,” he says. “When Luke is saying that, he could just be saying Jesus was in his 30s [when he started preaching]. It’s all approximate.”

Things get even more confusing when you add in the December 25th factor.

Scholars often point out that Luke’s gospel cites that there were shepherds in the field on the first Christmas, which suggests Jesus was not born in winter. “There’s also the idea that Mary and Joseph would have embarked on an epic journey from Nazareth to Bethlehem in the middle of winter with Mary being pregnant, which seems really implausible,” Goodacre says. As TIME put it in a 2004 cover story, “in the debates over the literal truth of the Gospels, just about everyone acknowledges that major conclusions about Jesus’ life are not based on forensic clues. There is no specific physical evidence for the key points of the story.”

In his 2012 book Jesus of Nazareth: The Infancy Narratives, Pope Benedict himself cast doubt on many recollections of Jesus’ birth as well.

And the Christmas confusion doesn’t stop at when Jesus was born. Many people imagine Mary and Joseph going to some kind of inn in Bethlehem, Goodacre says, and ending up in a stable because they could not get a room. “What Luke says is that they laid Jesus in a manger because there was no room for him in the accommodation, which is probably what he means as the family house. So they went to the lower level of the house,” he says. “They’re not in a stable. It’s one of the most famous pictures of Christmas there is, of him being born in a stable, and the textual basis for that is very weak.”

Read more