Deeply Isolated Neanderthal Groups?

A look at Neanderthal population structures. Did some stay isolated for 50,000 years?

The news this week that a particular Neanderthal named ‘Thorin’ may represent a group that remained isolated from the rest of their kind for around 50,000 years may come as no surprise to regular readers. In July last year I wrote about the slow development of human cultures within Europe, starting with the oldest apes and ending with the first modern humans. A preprint paper available at the time had presented tentative results from a low-coverage genome of Thorin, a late Neanderthal from Grotte Mandrin, a cave in southern France. To quote the researchers:

Our results nevertheless suggest a minimum of two distinct Neanderthal lineages present in Europe during the late Neanderthal period. In the absence of any detectable gene flow between Thorin and other Neanderthal lineages after its divergence, we conclude that Thorin represents a lineage that has stayed isolated for ~50 ka… Our results thus also shed light onto the social organization of Neanderthals, suggesting that small isolated populations with limited, and potentially without, inter-group exchange as a possibly more general feature of Neanderthal social structure.

Since then that preprint has been revised and published, led by the archaeologist Ludovic Slimak. He is an interesting guy, and it is worth reading an interview with him from June 2023 to get a feel for his vision of prehistory:

What happened? Why do we have all Sapiens in Europe with Neandertal DNA and not a single Neandertal have Sapiens DNA? So we know from Claude Lévi-Strauss’ Elementary Structures of Kinship that the question of the reproduction of societies is not a question of love. It’s a question of exchanges and alliances between populations. So that means that when two groups meet, it’s very important for them to exchange genes. And we know from DNA how they do it, it’s universal for both Neanderthal and Sapiens: through female mobility. That means: “My sister will go in your group, but your sister will come in my group”. And like that, we will build an alliance - we call this patri-locality. But if your sister comes in my group, my sister will have to come in yours. I can’t have your sister in exchange of flint or 10 horses.

What I explain in I Love You, Me Neither, is that in the case of the Sapiens and Neanderthals, it’s: “You give me your sister, but I don’t give you mine”. This is rare, but it happens. One possible instance when we see this is when there’s a total war between populations, and one group is going to seek to destroy another group. But in fact, it’s not really a genocide, because when that happens, traditionally what they do is that they keep the children and the women, and then they have children with these women.

The 50,000 years of isolation result has caused quite a buzz in the last few days. It seems almost unbelievable that a group of Neanderthals would stay totally isolated and separated from other Neanderthals who lived relatively close by for such a huge length of time. The result stems from a ‘low-coverage’ genome sequence of Thorin. Low-coverage refers to the number of times each DNA base is read, the higher the number, the higher the accuracy and chance of spotting rare variations. Low-coverage sequencing is common for population analysis and preliminary work - it is also cheaper. Obviously a higher-coverage genome sequence would be ideal for very rare specimens such as Thorin, but even so, the results seem to show a clear distinction between this individual and other sequenced Neanderthals.

Demographic model based on genome sequencing of Thorin and other Neanderthals, against Denisovans and Mbuti pygmy outgroups.

So if these results are correct, then the Thorin lineage represents a small population of Neanderthals which diverged from the rest of the main European population around 100-115,000 years ago. Previous Neanderthal work using DNA extracted from cave sediments had already pointed to a ‘branching’ of Neanderthal lineages around this time.

Partial figure from Vernot et al 2021, showing the divergence of multiple Neanderthal sub-populations: once around 135kya, and again around 105kya.

This period of time is significant climatically. It coincides with the middle of the MIS 5 interglacial period, which saw temperatures increase, ice sheets retreat and an increase in animals and plants for food. Indeed the 2021 paper previously referenced concludes that there was a population turnover in northern Spain around 100,000 years ago, based on the sudden decrease in mitochondrial DNA diversity. The split dates of these new sub-populations - Mezmaiskaya 1, Vindija and Chagyrskaya 8 - are re-assessed in the latest paper to around 80,000 years ago, and there remains the possibility of other ‘ghost’ populations which have yet to be sampled.

Research into sub-populations of Eurasian Neanderthals has been growing in the past few decades. This image of two Neanderthal cultures with a border in the Caucasus is derived from stone tool cultures rather than genetics, but the Eastern Micoquian lines up neatly with the Mezmaiskaya group mentioned above. How small each population must have been is becoming an important topic, now that such deep isolation has been suggested.

In this image from Skov et al 2022 we can see three caves - Denisova, Chagyrskaya and Okladnikov in southern Siberia. DNA analysis of Neanderthal remains from all of these again reveal profound levels of isolation and very small communities:

Some Chagyrskaya individuals were closely related, including a father–daughter pair and a pair of second-degree relatives, indicating that at least some of the individuals lived at the same time. Up to one-third of these individuals’ genomes had long segments of homozygosity, suggesting that the Chagyrskaya Neanderthals were part of a small community. In addition, the Y-chromosome diversity is an order of magnitude lower than the mitochondrial diversity, a pattern that we found is best explained by female migration between communities…

Denisova Cave was occupied by both Neanderthals and Denisovans around the same time that Neanderthals inhabited Chagyrskaya Cave. However, the stone artefact industry at Denisova Cave lacks the characteristics of the Sibiryachikha variant found at Chagyrskaya Cave. Accordingly, despite the proximity of the two caves and the presence of an offspring of a Neanderthal mother and a Denisovan father in Denisova Cave some tens of millennia before Chagyrskaya Cave was occupied, we find no evidence of gene flow from Denisovans to the Chagyrskaya Neanderthals in the last 20,000 years before the Chagyrskaya individuals lived.

Although not quite as isolated as the Thorin lineage, a pattern of small communities with infrequent levels of female migration seems to be emerging. Even more interestingly perhaps, is the result that Denisovans and Neanderthals were living near one another for perhaps 20,000 years without any gene flow. The one example of a hybrid individual with a Denisovan father and Neanderthal mother (Denisova 11 or ‘Denny’) is dated to around 90,000 years ago.

Slimak hints in this latest paper that by the time the first wave of modern humans arrived in Europe, around 55,000 years ago, the existing small groups of European Neanderthals were being replaced by the larger Vindija group. That the Vindija group were larger and perhaps healthier seems confirmed by looking at the ‘runs of homozygosity’ (ROH) in the different genomes. The higher the ROH, the higher the level of inbreeding or cousin mating and the smaller/less diverse the population.

It’s clear looking at this figure that the Vindija and the older Goyet Neanderthal groups were much larger or more genetically diverse than the others discussed. Thorin looks to have been part of a very small and closely related group. Thorin’s group re-occupied the Grotte Mandrin cave after the first wave of modern humans entered Europe (Neronian culture), which suggests they may have helped force them out, or waited to claim territory once the newcomers left or died out. What the relationship was between the Thorin group and the possibly expanding Vindija group was is something that will hopefully be revealed in time. Maybe Thorin was caught between modern humans moving in from the Levant and newer, healthier Neanderthals moving in from the north? For now we have some tantilising glimpses into the populations structures of the last Neanderthals, and perhaps some insights into their social structures and psychology. Small groups seems to be the recurring theme throughout much of their history, which would put them at odds with the more socially flexible modern humans. Language divides, if they spoke something like a modern language, may have been huge given these time depths of isolation. How mentally flexible were they? Could they adapt to changing circumstances quickly? So many questions.