For decades researchers have been uncovering the genes living people owe to the extinct Neanderthals—a genetic legacy that boosts our immune systems, helps our blood clot, and may play a role in depression. Now, a study of ancient DNA turns the tables on Neanderthals and asks: What did they get from us?

The Neanderthal-eye view allowed the researchers to date when the two groups mingled, finding they made babies together remarkably early: more than 200,000 years ago, not long after Homo sapiens coalesced as a species. The dalliances were repeated 105,000 to 120,000 years ago, and 45,000 to 60,000 years ago, the ancient Neanderthal DNA suggests. It argues mating was more common than previously thought.

This new picture further blurs the boundaries between Neanderthals and modern humans. And it identifies features of the Neanderthal genome suggesting our big-brained, heavy-browed relatives were pitifully rare, which could help explain why they went extinct. “It’s alarming to see how small the Neanderthal populations were—this is a very powerful result,”

Once seen as a separate species, Neanderthals have enjoyed a complete makeover in the 14 years since researchers first sequenced DNA in their fossils and found they interbred with modern humans. Most living people outside of Africa have inherited about 1% to 2% of their DNA from Neanderthals, perhaps from a prolonged period of mixing 45,000 to 60,000 years ago in Europe or the Middle East.