Reduced productivity leading to more land use (hence forest clearing) would be one of the negatives I can see.

Luckily we’re probably going to be on the downswing for population soon, but in the meantime, we’ll probably lose the Amazon to low productivity farming.