It was definitely part racism.

The Three-fifths Compromise was an agreement reached during the 1787 United States Constitutional Convention over the inclusion of slavesin a state’s total population. This count would determine: the number of seatsin the House of Representatives; the number of electoral votes each state would be allocated; and how much money the states would pay in taxes. Slave holding states wanted their entire population to be counted to determine the number of Representatives those states could elect and send to Congress. Free states wanted to exclude the counting of slave populations in slave states, since those slaves had no voting rights. A compromise was struck to resolve this impasse.

Emphasis mine, but the electoral college was originally disproportionate for more than one reason.