There was one difficulty however of a serious nature attending an immediate choice by the people. The right of suffrage was much more diffusive in the Northern than the Southern States; and the latter could have no influence in the election on the score of the Negroes. The substitution of electors obviated this difficulty and seemed on the whole to be liable to fewest objections.
https://founders.archives.gov/documents/Madison/01-10-02-0065