Wow great point, but you left out that James Madison was an aristocratic a-hole who was terribly afraid of people who didn't own land.
Whenever Madison talks about "the weaker party" or "the minority" it's obvious the minority he has in mind: wealthy landowners.
"the Constitution was intrinsically an aristocratic document designed to check the democratic tendencies of the period." - Gordon Wood
Or as Madison's close colleague John Jay put it,
"Those who own the country ought to govern it."
That's not really in dispute among Madison scholars.