There were also significant contributors such as Horace King, who, according to the New Georgia Encyclopedia, "was the most respected bridge builder in west Georgia, Alabama, and northeast Mississippi from the 1830s until the 1880s."
There were the enslaved rice farmers of the low country, without whose knowledge plantation owners could not have sowed their rice.
There were significant contributions of freed slaves in places like St. Augustine.
Monticello was probably mostly built by slaves, but history, in my opinion, has hidden some (but not all) of this fact from view.
And on and on it goes.