Blacks in America were excluded from voting, for the most part, until 1964. I suspect that did not happen to your grandfather, who arrived 200 years after, most likely, the descendants of most slaves.

This is a circular argument that those who don't want to acknowledge the effects of slavery love to present.