It is true, as I understand it, that the modest amount of progressivity left in the Federal tax code means that most income taxes are paid by people who are at least moderately well off. It is also true that Federal income taxes, proper, are only a part of the overall tax burden, and when you factor in other taxes (FICA, sales taxes, gas tax, state income tax, real estate taxes) the overall tax structure in this country becomes regressive (poorer people pay a higher percentage of their income in taxes, overall, than rich people, and MUCH lower than the super-rich. That is a reason activists often give in explaining why having rich people pay more in income taxes is fair.
But there's another reason I think is even more important. Wealth, ALL wealth, ultimately derives from profits - i.e., from something like a rent for the use of your property - and not from labor. This is true even if a high income is disguised as salary. NOBODY's labor is "worth" more than 200 times the median. Ultra-high CEO salaries come from their ability to control property (their own, and that of shareholders).
And profit is essentially a privatized tax - a tax that we pay on every good and most services that we consume, but that goes strictly to the benefit of a small class of individuals. (Small in percentage population terms; rather large in absolute numbers.) Profit represents the owning class blackmailing us by threatening to withhold the means of production, which would mean that we could not, with our labor, produce anything at all. And the more that wealth becomes concentrated, the stronger the stranglehold that property has on us, and the more they are able to squeeze out.
What progressive tax rates do, in a capitalist society that refuses to simply expropriate private owners, is to reclaim part of that private tax, for the public benefit. More progressive tax rates actually represent a LOWERING of the private tax that is capitalist profit.
And that is why progressive taxes are fair.