Elitism, by its nature, is exclusionist. It has as its origin the Calvinist idea of the Elect; those whom God has ordained from birth to be his chosen. It's the kind of belief that makes kings magically cure scrofula by exposing their ring, and purple be a colour reserved for royalty and the priesthood under pain of death.
If you mean that those blessed with an education and resources are envied, then I don't think you will find too many arguments. If you say that different pursuits have different social conventions that favour those brought up within them, then again I can't fault you - after all, Rex Harrison would have had just as many problems at a Dogfight as poor old Audrey had at the races.
"Go on, you huge canine; hasten the destruction of your opponent."
IF, however, you are implying that certain pursuits are beyond the reach of groups of people, and that these pursuits are somehow innately better, then you start to tread into uncomfortable territory. Some pigs are happy not to be Socrates, and Socrates ended up dying from Hemlock, whereas Saddlebottom went onto a decent career in the army.