Not only will the vast majority never be able to attend Pan Ams.......... they just don't care.
Than, the Olympics is not just an event. It is a brand that is recognized worldwide for what it is. If you polled 100,000 people in the US if they knew what the pan am games are and you'd have like 3 that did.