I can't comment on course or competitive skiing, but I have thoughts about recreational skiing at least in my small part of the world. I do think the sport is very expensive, but for arguments sake, let's say money is not a barrier. My wife and I both work part time, and live on a lake with our boat on a lift in northern Wisconsin where it's hard not to live on a lake. It doesn't get any easier for us to water ski. There are basically zero course skiing opportunities up here, but that's an entirely different issue. However, the biggest thing preventing me from skiing as much as I want is that this goofy sport always needs at least one other person to do it. No matter what, there has to be a driver. We're lucky in Wisconsin to no longer need a spotter, but in how many other states is that a requirement? So now you need one, if not two other people to go along with you.
I participate in a lot of different activities year round. Water skiing is the only one I can't do by myself. I no longer participate in team sports, but even a lot of those can be worked on by yourself (at least technique/fundamentals). I think slalom skiing is very difficult, and needs a lot of time spent doing it even to be a good free skier. I only get better when I'm behind the boat. It's very hard to do drills or work on fundamentals without being behind the boat. That's not the case with many other sports/activities.
I guess what I'm trying to say is that it comes down to access in a certain way. Even if you are the most driven skier alive, have a brand new tournament ski boat on a private lake with a course, 24 hours of daylight with 85 degree temps, you can't do anything without a driver. And the weather/wind is another variable that really impacts this sport especially when dealing with public water. I won't even try to address the topic of other boats sharing the water. Lots of things need to align in this sport. Not so with many others.