My practical mind tells me that weight is the only thing that's important. My engineering mind tells me that weight is critical but when stepping up your game, your height will play a large factor that may need to be addressed. See if you're a 190lbs person that's 5' tall, as you lean forward or backwards on the ski, it has an effect on how the ski rides in the water but not a large effect because your center of gravity is lower over the ski. As a shorter person, your head and shoulders can move more without affecting the way the ski rides. If you're 6'7" and 190lbs, your body position will be more critical because your center of gravity is further away from the ski. So smaller movements of your head and shoulders will produce the same rides changes of that ski. To minimize the effects of your body movement and give you more wiggle room, a taller person needs a slightly larger ski.
In case that didn't make any sense, I'll try to explain it a different way. We're all about leverage in skiing right? As a tall person, the boat has a greater leverage advantage with a tall person because the pull is coming from your shoulders which are further away from your ski. Think of it like pulling a small tree down by tying a rope at the top and tying a rope in the middle. It takes less effort to pull the tree down when pulling from the top. Well to help counter the boats additional leverage advantage, the skier can increase the size of ski which makes his base more stable. The bigger ski is like that same tree having larger roots which helps resist you trying to pull the tree down.