A 5 year review of world wide elite scores

[Horton]

I set out to review the data from the last 12 months. I expected to find that some countries or sites that had abnormally high scores a year ago did not this year. This turned out to be somewhat true but when I sliced the data a number of ways the bigger dataset turned out to be the most interesting and perhaps the most defendable.

The graph below was created with the following criteria:

• Data was mined from http://www.iwsftournament.com/homologation/idopenslalom0419.php
• Skiers #2 through #11 from the current world standings list - Smith was excluded because he skis in Europe less that the other skiers and because his scores tend to be an outlier
• Scores were achieved between April 2014 and August 2019
• Countries shown on the chart had at least one score of 70 balls (4 @ 41) or greater

How to read the Gray bars

• These 10 skiers posted scores from 1717 rounds between April 2014 and August 2019
• The grey bar is the number of rounds per country divided by the 1717
• The grey bar is the percent of total rounds per country
• Example – There are 220 scores in Italy. 220/1717 = 12.81% of all scores in the data set

How to read the colored bars

• The bars include the number of times any / all of the skiers scored at or above 68, 70 or 72 balls in each country.
• That total is then divided by the total number of scores from in each country.
• Example – in the USA, 68 balls (2 @ 41) or more was run 414 times and there were 775 total scores in the USA data set. 414 / 775 = 53. In other words these skiers ran 2 or more at 41 balls 53% of the time.
• The bars show the percentage of scores at or above 68, 70 or 72 balls per country.

[Horton]

A few addition comments

• There is no proof about what did or did not happen in Spain last summer but by all accounts the same event this year appears to be beyond reproach.

• Any conclusions drawn from this data should not be a criticism of the skiers. The point of looking at this data is to highlight any events that are not at the same standard as the rest of the world.

• It is my understanding there is a shift of culture and a desire to tighten up the specs at many of the European venues.

• If anyone wants to look at my source data all they have to do is ask.

 

[Horton]

Last thing.... I struggled to find a best way to show this data. If you think you are confused the Bold bullets in the original post should help. Otherwise just ask.

[Horton]

At least temporarily this post is in a section of the web site where only USAWS or members of other federations can comment.

[Luzz]

Interesting way of thinking about this data. Can you retrieve the number of sites and tournaments per country this data is based on? Probably a ton of extra work, but should be able to obtain it from tournament codes.

[bishop8950]

Interesting. What if you normalize the number of scores to a fixed number of events? In other worlds the gray bar is the same for all countries and the score bars adjust proportionately.

[Horton]

@bishop8950 I am can't figure out how to graph the data the way you are saying so here is the source data for anyone who wants to play with it. There are 1543 rounds shown. If you want to include rounds where 68 balls was not achieved the total is 1717.

 

 

	Number of rounds	68 balls or more	70 balls or more	72 balls or more
AUT	16	7	2	0
AUS	153	29	2	0
GBR	70	31	4	1
ESP	75	54	13	5
ITA	220	106	27	4
FRA	234	149	47	15
USA	775	414	70	13

[Than_Bogan]

@bishop8950 I actually think @Horton has already done what you've asked for. I want to loudly applaud him presenting this data, but I agree with his self-assessment that he hasn't quite figured out the best way to present it. The biggest problem is that everything is a percentage, but in a fundamentally different way! On the grey bars, it's a percentage of all tournament rounds. For each of the other bars, it's a percentage of success out of tournament rounds in that country.

 

When the denominators aren't the same, percentages don't relate to each other in any useful way.

 

To me, the grey bars are almost meaningless, except to suggest statistical significance.

And even that is indirect, because pure quantity is all you need for that: 200 rounds in one country is statistically significant even if there were to exist another country with a million rounds. And 2 rounds would not be statistically significant, even if there had only been a total of 4 rounds and thus the 2 represented 50% of the total.

 

What your eye should be drawn to is the unusual height of the green bars, which indicate the percentage of running -41. These are unusual in four countries: In Australia and Austria, they are outlying on the low side (both 0 in fact). In Spain and France, they are outlying on the high side -- more than triple the percentage of completion of -41 than in the US, Germany, and Italy.

 

As always, all sorts of explanations can be possible. For the low ones, Austria is not statistically significant, because at the US rate, the expected number would be about 0.3, making 0 well within the realm of probable outcomes. And Australia is probably heavily skewed by the extreme challenge of the Yarra River.

 

For the high ones, I do not have ready explanations.

[Than_Bogan]

Sidenote about statistical significance: When you are taking about improbable events, you need more data points than usual in order to make the type of conclusion one usually wants to make. Suppose after 100 trials, something has happened 48 times. It's extremely likely that the thing I've just sampled is in the range of 40% to 56% likely. But suppose after 100 trials, it hasn't happened at all. In this case, it's not crazy to think it might be 1% likely, but it also might be completely impossible. I thus don't have any insight into whether it is actually 1% or actually 0.00000001%. If I want to be sure I'm within say a factor of 2 of the real probability, I need to keep gathering samples until I have quite a few occurrences. And if the real probability actually is 0, then I can never use random sampling to get with a factor of 2 of that, and instead will eventually have to give up and say something like "it's 95% likely to be less than 0.001%."

[Horton]

@Luzz Here is sanctions per country. Is this what you are asking for?

 

AUS 17

AUT 2

cAN 12

chi 3

Col 3

cze 2

DOM 4

ESP 8

FRA 34

GBR 13

GRE 1

ITA 42

jpn 1

MEX 6

nor 1

PER 3

pol 1

ukr 1

USA 158

[bishop8950]

@Than_Bogan I would not dare to challenge your nerd skills when it comes to this sort of thing. But the way I was looking at this:

 

ESP: 75 rounds, with 5 over 72

FRA: 234 rounds, with 15 over 72

USA: 775 rounds, with 13 over 72

 

So per round, this says these skiers are ~4x more likely to ski 72 or higher in Spain or France than in the USA (without taking into account boats, drivers, sites, events, etc).

[Luzz]

@Horton kind of. I'd be interested in knowing the same event over the years. E.g., 18ITA001 and 19ITA001 would all fall within "San Gervasio Pro Am". As I said, lots of work, so no problem if it's unfeasible

[Than_Bogan]

@bishop8950 I agree. And the green bars show exactly that. The only slight caveat is that I'd say Spain is "borderline" in terms of statistical significance, because the expected number, at the U.S. rate, would be 1.2. So if 4 random people just happened to totally kick ass one time each, we could arrive at 5. That's a little improbable, which is why I call it "borderline." I would consider France to be statistically significant, because having 11 "random amazing performances" beyond the expected value is not nearly as plausible.

[Horton]

@Than_Bogan @bishop8950 @Luzz

 

There are really all kinds of problems with this data because it is a small data set and because of other unknown factors. This data is an indicator of a problem but not statistical proof of anything specific. My suspicion is that standards are more variable in parts of Europe.

 

One of the top pros in the world told me that he does not see it as a country issue but a site issue. One site might be dead on the rules and the next site 10 miles away is playing fast and loose with everything.

 

There are all kinds of downsides to the events with inflated scores. The World standings lists become meaningless. All scores become less meaningful because we do not know which are real and which are fake. Sites that do put on legit events are looked at with the same suspicions as those who do not. When skiers do achieve something special at a legit event it is undermined by fake scores. Freaking Standards Matter

 

From my office here in California I am trying exert pressure on the European federations to just pay attention.

[Horton]

@luzz here is the raw version of what I think you are asking for.

 

 

42 events	220 Rounds	68 balls or more	70 balls or more	72 balls or more
19ITA013	2	1	0	0
19ITA009	1	0	0	0
19ITA001	27	18	4	0
18ITA016	1	0	0	0
18ITA015	3	1	1	0
18ITA014	3	3	1	0
18ITA013	3	3	0	0
18ITA011	3	2	1	0
18ITA001	27	15	2	0
17ITA013	4	1	0	0
17ITA012	2	0	0	0
17ITA011	2	2	0	0
17ITA010	4	3	1	0
17ITA008	3	1	1	0
17ITA007	3	0	0	0
17ITA006	2	2	0	0
17ITA003	2	1	0	0
17ITA002	2	2	1	0
17ITA001	21	11	3	0
16ITA020	2	0	0	0
16ITA016	3	2	0	0
16ITA013	2	0	0	0
16ITA009	6	4	2	0
16ITA008	16	10	5	2
16ITA007	14	11	3	2
16ITA006	4	2	0	0
16ITA004	2	0	0	0
16ITA003	3	0	0	0
15ITA018	2	0	0	0
15ITA014	4	1	0	0
15ITA013	4	1	0	0
15ITA011	3	0	0	0
15ITA009	3	0	0	0
15ITA008	3	2	1	0
15ITA005	10	1	0	0
15ITA002	5	0	0	0
14ITA013	2	1	0	0
14ITA012	2	0	0	0
14ITA009	6	3	0	0
14ITA006	6	1	1	0
14ITA003	1	1	0	0
14EAC001	2	0	0	0

[Luzz]

@Horton pretty much. So, for example:

San Gervasio Pro Am

105/220 rounds

57/106 68+buoys

13/27 70+buoys

2/4 72+buoys

 

Pretty much 50% of the rounds are accounted by one pro event (the only one in that country). This data can initially make people raise eyebrows. However, I believe it highlights the simple fact that pro events at a good site are likely to generate a lot of high scores.

 

I'm pretty sure that if one was to see the data like this for the US, similar results would show up, namely all pro events at good sites would account for possibly 50% of the scores.

 

I believe the efforts of understanding why some tournaments in the past produced an unusual number of high scores are justified. At the same time, I believe that providing standards and making sure those standards are followed is the issue here, and a point that can be made without fishing for data interpretation.

 

Truth to be told, the E&A region assigns judges for RC events. While this may not be enough of a scrutiny and control system, it is already a start to make sure that no intentional rule bending occurs. I know this is not the case in the PANAM region, not sure about A&A. The issue is that not all tournaments are RC, and if you ski a score in any of those categories (including 0@43), as long as it is not a record, the regional or world TC doesn't have to review it.

 

One of the issue with this is that our sport doesn't have professional judges, something that if it were in existence would drastically diminish conversations about scrutiny. By the same token, we are a super small sport, so the whole volunteer judging system we have in place is somewhat necessary to run events.

 

In short, more scrutiny is needed, pro tournaments should all be RC, and thank you to the judges and drivers that invest their own time to help the sport :smile:

 

( @Horton I know you are not on the other side of the points I am making. These are just my thoughts after thinking about this. )

[Horton]

@luzz I hate that the above graph data infers that all sites in Europe or in any one country may not be within bounds but I am not sure how else to make my point.

 

EVERYTHING I have heard about the San Gervasio Pro Am is that the event is run by the rules and is a model tournament.

 

I also have to say that it appears that the Seseña event was by the book this year. It was the Seseña event in 2018 that brought the issue to a head.

 

The question I want answered is why some guy with a website in California has to point fingers before the people in charge of international start asking questions?

 

For the integrity of the sport and for the elite skiers lets all ski to the same standards.

[Luzz]

@Horton . I just talked about that tournament because you pulled up that data. I know my event is legit.

 

All I am trying to say is that there is more scrutiny in Europe than in the US, and more questions are asked than you or I or people we both know are aware of.

 

That doesn't mean that questionable scores can pop up anywhere, Europe, US, or elsewhere.

 

One of the things that could be considered is not to officialize any non-RC score at 43off until review. That is the case right now for 11k+ scores at non-RC events. However, that is also contributing to the lower number of scores in tricks, lower tournaments, etc.

 

Again, the point is well-taken and necessary, but I am not sure that an analysis like the one you are doing is going to strengthen it.

[Horton]

@Luzz my analysis might be very flawed but if I get people talking about the subject and looking for solutions then my job is done.

[Than_Bogan]

I think it goes too far to call your analysis "very flawed." I think about the harshest criticism that should be applied is "cursory" -- i.e. it hints at some things to look into but doesn't provide further details.

 

Well, also I'm deducting points for displaying things on the same scale that are totally unrelated, but that is trivially fixed by just removing the grey bars. :)

[mshaw200]

I would be curious to know how the scores in 2019 have been affected by the new MasterLine ropes.

[andjules]

Thread summary, courtesy of @Than_Bogan (what @Horton is really trying to say?):

What your eye should be drawn to is the unusual height of the green bars, which indicate the percentage of running -41… In Spain and France, they are outlying on the high side -- more than triple the percentage of completion of -41 than in the US, Germany, and Italy.

[HighAltitude]

In a former life, I taught probability and stats at the Air Force Academy and then used that knowledge to do test and eval and space systems. This looks like a perfect area to apply some more rigorous statistical analysis. With that said, I don't think it takes any serious stats skills to say that this sport can be "site" specific. The real question is what makes a specific site better than another? Thus, breaking down the data by country doesn't do the analysis any justice. You would have to look at all sorts of crazy data to see if this is truly site specific or if it is boat specific, driver specific, weather specific (temp, winds, etc), time of year specific, country specific or a multitude of any other factors.

It's an important question to answer if you are a world record holder but that's about it.

The most important thing is that everybody competes at the same site in a tournament. Golf is a perfect example. All the different courses are rated by slope and difficulty to make sure the handicap system works for people playing any course, but people also understand that some courses play hard or play easy. Relative performance is the only thing that determines a winner on any given day.

 

[Horton]

@HighAltitude

Your golf comments would be true if we eliminated the world standings list. The current expectation is that all events are run to as close to the same specs as possible. Your comments make sense if buoy width, boat speeds and rope length we're not exactly specified universally / worldwide

 

When some ski sites run fast and loose with the rules and inflate the scores the athletes who obtain those scores move up the standings list compared to athletes that only compete at sites that are exactly on spec. I have heard Elite skiers talk about the need to go to specific events so they can get ( illegitimate ) high scores to maintain their world standings. That is ridiculous and makes the sport look stupid.

[Horton]

@HighAltitude there is a link in my original post to all of the source data. If you're interested knock around in there and see if you find any patterns.

[Horton]

@Than_Bogan I don't really think that my analysis is flawed but I am clearly not a statistician. I would not fall out of my chair if somebody reviewed my work and found some error. I double-checked it more than once and I'm pretty sure my methods are correct but I did not get a second set of eyes to review.

 

What I am confident of is that no matter how you crunch the data you would come to a very similar conclusion.

 

The best thing for the sport is if more people took a hard look at the raw data.

[rfa]

Now that I have learned stats (and pole jumping) from @Than, I feel I can comment here...LOL.

I should disclose that I was never a fan the original tread about the "unusual scores" in Spain. I had a similar "bad taste" around the same time, about another tread on "unusual" high scores at an amateur tournament site in NJ. My discomfort with those was the innuendo that some type of "fraud" was responsible for the high scores. I have no idea if any "fraud" was present (narrow course, slow boat, bad boat path, lenient judges, whatever), so personally, I would not bring it up without some proof. Conversely, I agree with @Horton that "questions should be asked" and answered.

My comment on this analyses is not whether it is perfect or flawed. Looks like a good compilation of available data. (as a less nerdy R&D engineer now that I am old, I still love "data"). My view is that this does not increase the potential justification to ask whatever questions might be legit about the Spain tournament. It was a one tournament situation. I suspect that if no answers were given to date, this will not provide any further impetus. @Horton says "it appears that the Seseña event was by the book this year". Does "by the book" mean that no unexpected 41's were run this year? Or do we have some new relevant information about how it was run last year?

What if one looked at the graph and asked about the lack of 41's in AUS and AUT? Wide course, fast boat, "too good" boat path, over zealous judges, whatever).

Bottom line, great information compiled and presented but, in my opinion, not changing the need (or lack thereof) for questions about Spain 2018.

[Horton]

@rfa ideally I would have left Spain out of this data set but as soon as I start including excluding parts of the data I think it becomes less valid. Furthermore grouping the data by country is also not ideal. The suggestion that one particular nationality is honest and another is not is ridiculous. With a data set so small there has to be silos and unfortunately the logical silo is by country.

 

I can't say this enough. My goal is to make people at the high end of the sport look at the data and see the anomalies.

 

[HighAltitude]

@Horton I'll try and look things over when I have time, but just looking at what you have here, I'd say there isn't much more to analyze. Yes, we could show there is a statistically significant difference between sites and we might be able to use some sort of confidence interval to show how unlikely something is, but....I think it will become clear that some sites ski better than others. The real question is why. Are some playing "loose and fast" with the rules or is it consistently better conditions?

 

I think one boat manufacturer currently has more world records behind it than the others (I haven't seen those ads lately, so I'm not sure if its true anymore). Does that mean that boat is better? Could that be the difference? I don't believe that but it goes to show how many variables there still are despite an attempt to minimize them.

 

BTW... pro golf has world rankings as well.

[bigtex2011]

@Horton I'm intrigued by the new hard hitting Horton. Are you going to focus your new fact finding hard hitting expos'e to equipment that fails (broken skis, ropes that are too long or break, bindings flaws) How many and what kind. I'd love to see your stats on AWSA membership or what happened to TRex was it an asteroid or global warming.

 

Back in the DAY or Lapoint and Mapple I feel confident some sites were probably more skier friendly that others. In the end, it all works out.

 

 

 

 

[Horton]

@HighAltitude Boat manufacturer is not a factor. The brand with the most top skiers on salary has the most records. I do not remember the last time a world record was set behind a brand of boat who was not paying that skier.

 

Golf has ranking but the criteria is totally different*

 

( * assumption )

[Drago]

@mshaw200

You could plot Nationals

[Horton]

looks like I need to update...

 

http://www.iwwfed-ea.org/competition.php?cc=T-19FRA008&page=live

[ToddF]

using the data @horton gave this is a percentage break down of rounds run for 70 and 72 buoys I left out 68 because it skewed the chart from the higher end performances

 

 

[Horton]

@ToddF did you update the data or just reformat my old data?

[ToddF]

Old data from thread, I could easily do new data though

[ToddF]

Here is another way of looking at scores based on their % to the world average

still using original data. Not sure if Australia skews things with it being a river with obstacles

 

 

 

[JAS]

If a tournament was to have unusually high scores for competitors across the board (compared to averages), what areas of standardization would be suspect. Opinions?

[Horton]

course width

rope length

boat path

boat speed

 

[Stefan]

Someone seems to think the rules are stretched to the limit in the US...

Title to clip is ”Why does U.S. customs take 6 bouys?”

[MattP]

I disagree with his comment that we run courses long and narrow. Actual is the goal and the tolerances allow for natural deviation. I don’t know any TC that puts a course in that stretches the rules and does not strive for actual.

[TEL]

@MattP I believe he said the lakes are made long and narrow for perfect conditions

[MattP]

@TEL 1:17 mark. He says we run them long and narrow. @JoelHowley

[JoelHowley]

@MattP Not sure if I should get into this, was just trying to be honest and it was meant as a compliment.

 

I see you are of the opinion that “Actual is the goal and the tolerances allow for natural deviation”

 

99% of courses would be set to actual but records are set at the limits.

 

I don’t see many pro skiers with their handle length measuring actual, maybe I am morally wrong for pushing the limits on my handle length but I’m not sure rules designed to judge intent could ever be enforced.

 

Maybe actual should be the new minimum tolerance for width and maximum tolerance for length. Please don’t be offended by all this. Was just trying to be honest. I will try and return to being less controversial in the next vid, haha.

 

I’m sure this thread must mention all the sketchy European scores. I have never seen any attempt to push limits beyond the rules in the U.S. (apart from maybe buoy height which plenty of people justify as a safety issue) ?

[Horton]

@JoelHowley I appreciate you saying what you think. Perhaps there are some sites that were set up on the edge of tolerance. Generally speaking I think the current culture of USAWS is to tighten specs on every thing.

 

 

[RichardDoane]

"the boat pushes the cable down, and that draws the buoys in" @horton - How many of the suspect courses are not anchored to the bottom ?

[Horton]

I am aware or only one Record certified cable course in the US. That a cross course system at Okeeheelee.

[JoelHowley]

Thanks @Horton

 

@RichardDoane I said "down" not "in". I said down intentionally however if I am being completely honest they may come in a fraction also.

 

[Adam Caldwell]

@JoelHowley were you betting on the boat wake action on the two ball boat guides at worlds helping to pull 2 ball in/down when you went for broke at worlds? No pun intended :)

[JoelHowley]

@adamhcaldwell haha, yeah I would have turned 2 ball for sure if they had a cable course. I was only about 2ft inside it.