Just for Than and Eric – Partial Credit

[Bruce_Butterfield]

@Than made a comment in another thread that I thought warranted further discussion and didn’t want to hijack that thread, so here goes.

 

Than wrote:

" Than_Bogan Posts: 5,167 Mega Baller

1:09PM

I've said this several times over the years, whenever Eric brings up partial credit, but for anyone new to the discussion:

 

If you want an accurate score, thresholding is bad.

 

Any time you force all-or-nothing, the score becomes unstable. This principal extends way beyond trick skiing into all forms of scoring metrics used to solve all sorts of real-world problems. "

 

 

Interesting idea. It was brought up in the context for partial credit for tricks, but if an accurate score is what we are after, for the sake of inclusion and fairness, let’s apply it to all 3 events:

 

<>

 

For jump, the jumper would get full credit for his distance when he rides out a jump as normal, but would receive partial credit as follows:

- Lands on his skis but fails to ride away – 10% reduction

- Fall out the back - 20% reduction

- Falls out the back, but the helmet is the first thing to hit the water, 30% reduction

- Falls out the front – 50% reduction, but judges may award and additional 30% for “cringeworthiness” of the fall

 

For slalom, we currently have ¼, ½ and full buoys, but that doesn’t accurately score the skier relative to the others, so points will be awarded as follows:

- 1/8 point for crossing the wakes and skiing within 3 feet inside the next buoy

- ¼ as currently scored

- 3/8 if the skier hits the buoy and “has a yardsale”

- ½ as currently scored

- ¾ if the skier either gets back to the line of boat guides after the next set of boat guides, or begins his pull, but loses skiing position before reaching the line of boat guides

- 7/8 if he gets back to the line of boat guides, but loses the handle and doesn’t ski away

 

<>

 

Ok, so hopefully we can agree that the above would indeed more “accurately score" each skier, but would be a judging nightmare and would do next to nothing to distinguish the better skier of that day.

 

The same can be said of partial credit for tricks. In the other thread, there were discussions about pre-turning, turning before getting in the air, holding a side-slide for 2m (an absurd criteria BTW). Just imaging the arguments and variations in “judging” if some form of partial credit is implemented and the long list of criteria it would involve. Remember that the judges have to call tricks in real time.

 

The bottom line is that there has to be some sort of “threshold” in any event to determine the score – even if you apply partial credit, what is the “threshold” for partial credit? The current trick scoring criteria is reasonably clear once you study it enough. The last thing we need is more subjectivity in judging. Partial credit would be an absolute nightmare.

 

JMO, YMMV B)

 

 

[klindy]

@Bruce_Butterfield the suggested "partial credit" idea for tricks didn't work that way at all. In fact @eleeski and @Than_Bogan may not like it but it was certainly simplified.

 

The idea that was suggested to me didn't change the definition of credit/no credit for the trick. What it did was basically modify the score depending on the number of judges calling credit/no credit. So with 5 (or 3) judges calling the trick credit on a trick the skier would get full credit. If one of 5 said "NC" then the value of that trick would be reduced by 20%. So if 3 of 5 said "NC" the score would be 40% of max instead of zero. (The same logic applies for 3 judges).

 

Interestingly this approach tends to lower the top level of scores but raises the lower scores and, to some degree, the middle of the pack. So it's not a different judging approach but instead a different way to score tricks.

[Edbrazil]

Tricking used to have a provision for 10% or 20% off on a trick not quite done right. That

was in the 1960s and earlier, to my memory.

[Horton]

@Bruce_Butterfield If you can write this up in a rules format it can my April fools for next year. @skimom will not know what hit her.

[eleeski]

It's too bad when people don't understand basic statistics.

 

@Bruce_Butterfield 's sarcasm is just insensitive and thoughtless. Funnier than heck on April 1 but out of line today.

 

@klindy has perhaps the worst possible partial credit plan imaginable - it guarantees that full score can't happen.

 

As a senior judge with a lot of experience, I assure you that the hardest part of judging is deciding credit or not. This takes more real time than actually recognizing the trick or even recognizing what was wrong. No, the decision as to whether the form break is severe enough to warrant a score of zero takes the time and creates the difficulty.

 

Can I see a tip drag? Of course. Deciding how much weight that tip drag carries is tough. Real easy to assign a 10% cut for that and get on to the next trick. Some judges just consistently cut anything amiss - that's easy too, but not good if we want skiers to stay engaged and enjoy skiing tournaments.

 

Diving, gymnastics, figure skating and many other sports are subjectively judged quite well. Certainly better than the randomness of trick judging.

 

Eric

[Than_Bogan]

Disappointing. A post like this should be funnier.

[Bruce_Butterfield]

@Than sorry you didn't find the humor. I guess that's just one more data point that geeks have a strange sense of humor:)

But the serious side was taking disagreement with your comment about "thresholding" being bad. There is always some sort of threshold, even for partial credit.

 

Anyway, the point was that partial credit would further complicate an already complicated sport. Very few people outside of Eric think that going the route of gymnastics and figure skating would be an improvement.

[klindy]

The idea I presented was my understanding of the suggestion of the current world record holder. Dumb idea? I tend to listen to those who it affects the most.

[Chef23]

I thought the jumping section was pretty funny and appreciate that it was done with good intentions.

[RazorRoss3]

Trick is the anomaly in our sport because it's hard to be as objective as you are in jump and slalom. Jump, who went farther? We have computer measurement and landing vs not landing is a pretty clear distinction. Slalom, did he go through the gates and around the buoys? I understand that reasonable people can disagree on whether or not a gate was missed or 1/2 vs 1/4 buoy but usually its fairly apparent. Trick is odd because every trick looks a little different and has different rules applied to it and it is much easier for reasonable people to disagree. I'm not sure if it would create more or less accurate or more inflated or deflated scores to add more variability to the score of any individual trick. And can you imagine the challenges that would occur? I would generally prefer the binary method, you scored or you didn't but I've always been drawn to the more objectively scored sports anyways so maybe that's just me.

[ToddL]

In gymnastics, there are difficulty points and demerits for deviations from perfection. It is an objective potential score based upon the routine with subjective penalties for imperfection. They watch the routine and mark down instances of demerits or omissions. When it is over, the total potential routine score is reduced accordingly by the weight or number of the demerits.

 

The difference from that and trick skiing is that the gymnastics judges have the routines in advance and the maneuvers are not performed in rapid succession crammed into a 20 second time frame. That rapid succession is what makes trick judging hard.

 

Take Joshua Briant's Recent NCWSA Nationals Trick run:

Pass 1 Score 6820 Skis Trick Results Score 1 B Before 0 1 BFLBB Credit 800 1 RBFLBB Credit 800 1 F Credit 60 1 FFLO Credit 800 1 BFLB Credit 750 1 F Repeat 0 1 RBFLB Credit 750 1 FFLF Credit 850 1 SLO Credit 400 1 B Credit 60 1 SL5F Credit 550 1 SL5B Credit 550 1 SLBB Credit 450 1 F OOC 0

There are 13 scored tricks in 20 seconds. (Actually, he skied a few more that were out of time, but judges have to call everything and figure out what was in time later.)

 

This pass was actually well paced due to all of the flips. (Some Toe passes have tricks occurring faster than 1 second each!) Still...

 

So, on average the judges had less than 2 seconds to

see the maneuver,

recognize it,

correctly speak the name of it into a voice recorder,

determine if it was sufficient,

if not, say "no credit" possibly adding why

 

Judges don't need slow motion for judging. Judges need time between maneuvers to document their decision.

 

(It would suck and take too much effort to do this, but it would be ideal for the act of judging if the trick video was prepared with each maneuver isolated and blank time added between each one. In reality that would take too much time, technical skill and just isn't feasible.)

 

 

Joshua's Trick Run Video: (his starts at 25 seconds)

[klindy]

@ToddL not everyone uses a voice recorder. In fact, I'd say that the majority of us who can call 11K+ don't use a recorder or a secretary. I haven't used a trick secretary in 25 years.

[thompjs]

@klindy Back when I called a lot of tricks, I would just watch the pass and then write down what they did. That worked up to about 7K.

 

11K you need to do some sort of hieroglyphics.

[Chef23]

@klindy is that watching video where you can stop it or is that watching it live? Some of those toe passes the tricks are coming at you really quick and can be hard to keep up with.

[klindy]

@chef23 Watching live. I watch and write for myself. I do use some symbols especially for flips. Flips go slower so it's easier to write but the abbreviations are so long you can't write them fast enough. For 8-9K and above I typically leave off the "T" for toe tricks on my initial 'back page' notes.

 

I typically fold over the long edge of the pink sheet and write without looking down (keep my eyes on the water or the monitor). Then I transcribe so someone else can read them.

 

Some of the judges from around the world have some pretty high level codes that are somewhat universally known especially in the countries/areas where they call a lot of tricks. Others will even provide a 'cheat sheet' to translate.

[Chef23]

@klindy thanks. I am still improving as a judge and hearing this stuff helps.

[Edbrazil]

I sure appreciate the difficulty of calling tricking in real time. Or, writing it down. To

write down the full letter code for each trick is basically impossible for big runs. A few

years ago, I did experiment with voice recognition on a computer, but that was not

accurate enough. By now, that may not be the case.

Back in the 1960's, I did put together a verbal shorthand for tricks, where they are

called like they code looks. Worked for me; didn't get adopted for others, and also

generated a bit of humor. Once the Tour got Tour-ing and didn't have tricking, I

didn't keep up. Now, I'm just an Assistant in TR. But, still practice on video at times.

 

One story from the Olde Daze. This is when Al Tyll was coming into prominence, and

setting records, including the first score over 4,000 points. Sounds simple, but Al

was doing nearly one trick per second. So, here's this judge from the 50s and early

60s attempting to call Al's run: "He does a wake helicopter 360 front to front, and

then a reverse wake helicopter 360 front to front...uh...glub...glub..." By that time,

Al was about into his 5th or 6th trick.

[eleeski]

This is my partial credit proposal.

 

Trick recognition is the prime focus of judging. You can pass the regular test without cutting a trick. The threshold is recognition, not perfection.

 

So if you can recognize a trick, it should score most of its value. Not half or 2/3 but more like 90%. Of course some tricks are badly blown in many ways. Larger deductions need to be an option for the judges (50%) but the standard deduction should start at 10%.

 

Trick executed in compliance with current standards, 100%

Trick executed with minor form breaks, 90%

Trick executed with significant deficiencies but recognizable, 50%

Trick not readily recognizable or with significant intentional deficiencies, 0

 

In the few hundred milliseconds a brain needs to process data, there is plenty of time to select one of these options in judging the tricks. It is easier than making that all or nothing decision on a borderline trick.

 

A scoring average of the judges would give the final score. If 5 judges are used, throw out the top and bottom scores.

 

Note that the spectators are watching for trick recognition, while their view may not be as good as the judges' they can still recognize what is being done. Judging should at least come close to what the crowd sees.

 

Eric