Video Based Pose Estimation Example

[Obrienslalom]

A couple of years ago, I looked at predicting human poses from an image and/or video. It turned out to be more effort than I was willing to put in at the time (especially for water skiing, since there is a lot of noise and occlusion via ropes and spray).

 

Anyway, it led me to follow a few people's work in the space, specifically Michael Black's work:

https://github.com/mkocabas/VIBE

 

I ran his latest project on a clip from the 'Terry Winter at Quarter Speed' video from Horton's channel. Out of the box, this performed amazing well. Also, there isn't any specific training towards this type of movement. Very promising in my opinion.

 

I also tested it against some random clips from youtube, but I won't subject randoms to this discussion. Since this is in quarter speed, and very well zoomed in, I assumed it would be much better. Honestly, it's better in some areas, but much more jittery for some reason.

 

Here is the result:

 

You can find a ton of work in various sports doing this sort of thing. From the tracked joint data you can do a lot of automated analysis or comparisons between runs. It's also interesting to see the 3D pose, since we rarely get that view in this sport.

 

Thoughts?

[BG1]

The image on the right skis exactly like me.

[Chris Rossi]

Id be interested in seeing more of these.

[aupatking]

I want to make jokes about it, but when you slow it down even further, it really does look like there’s potential there. I’m sure with time spent teaching it to recognize the position of the feet is (fairly) static, and ironing some other little quirks out, there’s decent data. If you were able to run this with enough video, I wonder if you could factually establish what is truly the most important fundamentals? Basically score the skier

[jimbrake]

that was oddly disturbing to watch

[jetpilotg4]

I agree with you @jimbrake Hate to see what I’d look like on it Frightening I’m sure

[jpattigr]

Would like to see Robert Pigozzi overlapped with his unique style!

[Obrienslalom]

Hahah, well I'm glad I put minimal effort into working on something. Didn't know everyone would be creeped out by it!

[klindy]

That's fascinating. The potential for coaching, judging, even ski design could be huge.

[Tevskis]

I LOVE this!!! I also started laughing ridiculously hard this morning when I first saw this! I see in this a near perfect representation of why top short-line skiers run exponentially more buoys than the masses. Makes me smile hearing Terry say "if you don't feel like you're going to eat s@$% at any time you're probably not doing it right"

 

Following just the 3D figure, his mass dang near throws him out the front after loading through the gates.

Yet in the video, Still, Tall and early into the buoy-line.... Perfection :)