Collecting data (load, acceleration, audio, visual) while waterskiing for AI project

[david_quail]

Hi folks,

I'm an avid albeit fairly beginner slalom skier and I'm doing my PHD in Reinforcement Learning (a branch of artificial intelligence). I've mostly used streams of sensorimotr data coming from a mobile robot for my experiments. But for the summer, I thought it might be fun to study data generated while skiing. Fun and it would allow me to say that I'm working on my PHD while water skiing!!! Yay less guilt skiing mid day during the week!!!

 

Do any of you have thoughts / experience collecting such data (I'm thinking line load, accelerometer, visual pixel data, etc). This would involve spending money on robotshop (another passion of mine) in an effort to instrument some combination of the ski, handle, pylon, myself. I seem to remember other threads which talked about monitoring speed, acceleration, load etc. I have lots of ideas but wondering what I may learn from others who have done so before ...

[keithh2oskier]

I think @Horton had something that tracked acceleration and speed that he attached to a video at some point.

[Horton]

@david_quail

 

I spent a stupid amount of time on this before I understood that the hardware could not keep up with a skier and it was a waste of time.

[vtmecheng]

@tap should stop on by this thread. I recall he has/had done some data acquisition with skiing.

[BraceMaker]

@Horton the necessary data can all be extrapolated from video. You want to instrument the boat not the skier.

[eleeski]

A smart pylon might be a very useful tool. Measure load and angle while keeping track of position in course. I'd use it as a coaching tool. The boat companies might need it to justify 100k boats.

 

My experience with the fish scale (the kind with a little ring to show peak loads) is too outdated to be useful. Interesting data...

 

Eric

[Adam Caldwell]

@david_quail - Where will you be doing this? @TAP did some stuff here at Trophy Lakes with us. Maybe we can coordinate an AI weekend. I love data.

[tap]

@david_quail are you looking to setup a means to collect data real time to interface with the AI, or are you just looking for some available data sets to feed it?

 

Over the last couple years I've randomly experimented with a few different methods of data collection: pixel tracking, accels, load cells, rope angle, strain gauged pylon etc. I even got @adamhcaldwell to ski once with a data cable zip tied all the way down the rope connected to a couple of strain gauges on his ski. It got a little interesting when I realized I accounted for a quick release connector if he let go of the handle, but neglected to account for a quick release connection between his leg and the ski... good thing he didn't fall.

 

So far, the best system in terms of data quality I've been able to come up with is direct line load measurement using a load cell and line angle measurement using a homemade angle-to-voltage sensor and a portable DAQ. My opinion is if you have line load and line angle you pretty much have, or can calculate, everything you would want to know. It's been super interesting and insightful, but up until now I've only logged the data real time then did all the post processing off line. I actually just starting playing with this again a couple weeks ago to setup a real time graphical interface to make the data much more useful, i.e. real time feedback instead of taking a couple hours to generate all of the graphs. I've found that the data is way more interesting to people if they can see it real time. I also have aspirations to hook up a third sensor to the boat to measure rpm at the same time. I'm super curious to see if I can measure the lag and intensity of the rpm vs. skier load for the different ABC/123 zero off settings.

 

Feel free to email me, I'd be happy to share the setup if it means more data. I really like @adamhcaldwell 's idea of a data collection weekend at Trophy if you happen to be in the southeast. I can also send you some data files to get you started if it's anything that's meaningful to you.

[andjules]

A couple of resources, one cheap, one not:

- Metawear sensors: pretty flexible, fairly simple-to-program, can record at a fairly high frequency (although the trade off is the amount of data that can be stored). I think ski roll combined with line angle and line load vs acceleration/deceleration would be interesting to study.

- Footbed sensors: Originally part of a downhill ski system; it would be great to study the pressure differences at different stages - front vs back foot, as well as fore vs aft vs inside particularly in the front foot.

[dchristman]

@andjules Those metawear sensors look interesting. A couple years ago I experimented with using sensor fusion in Android to develop an app with that tracked angle/path and gps video start/stop control ala WakeEye. As you can see from the video, there was some drift in the angle sensed over the length of the course.

 

[andjules]

@dchristman very, very cool. I wonder if a better roll sensor + some data noise reduction would get closer to what you were hoping for.

[dchristman]

Maybe. The drift is caused by the gyroscope and can presumably be calculated away. I'm wondering if the Metawear sensors are improved over what's built into the old phone I used and if the drift is accounted for before the data is transmitted.

[andjules]

Well, at least in simple, prototype mode, the metawear stuff would just be recording the data to download and sync up later. I believe it can broadcast in real-time, but that adds some complexity.

[dchristman]

I'm after "instant", the "I" in IPA. It would be great to be able to have a coach scroll be through the video of a pass with overlayed data offline, but it would be fantastic if that could be done at the end of the pass while still in the boat. I find I'm much less interested in the data after the fact.

 

The data collection also has to be easy to do. If it's too much trouble to get set up to record, it's not going to happen consistently. A smart pylon with wireless interface to an app on a tablet or phone in every boat would be great. That would give consistent measuring between different skiers. MasterCraft could integrate it with their pylon camera mount. Every skier could pair their own phone and take the data and video with them for logging and review.

 

Barring built-in boat tech, a phone could maybe do an adequate job. @Jibbo please, steal my idea, charge more for your app, and send me royalties :)

[tap]

@dchristman that's pretty neat. I assume what I'm looking at is a phone on a pylon mount and you're tracking the orientation at which the phone is pointing (please correct me if I'm wrong). Were you using only the on board gyroscope or some combination of sensors?

[Adam Caldwell]

What actual data is REALLY going to be useful in a live, on-board coaching situation?

 

Data is hugely important and it has its place, but I really don't know if there anything that would be necessary to have in a live coaching situation. Even video is mostly useless to look at unless you actually know what it is you are looking for.

 

It takes me hours to mentally process and understand collected data and wrap my mind around its application. I don't think a 'student' would get anything from it on the back of the platform.

 

[dchristman]

@tap your assumption is correct. It uses the rotation vector sensor, which is a software-based sensor created using available hardware sensors - gyro, accelerometer, and magnetometer. Android motion sensor documentation:

https://developer.android.com/guide/topics/sensors/sensors_motion.html

Good presentation on sensor fusion. Uh oh, I'm headed down the RAT hole again when I should be out making money. What I need is a Kalman filter.

 

[dchristman]

@adamhcaldwell I don't know what data is really going to be useful in the boat, but I asked the question and suggested a few things in this thread:

https://www.ballofspray.com/forum#/discussion/14216/rat-ipa-swerve

 

Not much feedback received other than "that's cool". It seems everyone gets hot on this topic over the winter months and loses interest when it's time to ski, including me!

[david_quail]

@adamhcaldwell I'm based out of Edmonton (they somehow convinced the worlds most renowned Reinforcement learning researchers - Rich Sutton - to move here to teach - so I'm trying to sponge up as much from him as I can). But the sounds of a rendez-vous for a weekend of skiing / AI brainstorming with @tap sounds amazing if it ever panned out! We still have 18 inches of ice on our lakes here!

 

@dchristman @andjules - Awesome pointers to some sensors and work that has been done before!

 

For my particular branch of research, I'm really interested in low level continuous sensorimotor streams of data. Until now, it's mostly been turtlebot mobile robots handing me pixel, load, temperature, acceleration, etc. data. But I think there are several reasons why streams of data generated during slalom would be of particular interest (beyond it justifying my time at the lake!). While there's a lot of correlated data, there's also a lot of noise - which makes the discovery of useful predictive features interesting. In addition - the data is episodic in nature since a slalom pass has a clear start and finish. That's different from mobile robot data - and enables a different type of research because of it. Anyways - that's my long winded and nerdy way of saying that I'm mostly interested in the pure science, rather than creating some useful application.

 

That said, the discussion around a training tool is interesting. I can see value there (and as a former mobile developer am tempted to build such a thing). Bringing a product like that beyond a DIY kit though seems a rather daunting challenge given such a niche market.

 

[Mortyski]

Hi David. I spent a ton of time and a lot of money trying to do what you want. I had a gps system with 20 capture point per second and a load cell. I was going to put the data on a Dartfish interface to show how the video looked vs all the vital statistics. My concepts were good but the GPS could not pick up the points to give me location data from which I could derive velocity, direction and acceleration. The load cell would give forces. I was trying to help Chris Rossi and Radar design skis easier. Everything was synced with a magnetic timer. I’m from Calgary. Say hi to Ken Nelson for me because you likely ski at Shalom Park

[disland]

To me its not in the boat type of analysis you want at all. What is needed is post skiing analysis that can pin point where loss of efficiency occurs. this could be done by comparing to the current state of the art in efficiency Nate Smith and others. Use science to explain why he makes it look so easy

[dchristman]

@disland do you want me to tell you now or next week how you can ski like Nate? :) I guess I'd be happy with next week, but I'd rather know right now so I can work on it the next pass.

I rest my case for Instant Pass Analysis.

[dchristman]

@adamhcaldwell What's to dislike?! You can analyze your full data set later if you want. If I have predetermined data targets in the course I'd like to hit, how can it be a bad thing to know immediately after the pass if I'm getting closer or not? Not that I'm advocating coordinates skiing... I just think it could be advantageous to be able to aspire to some particular goal in the space of a set. How consistent was I getting high on the boat for the gate? Unpaid friend-coach: "Uh, I really wasn't paying attention that time, I'll be more attentive next pass". IPA: "you were 2 feet behind the pylon and 42 feet before the gate. Nate is 6" behind the pylon and 28 feet before the gate on that pass, Dave. Human coach - please tell Dave how he might achieve that".