Right now there are two systems for capturing motion capture data for animations in games/movies, inertial and optical. Inertial is easier and more affordable but comes with more errors/inaccuracies in the capture that requires manual correction. Optical is more accurate and requires less cleanup but has an expensive hardware and space requirement.

My thought is to have someone wear an inertial mocap suit and record both inertial and optical session at the same time and then use ml to learn how to automatically fix the mocap data. Then you could theoretically get the precision of optical capture from inertial recordings after passing it through the ml.

Curious if you think something like this is tractable for a first project? If you have any suggestions for how you’d solve this or if there are any existing projects you could point me I appreciate any help!

https://github.com/facebookresearch/co-tracker

This sounds like a perfect place for you to get started!

I'm assuming you've done some tutorials or courses on basics of DL and can program Python. At that point the easiest first step would be to just train an MLP to convert a single time step from inertial data to match the optical prediction (presumably they are in the same coordinate system and temporally close enough).

The crux of building something good would be in how you handle the temporal aspect I'd imagine. Clearly you want to use multiple samples over time from the inertial to get more accurate positional estimates. I'd imagine a fixed window of the past n inertial samples would be a good start. I wouldn't worry about more complicated temporal modeling, e.g. RNN or transformer, unless you can't get satisfactory results with the MLP.

My gut says there's probably a non-ML approach to this too, some sort of Kalman Filter etc. Always best to avoid ML if a simpler solution exists :)

I’d also recommend the detailed pytorch optimization case studies by Paul Bridger:

https://paulbridger.com/

Thanks!

[1] https://github.com/stas00/ml-engineering/blob/master/insight...

I'm not sure of the exact numbers, but I would guess that an overwhelming majority of the Top 500 Supercomputers [3] are running Slurm. And as others have noted, research computing centers in academia all mostly run Slurm. And Slurm also dominates in the DoE national labs in the US.

Oh, and as a [potentially apocryphal] fun fact, the name "Simple Linux Utility for Resource Management (SLURM)" is a backronym from the soda in Futurama! [4]

[1] https://en.wikipedia.org/wiki/Oracle_Grid_Engine

[2] https://github.com/adaptivecomputing/torque

[3] https://www.top500.org/

[4] https://futurama.fandom.com/wiki/Slurm

Do you mean outside of academia _and_ HPC? Industry HPC clusters using slurm are quite common.

It's not that different from learning how to program without already having a programming job.

(That isn't to say either of these two is easy. They both require a lot of dedication.)

There's a tiny job market for that - in comparison to, say, web dev - and these projects require professionals with very deep knowledge. This is not the kind of work where chatgpt or stackoverflow will help you a lot.

Important thing is to keep moving.