> "We obtain these times from MIDI files, though in the future I’d like to explore more automated ways of extracting them from audio."
Same here. In case it helps: I suspect a suitable option is (python libs) Spleeter (https://github.com/deezer/spleeter) to split stems and Librosa (https://github.com/librosa/librosa) for beat times. I haven't ventured into this yet though so I may be off. My ultimate goal is to be able to do it 'on the fly', i.e. in a live music setting being able to generate visualisations a couple of seconds ahead being played along with the track.
Not sure if this is unsavory self promotion (it's not for commercial purposes, just experimenting), but I am in the middle of documenting something similar at the moment.
Experiments #1 - A Mutating Maurer Rose | Syncing Scripted Geometric Patterns to Music: https://www.youtube.com/watch?v=bfU58rBInpw
It generates a mutating Maurer Rose using react-native-svg on my RN stack, synced to a music track I created in Suno AI *. Manually scripted to sync up at the moment (not automatic until I investigate the above python libs).
Not yet optimised, proof of concept. The Geometric pattern (left) is the only component intended to be 'user facing' in the live version - But the manual controls (middle) and the svg+path html tags (right) are included in this demo in order to show some of the 'behind the scenes'.
Code not yet available, app not yet available to play with. Other geometric patterns in the app that I have implemented:
- Modified Maurer
- Cosine Rose Curve
- Modified Rose Curve
- Cochleoid Spiral
- Lissajous Curve
- Hypotrochoid Spirograph
- Epitrochoid Spirograph
- Lorenz Attractor
- Dragon Curve
- Two Pendulum Harmonograph
- Three Pendulum Harmonograph
- Four Pendulum Harmonograph
This is the Typescript Maurer Rose function (that is used with setInterval + an object array of beat times which determine when to advance the 'n' variable):
export const generateGeometricsSimplemaurer = (n: number, d: number, scale: number = 1) => {
const pathArray: TypeSvgPathArray = [];
for (let i = 0; i <= 360; i += 1) {
const k = i \* d;
const r = Math.sin(n \* k \* (Math.PI / 180));
const x =
r \*
Math.cos(k \* (Math.PI / 180)) \*
40 \* // base scale
scale +
50; // to center the image
const y =
r \*
Math.sin(k \* (Math.PI / 180)) \*
40 \* // base scale
scale +
50; // to center the image
pathArray.push(\${i === 0 ? "M" : "L"} ${x} ${y}`);`
}
const pathString: string = pathArray.join(" ");
return pathString;
};
The geometric patterns (with their controls) will have a playground app that you can use to adjust variables... As for the music sync side, it will probably take me a long time.
*Edit: I just noticed that the author (Victor Tao) actually works at Suno