Hacker News new | past | comments | ask | show | jobs | submit login

Do any of these generative music ML framework export as midi instead of .wav or .mp3? That would be 1000x times more useful as the quality we're reaching is good enough.

I can imagine an VSTi that just takes a prompt and generates midi tracks. Something like this is surely coming in the next couple of years.




That was the way almost all ML research on music was done until recently- train for MIDi generation with input of other MIDI, or, at an even more fundamental level, just notes. If you go back, tons of ML music papers were written on generating believable sequences of Bach, Mozart, etc. because it was just note prediction.

Since the advent of transformers, and this idea of using text models mapping the natural language space to tagged music samples, and the music tokenizer acting directly on sampled audio stream (the bits of a .wav file, essentially) all the cutting edge work is going that route. Because it is producing high quality, finished audio streams directly. And I think part of it is because there is way, way more training data for actual audio than there is for MIDI alone (there are tons of free midi sites out there but a lot of it is garbage and it pales in comparison to what is already sampled and tagged in real audio libraries).

I imagine what will happen is... within two to three years these LM-transformer-music models will get so good that the audio will be damn near spotless sounding, and there will be additional methods developed to synthesize with more control directly with the models, to the point where wanting MIDI so you can use your own HQ synth isn't needed, because if you want "the lead synth to sound less digital and more like a classic Minimoog Model D" you just add that to another 'Music2Music" pass and out pops your sound.

For those who still want MIDI there is still work being done on traditional audio-to-MIDI modeling and I think you'd wind up just using that in the chain.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: