Input / Provide any music clip, automatically “learn” its timbre, and generate a playable preset.
Use Cases
Producers provide in 3 - 10 seconds of audio and instantly get a playable timbre in the same style.
An integrated instrument separation feature can be added later.
Sampling compliance: replace direct sampling with the generated timbre.
Framework
- Input: clip trimming, loudness and noise processing, automatic segmentation (based on transients, etc.).
- Compute the current Score with the model, then adjust to the target Score.
- Plugin: provide a playable preset as a synthesizer-type VST.
Training
Try to process all of Serum’s presets first, then add acoustic instruments (such as violin, piano, etc.). Especially instruments like the violin that have transient variations. Then add some sound effects. Keep the overall training set above 1,000 if possible, and continue to expand as needed. Each data item includes the synthesized audio .wav file and the pre-synthesis MIDI file.
Paragraph
For different instrument in a music there are different types of timbres. In order to compose an electronic music you need a “virtual instrument” to generate the sound with the score / notes you made. In standard ways composers will need to design the timbres by themselves which is really a complex job.
My plan is to develop a plugin that are able to “analyze” and “reproduce” the timbre in a certain music clip. This plugin reduce the sound design process in most cases.