Comments (15)
Glad to see the interest here! Just to clarify: what kind of MIDI support are people looking for? It seems @MaxGodTier's request is around real-time MIDI, @Muzz is looking for static rendering of MIDI files, etc. This might be a number of different issues in one:
- add support for virtual instrument plugins in addition to effects (i.e.:
MIDI -> Audio
instead of the currentAudio -> Audio
) - add support for MIDI file I/O and MIDI-to-Audio rendering (maybe via integration with Mido)
- add support for sending MIDI messages to effect plugins during effect rendering (i.e.: Audio + MIDI -> Audio)
It'd be good for Pedalboard to support all of these eventually, but I'm curious to know which to prioritize.
from pedalboard.
@psobot - thanks for following up on this!
Use case I would be most interested in would be static midi to audio rendering with VSTi plugins. Iโm interested in ML research with synthesizers, so being able to programmatically adjust parameters and render sounds using simple midi files / objects would be ideal. There are a couple other python options out there that support this, but they arenโt pip installable. Having a pip installable package for VSTi rendering would be amazing!
from pedalboard.
There isn't yet, but this is something that would be possible to add.
One minor complication: Pedalboard doesn't run in real-time on its own, so communicating with the plugin via MIDI would have to happen via some new API, rather than a virtual MIDI device. JUCE (the C++ library underneath Pedalboard) allows this kind of thing by effectively passing a list of MIDI messages in with every process
call - something like:
my_plugin = pedalboard.load_plugin("...")
# Prepare a list of MIDI messages to send to the plugin as the audio is processed, of the format:
# (time_offset_in_seconds, [midi_byte, midi_byte, ...])
# For example, this sends a single Note On message at a time index of 0 seconds:
midi_messages: List[Tuple[float, List[int]] = [(0.0, midi_note_on(C4, 127))]
output = my_plugin.process(audio, sample_rate=sample_rate, midi_messages=midi_messages)
This could work, but I'm considering if there are more ergonomic APIs that could be used rather than just overloading the process
method with additional arguments.
from pedalboard.
+1
really interested in virtual instrument plugins supporting MIDI -> Audio
from pedalboard.
It seems @MaxGodTier's request is around real-time MIDI
Actually I find it more convenient to render the results directly (faster) to a file than a real-time solution (slower), I just didn't know that was the easiest solution to implement!
In my case scenario, the VST plugin is adjusting the pitch from an existing audio signal (. wav) to fit the notes being played from a midi track (.mid) like in this video I showed (wav file contains only static noise, mid file is playing music), I'm not sure if that's more like 2 or 3, however I don't require real time (it's actually better if it renders it to a file directly!)
from pedalboard.
+1 please support loading instruments :-)
from pedalboard.
Yes to all of the above, though i feel that you should probably prioritize in terms of what functionality matches what is there already.
Offline vst instrument midi rendering is a little served niche so would be a great first stop.
from pedalboard.
+1 to all of three above.
I'm interested in sending midi messages to effect plugins in continuous mode. Example: I want to use pitch corrector and I want to send midi notes to it like:
and as result I want pitch corrector to work continuously, like it works now when I just pedalboarding whole file with fixed settings. I mean not like cycle (https://github.com/spotify/pedalboard#does-pedalboard-support-changing-a-plugins-parameters-over-time) , because cycle with every iteration will reset parameters of pitch corrector and pitch of vocal will fluctuate.
So somehow whole pedalboard needs to "feel time", bars, frames, milliseconds.
from pedalboard.
Same here, I would like to be able to read midi files that I generate and map them to VST synths as to create a full track
from pedalboard.
Just want to pipe up, and say that midi support would really be helpful here, and really open up the possibilities with this library.
Even just static midi rendering would be great.
from pedalboard.
+1 !
from pedalboard.
Is there any branch in this repo where this issue is being developed? @psobot
from pedalboard.
@valleyofblackpanther Somewhat - @robclouth has an open pull request that started to add MIDI message support, but that PR has been abandoned for some time and I don't know of any other efforts to add MIDI support at the moment. Pull requests are welcomed though!
from pedalboard.
@psobot do you think this python midi api could be used in the process of building the midi support. I think this api could be used to build the midi messages. But still I think it would be great if you could take a look at it.
from pedalboard.
@valleyofblackpanther If we were to build in MIDI support, it'd be great to use Python's duck typing avoid adding an explicit dependency on any specific MIDI library. MIDI messages are just tuples.
from pedalboard.
Related Issues (20)
- Is threading supported for `VST3Plugin` instances? HOT 2
- Need help with parameters for improving TTS generated audio
- Does WriteableAudioFile handles creating a WAV file with the standard header? HOT 2
- Unable to load VST3 when running through CircleCI HOT 1
- Limiter Plugin seems also to normalize? HOT 1
- is there a way to overplay 2 audio together using pedalboard HOT 1
- Actual resampled audio length != expected resampled audio length
- Audio Stream HOT 2
- Parameters with the same names overwrite each other in VST3Plugin.parameters
- AudioFile in input of AudioStream HOT 2
- [Feature Request] Add support for playhead tracking HOT 4
- Playing specific segment of audio HOT 1
- Change VST editor params while streaming audio? HOT 2
- [wish] Expose additional AU/VST plugin metadata, esp. manufacturer name HOT 1
- Applying plugins on multi channel tracks HOT 1
- Is it possible to use VST3 instrument plugin in AudioStream for Live Audio
- Question: Is it possible to make the universal (x86_64 + arm64) binary for macOS? HOT 1
- Is it possible to run pedalboard in a Docker container with MacOS VST3 plugins? HOT 1
- Unable to Change Instruments in BBC Symphony Orchestra VST via Midi messages
- Plugin .parameters for builtin plugins
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pedalboard.