jameshball / osci-render-juce Goto Github PK

• Could be just detecting it and then having a button to refresh the file
• Could have a setting that automatically syncs the file between osci-render and the file system

CONTEXT AS TO WHAT THIS IS: https://www.youtube.com/watch?v=pCn297Ejvlw

Should be relatively straightforward to implement by copying the implementation from legacy osci-render https://github.com/jameshball/osci-render/blob/master/src/main/java/sh/ball/engine/ObjectServer.java

JUCE has a StreamingSocket class that should make setting up the TCP server easy, and a JSON class that will let you parse the message.

For completeness, this is the client-side code for connecting to the server in Blender https://github.com/jameshball/osci-render/blob/master/blender/osci_render/__init__.py . This will let you see the JSON format that is sent to osci-render.

The only complex part of this will be routing up the lines that are drawn to the PluginProcessor, either via the frameFifo directly

Project files should be compatible with Java osci-render and be in the exact same format.

We might run into issues with effect precedence being changeable in osci-render 2.0, unlike in Java osci-render. We should hard-code the precedence to be the same as the Java osci-render one in the case it isn't given in the project file.

Increase stability

Lua live-coding is unstable for certain scripts. Some examples I've noticed are:

• Empty scripts completely crash osci-render (anything with only blank space or only comments)
• Scripts that are just a return with no expression crash osci-render

But there are probably several others.

The code that handles parsing Lua scripts is in LuaParser.cpp, specifically in the parse() function

I've noticed that there are a lot of abort() called in the lua library, which we NEVER want to happen as this crashes the entire program. We want to just fail parsing and return silence from the function. This is already done in parse() but clearly is doesn't catch every case. Just need to have a look at the stack trace to see where it's crashing and either change what functions we call to parse the code, or in the worst case change the lua library code.

The Lua C API is fairly complicated, but there is a good reference here https://www.lua.org/manual/5.3/manual.html

Save last successful script

Similar to the prev. issue, if parsing ever fails, we should save the script/function of whatever was previously successfully parsed and either save it or reparse it. That way, the audio keeps playing and doesn't constantly cut out whenever you make an edit to it.

This should be fairly straightforward and can be done in the LuaParser.cpp/.h files.

This is elaborated more in this JUCE tutorial: https://docs.juce.com/master/tutorial_sine_synth.html under "Smoothing frequency changes" which also explains how this can be implemented.

Doing this vastly improves the usability of osci-render in live performances and when being controlled with MIDI controllers that jump in large steps.

CONTEXT FOR THIS: https://www.youtube.com/watch?v=PKVCx7rs_r8&t=269s

This gives good context as to what MIDI CC channels are (ignore anything specific to Ableton) https://help.ableton.com/hc/en-us/articles/360010389480-Using-MIDI-CC-in-Live but basically, they are used to control effects/parameters such as adding reverb, or a low/high-pass filter etc. In osci-render, it can control any effect/slider seen.

This involves the following:

• Add MIDI SvgButton to EffectComponent so that every single effect has a button they can click to arm a MIDI channel.
  • Linking a parameter to a MIDI CC channel has three states: unlinked (white icon), listening/armed (red icon), linked (green icon)
  • SvgButton colour must be changed accordingly. Currently it only has support for changing colour when toggled on/off, so maybe add a function to it that lets you change the colour arbitrarily.
• Only one parameter can be armed at once, so store a reference to the armed effect that is overwritten whenever the MIDI button on an unlinked effect is clicked
• Then, look for any MIDI CC channel messages in PluginProcessor processBlock

  osci-render-juce/Source/PluginProcessor.cpp

  Line 453 in 9293214

  void OscirenderAudioProcessor::processBlock(juce::AudioBuffer<float>& buffer, juce::MidiBuffer& midiMessages)

  and if there are any, link the CC channel to that effect, and update the value of the parameter to the value in the CC message
• Set the effect's MIDI button to green to indicate it's linked to a CC channel
• Now, any time that MIDI CC channel is seen in the future, it is assumed to be linked to that effect and the value should be updated
• You should be able to unlink the effect and CC channel by clicking on the button again

Try and use legacy osci-render as a reference, but this might be a bit hard if you don't have a MIDI controller!

The bulk of legacy osci-render's code for this is here: https://github.com/jameshball/osci-render/blob/0330152b4e951ccf15d6fee9736ea27576805155/src/main/java/sh/ball/audio/ShapeAudioPlayer.java#L687

BUT I am generally unhappy with how it works here. This might need to be reworked from scratch.

MIDI in legacy osci-render works as follows:

• A note on MIDI message is sent for a key and channel
• We register the target volume/velocity in an array (keyTargetVolumes)
  • This is the 'target' volume that we will gradually reach after a period of time defined by the attack
  • This is done in the main loop within applyEffects here: https://github.com/jameshball/osci-render/blob/0330152b4e951ccf15d6fee9736ea27576805155/src/main/java/sh/ball/audio/ShapeAudioPlayer.java#L335 and is quite expensive to run every sample!
• Osci-render chooses the baseNote in notesChanged() https://github.com/jameshball/osci-render/blob/0330152b4e951ccf15d6fee9736ea27576805155/src/main/java/sh/ball/audio/ShapeAudioPlayer.java#L667
  • This is defined as the highest pitch note that is part of the main MIDI channel (which defaults to 0 but is chosen by the user)
• The base note is used to control the frequency of the oscilloscope graphics image that will be rendered, because only one image can be rendered at a time
• Any other notes that are currently being held down that are not baseNotes are just played as sine waves
  • The code for all the sine waves being added is in applyEffects()

The major problems with this are:

• It sounds terrible for anything other than monophonic MIDI because you have one main note that has a usually very harsh oscilloscope-music-sounding timbre, mixed with several very smooth sine waves
• The simple algorithm for choosing the base note leads to some jarring switches in what melody sounds like the main melody due to the oscilloscope graphics image being the prominent sound you hear

I think the best solution could be to convert https://github.com/jameshball/osci-render-juce/blob/master/Source/PluginProcessor.cpp into a Synthesiser which would require a lot of refactoring, BUT means that handling MIDI should be super simple because the Synthesiser class handles most of it for you.

This would also allow the user to easily choose between monophonic and polyphonic audio and the SynthesiserVoice interface is a very easy way to add a lot of customizability in the additional sounds that are played, rather than them all being sine waves.

We could also try and support full polyphonic oscilloscope graphics, however this would likely be extremely expensive to run in real-time. This COULD work by having a separate FrameProducer for every SynthesiserVoice so that it is fully multi-threaded...

In the simplest case:

• Convert PluginProcessor to a Synthesiser
• Have SynthesiserVoices for the main oscilloscope graphics as well as any additional sounds, e.g. the sine wave sound we had in legacy osci-render
• Let the user choose what SynthesiserVoices they use and whether it's mono/polyphonic

In the complex case:

• Completely refactor PluginProcessor to allow many instances of it as a SynthesiserVoice so that you can have many oscilloscope graphics voices at the same time
• Each SynthesiserVoice has its own FrameProducer, meaning samples are produced in multiple threads

I would advocate for the complex case as it would make it very powerful :)

Currently this looks laggy on lower sample rates. It shouldn't be laggy, it should just be more jagged. The buffer size should be a fixed period of time, changing depending on sample rate, rather than constant.

see https://www.reddit.com/r/oscilloscopemusic/comments/16pct38/turn_off_lua_interactive_mode_in_oscirender/

maybe add a 'compile' or 'run' button or something?

Chinese postman is extremely expensive to run. We can use Fleury's algorithm and pair odd degree nodes suboptimally as a heuristic for Chinese postman.

Fleury's algorithm: https://slaystudy.com/fleurys-algorithm/
Working implementation of this: https://github.com/Eric-Lennartson/pd-osci/blob/dev/Classes/Source/tsp.c

To implement this, we should replace the code in

As an aside, we should look to improve the performance of the graphs used to perform this algorithm as they can probably be improved a decent amount.

Currently, they are hideous and light-theme-oriented! Just change it to use similar colours to the darcula theme https://github.com/dracula/dracula-theme by creating a ColourScheme and setting it on each CodeEditorComponent here https://docs.juce.com/master/classCodeEditorComponent.html#a4faca1aad5f908a0e35a71ca25e784b1

All the CodeEditorComponents are initialised here:

• .lua file parsing working
• SampleSource needs to be introduced, like in Java osci-render

Legacy osci-render lets you open the software oscilloscope ( https://james.ball.sh/oscilloscope ) from within osci-render by clicking on Window and then Open Software Oscilloscope.

We should mimic this in osci-render 2.0, but possibly with better UX! It's a bit weird atm that it's part of the title bar under Window when we don't use Window for anything else. So, either think of a better way of doing it with a button or something or just replicate it.

This should be trivial to do in JUCE: https://docs.juce.com/master/classURL.html#a2e0faa22cab2447d64c8ac0d01bdccda

We obviously depend on #8 before we can implement this.

Something like this would be amazing:

Can download here to reverse-engineer / for a case study https://vital.audio/ and it's also open source: https://github.com/mtytel/vital

This explains what an envelope is and what all the parameters do: https://en.wikipedia.org/wiki/Envelope_(music)#ADSR

To see roughly how you'd implement a basic version of this, take a look at the volume visualiser component: https://github.com/jameshball/osci-render-juce/blob/9293214943af555d2aaa15eb4f71dc3771a66a9d/Source/components/VolumeComponent.cpp

This has a paint() function that you draw the component for. We could quite easily draw a JUCE Path that goes between the attack/decay/release/sustain etc. points. The harder part would be supporting dragging the points around which I don't immediately know how to do, but the JUCE Slider class will give some ideas. The DragAndDropContainer also looks very promising - we could have the knobs as separate components that are DragAndDropContainers and have the parent ADSR component be the DragAndDropTarget that has callbacks for when the knob is being dragged and can update values accordingly.

Legacy osci-render supports changing the font and style (i.e. bold/italic etc.) globally for all text files, but this is pretty limiting.

A better solution would be to add these settings as a separate TextComponent, in exactly the same way LuaComponent and ObjComponent have been added. i.e. the settings should control just the current txt file open, and should appear when a txt file is opened.

When this has been created, it just needs to be added to the PluginEditor and then a few bits need to be changed to toggle whether it is visible or not:

Changing the font should trigger the font file to be re-parsed, as it takes in the Font

The style of the font is part of JUCE Font itself, so it should be fairly trivial to set using https://docs.juce.com/master/classFont.html#a52a35b5dc1e6f782e25283c9283bae4f

JUCE has a very easy way of getting all the installed fonts on the machine: https://docs.juce.com/master/classFont.html#a95f7587d2f1bb76329f4f09444b09cdb so having a ComboBox that contains all the fonts that a user can choose from makes sense here.

The font and font style should be stored with the project file and used in both getStateInformation() and setStateInformation()

Legacy osci-render also supports custom fonts/styles so https://github.com/jameshball/osci-render-juce/blob/9293214943af555d2aaa15eb4f71dc3771a66a9d/Source/LegacyProject.cpp should be updated to recognise custom fonts from the legacy project file. To get an example legacy project file have a look here https://james.ball.sh/docs/spiral_dots.osci and it's under fontFamily and fontStyle XML tags.

IMPORTANT: The fonts that are installed are not the same on every computer, so it's important that there is always a fallback font if the font saved to a project file is not available on the machine.

Hello! On my osci-viewport project, which is based on osci-render, I have improved the software oscilloscope drastically and I want to do the same for osci-render. I'll create a fork and make a pull request once osci-viewport has been detached.

I'm generally very happy with the project select screen in legacy osci-render, the only thing that would need changing is thinking about WHEN the project select screen is opened by default.

Currently, legacy osci-render opens this screen whenever osci-render is opened. But, now we are dealing with audio plugins, I think it only makes sense to open the project select screen when the DAW doesn't try loading the previous state of the program using setStateInformation() first.

For clarity, project select screen should NOT open when osci-render opens if:

• setStateInformation() is called
• The project passed to setStateInformation() is actually a valid project
• The project is non-default (this is fairly hard to identify so maybe ignore this)

The UX for an example first-time-user of osci-render within a DAW could be:

• Greeted with the project select screen
• Closes osci-render on the project select screen
• Opens again, and greeted with project select screen
• Starts a new project
• Closes osci-render (DAW saves the state when closing)
• Opens again, now the DAW has saved the state, so they bypass the project select screen and go straight to their previously started project

If the project select screen is visited again, and the user closes osci-render, then they should be greeted with the project select screen upon reopening.

To achieve this, we could do something like save the page that the user is currently on (i.e. project select or main page) with the project state and look at this to determine what page to show them.

SIDE NOTE: We should probably be storing currentProjectFile with the project file so that we can restore this when saving.

Port the web-based software oscilloscope to C++ and and have it work either within osci-render or as a separate VST.

Should be trivial to create the logger itself using FileLogger which should be a member of the PluginProcessor class so that it persists.

The hard part is finding all aspects of osci-render where errors could possibly occur and logging them rather than silently ignoring them or hard crashing like we are at the moment!

Things that immediately come to mind are:

• Server that sends data to software oscilloscope can fail to start
• Server that receives Blender line data can fail to start ( #36 needs to be implemented first obv )
• Critical errors with Lua
  • Could involve adding extra safety checks like making sure the lua script doesn't take too long to run and cancelling if it does and logging this
• IF POSSIBLE: checks in any threads that use locking to wait on the audio thread to log to a file if they have waited for too long for audio (i.e. suspicion of deadlocks)
  • Examples of components that wait for the audio thread are anything that has a BufferConsumer, so AudioWebSocketServer, PitchDetector, VisualiserComponent, VolumeComponent.

Goes without saying that over-logging is much more important than under-logging as this is the number 1 most useful tool for debugging when you don't have access to a user's machine.

ASIDE: There should be a max log file size to prevent somehow using up all space on user's machine! (this mistake was made with legacy osci-render, oops)

• Should mimic the octave slider from legacy osci-render

There should be one global frequency analyser that other classes can listen to, the same as in Java osci-render. We need this for the wobble effect