Scintillator User Guide

See also: ScinthDef ScinServer

Overview

Scintillator is a video synthesis extension for SuperCollider. It is distributed as a Quark, but requires an additional download and installation step to get the synthesizer program.

Scintillator is designed to be intuitive to users already familiar with SuperCollider idioms. It follows the client/server archiecture established by SuperCollider, accepts ScinthDefs in a similar manner to SuperCollider SynthDef s, provides facilities to invoke and control Scinth instances similar to Synth , and so on. For a detailed list of classes with analogous SuperCollider classes see Scintillator Parallel Classes.

Installation Instructions

1. Install Scintillator Quark

Run the following code in the SuperCollider IDE:

(
Quarks.install("Scintillator");
)

Or you can use the Quarks GUI to pick out Scintillator and install it. See UsingQuarks for more information.

2. Recompile Class Library

After any Quark installation you need to recompile the SuperCollider class library. In the IDE menu select Langage -> Recompile Class Library. This should also kick off the automatic installation of the ScinServer binary.

Tutorial

This section can serve to validate your Scintillator installation, as well as to establish some of the basic concepts and get some pixels lighting up on the screen. The first step will be to get an instance of the video server running, which will require installing the correct server binary for your platform of choice. To validate that the server binary is installed correctly we’ll start the server, define a ScinthDef, and render the definition with a Scinth.

Starting the Server

SuperCollider by convention stores the default audio synth server in the s environment variable. While developing Scintillator I have often found it convenient to store the video synth server in the v environment variable, but storing it in any suitable variable is fine.

(
~v = ScinServer.new.boot;
)

This should bring up an empty window with a black background. By default the window is 800 pixels wide and 600 pixels tall. It is configured to float on top of all other windows on the screen, but not to steal keyboard focus. The hope was that you could pop open these windows and still continue typing away in your SuperCollider IDE (or other editor of choice) uninterrupted. The server has many boot-time configuration options, for more details see the ScinServerOptions documentation.

Creating a ScinthDef

The simplest imaginable ScinthDef sets the same color everywhere on the screen. We’ll use the VRGBOut VGen for that.

(
~red = ScinthDef.new(\red, {
    VRGBOut.pr(1.0, 0.0, 0.0);
}).add;
)

You’ll notice after running this code the screen is still black. This is because, just like the audio class SynthDef , the server has only received a template for creating Scinths. We’ll create a Scinth now from the \red template.

(
~redScinth = Scinth.new(\red);
)

What’s happening here is that for every frame, and at every pixel, the graphics hardware is running an instance of the \red ScinthDef to compute the color of that pixel. Since \red defines that pixel as a constant, that number defines the output everywhere, giving a field of solid red. We’ll make a more interesting ScinthDef soon, so let’s turn this one off for now:

(
~redScinth.free;
)

Time-Varying Video Synths

The VSinOsc oscillator is similar to the similar class SinOsc . Note that Scintillator VGens will always start with a V, to allow their easy identification. There are some differences in a video synth from an audio one, however. The first consideration is that signals don’t have to vary over time to impact the output. The first Scinth, \red, should prove that. A similar constant output in an audio context would be inaudible, except for that it might damage certain audo setups, so caution is advised in testing that assertion!

Another point to note is that while audio signals normally operate in the domain from -1 to +1, with negative signals indicating a moment of underpressure in the sound wave, video signals only vary from 0 to 1. Light can only be present or absent in this video synthesizer, with 0 indicating no light and 1 indicating maximum brightness of light. Most video cards will clamp output between 0 and 1, but Scintillator doesn’t take any special steps to limit signals, and video signals outside of the range are typically clamped. As a result of the different range, many of the default inputs on VGens with analogous UGen s are are adjusted so that instead of providing a signal input from -1 to +1 they output from 0 to 1. This can be seen, for example, in the defaults to VSinOsc, where the mul and add arguments are both 0.5, constraining the video signal between 0 and 1.

(
~w = ScinthDef.new(\wave, { |f=1|
    VBWOut.pr(VSinOsc.pr(freq: f));
}).add;
)

(
~k = Scinth.new(\wave);
)

This should produce a 1 Hz wave, with the entire screen lightening and darkening in unison. Each frame, each pixel is subject to the same computation, and so will produce an output pixel with the same level of brightness. A few other things to note from this example. First is that the video signal at 1Hz is easily observeable, whereas a 1 Hz audio signal is below the range of human hearing. The second is to note that this Scinth has a parameter, just like audio Synth objects. Try playing around with the parameter to see the frequency change in the output.

(
~k.set(\f, 0.2);
)

Another thing to note is that Scintillator does its best to track your visual display refresh rate. Typical consumer displays refresh at 60 Hz, sometimes slower at 30 Hz. Some gaming monitors go as high as 240 Hz. So setting a frequency higher than about 30Hz is not going to notably change the flashing frequency. This can be considered a rough visual proof of the Nyquist Theorem.

Space-Varying Video Synths

So we’ve seen time-varying signals, what about spatial variation? Let’s clean up the current Scinth, and define a new Scinth using the VNormPos VGen:

(
~k.free;
)

(
~spot = ScinthDef.new(\spot, {
    VBWOut.pr(VLength.pr(VNormPos.pr));
}).add;
)

(
~d = Scinth.new(\spot);
)

What’s going on here? At every pixel, VNormPos is producing a 2-dimensional constant signal that varies from -1 to +1 in the y (or vertical) dimension, and from around -1.33 to +1.33 in the x (or horizontal) dimension. The VNormPos documentation has details about how the coordinate system is set up. That 2D signal is then being converted by the VLength VGen into a single, mono signal, which varies from 0 or black at the origin in the center of the screen, to 1 (or greater than 1) at the edges of the screen. The video hardware is clamping the signal at 1, which is why outside of the top and bottom of the image the gradient stops at the edge of the unit circle.

Dimensions in Video Signals

Astute readers may have noticed in the previous discussion that VNormPos produces a 2D output signal for consumption by the VLength VGen. There’s a longer discussion about signal dimension in the “Dimensional Analysis” section of the ScinthDef documentation. In short, video signals at output are always four-dimensional signals, and in Scintillator are always packed as red, green, blue, alpha signals. The multichannel expansion mechanism used in SuperCollider is flexible in that it allows SynthDefs to be defined for audio signals varying from a signal channel to complex multichannel ambisonic arrangements. Video signals trade a lot of this flexibility for the massive parallelism required to compute a color at each pixel 60 times per second. Furthermore, video cards are optimized to handle mathematical operations on all four channels at once in a single instruction. So instead of treating every ScinthDef as a hard-coded 4-channel signal chain, Scintillator tracks which combination of dimensions in input and output each VGen will produce, and validates at definition time if the combination of VGens and signal dimensions is valid. Furthermore, each VGen includes in its documentation a discussion of the supported pairs of input and output channels.

So, for example, the VNormPos VGen accepts no inputs and produces a 2D output always. The VLength VGen accepts inputs from 1 to 4 channels and computes a vector length on the input. The result of the operation is a scalar, so regardless of the dimension that the input was VLength will always produce a single-dimensional output. This single-dimensional output is accepted by VBWOut, which splats the single input into the first three red, blue, green output channels, and adds the hard-coded alpha channel at 1.0, or completely opaque.

There are helper VGens that can pack single-channel signals into multi-channel ones, these are the VVec2, VVec3, and VVec4 classes. There are also VGens for extracting single-channel signals from individual channels from multi-channel signals, these are the VX, VY, VZ, and VW VGens.

Scintillator supports most common mathematical operations from 1-4 dimensions. Additionally, many vector operations support combining with scalars. For instance, it is possible to multiply a single-channel value against 1, 2, 3, or 4 channel values. For the current list of supported operations see the VGens Overview. The important thing to understand is that multi-channel operations typically operate per-channel. An example might make this more clear:

// Clean up the \spot Scinth from the above example
(
~d.free;
)

(
~rings = ScinthDef.new(\rings, {
    var rad = VLength.pr(VNormPos.pr);  // rad is one-dimensional
    var rgb = rad * VVec3.pr(31, 41, 61); // rgb is 3D
    rgb = 0.5 + (rgb.sin * 0.5); // Can do arithmetic with vectors and scalars
    VVec4.pr(VX.pr(rgb), VY.pr(rgb), VZ.pr(rgb), 1.0);
}).add;
)

(
~rg = Scinth.new(\rings);
)

Important to understand that the sin operation is happening on each channel independently. It can be instructive to re-run this example with only one color channel enabled at a time, to understand what is happening in the red, green, and blue channels independently before trying to understand how the colors are mixing in the combined image. This example also demonstrates that any 4-D vector is accepted as valid output, the VRGBOut, VRGBAOut, and VBWOut classes are just convenience methods.

Variation in Time and Space

The last experiment in this quick start guide is to combine variation in both time and space together:

// Remove the \rings Scinth from the above example
(
~rg.free;
)

(
~zoom = ScinthDef.new(\zoom, {
    var pos = VNormPos.pr;
    var box = 1.0 - max(VX.pr(pos).abs, VY.pr(pos).abs);
    VBWOut.pr(VSaw.pr(phase: box));
}).add;
)

(
~z = Scinth.new(\zoom);
)

(
~z.free;
)

It can be instructive to think about why the phase argument to VSaw is needed to create the zoom effect. Remember that all pixels run this program independently of each other, every frame.

Next Steps

Scintillator is still in active development. The Development Blog typically has the most up-to-date information.

It’s definitely worth perusing the VGens Overview to get a better understanding of currently supported VGens. There are a lot of additional features planned for Scintillator, which will likely be documented in separate, independent guides. Lastly, join the conversation! Post your feedback and questions on the GitHub page , find me on the SuperCollider Slack, there’s a dedicated #scintillator channel, or drop me a line over email. I’d be very happy to hear from you!