Nikolai Kozak

About
Contact Me

Multidisciplinary artist and wooden boatbuilder, 
interested in memory and interfaces.

Former partner at 
Bede-Fazekas & Kozak in Budapest, Hungary

Currently attending ITP @ NYU.


Code of Music 

ITP Spring 2025




Assignment 1.1 & 1.2 - Audio/Visual Instrument

Given this is a browser -> node -> oF sketch, I cannot provide a sketch link, but I’m providing a github link in its stead.

This week’s assignment was to create an audiovisual instrument - a sketch that would react visually to samples played via the keyboard. It was also specified that using the Tone.js library was a requirement, in order to familiarize ourselves with audio analysis (FFT, waveforms, etc.).

In passing, Luisa mentioned the possibility of creating a sketch using openFrameworks, which I found intriguing given it had been about a decade since I had last used it. I also found the technical challenge worthwhile, given that I’d have to figure out a way to pass data from a p5.js sketch running in the browser to an openFrameworks application running locally.





The architecture of the final sketch looks like this:

P5.js & Tone.js (Browser) --- (websocket) --- > Node.js (Local) --- (OSC) --- > openFrameworks

If you look closely at the top-right of the openFrameworks sketch in the screenshot below you’ll see the actual OSC message (address and data) being sent from the Node server. This OSC message contains an array of float values produced by the Tone.js Analyzer class, more specifically FFT values.

A total of 6 samples can be played on the browser, and they all affect the spinning “disk”. This disk is really just a polyline whose y points correspond to the fft values. In order to draw a smooth polyline, I’m interpolating the FFT values, creaing around 100 points so that we see smooth waves as opposed to jagged ones. The sketch is completely dynamic - I can alter the resolution of the geometry as well as the radius of the disk, and set the upper and lower dB bounds to which height is mapped. The FFT values towards the center correspond to the FFT values at the beginning of the array, so (lower?) frequencies.



Some gifs illustrating the working program:














Assignment 2.2 - Composition


This week’s assignment entailed the creation of a composition using a queueing system based on Tone.js. As opposed to last week where I explored the transfer of data between a p5 sketch and openFrameworks, this week I decided to stick solely to Tone.js and p5 in order to compartmentalize and keep everything easily accessible for end users. 

The project started by thinking through what exactly queueing audio might mean, and what interfaces (or rather shapes of interfaces) might look like. in this context. How can we create intuitive systems that are also extensible and fun to use?

Here are some sketches from the thought process at the very beginning:

[Upload them!]

Eventually what emerged was a tendency towards a grid-based system. This is a direction I’ve taken before when thinking of designing interfaces. There’s something very clean and appealing about the grid - a simplicity that makes it attractive in that it sets very clear ground rules and yet invites you to build on them.

I have another grid-based project called Rethinking CAD where I define a clear aesthetic, and it ended up being one I borrowed for this exercise. I also ended up cannibalizing a lot of the code structure. Towards the end, I leaned quite heavily on CursorAI to help refactor as the code became bloated: it’s the first time in all my years of coding that I’ve pair-programmed with an AI in earnest. I found that if I kept reminding it to split concerns according to Volatility Based Decomposition (https://rightingsoftware.org/), it would end up doing a nice job. That said, it also produced a crazy amount of fluff, and I’m eventually going to have to go back and cut through all of it to clean it up. I was also somewhat saddened that it always biased towards an OO style of programming instead of a functional style, which is more my jam. I guess it’s to be expected given the data it’s trained on.

Regardless. The system looks like this:

 


What we see here is a grid, with customizable columns and rows. The columns represent the beats according to the time signature we set (can be 
changed interactively), and a playhead moves through the grid at a given BPM which can also be changed. Pay attention to the ‘Mode’ indicator - this little application follows a modal pattern (like Vim!), and we can cycle through modes by pressing specific hotkeys listed on the right.





This particular screen is the “arrange” screen: its purpose is to queue samples at a given position. The samples that are currently loaded are listed above, and each sample corresponds to a given key. You can move around this screen (and all the other modes for that matter) with a cursor (in red, with an ‘@’ sign in the middle).




 Moving to a location and pressing the key of a sample lays down a block, which just means the sample will play when the playhead hits it at that position. The sample can be removed with the ‘x’ key. The vertical position of the samples in this case is meaningless. 

We can move into the ‘volume’ mode by typing the ‘v’ key. What greets us looks like this (trimmed somewhat for visibility):





In this mode, unlike the previous ‘arrange’ mode, the horizontal axis means nothing, but the vertical axis controls the volume of a given sample. Changing volume is easy, we simply press the key corresponding to the given sample and the block is repositioned upwards or downwards, according to where our cursor is. The volume is adjusted live, as the playhead is cycling.

Another mode is the ‘trim’ mode. 





The trim mode allows us to set in-out points in our sample, which again is reflected in the live playback. As with other modes, in this mode the vertical axis does not matter - the leftmost and rightmost blocks dictate the in and out points of the given sample:





Finally, we have the ‘pitch’ mode. The pitch mode allows us to shift the pitch of a sample by a given amount, upwards or downwards. This mode has a series of additional rules - pitch blocks can only be drawn above or below an existing ‘arrangement’ block, and doing so will shift the pitch of that block only. 





Sometimes there are multiple different blocks in one column. The shift mode keeps track of blocks that have the *same* key correspondence, and so multiple blocks can be shifted in the same column. 

This project was a quick prototype for class, which started out with a simple grid and a basic queueing/positioning system, and eventually evolved into an outright application. The work I have to do now has to do with cleaning up a lot of the generated code, moving towards a more functional style, and then setting up neater abstractions that will allow me to add more modes. I’m already thinking of being able to create multiple arrangements and cycle through them, or toggle them on and off.





Sources:
- Legowelt for samples
- CursorAI for AI pair-programming, particularly their new “Composer” mode, which allows for greater context awareness.




Assignment 3.4 - Interactive Rhythm Study

Design and implement an interactive exploration of rhythm.

I’m very interested in the possibilities of Domain-Specific-Languages. This stems from a believe in Lisp-superiority, and the idea that DSL’s help us regain control over the systems we are often forced into using out of necessity.

And so, I decided to look at the possibility of creating a little lisp-like language in order to live-code a step-sequencer using Tone.js.

I’ve had some experience with these projects, and thought this would be a fairly self-explanatory build:

1. Make a tokenizer and a parser.
2. Make an interpreter.
3. Create the interop function-table to mix-in Tone.js.
4. Live code!

Of course this was extremely naive, but that’s most of the fun.

The language

I wanted to accommodate few functions outside of the scheme/lisp primitives (i.e. ‘+’, ‘let’, etc.). I also wanted to keep the style as functional as possible, implementing scopes as closures and not objects/classes which is the JS approach.

I’ve gone through the exercise of scaffolding the tokenizer->parser->interpreter flow many times, and so I decided to pawn this off to Claude (3.7!).





^ Our dependencies: we’re using Tone.js and prism.js with codejar.js as our code editor.





^ This is the tokenizer, nothing too crazy. What’s cool is the implementation of a [getLineAndColumn] method, which gives us some pretty nice conveniences when throwing tokenizer errors.





^ This is the meat of the parser, and should be recognizable to anyone that’s written a lisp-style interpreter before. Ultimately we return an AST where each nested array is initiated via the presence of parentheses. The getLineAndColumn function also does some work here, giving us nice debug info.





^ We move into interpreter territory now, where we create scope environments using closures. This keeps things relatively clean and simple (implementing this with a Class feels like it goes against the spirit of a functional-language implementation).





^ We declare our “primitive” functions, namely math stuff. It’s nice to work with reductions; it always seems to me such an elegant way of dealing with function parameters, and a nice reference to the core of lisp functionality (list processing).






Here is where things begin to get a bit more specific - evaluating our “music” primitives will actually yield further tokens (which we use to get around the absence of a real symbol table).





^ Finally, we arrive at the evaluator; this ties everything together, outputting our populated tokens with their arguments resolved (via variable lookups in our closures, or parsing numbers, or evaluating further functions). 




^ Lastly, here are our samples, which are being loaded in at runtime.

All in all, there’s nothing too interesting to look at, at least not until we get to our player file, which contains the glue tying our AST and tokens to Tone.js and tone’s specific logic:





What’s happening here is that, as the ‘data’ (our evaluated, interpreted AST result - a large state-object) is passed into [extractData], it is then decomposed according to Tone’s logic. This is actually quite a nice way to work: the interpreter is only really focused on turning all of the information we’ve encoded in language, and creating a state-object that can then be used by some other process.

While this is probably the simplest approach we could take, it also means that the language is not a “true” language. I mean, it *is* technically, but in reality the rules that govern its syntax in order to be somewhat compatible with Tone’s idiosyncracies are borderline unworkable.

For ease of use, I’ve also incorporated a “macros” module which I’m ommiting here, but which allows us to expand certain lines into pre-defined code, saving us time when creating new arrangements and patterns.





This is what the parser looks like in the end. 






Assignment 4 -  Melody


This week’s assignment was to play with melody, and create a composition using melodic principles.

The past weeks of building “instruments” had left me wanting to make something that was more data-driven, and so I turned to find data sources that could be turned into some sort of melodic progression.

I made two attempts: the first, using NOAA buoy data, and the second using the UK Shipping Forecast by the MET Office.

First Attempt - Buoys

NOAA has a site where it collects all of the real-time wind and weather data from buoys located around the world. This data is freely available as a plain-text file that can then be parsed. 





This is what the data looks like:




I put together an instrument that maps wind direction (0-360) to either a Pentatonic, Minor, or Major scale. Multiple files can be loaded, and in this way, multiple “buoys” can be played. Each buoy has its own Synth instance, meaning its sound can be adjusted to better complement the other buoys.





The result is interesting, but as I’d quickly find out with this type of input, quite boring.

Second Attempt - UK Shipping Forecast

The UK Shipping Forecast is near and dear to my heart, having had to transcribe it for a couple of months while sailing in the English Channel. It follows a pre-defined format, and is read multiple times during the day over the radio, covering all of the coast of the UK. 

The nice thing about the Shipping Forecast versus the buoys is that, because it covers a wide area, you see many shifts happen in the wind direction as well as sea state, making for more interesting compositions.

Like with the buoys, the forecast is parsed and offsets from a base note are developed. These offsets are then translated to notes on a pentatonic scale, and played in sequence. We can optionally choose for the tempo to be affected by the sea state, but this is turned off whwne we’re playing everything sequentially.






This is what the final program looks like - a bit more sparse than the first, but it’s done for simplicity’s sake as well. You navigate using the keys, and you can choose to play each forecast area indipendantly or together in sequence.