XJ music enables composers to realize new possibilities in dynamic music for video games.
The overall XJ music system is implemented by the XJ music workstation, a DAW dedicated to preparing dynamic video game music and the XJ music engine to integrate the product within your game audio system.
Download and install the XJ music workstation on your computer to start creating dynamic music for video games. The latest version of the XJ music workstation can be downloaded from xjmusic.com/download.
You may use the XJ music engine completely free for projects with a budget under $250K. For larger projects, contact licensing@xjmusic.com for a license.
Open-source Codebase
The entire codebase is open source and available on GitHub
The best way to get in touch with the core engine developers is to join the XJ music Discord.
To get started contributing to the project, see the contributing guide. This document also includes guidelines for reporting bugs.
Subsections of XJ music documentation
What is XJ music?
XJ music enables composers to realize new possibilities in dynamic music for video games.
XJ includes the XJ music workstation, a DAW dedicated to preparing dynamic video game music and the XJ music engine to integrate the product within your game audio system.
The XJ music workstation and XJ music playback engine are software that implements the patented XJ music system (“XJ”) for generating musical audio compositions, protected by US Patent Grant 10,446,126. This music software is intended for creating musical compositions for playback on a wide variety of devices.
XJ is founded on the core principle that music is human, enabling composers to realize new possibilities in dynamic music for video games. We seek to innovate what’s possible. XJ is a new algorithmic medium. We are building a new school of cutting-edge musicians who will put this system to work.
We work tirelessly on the XJ music platform to cultivate a compelling world for composers in our new music engine. In the XJ music Podcast, the XJ team shares highlights from our journey.
XJ utilizes a versatile template system that accepts various inputs, such as audio stems, programs, and instruments from many libraries.
XJ’s playback engine is more complex than it seems. The engine meticulously reads the content and constructs an ongoing sequence of segments. Each segment, unique in its timeline placement, consists of all the content employed to forge it.
Creating a segment requires several elements. Initially, you will need Macro Programs, which are comparable to the sequence of songs a DJ would pick to play one after another. Following this, you will need Main Programs - these are analogous to the individual pieces in a DJ’s set, complete with chord progressions and specific voicings for each instrument in the song.
As the XJ moves through the creation process, it seeks out Beat Programs. These programs house a collection of drum events that construct the underlying rhythm of the piece. XJ further scours for Detail Programs that execute the voicings derived from the Main Programs, such as a bassline or stab pattern.
To fully realize the sonic details in the programs, XJ will search for the appropriate Instruments, each packed with audio content. For instance, a bass detail pattern from the Detail Program merges with bass voicings from the Main Program. This combination then aligns with a polyphonic bass-type instrument selected to curate and choose the final bass audio for the segment.
Some types of instrument audio are utilized formulaically within XJ, such as transition audio, percussion audio, and ambient audio. This creates a cohesive, dynamic, and robust sonic experience. Hence, XJ combines all these elements to generate a beautifully organized and harmonious audio segment, providing a user-friendly, intricate, and comprehensive approach to music creation.
Workstation
The XJ music workstation is a digital audio workstation (DAW) that is dedicated to preparing dynamic video game music. It is the primary tool for creating music in the XJ music system.
View the source code in the workstation package within the XJ music code repository.
Engine
We’ve built the playback engine for XJ music as a shared library, which can be used in any language that can interface with C++. The engine is designed to be as simple as possible, and to be able to run on any platform.
View the source code in the engine package within the XJ music code repository.
Installation
Depending on your operating system, download the appropriate binary from https://xjmusic.com/download and follow the corresponding instructions:
Download the XJ music workstation Mac OS (Intel) installer ZIP file to your chosen directory.
Extract the contents of the ZIP file with your extraction tool of choice.
Double-click the DMG file to mount the installer disk image as a volume.
Open the XJ music workstation installer volume that appears on your desktop (unfortunately, it is not named “installer” nor does it have an installer icon appearance)
Inside the installer volume, drag the XJ music workstation application into your Applications folder.
After installation, the XJ music workstation will appear in the Applications folder as well as in the Spotlight Search results.
The first time you try to start a new installation of the XJ music workstation, Mac OS will refuse to open the application because this app is not signed. Choose the “View in Finder” option.
In Finder, inside the Applications folder, right-click (option-click) on the XJ music workstation application and choose Open- this will override security and run the application.
Repeat the previous step. For some reason, Mac OS forces this.
Mac Installation (Silicon)
Download the XJ music workstation Mac OS (Silicon) installer ZIP file to your chosen directory.
Extract the contents of the ZIP file with your extraction tool of choice.
Double-click the DMG file to mount the installer disk image as a volume.
Open the XJ music workstation installer volume that appears on your desktop (unfortunately, it is not named “installer” nor does it have an installer icon appearance)
Inside the installer volume, drag the XJ music workstation application into your Applications folder.
After installation, the XJ music workstation will appear in the Applications folder as well as in the Spotlight Search results.
The first time you try to start a new installation of the XJ music workstation, Mac OS will refuse to open the application because this app is not signed. Choose the “View in Finder” option.
In Finder, inside the Applications folder, right-click (option-click) on the XJ music workstation application and choose Open- this will override security and run the application.
Repeat the previous step. For some reason, Mac OS forces this.
Windows Installation
Download the XJ music workstation Windows installer ZIP file to your chosen directory.
Extract the contents of the ZIP file with your extraction tool of choice.
Double-click the Application file in the folder to begin installation.
Windows Defender may stop the installation process- select More Info and then Run Anyway.
Follow the steps in the Setup Wizard to choose where you want the workstation to be installed.
Linux Installation
Debian/Ubuntu users simply install the .deb package to make the XJ music workstation available from the launcher.
Getting Started
When you first open the XJ music workstation, you’ll see this empty space, because no project is open.
Project Menu
The Project menu provides a comprehensive set of options for managing project files efficiently. Creating new Projects, opening previously created Projects, saving your work, changing directory settings, and quitting the workstation is done here.
Fabrication Menu
The Fabrication menu displays two commands for the fabrication of musical segments, Start (CTRL+Space) and Follow (CTRL+ALT+Space).
View Menu
The View menu contains navigational shortcuts between the Content, Templates, and Fabrication tabs, Log controls, and backward/forward buttons to visit their previously viewed locations.
Content Tab
In the Content section of the workstation, you will see the Content Browser which lists
all the the Libraries in the current project. Inside of each Library
is a list of Programs and Instruments.
In the Templates section of the workstation, you will see a list of available Templates in the current project. Templates contain bound Libraries- your Programs, and Instruments.
Fabrication Tab
In the Fabrication section of the workstation, you will see another empty space.
If all Libraries have finished loading their data, you can prompt the workstation using the Start button in the top right-hand side, and you will see the playback Timeline displayed!
Subsections of Getting Started
Menu Bar
When you first open the XJ music workstation, no project is open. From the Project menu you can create a New Project, Open a Project, or clone one of the Demo Projects to get started right away.
Demo Projects
When first opening the XJ music workstation, the user will have the option of demoing four demo projects from our in-house artists: Bump, Slaps, Space, and The Hero’s Journey. These demos offer users a firsthand experience of the workstation’s capabilities as a platform and can inspire creativity in their own sessions.
To open a demo project, access the Project dropdown menu in the top left-hand side of the workstation, and choose Demos. Here you will find a list of four projects created by our team that will help you get acquainted with what the workstation can do. Select one and click OK to begin loading the project’s data, this could take a few minutes as you will be downloading the included audio files.
View Menu
View Menu
Description
Content (F3)
Switches to the Content tab.
Templates (F4)
Switches to the Template tab.
Fabrication (F5)
Switches to the Fabrication tab.
Show Logs (F10)
Opens the Log window at the bottom of the Workstation, allowing the user to view the status and progress of ongoing processes.
Logs Auto-scroll (Ctrl+F10)
Access tokens from the Lab are pasted here to establish a link between the XJ workstation and the Lab.
Log Level
Allows the user to select which levels of information they will be shown in the logs. There are options to toggle Debug, Info, Warning, and Error logs on and off.
Navigate
Allows you to navigate Back (Ctrl+ALT+←) and Forward (Ctrl+ALT+→)
Help Menu
Help Menu Item
Description
About
Displays the current version of the XJ music workstation.
Tutorial Video
Open a walkthrough video to help you get started with a demo project.
The Content Browser is the central hub for navigating and managing various types of content within the XJ music workstation. Accessible via the Content tab at the top right-hand corner, this main window offers an organized and intuitive view of the available resources, including Libraries, Programs, Instruments, and Audios.
Program Editor
The Program Editor allows you to edit music notation by placing Events within Patterns.
There are different modes of editing available. The editor will automatically show the panels corresponding to the type of program you are currently editing: Macro Program, Main Program, Beat Program, or Detail Program.
Each sequence of a macro program corresponds to a song a.k.a. main program that will be chosen.
The content of a macro program sequence is irrelevant. We only use a sequence to store attributes such as tempo and intensity. For example, this is how tempo is driven. However, we are waiting on some software enhancements such as XJ Tempo Behavior v2 and a more user-friendly editor mode for macro programs.
First, you’ll create some number of sequences containing attribute values for total, key, tempo, and intensity:
Next, you’ll make sure to be in BIND mode (see the upper left corner) and then bind your sequences into some order. The bindings can have unique memes.
Editing a Main Program
Main Programs contain musical notation describing chord progressions and voicings of those chords for various types of instruments. XJ will look for the chord progression/voicing combinations when the main program has been bound to a template and chosen during playback.
Editing a Beat Program
Beat programs utilize a grid system to graphically represent time, divided into equal segments and measured in bars. When editing a beat program, users place events on the grid by left-clicking within the timeline. To change the positioning, left-click and drag.
With the snap option enabled, events will be dragged to the nearest grid line during placement and modifying duration.
To extend or shorten the length of the event, bring your cursor to either side of the event, click and drag the edges. Double-clicking on an event brings up its event properties menu, which contains Tones, Velocity, Position, and Duration options.
To delete an event, right-click the event and choose delete.
Editing a Detail Program
Detail programs utilize the same grid as Beat Programs, but to coordinate the position, duration, velocity, and notation of melodic events.
Instrument Editor
In XJ music, an instrument comprises a set of Instrument Audios. This section covers the importation of audio stems and the corresponding properties for uploaded audio.
Importing Audio
Click the Import Audio button at the top right above the main Instrument window to open your file explorer and choose an audio file to upload. The fiile types currently supported are .wav, .aiff, and .mp3. Upon importing a new audio file, the user will be taken to the audio editor.
Audio Editor
Audio Properties
The properties of the selected audio are listed in the side panel to the left of the waveform viewer and display the name, event, volume, tones, tempo, intensity level, and loop beats length. Underneath the main audio properties panel is a highlighted section containing the numeric value for the transient offset.
Open Audio Folder and Open Audio File
The two icons in the bottom left of the Waveform Viewer will open the selected audio’s folder and open the selected audio file, respectively. The audio’s original filename is displayed to the right of the open selected audio icon.
Setting a Transient Offset
A transient offset is a floating integer that determines in measurements of seconds where the chosen audio will be starting from when its event is slated to trigger. The default transient offset is 0.0, indicating that the audio will start playing at the beginning of its file when triggered. A transient offset of 0.5 will include a pre-roll of 0.5 seconds and begin playing there. To set a transient offset, click the + symbol and place the new offset on the waveform in its window. You can also manuially enter the exact time in the Transient text field.
Waveform Viewer
The waveform viewer component gives the user a graphical representation of the amplitude of selected audio over time, allowing the user to pinpoint peaks and valleys, monitor levels, and synchronize timing.
Use the magnifying glass icons at the top left of the waveform viewer to zoom in and out.
Fabrication Tab
In the Fabrication tab is where you can control the playback of your music. The Fabrication timeline is the command center of the XJ music workstation, offering precise control over your compositions.
There are a number of Fabrication Settings that can be modified to change the timeline behavior and audio output.
In XJ music, an Instrument is a collection of Audios. While all instruments are stored according to the Instrument Model, different Types and Modes of instruments have unique purposes and behaviors.
The buttons on each row can be used
to Edit, Duplicate, or Destroy that Library.
Programs
In XJ music, a Program is a collection of musical instructions. While all programs are stored according to the Program Model, different Types of programs have unique purposes and behaviors.
Each program contains a configuration, which may affect the music when that program is chosen.
The program menu contains a list of all programs.
Here you can see the attributes of each Program, as well as the memes assigned to it.
To compose the content of any given program,
click its name in the list to open the Program Editor.
Type Purpose Macro Programs Like a playlist, this describes an order in which to play songs a.
Subsections of Programs
Events
Automatic fabrication performs a search for matching Instrument Audio for the chosen Sequence Pattern Voice-Events.
That is how an Audio is chosen inside of an Instrument, based on its Audio-Events.
We use a score-based stochastic model for event audio picking, as opposed to the marble bag model we use for choice.
Drum Beat Events
The beat program is the center of the groove during any given main program. XJ chooses a single beat and keeps that beat for the duration of the main program.
It is important for percussive-type instruments to use the following standardized set of Event Names, which have far more influence on the outcome of percussive instrument-audio choice (than, for example, Event Notes).
Currently, we are using a very narrow set of events for in Beat-type programs.
We currently use these codes, for beat program events, and drum instrument audio events:
CODE
Name
HIHATCL
High Hat Closed
HIHATOP
High Hat Open
KICK
Kick
SNARE
Snare
Drum Detail Events
In the future, XJ will also choose Detail programs having additional drum events to decorate the groove. These drum detail programs serve a similar function as percussion loop and transition instruments.
We expect that these events may include various types of drums or other percussion instruments.
We plan to use these codes for drum detail program events, and drum instrument audio events:
CODE
Name
BONGOHI
Bongo High
BONGOLO
Bongo Low
BLOCKWDLO
Block Wood Low
CABASA
Cabasa
CLAP
Clap
CLAVES
Claves
CONGALO
Conga Low
CONGAHIOP
Conga High Open
CONGAHIMU
Conga High Mute
COWBELL
Cowbell
CRASH
Cymbal Crash
CUICAOP
Cuica Open
CUICAMU
Cuica Mute
GONG
Gong
GUIROSH
Guiro Short
GUIROLN
Guiro Long
HIHATCL
High Hat Closed
HIHATOP
High Hat Open
KICK
Kick
MARACAS
Maracas
RIDE
Cymbal Ride
SHAKER
Shaker
SLAP
Slap
SPLASH
Cymbal Splash
STICKSIDE
Stick Side
TAMBOUR
Tambourine
TIMBALEHI
Timbale High
TIMBALELO
Timbale Low
TOMFLRHI
Tom Floor High
TOMFLRLO
Tom Floor Low
TOMHI
Tom High
TOMHIMID
Tom High Mid
TOMLO
Tom Low
TOMLOMID
Tom Low Mid
TRIANGMU
Triangle Mute
TRIANGOP
Triangle Open
VIBRASLAP
Vibraslap
Percussion Loops
Percussion Loops have an Instrument Type of Percussion, and an Instrument Mode of Loop.
They are assigned for each fabricated segment during percussion loop craft. It’s going to be used to determine a target # of perc loops.
Assumed that one or more loops will be chosen for any segment— minimum and maximum are configured via template config. For each MP, setup delta arcs for N number of percussive loops.
Perc loop instrument behavior selects one audio per section. Consolidating audio with the same sound set under one instrument. Then setting the default length an audio stays in the stream to a full section rather than switching every 4 beats.
Percussive Loops can specify # beats in the instrument or audio editor.
Loop Priority
Some Perc Loops are killer by themselves. In the case of house music, this might be a fully fleshed out 16x 16th notes of house locomotion, requiring only an added kick and snare to be complete. Let’s call these PRIMARY perc loops.
Other Perc Loops are intended to be added on top of a stable base of Primary perc loops.
Other Perc. Loops are intended to support the known structure of the rhythm.
Based on the intensity value of any given perc loop and the current mix intensity, XJ will fade each perc loop into the mix.
Configuration of Programs
Type
Purpose
barBeats
Sets the length of a bar, as measured in total number of beats.
cutoffMinimumBars
Sets the minimum length of bars that a sequence can transition into another in.
doPatternRestartOnChord
Determines whether patterns restart on chord changes.
Types of Programs
Type
Purpose
Macro Programs
Like a playlist, this describes an order in which to play songs a.k.a. main programs.
Main Programs
Like a song, this musical notation describes chord progressions, as well as voicings of those chords for various types of instruments.
Beat Programs
This describes a beat, as in the foundation of a groove. XJ chooses one beat for an entire song (Main Program). This musical notation is used directly for a Drum Instrument to play.
Detail Programs
This involves transcribing the rhythmic pattern for a specific instrument, allowing XJ to interpret and apply it to any song based on its chord voicings.
Instruments
In XJ music, an Instrument is a collection of Audios. While all instruments are stored according to the Instrument Model, different Types and Modes of instruments have unique purposes and behaviors.
Different Types and Modes of instruments have unique purposes and behaviors.
Option A, is to tag audios with a single chord symbol (ie Cmaj7, Emin7b5) which is itself a shorthand for a list of notes (C, E, G, B and E, G, Bb, D for these examples)
With some clever voicing list tricks you could have a lot of freedom with this route. For example if you wrote a voicing list that included both Fmaj7 and Amin7 you would be outlining Fmaj7/9
The more complicated and versatile route, Option B, is what you’re saying—tagging audios with comma-separated lists of notes and having XJ cross-reference those with Main Program voicing lists to choose audios.
I think both will be useful, but to me the chord symbol tagging seems like a faster route to get multi-note audio happening because it’s the same process as the current single-note tagging. I could be underestimating what’s going on currently though
Theoretically, option B is closer to what exists today, but that’s not an obstacle.
I believe that there is a simple formula to tell us whether Option A or Option B yields more freedom a.k.a. modularity a.k.a. less work for more payoff.
Starting with Template Analysis (soon to be built) we can see how many different chords are present in all the main programs of the given template
(For each type of instrument independently?) Calculate the relative complexity of using Chord-based (option A) or Note-based (option B) audio
Chord-based complexity = How many different audios would be required to cover all the chords present in all main programs
Note-based complexity = How many different audios would be required to cover all the voicing lists present in all main programs
I suspect the answer (A vs B) varies between templates and instrument types
How Craft Phases, Instrument Types, and Instrument Modes Interact
Instrument Configuration
Setting
Description
isMultiphonic
Toggles whether multiple notes can be played at once.
isAudioSelectionPersistent
Toggles between persistent and randomized audio selection.
isOneShot
Toggles Instrument between one shot sounds and loops.
isOneShotCutoffEnabled
Enables one shot sounds to cut one another off when played.
isTonal
Determines whether audios are tonal or atonal.
oneShotObserveLengthOfEvents
Determines whether one shots play out fully or are influenced by the length of their events.
releaseMillis
Sets release length in milliseconds.
Instrument Menu
The instrument menu contains a list of all instruments. Each instrument contains a configuration, which may affect the music when that instrument is chosen.
Here you can see the attributes of each instrument, as well as the memes assigned to it.
To compose the content of any given instrument,
click its name in the list to open the Instrument Editor.
Pro Tip
The buttons on each row can be used
to Move, Edit, Duplicate, or Destroy that Instrument.
Instrument Modes
Chord-mode Instruments
A chord-mode instrument such as a Pad-type instrument will provide a single audio per chord. In that case each row (audio) in the instrument will use the tones column to store the chord. The event column is ignored.
When choosing instrument audio (from a chord-mode instrument) for the chords of a main program, XJ will prioritize (see: marble bag) any available exact match. In lieu of an exact match, XJ will allow an acceptable substitute of the pre-slash description of a chord to a non-slash chord (e.g. “Eb/G” is an acceptable substitute for “Eb” and vice versa).
While the presence of voicings is used to make the decision “I’m going to look for a ___-type instrument” once XJ has arrived at the decision “I’m going to craft this ___-type chord-mode instrument now for this segment” there are zero references to voicing content. The content of the chord voicings is ignored. E.g. It’s possible to tell XJ to seek Pad instruments for a given main program even if the content of those voicings are all just “X” but it’s critical to understand the whole template you are composing. If the design of the template calls for both Note- and Chord-type instruments for a given instrument type, then it’s necessary to write whole voicings (comma-separated lists of notes voicing each chord) for interoperability.
A polyphonic instrument such as a Bass-type instrument will provide a single audio per note. In that case each row (audio) in the instrument will use the tones column to store the note. The event column is ignored.
A Drum-type instrument will provide a single audio per drum event. In that case each row (audio) in the instrument will use the event column to store the event. The note column is ignored.
Loop-mode Instruments
Audio in loop-mode plays through its total length continuously, repeating at the start of each Main Program selection in a segment.
Instrument Properties
In the top header of an instrument are included properties that apply to the entire instrument, such as names, memes, type, mode, state, and volume.
Instrument Property
Description
Name
The name of the audio file. This defaults to the filename of the audio after importing but can be changed in the audio editor menu.
Event
Determines the type of instrument audio event that the chosen audio will be, which influences the mixing choices of a segment.
Volume
This value is a floating point integer that sets the volume of the chosen audio. 0.0 is completely silent and 1.0 is the original volume of the imported audio file. The volume value can be increased beyond 1.0 and its original volume, for example, an audio set to 2.0 will be twice as loud as the original file.
Intensity
A floating integer from 0.0 to 1.0 that determines the intensity level at which the audio will get introduced into the overall mix in a segment.
Tones
This value sets the note for the chosen audio. It can be a single note or chord.
Tempo
Sets the audio’s tempo in beats per minute.
Transient
A floating integer that determines in measurements of seconds where the chosen audio will be starting from when its event is slated to trigger. For example, the default transient offset is 0.0, indicating that the audio will start playing at the beginning of its file when triggered. A transient offset of 0.5 will include a pre-roll of 0.5 seconds and be that far into the audio when previously it would just be starting.
Loop Beats
An integer that determines the length of a given audio, measured in bars.
Pro Tip
The buttons on each row can be used
to Edit or Destroy that Audio.
Instrument Types
Drum
Drum instruments contain audios that XJ will pair with Beats during the beat-craft phase. This can be any one shot drum event, including kick drums, snare drums, hi-hats, toms, cymbals, and more.
Bass
Bass instruments contain melodic bass audios and can be played in all three instrument modes- event, chord and loop. Bass instruments are introduced during the detail-craft phase.
Pad
Pad instruments contain sustained melodic pad audios that are typically used as underlying texture. Pad instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Sticky
Sticky instruments contain one shot audios that “stick” closely to the beat, providing a supportive, repeating, syncopated single note phrase that is usually done by a guitar but could be approximated by any sound provided that the notes are relatively short in duration. Sticky instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Stripe
Stripe instruments contain melodic audios that punctuate transitions, highlight important beats, or enhance climactic moments in the music. Stripe instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Stab
Stab instruments contain short percussive or chordal audios that are often created using synthesizers, electric pianos, or brass instruments. Stab instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Hook
Hook instruments contain memorable musical phrases or riff audios played by instruments such as guitar, bass, or keyboards. These hooks can serve as the foundation of the song’s melody or provide a catchy motif that recurs throughout the composition. Hook instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Percussion
Percussion instruments contain audios that consist of rhythmic patterns or sequences of percussion instruments that are often used to add groove, texture, and rhythmic complexity to a track. These instruments typically consist of various percussive elements such as drums, cymbals, shakers, tambourines, congas, bongos, and other percussion instruments. Percussion instruments can be played in event, chord and loop modes, and are introduced during the detail-craft phase.
Transition
Transition instruments contain audios that are used to smoothly move between different sections or parts of a song, scene, or composition. These sounds help to create cohesion and flow within the production, guiding the listener from one part to another and maintaining interest and engagement throughout. Transition-instruments can only be played in event mode and are introduced in the transition-craft phase.
Background
Background instruments contain audio textures that are at least 1 minute (longer than any section). They are used to create an atmospheric backdrop and establish a sense of place, mood, or environment without necessarily being the focal point of the sequence. When chosen, XJ uses a random piece from within the available audio to fill each section when the instrument is chosen and builds in and out of layers of background audio the same way it does perc loops. Backgorund instruments can only be played in loop mode and are introduced in the background-craft phase.
Memes
A meme is a tag that artists apply to programs and instruments in order to guide the choices that XJ makes.
XJ factors in the memes it sees in order to make choices about which programs and instruments will be included in a Segment.
Pick a Marble from the Bag
In order to make random choices based on weighted options, XJ uses a stochastic mechanism called a Marble Bag.
The example figure to the left represents a choice between three possible outcomes: Red (R), Green (G), and Yellow (Y). The likelihood of a choice is increased when we add more of that choice’s marbles to the bag.
In the following section, we’ll discuss how the presence of different kinds of memes either increases or decreases the chance of a selection, or prevents a particular option from being chosen at all. These are in fact expressed by adding and removing marbles from a virtual bag.
For example, to make sure an option is never chosen, we remove all of its marbles from the bag.
XJ’s marble bag is actually divided into phases. When a marble is put into the bag, it is assigned a phase. For example, if the phase 1 bag contains any marbles, we will pick from only the phase 1 bag and skip phases 2 and beyond. This supports functionality such as “XJ always chooses a directly-bound program or instrument when available”
Meme Type
Example
When meme matches
Regular Meme
PEACHES
Increase likelihood to choose content that matches regular memes XJ has already chosen for the segment.
Anti-Meme aka Not-Meme
!PEACHES
Do not choose this content, if we have already chosen content with the corresponding regular meme. If XJ has already chosen content for the segment having anti-meme !PEACHES, then XJ will not choose content having regular meme PEACHES
Unique-Meme aka Solo-Meme
$PEACHES
Do not choose this content, if we have already chosen content with this unique-meme. If XJ has already chosen content for the segment having unique-meme $PEACHES, then XJ will not choose any other content having this same unique-meme.
Number-Meme
5PEACHES
Do not choose this content, if we have already chosen content with a different number of the same meme. If XJ has already chosen content for the segment having number-meme 5PEACHES, then XJ will not choose content with number-meme 7PEACHES.
Strong-Meme
PEACHES!
Only choose content if we have already chosen content with the corresponding regular meme. Only if XJ has already chosen content for the segment having regular meme PEACHES will XJ consider choosing content with strong meme PEACHES!
A set of memes is referred to as a meme theorem (a.k.a. meme stack) and a meme theorem is valid if all the memes
in the set are compatible. For example, the set of (PEACHES,!PEACHES) is invalid because you cannot simultaneously allow
and deny PEACHES. As XJ crafts a segment, it adds content one at a time. First a Macro-type program, then a Main-type
program, then a Beat-type program, then Instruments and other programs. As each piece of content is added, XJ checks the
memes associated with that content and adds these memes to the segment. If the memes of a new piece of content would
create an invalid theorem when added to the segment, XJ will not add that content at all. Ultimately, the memes of any
given segment should add up to a valid theorem.
Note: Strong memes are not added to the stack.
Meme Taxonomy
A template configuration has a field called memeTaxonomy which defines the taxonomy of memes.
That would tell XJ about the existence of a meme category called COLOR with values RED, GREEN, and BLUE, and a meme category called SEASON with values WINTER, SPRING, SUMMER, and FALL.
For example, after content having RED is chosen, we can choose nothing with GREEN or BLUE.
*NOTE: Use a semicolon to separate the categories. Each category has a comma-separated list of memes.
Chord Interpretation
Chord Interpretation
Using 12-tone equal-tempered octaves.
Chord
Synonyms
C
C maj, C major, CM
C 7/9/13
C dom 7/9/13, C dominant 7/9/13, C 7/6/9, C dom7 6/9, C dominant 7 6/9
C +7
C aug7, C augmented7, C 7#5
C dim
C diminished, C°
C 7#9
C dom7#9, C dominant, 7#9
C 5
(N/A)
C 6/9
C maj6/9, C major6/9, CM6/9, CM6add9, C 6add9, C maj6add9, C major6add9
C -13
C min13, C minor 13, Cm13, C -7/13, C min7/13, C minor7/13
C -7/11
C min7/11, C minor7/11, Cm7/11, Cm11, C -11, C min11, C minor11, C mi11, C mi7/11
C maj7#11
C major7#11, CM7#11, C∆#11
C -6
C min6, C minor6, Cm6, C mi6
Chord
Synonyms
C -
C min, C minor, Cm, C mi
C 7/13
C dom 7/13, C dominant 7/13, C 7/6, C dom7/6, C dominant 7/6, C 13
C 7
C dom7, C dominant 7, C dominant
C 7b9
C dom7b9, C dominant 7b9
C 7#9b13
C dom7#9b13, C dominant, 7#9b13
C dim7
C diminished7, C°7
C add9
Cmaj add9, C major add9, CMadd2, CMadd9, C add2, C maj add2, C major add2
C -7b5/11
C -7b5/11, C minor7b5/11, Cm7b5/11, C Ø11, C half dim 11, C half diminished 11
C min/maj7
C -maj7, C minor/maj7, CmM7, Cmin∆, C-∆, C mi/maj7, C minor/major7, C minor major 7
C 7sus4
C dom7sus4, C dominant 7sus4, C 7sus
C -b6
C minb6, C minor b6, Cmb6
Chord
Synonyms
C maj7
C major7, CM7, C∆
C -7b5
C min7b5, C minor7b5, Cm7b5, CØ, C half dim, C half diminished
C 6
C maj6, C major6, CM6
C 7b9b13
C dom7b9b13, C dominant 7b9b13
C 7/9b13
C dom7/9b13, C dominant 7/9b13
C sus4
C sus
C add4
C maj add4, C major add4, CMadd4, CMadd11, C add11, C maj add11, C major add11
C 7/9
C dom7/9, C dominant 7/9, C 9, C7add9
C aug maj7
C +Maj7, C augmented major 7, C+M7, CaugM7, Caug∆, C+∆, C maj7#5, C major7#5, C+♮7, C∆#5
C dim maj7
C diminished major 7, CdimM7, Cdim♮7, C°M7
Chord
Synonyms
C -7
C min7, C minor7, Cm7, C mi7
C 7add4
C dom7add4, C dominant 7add4, C 7/11, C 11, C dom7/11, C dominant7/11
C +
C aug, C augmented
C #5
C 7b9/13, C dom7b9/13, C dominant 7 b9/13
C sus2
C 5add9, C 5add2
C maj7/9
C major7/9, CM7/9, CM7add9, C∆9, C maj7add9, C major7add9, C maj7add2, C major7add2
C 7#11
C dom7#11, C dominant 7#11, C 7b5, C dom7b5, C dominant 7b5
C -7/9
C min7/9, C minor7/9, Cm7/9, Cm7add9, C min7add9, C minor7add9, C mi7/9, C mi7add9, C -7add9
C maj7/13
C major7/13, CM7/13, C∆13
Fabrication
XJ music fabrication is the command center of the XJ music workstation, offering precise control over playback of your compositions.
Segment Model
During fabrication, XJ constructs a Chain made of a series of Segments. Each Segment is a self-contained unit of music that can be played back in real-time or recorded to a file. Segments can be combined to create more complex compositions.
The fabrication settings can be opened from the top menu or the hamburger icon in the Fabrication tab.
Subsections of Fabrication
Delta Arcs
In the XJ music system, the main program is divided into sequences. Each segment is a musical composition that is based on a sequence.
The delta value of any segment counts from 0 (at the beginning of the main program) incrementing each segment by
the total number of beats in that segment. The delta value is used to calculate the relative position of a segment in the main program.
The delta arc value of any segment is the intensity of the segment. The arc value is used to calculate the intensity of the segment in the main program.
Fabrication Settngs
The fabrication settings can be opened from the top menu or the hamburger icon in the Fabrication tab.
The Fabrication Menu is the command center of the XJ music workstation, offering precise control over your compositions. The table below will walk you through each feature, ensuring you make the most of the creative possibilities at your fingertips.
Fabrication Menu Settings
Description
Input Template ID
Input the specific template ID to instruct the workstation on which template to play.
Output Mode
Playback Mode plays the selected Template back in real-time. File Mode generates a recording of the Template faster than real time, but without playback.
Output File Mode
Controls how outputs in File Mode are divided. Continuous outputs one file, while Segment, Macro, and Main output files are based on the lengths of each of the three options.
Output File Seconds
Sets the length of a file generated using the Continuous file mode.
Output File Path Prefix
The location where files will be saved.
Output Frame Rate
The frame rate at which sounds associated with the Template will be used or converted to before playback or output.
Output Channels
The number of channels utilized in playback and output.
Craft Ahead Seconds
The amount of time XJ will Craft ahead in seconds.
Dub Ahead Seconds
The amount of time XJ will Dub ahead in seconds.
Ship Ahead Seconds
The amount of time XJ will Ship ahead in seconds.
Content Storage Path Prefix
Sets the location where sound files associated with the chosen Template will be saved and preloaded from.
Timeline Segment View Limit
Limits segments available to view in the timeline to this value.
Templates
Templates are packaged compilations of Libraries, potentially highlighting specific Instruments and Programs that can be previewed and/or broadcast.
Each template contains a configuration, which may affect the music when fabricating that template.
Click on the name of a Template to edit that template.
Pro Tip
The buttons on each row can be used
to Edit, Duplicate, or Destroy that Template.
The Bindings menu allows you to specify which Library, Program, or Instrument can be utilized in a Template. When creating a Template it is important to Bind all of the necessary Libraries, including Main and Macro Programs, Beat Programs, Detail Programs, and Instruments.
Configuration of Templates
Each Template contains a Template Configuration, a set of values and definitions artists provide to XJ in addition to their chosen Bindings.
The Template Configuration menu is accessed by clicking the gear icon in the middle of the Template header.
Template Configuration Values
Description
backgroundLayerMax/backgroundLayerMin
The maximum and minimum number of background Ambience Loop events selected
bufferAheadSeconds/bufferBeforeSeconds
The amount of seconds that the Template will buffer ahead and before*
choiceMuteProbabilityInstrumentType
The chance that the Instrument type will be randomly muted in a sequence, from 0.0-1.0 (0% to 100%)
deltaArcBeatLayersIncoming
The number of Beat Layers being input into the Delta Arc function
deltaArcBeatLayersToPrioritize
The Events in the Sequence to be prioritized in the Delta Arc function
deltaArcDetailLayersIncoming
The number of Detail Layers being input into the Delta Arc function
deltaArcEnabled
Enables or disables the Delta Arc function
densityCeiling/densityFloor
The upper and lower limits of Sequence Density (0.0-1.0)
detailLayerOrder
The order in which Instrument types are prioritized
dubMasterVolumeInstrumentType
The master volume of an Instrument type
eventNamesLarge/eventNamesMedium/eventNamesSmall
Defines the naming convention for Event sizes
instrumentTypesForAudioLengthFinalization
Determines which Instrument types are muted when another instance of itself plays or the sequence ends
instrumentTypesForInversionSeeking
Determines which Instrument types are able to have their chords inverted
mainProgramLengthMaxDelta
Sets the length of Main Program as they relate to Delta Arcs
memeTaxonomy
The words or phrases XJ will define as Memes (Meme) and the categories that Memes fall into (Category)
mixerHighpassThresholdHz/mixerLowpassThresholdHz
The upper and lower thresholds for the High Pass and Low Pass EQs
outputChannels
The number of output channels
outputContainer
Defines the output container used, e.g ‘OGG’
outputFrameRate
The output frame rate
outputSampleBits
The output bit-depth
percLoopLayerMax/percLoopLayerMin
The maximum or minimum amount of Percussion Loop Layers that will play in the Chain
stickyBunEnabled
Enables or disables the Sticky Buns function
transitionLayerMax/transitionLayerMin
The maximum or minimum amount of Transition Layers that will play in the Chain
vmResourceLimitCpu
* it isn’t recommended that users make changes to this setting.
vmResourceLimitMemoryGb
* it isn’t recommended that users make changes to this setting.
vmResourceRequestCpu
* it isn’t recommended that users make changes to this setting.
vmResourceRequestMemoryGb
* it isn’t recommended that users make changes to this setting.
Dubbing of Output Audio
We are still working with the original simple XJ audio mixer, to combine all source audio into the final output audio. Following is the design for our v0.7 Mixer.
Mix each Layer from arranged Instrument Audio sources
Mix Layers into Segment
Add Segments for Streaming
Ship Keys
Chains are shipped with unique identifiers called keys. Fabrication of Segments
XJ is essentially a traditional sampler-sequencer,[1] extended with new generative features.
Artists create a Template (Libraries of Programs, Instruments, and their Recorded Audio).
XJ fabricates a continuous series of audio Segments.
For each Segment, XJ chooses and arranges instrument audio, then modulates sources and mixes output audio.
Glossary
Composition Artistic musical production showing study and care in arrangement. The act of composition is the process of forming a whole or integral, by placing together and uniting different parts.
Artist (Musician) A skilled practitioner in the art of composition and/or performance of music.
Engineer Person skilled in the principles and practice of music technology, including but not limited to audio engineering, and the operation of musical generation systems.
Digital Audio Workstation (DAW) Electronic device or software application used for recording, editing and producing audio files.
Audio Signal (audio data, audio sample, signal, audio, or sample) Information that represents audible sound, such as a digital recording of a musical performance, persisted in a file on a computer.
Generation A process by which data is created, including but not limited to recording the output of a microphone or performing complex mathematical operations.
Modulation A process by which data is modified in such a manner as to alter at least some property, including but not limited to the amplitude, frequency, phase, or intensity of an audible signal.
Configuration (config) The arrangement or set-up of the hardware and software that make up a computer system.
Audio channel (audio track, track, or channel) A single stream of audio data. Optionally, two or more channels may be played together in a synchronized group. For example, stereo output is comprised of a left channel and a right channel.
Audio composition (audio mixing or mixing) The process of forming new audio by placing together and uniting at least two source audio samples or channels. In the process, each source audio sample may be modulated such as to best fit within the composition of the final audio output.
Audio mixer (or mixer) An apparatus used to perform audio mixing.
Audio event An event which occurs at a specific position in time within a piece of recorded audio.
Metadata Information describing musical properties, including but not limited to events, selections, notes, chords, or the arrangement of audio samples.
Series Two items succeeding in order.
Next Being nearest in time, or adjoining in a series. In an empty series, the next item would be the initial item added to the series.
Terminus Either the initial or final item in a series.
Static Having a permanently constant nature.
Dynamic Having a changing or evolving nature.
Permutation The arrangement of any determinate number of things, in all possible orders, one after the other.
Note A musical sound, a tone, an utterance, or a tune. It may refer either to a single sound or its representation in notation.
Pitch The frequency of vibrations, as in a musical note. The exact pitch of notes has varied over time, and now differs between continents and orchestras.
Interval The distance in pitch between two notes. The violin, for example, is tuned in intervals of a fifth (G to D, D to A and A to E), the double bass in fourths (E to A, A to D and D to G).
Harmonic intervals The distance between two notes which occur simultaneously, as when a violinist tunes the instrument, listening carefully to the sound of two adjacent strings played together.
Melodic intervals The distance between two notes played in series, one after the other.
Chord At least two notes played simultaneously at harmonic intervals.
Scale At least two notes played in series at melodic intervals.
Musical event An action having been, or intended to be performed by a musical instrument, beginning at a specific moment in time, continuing for some amount of time, having characteristics including but not limited to chord, pitch, or velocity.
Harmonic event A single occurrence of an action having been, or intended to be performed by a harmonic instrument.
Melodic event A single occurrence of an action having been, or intended to be performed by a melodic instrument.
Harmonic progression The placement of chords with relation to each other such as to be musically correct and emotionally evocative.
Key (root key or key signature) A musical composition indicating the scale to be used, and the key-note or home-note. Generally, a musical composition ends—evoking resolve—on the chord matching its key. The key of a musical composition determines a context within which its harmonic progression will be effective.
Voice A single identity within a musical composition, such as might be performed by a single musical instrument. A voice is either percussive, harmonic, or melodic.
Voice event A single occurrence of an action having been, or intended to be performed by a single voice of a musical composition. An event has musical characteristics, representing a particular note or chord.
Song A musical composition containing a beginning, a middle, and an end.
Section A distinct portion of a musical composition.
Partial musical composition (part) A subset of a complete musical composition, such as to be interchangeable with other subsets of other compositions.
Composite music A work of musical art created dynamically from distinct parts or elements, distinguished from traditional recorded music, which is mastered and finished statically as a deliverable record.
Aleatoric music (music composed “aleatorically”) Music in which some element of the composition is left to chance, and/or some primary element of a composed work’s realization is left to the determination of its performer(s).
Program (musical program or main program) A partial musical composition comprising or consisting of a progression of chords and corresponding musical events output of said related thereto and/or represented by stored musical notations for the playback of instruments. A program is comprised of at least some section representing a progression of musical variation within the program.
Composite musical program An integral whole musical composition comprised of distinct partial musical programs.
Macro Program This is like a playlist. Musical notation describing the order in which to play Main Programs.
Main Program This is like a song. Musical notation describing chord progressions and voicings of those chords for various types of instruments.
Beat Program This describes a beat. XJ chooses one beat for an entire song (Main Program). This musical notation is used directly for a Drum Instrument to play. The beat is also used to make choices about other arrangements.
Detail Program This is writing down the rhythm of a part for one instrument to play, in a way that may be interpreted by XJ in for any song according to its chord voicings. The most atomic and portable sort of partial musical composition, and is intended to be utilized wherever its musical characteristics are deemed fit.
Instrument A collection comprising or consisting of audio samples and corresponding musical notation related thereto and/or represented by stored audio data for playback.
Library A collection consisting or comprising of both programs and instruments, embodying a complete artistic work, being musical composition which is intended by the artist to be performed autonomously and indefinitely without repetition, by way of the present disclosed technology.
Chain An information schema representing a musical composite.
Segment An information schema representing a partial section of a chain. A chain comprises a series of at least one
References
Kaye, N. Charney. (2019). System for generation of musical audio composition (U.S. Patent No. 10,446,126). U.S. Patent and Trademark Office. https://patents.google.com/patent/US10446126B1
