Graph-based concordance and contextual tonicity/root-perception analysis with pitch memory for large chords

in 5D with multiverse and timetravel

This article is a journal for these projects: github.com/euwbah/dissonance-wasm & github.com/euwbah/n-edo-lattice-visualiser

# tl;dr

A new algorithm for modelling the perception of complexity and tonicity of chords up to 8 notes. Tonicity is the measure of how likely the note is to be perceived as the "tonic", which may or may not correspond to the chord. This is done in real-time which, when used in conjunction with my visualizer, updates a model of pitch memory that tracks the listener's interpretation of up to 8 unique notes over time.

This model is not biased towards any tuning system or musical culture, in the sense that there was no hard-coding of weights/penalties/points from existing musical patterns/vocabulary I am familiar with. Its only inputs are frequencies and the tonicity of each note from the previous update tick. The core model has no machine-learned/statistically regressed parameters and every step is explainable in terms of the frequencies between pairs of notes. Despite this, the results mostly agree with my subjective perception of consonance and root perception in Western/European musical harmony, more than any other model I have seen (however, this is by definition, subjective, but I just needed a model that I can agree with for my personal use).

The only assumption made is that the instrument has a harmonic timbre (although this setting can be changed easily by modifying dyad_lookup.rs).

The model is only based on dyadic relationships between notes, but unlike most other chord complexity models with dyadic-based approaches, it:

1. Captures the gestalt of the chord at different hierarchies of detail, without having to precompute the space of -note chord combinations
2. Does not have harmonic duality (major and minor triads do not have the same complexity score)
3. Depends on the current musical context by keeping track of pitch memory over time

Two key ideas that make it work:

1. Modelling subjective interpretations of chords as trees, where a (parent, child) edge is used to model the listener hearing the child note with respect to the parent note, e.g. C->E means E is heard as M3 of C. By aggregating over the space of interpretations, the model considers different permutations of substructures present in the chord that all contribute to the gestalt. (See Interpretation Trees)
2. Using the asymmetry of the complexity of intervals about the octave to break the symmetry of chord duality/negative harmony. E.g., it is generally accepted that P4 and P5 do not have the same complexity, even though they are octave-duals of each other. This asymmetry is exploited so that intervals have a preferred natural order, and this preference contributes to the likelihood of choosing one interpretation of the chord over another. (See Dyadic Tonicity)

Because it only depends on dyadic relationships, it is computationally efficient and can run in realtime. This model powers the automatic just intonation detemperament, root detection, and dissonance scoring logic in my visualizer which responds as I play.

Example of previous algorithm (less accurate root detection, requires hard coding rules based on circle of fifths which does not work for non-standard tonalities):

Example of new algorithm (more accurate root detection, no hard coding any cultural parameters besides octave equivalence and harmonic timbres):

See more examples on my channel

# Introduction

So far, I haven't found a measure of harmonic complexity/concordance/consonance that agrees with my own subjective perception of chordal and melodic tension and release. I started a bit of initial research towards this goal, and I wanted to document my current progress and thought process in a relatively informal and narrative manner here.

One year ago, I started work on polyadic-old.rs, which is an attempt to compute polyadic chord complexity and do root detection in a way that I am satisfied with. However, the computational cost of the full version of this algorithm is where is the number of notes in a chord, and is the number of candidate ratios that a new tempered note could be relative to the current harmonic context, and I relied a lot on aggressive pruning (via beam-search) to allow this algorithm to run in real-time at as I play, but this aggressive pruning caused the results to deteriorate from the full version quite significantly.

However, in the old algorithm, I found the idea of using spanning trees to represent possible interpretations of a chord promising. This article serves as a journal of my thought process for my second (or third) attempt developing this idea, where complexity is computed as the aggregate complexity over possible interpretation trees.

For context, I needed an algorithm for my real-time detempering music visualizer which heuristically detempers notes played in equal temperament into just intonation (JI) ratios, that is, finding the "most appropriate" JI ratio interval to represent a note that I played which could represent various possible JI ratios. On average, , and so a full search would require around 30 million computations per frame.

I needed a chord evaluation algorithm that can:

1. Be ridiculously fast. Max 16ms acceptable latency per note played, preferably under 10ms. Each update tick needs to be able to run at least 5 times per second, all while displaying a three.js visualization, OBS screen recording, video streaming, and running VSTs.

2. Keep track of the current harmonic context and consider the element of time. Since music is contextual, prior notes must be able to affect the current model of concordance and tonality perception, and the root detection/concordance score should evolve over time even as a single chord is held.

   • Ideally, rhythmic entrainment and strong/weak beat detection should be added too, but that's an equally huge problem to tackle later on.

3. Does not have symmetry/negative harmony/duality assumptions. The same chord voicing spelt in upwards intervals (e.g. C E G is a major triad spelt as M3 + m3 up from C) and downwards intervals (e.g., C Ab F being - M3 - m3 down from C) should not necessarily output in the same concordance scores or detected roots.

4. Perform some sort of root detection. Doesn't have to be perfect (which requires encoding cultural entrainment perfectly), but good enough to detect the root within pm 1 fifths away from the actual root, so that my visualizer can display the correct enharmonic spellings of notes played. E.g., in 31edo, if I play a C major chord with A# approximating , I would rather have the A# displayed as the enharmonic equivalent Bb< (HEWM notation).

5. Work in any tuning system.

6. (Not yet implemented) Has proper modelling of the physiology of hearing and psychoacoustic phenomena, e.g., octave non-equivalence (tuning curves), combination/sum-and-difference tones, etc... So far, only a model of lower interval limit is implemented.

7. (Not yet implemented) A model of cultural entrainment, so that certain harmonic patterns (e.g., 2-5-1s, common licks/riffs/language/vocabulary) that are known to imply a certain tonality/root/mode can be turned off/on in the model.

8. Except for rhythmic entrainment/beat detection and cultural/vocabulary entrainment models, all of the above features should not be implemented with a black-box machine learning model, but rather a fully explainable and interpretable algorithm that can hopefully give insight into (or at least a description of) the underlying logic (if any) of the harmony derived from the European musical tradition.

If you find any glaring shortcomings in the assumptions, find some part of the algorithm (or its implementation) redundant or incorrect relative to what I claimed it does, if you have a fresh perspective on this problem that can significantly speed up the algorithm, please reach out to me over Discord (@euwbah), Instagram (@euwbah), or email (euwbah [ａ𝐭] ġｍаíḷ [ɗօt] ċοm). I'd very much appreciate and enjoy any discussion on this topic!

I understand that the definition of "tonic", "dissonance", and "root" is very subjective. To help narrow down the scope of my definition, I will identify some musical biases that affects my definition of these words:

• Main influences: Romantic, Impressionist, Black American Music & its derivatives (including, but not exhaustively, bebop, swing, great American songbook, R&B, rap, hip-hop, blues, bluegrass, neosoul, ...), and many other contemporaries (Immanuel Wilkins, Soweto Hinkch, Peter Evans, Nick Jozwiak, Micah Thomas, Joel Ross, Jon Elbaz, Guinga, Mid-Air Theif, Brendan Byrnes, Tennyson, Mei Semones, Becca Stevens, Arthur Verocai, ...)

• Secondary influences (partial understanding and practice): Renaissance polyphony (mainly Frescobaldi), Bach, Spectralism (mainly all the composers of the Plainsound Music Edition), and Algorave/live-coded music.

• Tertiary influences (significant exposure but no formal practice): Early music, Gamelan (especially Balinese), Maqam (especially Taqsim) and Carnatic music.

# Definitions

For the context of this writing,

Consonance is the subjective/cultural perception of pleasantness or stability in musical intervals. It can be influenced by context, culture, and individual listener preferences. When I refer to "consonance", I mean my own subjective interpretation of what I think is consonant, which this algorithm will try its best to model.

Concordance refers to a numerical heuristic score of the "objective/physiological/psychoacoustical complexity" of a musical interval, i.e., we assume that every human has an innate understanding of "complex" vs "simple" harmonies, and the concordance refers to a model of whatever "simple" is. Specifically, in Harmonic Entropy, the presence of timbral fusion, virtual fundamental, beatlessness, and periodicity buzz are signifiers of psychoacoustical concordance. It is assumed that concordance is a contributing factor to consonance, but not the only one. Additionally, concordance is usually measured with the absence of context/time.

Complexity is more specific than concordance. It usually refers algorithm/math used to compute the concordance score, rather than focusing on the perceptual/modelling aspect. In this document I will use concordance and complexity interchangeably, but I will be referring to mainly the algorithmic aspect, not the attempt at modelling a universal human perception of concordance.

Tonicity is a heuristic measure of "root"-perception in chords — a probability distribution of how likely a given pitch (either perceived in the present as external stimuli, in a listener's short-term memory, as a psychoacoustic phenomenon, or implied by cultural entrainment) will be perceived as the "root" or "tonic" of a chord (not necessarily the lowest note). I note that "root" is not rigorously definable, so I give my working definition below. Tonicity can be calculated for a single chord in stasis (played forever), or over time and in context (a chord progression/song).

The tonicity distribution is an attempt to model my own understanding of Western/European harmony, something I have not yet seen in other literature (this is not a softmax of a convolution kernel over pitch classes/FFT spectra, which I find more common in tonality/key-detection machine learning models). Though, my reading is probably outdated, so if you know of any references/people who share similar ideas, please let me know!

Root, as in the root of a chord/tonality is a reference pitch that is defined by musical culture and context. To give a more quantifiable definition, a note is more rooted/tonic if it is more likely for other notes in the context to be heard with respect to that note as a baseline. Conversely, a note is less rooted/tonic if it is more likely for it to be heard in relation to other notes as a baseline. Ideally, this should correspond to the "local key" of a song, i.e. a progression iii-vi-ii-V-I in C major would (probably) have C as the root, but in my experience, the melody and rhythmic phrasing can have a stronger influence on root perception than the harmony itself. For the purpose of this document, since I only need root perception to be accurate within fifths of the actual root (i.e., actual root is C but guessing G or F is also acceptable).

• E.g., suppose there are two notes, C and a major third E above it, and suppose this is the first thing you hear in a song. Will you hear C as scale degree 1 and E as the 3rd of C; or will you hear C as scale degree b6 of the tonic E? Whatever note you hear as "1" is the root (it can also be neither, but we will not consider that possibility for simplicity's sake).

# Inspiration & references

# A History of 'Consonance' and 'Dissonance' — James Tenney

• Cultural and historical aspects of how the perception/definition of what is "consonant" evolves, tracing specifically the lineage of the European classical tradition.

• I also like the YouTube channel Early Music Sources.

# Height or complexity functions (see also: D&D's guide to RTT/Alternative complexities)

A purely mathematical, usually number-theoretic (like lowest common multiple, or sum of prime factors) or norm-based/functional-analytic measure of complexity of a JI ratio . When I first encountered the concept of a height function, some limitations immediately stood out:

• No standard generalization to tempered non-JI intervals in . It is possible to use smoothing techniques (e.g., compute up to a certain limit and then interpolate using splines/linear methods), which I have used to some extent in my own implementation. Harmonic entropy (see below) solves the issue using a spreading function.

• Any (commutative) generalization to chords of more than 2 notes results in negative intervals having the same complexity score as their positive counterparts. E.g., a 4:5:6 major triad and a 10:12:15 minor triad will score the same if we consider simple generalizations like taking the LCM of the 3 integers in the ratio, or just summing up the norms for each of the 3 intervals between the 3 notes. This doesn't reflect my (possibly subjective) experience of 4:5:6 sounding more simple than 10:12:15 (see A note on complexity and duality/negative harmony)

• No direct link to psychoacoustic perception of consonance/dissonance, it measures literally the complexity of a number in itself, which works for the first few primes (modelling periodicity and the critical band Plomp & Levelt's model of concordance), but quickly diverges from human perception for higher primes as human perception cannot perfectly distinguish complex ratios.

# Tonal Consonance and Critical Bandwidth & William Sethares' dissmeasure

See also: Python implementation by endolith.

1. No restriction to JI-only intervals. It considers the pure frequency content of two complex tones (i.e., musical notes with harmonic partials) and measuring the "roughness".
2. Introduces the idea of "roughness" caused by the interaction of beating partials within the critical band: the gray area of when a beating/warbling sensation frequency is too low frequency to be heard as a pitch, but too high frequency to be heard as beats/rhythm, is claimed to be a major contributor to the perception of dissonance in intervals. This is claimed to model the physiology of the basilar membrane, where two frequencies that are too close together will cause physically overlapping excitation patterns on the membrane, causing the indistinguishability of pitches.

I appreciated that this model ties back to the physiology of hearing, which gives a sense of universality (even though in reality much more experiments are needed to confirm this).

It's also great that this is intrinsically a polyadic model (Sethares' algorithm by default considers partials of each individual note) and works really fast (relatively speaking, is fast compared to whatever will be presented here).

However, the final result is effectively an unordered summation of (min-amplitude-weighted) critical band roughness scores over each pair of (sine) frequencies present in the spectrum, meaning that positive and negative chords still gave almost the same output, only disagreeing after a few decimal places due to some modelling of the lower interval limit curve. To me, that small difference is not sufficient to represent the perceived gap in complexity between major and minor.

• E.g.: Dissonance (Consonant Triads) — Aatish Bhatia, clickable demo diagrams showing that both major and minor triads have the same dissonance score.

# Harmonic Entropy

This addressed several limitations of the above methods:

1. By reframing the problem as a probabilistic one, this allows inputs which are non-JI intervals, any real-valued cents interval will do — the ear hears the external stimuli of an interval of cents, but accepts detunings (or mishears) with probability where is a spreading function, a probability distribution (e.g., Gaussian, Laplace, etc...) centred at 0 where is the probability of hearing an interval as cents when the actual stimulus is cents. This spreading distribution is the basis of the rest of the approach.

2. By considering three-note combinations (3HE) instead of dyads, which is done by populating the search space with 2-dimensional points representing the 2 intervals between the 3 notes (as opposed to searching through points on the real line up to some Weil/Tenney height or Farey sequence iteration), this model is able to break the negative-harmony "curse" of using purely dyadic heights. The major 4:5:6 and corresponding "negative", minor 10:12:15, have vastly different entropy/concordance scores.

I initially implemented a generalized -HE algorithm in Ruby for livecoding in Sonic Pi, but found that any quickly becomes intractable for real-time purposes. Every configuration of intervals need to be considered. If I wanted 3-cent resolution (with intermediate values interpolated) for intervals up to a 1 octave span, that still quickly blows up to 25600000000 pre-computed points for only 5-note chords. There were some heuristic optimizations (e.g., lazily populating lattice points in direction of most entropy, assuming the inverse Fourier transform of the Riemann-zeta analytic continuation of HE was smooth/analytic for arbitrary dimensions), which I tried doing in Ruby, then Java, then Rust (though I lost the most of the projects in a corrupted old SSD, sad...). I did to get it working up to 7-HE, but the optimizations deteriorated the results until they were no longer useful.

Things I really liked about HE:

• The probabilistic treatment for modelling the uncertainty of human perception of pitch. This idea of extending to intervals in via probabilistic interpretations/kernel smoothing functions was something I wanted to take further.

• It generalized to multiple intervals naturally, considering not just dyadic relationships but the entire gestalt of the chord by mapping chords to points in a higher-dimensional space.

What I didn't like was how quickly computations blew up when increases, and I was desperately looking for alternatives that preserved these two properties I needed in a complexity function, while not requiring populating a huge search space (even after applying threshold cutoffs/interpolations) for 7/8-note chords.

# Statistical/machine learning methods

These are very strong and computationally efficient methods for root/key detection, but only viable for 12edo, as they are trained/fitted for an existing corpus of music.

Some general ideas:

• Training an ML model on a melspectrogram (log frequency scale spectrogram/FFT)
• Using moving average amplitude of pitch classes over time, then fitting the distribution of the amplitudes of pitch classes to a known key profile via multilinear regression, machine learning, or musician's intuition, and comparing distributions by Kullback-Leibler divergence or -norms (e.g., Local Key Estimation from an Audio Signal Relying on Harmonic and Metrical Structures)

# A note on complexity and duality/negative harmony

It is possible for the negative variant of a chord to have the exact same set of intervals in the same order, take for example, in 12edo, C E G B (Cmaj7), whose negative is C Ab F Db (which then is Dbmaj7). If we ignore the absolute frequency and things like the lower interval limit, then these two chords are indistinguishable up to transposition by a major 7th (or minor 2nd). Also in JI, the negative of 8:10:12:15 is itself (up to transposition). In this case, it would make sense for any complexity algorithm that is invariant over transpositions to assign the same complexity score to both chords.

However, for non-mirror symmetric chords, this property does not hold, e.g. the negative of 4:5:6:7 is 60:70:84:105, which I think should deserve a much higher complexity score. To demonstrate what I mean by "complexity", consider:

Even though all 6 pairwise dyads are the same intervals, there are other psychoacoustic features like timbral fusion/combination tones/virtual fundamental that are much weaker in the latter than in the former, which contributes to the second "negative" chord being more complex.

Put another way, if all four notes were played together (in the context of a song with other instruments playing),

...a chord would be more complex if, for an ear-trained musician/transcriber, it would be:

1. Harder to find all 4 notes in 4:5:6:7 than in 60:70:84:105, i.e., some inner voice may be missed out when attempting to transcribe. (Play the following examples with other songs playing in the background to simulate the experiment).

2. Easier to correctly identify any of the 4 notes in 4:5:6:7 than in 60:70:84:105;

(Not a conclusive demo, just some anecdotal examples that may or may not work for you.)

To me, these two properties signify that 4:5:6:7 should be considered "simpler" than 60:70:84:105. This intuition guides what I want this complexity score to output.

# Algorithm v1

I will describe my algorithm using the narrative steps that I took to arrive at the final version. Though long-winded, I hope that showing what did not work can be as useful as showing what did.

The first version of the algorithm described is implemented in polyadic-v1.rs and uses dyad_lookup-old.rs for dyadic complexity/tonicity lookups.

# Goal 1: Crossing the dyadic-polyadic and micro-macro scale gap

I noticed that complexity scoring metrics either zoom in to the microscopic perspective of the complexity of each dyad and interactions between their partials (e.g., Sethares' dissmeasure, height functions) or zoom out to the macroscopic perspective to compute the score of the entire chord as a whole (e.g., HE or machine learning models).

It didn't make sense to have two fundamentally different models to describe the perception of one thing, and I want to explore the idea of the polyadic complexity/tonicity perception being an emergent property of the microscopic interactions between each pair of notes.

How can we evaluate the perception of the whole chord by only evaluating dyads (2-note chords) each time?

# Goal 2: No AI/ML (besides regressing simple statistical models)

There are models out there large enough to make sense of music. However, the two main problems I have with resorting to ML (machine learning) models are:

1. No explainability. There are models that are a working black-box that spits out sensible harmony, but the point of this exercise/experiment is to find whether there are any principles that influence complexity and root perception that can be made objective enough for a computer to evaluate. Of course, not everything can be explained through computation alone, but I am interested to find out the exact extent of what can and cannot be done. Hence, ML won't work for this purpose.

2. Not enough training data. The goal of this algorithm is to be able to analyze the complexity and tonicities of any arbitrary tunings. The inputs to the algorithm are raw Hz values, and the point is for any arbitrary frequencies from whatever tuning to be supported. The issue with most models that train on MIDI/FFT melspectrograms is the 12-centric quantization of pitch, which makes it impossible to generalize to arbitrary tunings in a way that is true to the "hidden underlying human model" of pitch and complexity perception. Ideally, the algorithm should see 12edo as a special case of the underlying logic, and any other tuning system should have exactly the same underlying logic applied.

# Base case: Dyadic complexity

Suppose we only have two notes. There are already many existing methods that give a satisfactory complexity score for dyads. I will fall back to some variant of Sethares' dissmeasure that considers the first 31 harmonics. Depending on the specific part of the algorithm, I use a version that is better suited for compounding additively or multiplicatively, or normalized (with polynomial regression) so that all octaves (besides unison) have the same dyadic complexity score. All variants look more or less the same:

For the code that generated this plot, see the implementation of DyadLookup in dyad_lookup.rs. The same results are stored in sethares_roughness_31_5_0.95.csv.

# Base case: Dyadic tonicity

The initial tonicity heuristic can be thought of as an attempt to answer this question: Assume nothing has been played yet (no harmonic context) and the listener has no prior expectations. If both notes are played simultaneously for long enough for the listener to initially form an opinion, but short enough so that the initial opinion is not changed, which note will the listener hear as the root/tonic?

For root perception, otonality first comes to mind. First a vague definition: we consider an interval to be otonal (as in overtones) if the top note can be seen as part of the harmonic series built from the bottom note. For various possible reasons, the top note in an otonal dyad is, more often than average, perceived relative to the bottom note. In a context-free vacuum, the bottom note of an otonal dyad is more "rooted", the same way a dyad is utonal (as in undertones) if the top note is more "rooted". (E.g., sing Do-Mi, and sing Mi-Do (or Le-Do) in a different key. Repeat a few times in random keys. Which note feels more like the root?)

E.g., in a perfect fifth, C-G, we think of C as the root and G as the fifth. If we invert the interval across the octave so that G is now below C, then the perfect fourth G-C is utonal, and we still may think of C as the root and G as a fourth below the root (or historically, the C is a dissonant suspension that should resolve to B and/or D, then G is the root).

Notice the issue of duality when it comes to root detection, a V-I progression in C major can also be a I-IV progression in G major. This can be solved by applying a cultural model (e.g., require two unique clausulae, use scale/mode/melody, etc...), but for this algorithm, as long as it answered within fifth of the "culturally correct" answer, I would consider it a success.

Now I say that the above definition of otonality is vague because I notice that otonality can be defined at multiple levels, sorted in decreasing strictness:

1. Strictly part of the harmonic series: only dyads are otonal for , i.e., 3/1 is otonal but 3/2 is not.
2. Part of the harmonic series up to octave displacements while preserving the direction of the interval: any dyad is otonal as long as . E.g., 27/16 is otonal but 5/3 is not. 5/4 is otonal but 4/5 is not (direction flipped).
3. Any dyad whose higher note is higher up the harmonic series than the lower note, up to octave displacement, is otonal: any for any where and is otonal, and are such that are not even. E.g., 7/5 is otonal but 10/7 is not.
4. Any dyad which is close enough to any interval that is otonal/utonal by the above definition is otonal/utonal respectively. E.g., we can consider 14/11, and also 400 cents, otonal because it is close to 5/4 and 81/64. The closer the interval is to other simple JI otonal/utonal intervals, and the simpler (lower height) of those JI intervals, the stronger the pull towards that classification.

These levels of definitions create a continuum of otonality/utonality. @hyperbolekillsme on the Xenharmonic Alliance Discord generated a plot using this python script which evaluates the otonality of intervals (otonality in blue line):

This was done by computing the otonality component for the numerator and denominator of each JI up to a fixed height using multiplicative Euler complexity (as @hyperbolekillsme calls it): e.g., , where and are primes. Then, raw otonality for a JI interval is computed as

For any interval (not necessarily JI), its otonality is computed by finding a set of JI approximations via find_top_approximations, then those approximations are scored/weighted by score = complexity_score * accuracy_score. The otonality is the weighted average otonality(a, b) * score.

This otonality scoring metric is consistent with definition 4. of otonality above.

One easy method to apply otonality to obtain an initial heuristic of tonicity is to sum up, for each pair of notes in a chord, how otonal each pair is. The more otonal a pair is, the more we increase the tonicity score of the lower note in the pair.

However, there are two things that prevented me from taking this approach:

1. The otonality curve itself didn't agree with my intuition for rootedness. For instance, I would think that in a vacuum, a 3/2 perfect fifth C-G, or even more so, a stack of perfect fifths C-G-D-A, would imply more tonicity for C than a single 5/4 major third. This does not work with the otonality curve which ranks 5/4 and 7/4 much more otonal than 3/2.

2. I don't think that otonality can substitute for tonicity/root perception — e.g., the minor tonality is built of the minor third, but 6:5 is a utonal interval (unless we are considering the minor third 7:6 or 19:16)

Rather than otonality, I thought about what other properties I could exploit to highlight the asymmetry between "root" and "non-root" notes. Specifically, I was drawn to the asymmetry of the roughness curve itself.

To show what I mean by "asymmetry", let's suppose that:

1. The duals of negative harmony are equivalent.
2. Octaves are equivalent.

Then, P5 = 3/2 and its dual -P5 = P4 - 1 octave = 2/3 must be equivalent. And because of octave equivalence, 2/3 should be equal to P4 = 4/3.

However, the roughness of 3/2 and 4/3 are not equal. This is the asymmetry that I want to exploit.

From the bias of my musical cultural entrainment, 3/2 reinforces the lower note as root and 4/3 reinforces the higher note as root. Since 3/2 has lower roughness than 4/3, I hypothesize that if an (octave-shifted) inverted interval has higher roughness than the original interval, then the original interval reinforces the lower note as root more strongly than the higher note, and vice versa.

Intuitively: If an interval has lower roughness than its octave-inverted counterpart, then the lower note of the original interval "wants" to be the lower note. Conversely, the inverted configuration having a higher roughness than the original interval could indicate that the inverted configuration is less stable.

Using this idea, I generated a plot for the initial heuristic tonicity score of a dyad in vacuum:

The code that generated this plot can be found in the implementation of TonicityLookup in dyad_lookup.rs, and the results are stored in dyad_tonicity_19_5_0.95.csv.

The higher the tonicity, the more likely the lower note is to be heard as root/tonic.

The blue line is the raw dyad tonicity, computed as

The orange line is the normalized tonicity. Because Sethares' roughness intrinsically decreases as two intervals get further and further apart, there is a drifting bias such that larger intervals always have lower roughness and thus higher tonicity than smaller intervals. To correct for this, each octave is fitted to a degree 5 polynomial and that is subtracted from the raw tonicity to obtain a flatter version, then the result is normalized with mean 0.5 and variance 0.0001. The choice of variance here is arbitrary, but I just needed to make the tonicity scores fit within 0.4-0.6 for numerical stability in the later time/context-sensitive parts of this algorithm.

Finally, the green line (smoothed tonicity) is obtained by applying Gaussian kernel smoothing with 21 bins ( cents) at cents, since humans don't perceive pitch with infinite precision, a lot of the sub-cent jitters are not very meaningful. This smoothed tonicity is used in the rest of the algorithm and referred to as dyadic tonicity.

Generally, I am quite satisfied with the smoothed tonicity (green), except for the fact that 5/3 scores higher tonicity than 5/4 (to my cultural bias, 5/3 in a vacuum should imply a third note 4/3 as tonic instead). At this point in time, I didn't think it would be an issue, so I just moved on.

# First step: Major vs minor triads

Now that I had a way of initially guessing which note is more likely the root in a dyad, I can move on to triads.

The main challenge of dyadic methods is to ensure that the 5-limit JI major triad 4:5:6; and the minor triad 10:12:15 do not have the same complexity score. The minor triad should be more complex.

My thought process:

• In a vacuum, when I hear both 4:5:6 and 10:12:15, I would instinctively hear the lowest note as the root, the top note as a fifth coloring above the root, and the middle note as the note that helps identify the quality of a chord.
• I would judge the notes in the triad relative to the most rooted note, followed by the fifth, then lastly the middle note. E.g., if I hear C-E-G in a vacuum, I wouldn't instinctively think of judging C as the b6 of E unless forced to by some other context.
• Therefore, there must be a way to ascertain how "tonic" each note in the triad is, then use that to weight the importance of each dyad's complexity score.

First, suppose we don't weight by tonicity, and let be the complexity scores of the dyads C-G, C-E, and E-G respectively, where .

Also (assuming complexity is invariant to transposition), will be the complexity of C-G, Eb-G, and C-Eb in the minor triad.

If we only sum up the dyadic complexities, both C-E-G and C-Eb-G will have the same total complexity of .

However, now suppose we have tonicity scores for each note. Let be tonicity scores for the major triad and let be tonicity scores for the minor triad.

If we have a function that takes in dyadic complexity and tonicity scores of notes and and spits out a weighted complexity score, then the total complexity score for the major triad can be

Some choices of :

Let's work out some numbers to see if this works.

For exaple, we can try this simple heuristic to get the tonicity of the notes in the triad:

1. For each dyad, add the dyadic_tonicity to the lower note's raw tonicity score, and add 1 - dyadic_tonicity to the higher note's raw tonicity score.
2. Take the average tonicity score for each note by dividing by the number of dyads it is part of (i.e., divide by where is the number of notes in the chord).
3. Apply softmax on average tonicity scores to get a probability distribution summing to 1.

According to the comutations in Base case: Dyadic complexity and Base case: Dyadic tonicity, we have:

Raw tonicity scores (Major):

• C:
• E:
• G:

Raw tonicity scores (Minor):

• C:
• Eb:
• G:

Softmax tonicity scores (Major):

• C:
• E: 0.3323
• G: 0.3253

Softmax tonicity scores (Minor):

• C: 0.3414
• Eb: 0.3343
• G: 0.3243

Using , we have:

Or using :

Note that the minor triad is only very slightly more complex than the major triad. The major-minor complexity gap can be widened further by decreasing the softmax temperature to give more opinionated tonicity scores, or by tweaking .

This is just an initial heuristic tonicity score, but I will later use a contextual model of tonicity that becomes more opinionated over time, so the small difference between tonicities is not of concern for now.

# Generalizing: Polyadic complexity using interpretation trees

We can trivially extend to 7-note chords from the triad case by repeating the same sum-of-tonicities and softmax normalization to get tonicities of each note in the chord, then summing up the weighted dyadic complexities for each pair of notes.

However, in my experiments, this completely misses the gestalt of the chord. E.g., a common voicing for a C13b9 chord is C Bb for the left hand and E A Db in the right hand. Now, the E A C#/Db forms an A major triad upper structure that easily stands out in the chord (because of its consonance). At the same time, I recognize C Bb E as a dominant 7th fragment.

The interpretation tree above implies that we hear A and C# with respect to E as a "subroot", and we hear Bb and E with respect to C as the root. Of course, this is not the only way to interpret this voicing, so the algorithm should also aggregate over different interpretations later.

To evaluate complexity based on this particular interpretation tree:

1. Compute the initial tonicities of the notes in this set-up, we use the heuristic initial tonicity computation in the first step. Though in practice, notes are only added one at a time, so we can assume that the contextual tonicities is already given except when adding new notes.
2. Perform a DFS (depth-first search) starting from the root node (C3):
   1. Compute the local relative tonicity amongst all children. I.e., assuming the current tonicity scores of the entire chord from context, we take the distribution conditioning on the parent node being the root. See below for various methods for obtaining local tonicity scores.
   2. For each child of the parent:

      1. Compute the complexity of child subtree recursively. If the child is a leaf node, it has complexity 0. This value should be contained in .

      2. Obtain the edge complexity in , which is obtained from a lookup table from the pre-computed dyadic complexities between the parent and the child. This complexity should model perceived roughness, thus the same interval at different octaves should generally have lower complexity the further apart they are. However, the raw additive Sethares' roughness is normalized to have a peak roughness that halves every octave. Instead, I want to aim for peak roughness to be at the -th octave, so we simply multiply the raw additive roughness by for octaves.

      3. The child's complexity (in the range 0-1) is computed as:

         where is the dyadic complexity between parent and child and is the subtree complexity of the child subtree.

         Link to desmos

         This formula was chosen so that:

         • : neutral complexity of both edge and subtree should return neutral 0.5.
         • : average complexity 0.5 should be preserved in recursive steps.
         • If , : edge complexity should have more effect than subtree complexity on overall complexity. This ensures more intuitive root choices will score lower complexity. If this inequality was flipped, optimizing for low-complexity roots will optimize for interpretation trees where the root is the largest "dissonance contributor" — in the sense that if the root is removed, the remaining notes (which are siblings/descendants of the children) will be the most consonant.
         • and : bounding min/max cases should give min/max values
         • is bounded with the same bounds as inputs.

         Question

         Is there a better way to combine edge complexity and subtree complexity, in a way that is explainable by human perception, rather than just using intuitive mathematical properties?

         This is not the final edge complexity vs. subtree complexity balancing method, a modification will be made later.

   3. Then, the total complexity of the parent's subtree is computed as the weighted sum of all its children's complexities, weighted by local tonicity of each child.
3. The final complexity, fixing this interpretation tree of the voicing, is obtained when the root node (C3) is reached.

I have considered different options for evaluating the local tonicity distribution, conditioning on the parent note as root:

1. Normalize tonicity scores of child nodes to 1. This is computationally efficient, but the issue would be that if a child node has very low tonicity, the complexity of the child's subtree (if any) will be discounted.

   • Problem: If we optimize root choice and tree structure for low complexities at the subtree level there would be a feedback loop where low-tonicity subtree roots are preferred, which doesn't make sense since the main point of the model is for parents to have higher tonicity than their children.
     • Solution 1: decouple tonicity calculation from subtree complexity computation, then this issue is avoided.
       • E.g., we can compute the target tonicity by comparing the distributions of complexity scores obtained by fixing each note as root. E.g., increase the tonicity of a note if the minimum complexity score when that note is root is lower than the rest, and scale the increase by the confidence (e.g., we can measure confidence as the entropy of complexity score distributions fixing each note as root, or as the inverse of the variance of complexity scores)
     • Solution 2: Intuitively, any node with low (global & local) tonicity should not have many children, as the point of tonicity is to model how likely a note is to be heard as the baseline reference for other notes. To regularize this, we can penalize configurations where low-tonicity parents have many children which evades the complexity contributed by edges formed between the parent and their children. Alternatively, we can penalize when children have higher tonicity than their parents.

2. Instead of computing local tonicity scores using only tonicities of direct children, the relative tonicity of each child will be the sum of tonicities of the child itself and all its descendants (or some kind of aggregation function that increases if the tonicity of all elements in the subtree increases, e.g. softmax of values where is the sum of tonicities for the -th child's subtree).

   • This will be the method I am currently developing for this article. This method comes with its fair share of problems which we will go through later.

3. Compute local tonicity of each child using the reciprocal of the child's subtree complexity. The intuition here is that if a subtree is a concordant/stable upper structure, it is more likely to be heard as a reference point over other substructures.

   • Problem 1: If the child is a leaf node, there is no complexity scoring for just a single note. When comparing the tonicities of child leaf nodes to each other, we can use the global tonicity context and normalize their per-note global tonicities to get a local tonicity distribution. However, how do we compare a leaf node to a non-leaf node, such as in the C13b9 example where Bb is a child leaf node of C but E-A-C# is a child subtree of C?
     • Solution: mix both local tonicity from global context and reciprocal subtree complexity. For a non-leaf note node , let be its global tonicity and be the subtree complexity, then we can define the mixed tonicity as where is a parameter controlling how much bonus we give to low-complexity subtrees, and is the expected subtree complexity score of all subtrees with the same number of notes of the subtree at .
     • This solution introduces more parameters and the computation of for each subtree size is memoizable but not trivial, so I have not experimented with subtracting yet.
   • Problem 2: The subtree complexity is computed as the sum of child subtree complexities weighted by the child subtree's tonicity. However, now the child subtree's tonicity is simply the reciprocal of its complexity, so now multiplying the local tonicity and complexity simply cancels out to a constant!
     • Solution: from the distribution created from the mixed tonicities as a solution to Problem 1 above, we combine the reciprocal subtree complexity with an edge-specific weight in a non-linear way (e.g., multiplying the dyadic complexity and the reciprocal subtree complexity).

This completes the first part — now we can evaluate the complexity given a single subjective interpretation of how a polyadic chord voicing is broken down.

# Generalizing: Polyadic tonicity evolving with time

The next challenge: How to aggregate over different interpretations of the same voicing to form a single complexity score?

1. Going back to the C13b9 voicing example, note that there are many other ways of interpreting this particular voicing, and not necessarily with C as the root. Though, certain ways of interpreting will feel more intuitive than others. How do we model this?

   • E.g., we could also interpret the voicing as an Bbdim (Bb Db E) over an Am dyad (A C), but this does not feel intuitive to me.

   In hindsight

   At this current stage, I assumed that explicitly scoring the likelihoods of the tree interpretation was not necessary since the complexity scores aggregated from each root of the interpretation tree are fed back in to the algorithm to update the tonicity scores, which I assumed meant that tonicities should converge to a value that depends on the likelihoods of the trees with that root.

   In the next step, I will show why this is not sufficient.

2. How do we update the perceived tonicity of notes according to the aggregated polyadic complexity calculations (instead of relying on the heuristic polyadic tonicity in First step: Major vs minor triads)?

To give another example, compare this voicing of 13b9 to other voicings with octaves shifted around:

I think it's fair to say all these voicings of C13b9 have differing perceived complexities, and there is a reason why some are more commonly played than others.

When there are more notes, certain subsets or upper structures stand out as strong units on their own, and usually depends on the exact voicing being played. This may very well just be a cultural artifact of how contemporary western harmony is organized, but I wanted something in the model that can capture this idea.

This is what a full single-tick update of the algorithm should look like, assuming no new notes were played, no old notes were deleted from the context, and the tonicity values of all notes in the chord are known and correct:

1. Iterate over different interpretations of the same voicing (finding different substructures)

2. Identify which choices of tree organization and root choices are more likely than others

   • The likeliness of root choices are used to update the global tonicity values.

3. Aggregate substructures into individual units, where each substructure has a "structure tonicity" and "structure complexity" score, weighted amongst other sibling leaf nodes/substructures at that level.

Step 3 is already handled by the single-interpretation case.

For step 2, we have to devise a measure of likelihood. For now, we focus on the simpler problem of measuring the likelihood of root choices, which directly corresponds to the tonicity scores. (In the next section, we find that we still have to model the intuitiveness/probability of perceiving each tree, see The big problem: Duality is still hiding)

Working through an example for triads, we have to think about 3 tree configurations and 3 different roots each:

Using the interpretation tree complexity evaluation from Generalizing: Polyadic tonicity with substructures, we can compute the complexity score for each of the 9 configurations above. An example of how this can be done using depth-first search (DFS) in the triadic case is provided in triad_sts_computation_example.py. Running test_one_iteration() gives the result:

C
|---E
|---G
Complexity: 0.63550
___________________________
C
|---E
    |---G
Complexity: 0.75960
___________________________
C
|---G
    |---E
Complexity: 0.55761
___________________________
Arithmetic mean complexity for root C: 0.65090
Harmonic mean complexity for root C: 0.64056
Inverse exp. weighted mean complexity for root C: 0.64407
Exp. weighted mean complexity for root C: 0.65789



E
|---C
|---G
Complexity: 0.76150
___________________________
E
|---C
    |---G
Complexity: 0.71398
___________________________
E
|---G
    |---C
Complexity: 0.71839
___________________________
Arithmetic mean complexity for root E: 0.73129
Harmonic mean complexity for root E: 0.73067
Inverse exp. weighted mean complexity for root E: 0.73083
Exp. weighted mean complexity for root E: 0.73175



G
|---C
|---E
Complexity: 0.63800
___________________________
G
|---C
    |---E
Complexity: 0.55702
___________________________
G
|---E
    |---C
Complexity: 0.76340
___________________________
Arithmetic mean complexity for root G: 0.65281
Harmonic mean complexity for root G: 0.64204
Inverse exp. weighted mean complexity for root G: 0.64569
Exp. weighted mean complexity for root G: 0.66008

Now this test assumes an initial uniform tonicity, where all C, E, and G have the same tonicity probability of .

Even then, we notice that the average complexity scores per choice of root are not equal. The lowest average complexity is obtained when C is the root, followed by G, then E, which I find is an acceptable answer to the question: "With no prior musical context, if you hear a simple C-E-G triad in root position as the first stimulus of a song, and given that the key of the song is either C, E, or G major or minor, which key do you think the song will be in?". I find the slight ambiguity between C and G acceptable because of the duality problem mentioned in Base case: Dyadic tonicity.

The results of this test hints that the per-root aggregated polyadic complexity scores can be used directly to nudge the tonicity context towards favouring roots with lower complexity scores.

Four different aggregation methods were tested: Where is the complexity score for the -th interpretation tree of the same root,

1. Arithmetic mean:
   • Complexities are weighted equally.
2. Harmonic mean:
   • Complexities with smaller values are weighted more heavily.
3. Inverse exponential weighted mean:
   • Complexities with smaller values are weighted more heavily, but less aggressively than harmonic mean.
4. Exponential weighted mean:
   • Complexities with smaller values are weighted less heavily.

Initially, I was inclined to use either the harmonic or inverse exp weighted mean, because intuitively I thought that if a particular root interpretation allows for a few interpretation trees to have significantly lower complexity than the rest, then the listener should update their subjective model of tonicity to favor hearing the current music with respect to whichever root that allows for an interpretation that obtains the least complexity.

However, in terms of raw numbers, I wanted to widen the gap between the probability of C being the root and the probability of G being the root. Hence, I at this point I considered using the exponentially weighted mean to aggregate per-root complexities. Musically speaking, this means that if any root choice allows for the listener to construct a high-complexity interpretation, that high-complexity interpretation would affect the overall complexity score of that root choice more significantly than a low-complexity interpretation would, e.g., multiple low-complexity interpretations are needed to "balance out" a single high-complexity interpretation.

Now that the per-root aggregated complexities are computed, we can update the global tonicity scores as follows:

1. Compute target tonicities as the softmax of negative per-root complexities (adding 1 for numerical stability). Where is the target tonicity of the -th note, is the aggregated complexity scores of interpretation trees with note as the root, and is the softmax temperature (which is lowered from the baseline of 1 to make the tonicity distribution more opinionated):

2. Perform smooth update of global tonicity context towards the target tonicities. is the current global tonicity of note , and is the smoothing factor (higher = slower update), we compute the next iteration's global tonicity using

   then we normalize with
   such that is a tonicity distribution.

The updated tonicity scores are now fed back to the complexity computation for the next tick.

This process continues indefinitely until the music stops.

An example of this computation is provided in triad_sts_computation_example.py in the function test_tonicity_update(). Running it to update the tonicities of a simple C-E-G triad with parameters iterations=30, smoothing=0.7, temperature=0.5 and assuming an initial uniform tonicity of gives the result:

Iteration 1      target: ['0.34986', '0.30181', '0.34833'] ctx: ['0.33829', '0.32388', '0.33783']
Iteration 2      target: ['0.34994', '0.30163', '0.34843'] ctx: ['0.34179', '0.31720', '0.34101']
Iteration 3      target: ['0.35000', '0.30150', '0.34850'] ctx: ['0.34425', '0.31249', '0.34326']
Iteration 4      target: ['0.35004', '0.30141', '0.34854'] ctx: ['0.34599', '0.30917', '0.34484']
Iteration 5      target: ['0.35007', '0.30135', '0.34858'] ctx: ['0.34721', '0.30682', '0.34596']
Iteration 6      target: ['0.35009', '0.30130', '0.34860'] ctx: ['0.34808', '0.30517', '0.34676']
Iteration 7      target: ['0.35011', '0.30127', '0.34862'] ctx: ['0.34869', '0.30400', '0.34732']
Iteration 8      target: ['0.35012', '0.30125', '0.34863'] ctx: ['0.34912', '0.30317', '0.34771']
Iteration 9      target: ['0.35013', '0.30123', '0.34864'] ctx: ['0.34942', '0.30259', '0.34799']
Iteration 10     target: ['0.35013', '0.30122', '0.34865'] ctx: ['0.34963', '0.30218', '0.34819']
Iteration 11     target: ['0.35014', '0.30121', '0.34865'] ctx: ['0.34979', '0.30189', '0.34833']
Iteration 12     target: ['0.35014', '0.30120', '0.34866'] ctx: ['0.34989', '0.30168', '0.34843']
Iteration 13     target: ['0.35014', '0.30120', '0.34866'] ctx: ['0.34997', '0.30154', '0.34850']
Iteration 14     target: ['0.35014', '0.30120', '0.34866'] ctx: ['0.35002', '0.30144', '0.34854']
Iteration 15     target: ['0.35014', '0.30120', '0.34866'] ctx: ['0.35006', '0.30136', '0.34858']
Iteration 16     target: ['0.35014', '0.30119', '0.34866'] ctx: ['0.35008', '0.30131', '0.34860']
Iteration 17     target: ['0.35014', '0.30119', '0.34866'] ctx: ['0.35010', '0.30128', '0.34862']
Iteration 18     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35011', '0.30125', '0.34863']
Iteration 19     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35012', '0.30123', '0.34864']
Iteration 20     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35013', '0.30122', '0.34865']
Iteration 21     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35013', '0.30121', '0.34865']
Iteration 22     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30121', '0.34866']
Iteration 23     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30120', '0.34866']
Iteration 24     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30120', '0.34866']
Iteration 25     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30120', '0.34866']
Iteration 26     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30119', '0.34866']
Iteration 27     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30119', '0.34866']
Iteration 28     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35014', '0.30119', '0.34866']
Iteration 29     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35015', '0.30119', '0.34866']
Iteration 30     target: ['0.35015', '0.30119', '0.34866'] ctx: ['0.35015', '0.30119', '0.34866']

And we can see that the tonicity scores being fed back to the complexity algorithm converges to tonicities:

• C: 0.35015
• E: 0.30119
• G: 0.34866

The variance/opinionatedness/confidence of tonicity scores can be increased by decreasing the temperature parameter further, and the rate of convergence can be adjusted by changing the smoothing parameter. Ideally, we want to run this at 60 fps — in practice, the smoothing parameter is scaled by delta time of each frame to ensure a constant rate of update.

# "Final" tonicity calculation

Now it is possible to evaluate the complexity of each interpretation tree, and update tonicities of each note based on the per-root aggregated complexity scores. The final dissonance score of the entire voicing is computed as:

where is the aggregated complexity score (as per the above section) of all interpretation trees rooted at note , and is the tonicity of note from the existing tonicity context.

After the harmonic analysis algorithm is finalized, the plan is to add rhythmic beat entrainment and harmonic rhythm entrainment to the model, such that the tonicity model becomes more sensitive (lower smoothing and lower temperature) when it is near a strong downbeat or an expected harmonic/chord change based on the rhythmic entrainment model.

# Algorithm v2

# The big problem: Duality is still hiding

It seems like the above methodology is complete and achieves the initial goals of non-duality, polyadic gestalt, and building a model of complexity and root perception as an emergent property of dyadic relationships in a tree structure.

However, after fully implementing the above algorithm in Rust, certain tests reveal huge red flags. In the following tests, I have initialized the tonicity context using the dyadic-sum-softmax heuristic from First step: Major vs minor triads, then ran one iteration of the tree-based update algorithm (at 1s delta time). Each chord voicing is evaluated in a vacuum with the context reset to the initial heuristic value.

============  Graph diss: P4  =====================

Voicing:
       0.00c
     500.00c
Diss: 0.4430
2.5s: [
    Dissonance {
        dissonance: 0.44301372910812864,
        tonicity_target: [
            0.5,
            0.5,
        ],
        tonicity_context: [
            0.49898347292521467,
            0.5010165270747854,
        ],
    },
]
Lowest 3 complexity trees for root    0.00c:
 -> comp 0.4430:
0
└── 1

Lowest 3 complexity trees for root  500.00c:
 -> comp 0.4430:
1
└── 0

============  Graph diss: P5  =====================

Voicing:
       0.00c
     700.00c
Diss: 0.3195
2.5s: [
    Dissonance {
        dissonance: 0.31948576531662115,
        tonicity_target: [
            0.5,
            0.5,
        ],
        tonicity_context: [
            0.5010165261177496,
            0.4989834738822504,
        ],
    },
]
Lowest 3 complexity trees for root    0.00c:
 -> comp 0.3195:
0
└── 1

Lowest 3 complexity trees for root  700.00c:
 -> comp 0.3195:
1
└── 0

In the above dyadic scenarios of the perfect fourth and fifth, the glaring problem is that the tonicity_target is perfectly uniform, i.e., 50% chance that either the lower or higher note is the root.

This was not what the initial heuristic model predicted (we can see the smoothed tonicity_context), and this does not agree with my intuition that I wanted to model in this algorithm.

It's easy to see why: in this case, there are only two possible interpretation trees, one for each root. Both trees have the same complexity score since it is one parent and one child (so the local tonicity of the child is always 100%), so the only factor that determines the final complexity of each interpretation tree is the edge dyadic complexity between the two notes.

This edge dyadic complexity is fully symmetric/dual, so both choices of root note always give the same complexity score (0.4430 for P4 and 0.3195 for P5).

Initially, I thought that the easy fix was to implement a special case for dyads, after all there was the dyadic tonicity model that I was already happy with in Base case: dyadic complexity.

However, the following triadic test cases revealed deeper issues:

============  Graph diss: C maj  =====================

Voicing:
       0.00c
     400.00c
     700.00c
Diss: 0.4885
2.5s: [
    Dissonance {
        dissonance: 0.48849305288177675,
        tonicity_target: [
            0.34949708791695255,
            0.30976802861030506,
            0.3407348834727424,
        ],
        tonicity_context: [
            0.3492957038643127,
            0.31039683349660957,
            0.3403074626390778,
        ],
    },
]
Lowest 3 complexity trees for root    0.00c:
 -> comp 0.3954:
0
└── 2
    └── 1

 -> comp 0.4403:
0
├── 1
└── 2

 -> comp 0.5691:
0
└── 1
    └── 2

Lowest 3 complexity trees for root  400.00c:
 -> comp 0.4932:
1
└── 0
    └── 2

 -> comp 0.5249:
1
└── 2
    └── 0

 -> comp 0.5804:
1
├── 0
└── 2

Lowest 3 complexity trees for root  700.00c:
 -> comp 0.3870:
2
└── 0
    └── 1

 -> comp 0.4580:
2
├── 0
└── 1

 -> comp 0.5924:
2
└── 1
    └── 0

============  Graph diss: C min  =====================

Voicing:
       0.00c
     300.00c
     700.00c
Diss: 0.4890
2.5s: [
    Dissonance {
        dissonance: 0.4889643528743752,
        tonicity_target: [
            0.3401420078466398,
            0.30971626450672535,
            0.3501417276466349,
        ],
        tonicity_context: [
            0.34019583693360894,
            0.3103994755135822,
            0.34940468755280896,
        ],
    },
]

Three glaring problems:

1. In the C minor triad (C-Eb-G) case, it is saying that the top note (G) should have the highest probability of being root. Clearly this does not model the average musical intuition.

   • I have left out the lowest complexity trees for C minor for brevity, but G being the most probable root implies that the trees rooted at G are deemed simpler than trees rooted at C by the current model. The reason for this is found in the second problem:

2. Looking closely at the lowest complexity interpretation trees for C major with root C, and comparing that of root G, we see that the tree 0->2->1 (interpreted as: C is root, G is P5 of C, E is m3 below G) has complexity 0.3954, and the tree 2->0->1 (interpreted as: G is root, C is P5 below G, E is M3 above C) has a lower complexity of 0.3870. Even worse still, both of these interpretation trees score with a lower complexity score than what I think would be the most intuitive interpretation: C->(E, G), i.e., C is the root, E is the M3 of the root, and G is the P5 of the root.

   • The discrepancy between C->G->E and G->C->E happens because our current algorithm only considers local tonicities between siblings, but these "trees" are simply paths/linked-lists where each parent only has one child, so there are no siblings to compare local tonicities with, resulting in tonicity scores being completely ignored in the tree's complexity calculation. The only time a note's tonicity is being used is in the final calculation of overall dissonance where the per-root aggregate complexity is weighted by the note's global tonicity.

     Notice how the algorithm is not modelling the fact that C->G is a much more sane interpretation than G->C for a basic 1-3-5 triad.

   • The single-child depth-2 paths C->G->E and G->C->E both score lower in complexity than the intuitive C->(E, G) interpretation because:

     1. The current function that aggregates a note's subtree complexity with its dyadic edge complexity between its parent and the note itself (see step 2c of tree complexity computation) penalizes edge complexity more than subtree complexity, i.e., if , then : edge complexity should have more effect than subtree complexity on overall complexity
     2. There is no penalty for deep/nested interpretations, when intuitively, deeply nested interpretations (note A is seen with respect to note B seen with respect to note C, etc...) are generally more complex than flat interpretations (notes A, B, C are seen with respect to some root directly), unless there is a good reason to use a nested interpretation (e.g., the A maj triad upper structure in the C13b9 voicing discussed earlier).

3. The dissonance score of the major and minor triads are still nearly identical! One of the core criteria of this algorithm is to break the curse of harmonic duality in dyadic-based models, but this issue is still here.

Finally, one last test case that really makes no sense:

============  Graph diss: C maj7  =====================

Voicing:
       0.00c
     400.00c
     700.00c
    1100.00c
Diss: 0.4638
2.5s: [
    Dissonance {
        dissonance: 0.4638236840323294,
        tonicity_target: [
            0.25847518240587736,
            0.24130394436259714,
            0.24142014916913435,
            0.2588007240623913,
        ],
        tonicity_context: [
            0.2583230183357106,
            0.24164858173458328,
            0.2416042487186883,
            0.2584241512110178,
        ],
    },
]

This one just says that the most probable root in a Cmaj7 voiced as a plain old C-E-G-B is B. Clearly something isn't right.

# Analyzing information flow of the flawed model: Dyadic tonicity is vanishing

The solution to these issues becomes clear when we map out exactly which variables are allowed to affect which other variables. The flowchart below shows the flow of information in the above flawed model:

The main issue is that the dyadic tonicity is not part of the recursion, it is only used to initialize the model. Recall that the tonicity score between two notes is the only source of asymmetry/non-duality in this model — this asymmetry was the entire motivation for introducing the concept of tonicity in Base case: Dyadic tonicity.

Since dyadic tonicity is not part of the recursion and every other part of this model is symmetric (as in, harmonic duality of major/minor), over time the model will always converge to a dualistic model.

To solve this, we have to find a way to incorporate dyadic/edge tonicities in the recursion.

My first instinct is to look at the weak spots of this model to find where dyadic tonicities can be added:

1. The function for combining edge and subtree complexities.

   • This function was heuristically made to combine the edge and subtree complexity scores, but the arbitrary precedence of edge complexity over subtree complexity just to get better triadic tonicity results was suspicious.

2. The subtree tonicity score is solely computed as the sum of global tonicities of all nodes in the subtree, there is not enough interaction between the subtree tonicity and the rest of the model.

   • The analogue of subtree tonicity is the subtree complexity. Notice that subtree complexity is directly affected by three information sources at each subtree: edge complexity, subtree complexities of its children, and local tonicity scores obtained from subtree tonicity.
   • Compare that to the active information sources that affect subtree tonicity: subtree tonicities of its children and global tonicity context. There is a very indirect recursion from the aggregation of tree complexities that affects the global tonicity context which in turn affects the subtree tonicities, but this only happens once every update step, rather than at every node traversal.

3. The meaning of "tonicity" is not consistently interpreted. For individual notes, tonicity is the probability of that note being interpreted as the root, but subtree tonicity is defined as the sum of global tonicities, but it doesn't make sense for an entire subtree to "be a root". When interpreted mathematically, the sum of global tonicities of notes in a subtree is equal to the probability of the root being contained in that subtree. The current flawed algorithm uses "the probability of the root being contained in a subtree" to weight the complexity contribution of that subtree .

# Decoupling likelihood, tonicity, and complexity

To glean some remedies from the first weak spot, recall that the rationale for weighting edge complexity more than subtree complexity was to discourage preferring interpretation trees whose roots are "dissonant offenders", i.e., notes that have high dyadic complexity with respect to many other notes in the voicing. E.g., consider the C-E-G triad interpreted two ways:

The first interpretation (left) will have the same complexity score whether we penalize edges or subtrees more, since both E and G are leaf nodes.

The second interpretation (right), however, will have a much higher complexity score if we give edge complexity precedence, since the first edge is E->G (m3) which is the highest-complexity dyad out of the three options (P5, M3, m3). Naively, I thought this was good since I did not want E to be considered a root. However, this also means that even though a subtree contains many more edges than a single edge, that single edge's complexity can override the entire subtree's complexity, which is not desirable.

Instead, if we give subtree complexity more precedence, we encounter a new problem: the second interpretation will have a lower complexity score, even lower than the first interpretation, since the subtree G->C is the lowest complexity dyad (P5) out of all three options, which cancels out the high edge complexity of E->G.

Neither of these options help to directly increase the likelihood of the first (left) interpretation without collaterally affecting other interpretations.

The underlying problem is that this model directly conflates complexity, root tonicity, and likelihood of trees. If we didn't have this direct relationship, there would be no need to fine-tune the complexity scores of each tree structure to yield the "correct" choice of root.

The fix is to decouple likelihood from complexity. Let's forget the current model, and now consider how to recursively compute both the likelihood and complexity of trees. Once we know how to compute the likelihood of a tree, then the tonicity of a note is simply equal to the sum of the likelihood of trees with that note as its root.

Now we have to rework the meaning of subtrees, edges, nodes, tonicity, and complexity in the interpretation tree to include the measure of likelihood. The table below summarizes which terminology applies to which part of the tree:

Listing ideal behaviours of each component in the new model:

1. A subtree with high complexity, regardless of whether the edge that connects it to the parent is simple or complex, should both:
   1. Contribute more to overall complexity of the tree (compared to simple subtrees), and
   2. Decrease the overall likelihood of this interpretation tree being a model of how the listener perceives the chord, since the main purpose of having the subtree is to identify consonant substructures.
   3. Vice versa for low-complexity subtrees.
2. An edge with high dyadic complexity, regardless of the subtree (if any) it connects to, should:
   1. Contribute more to overall complexity of the subtree (compared to simple edges), but
   2. It should not directly affect the overall likelihood of the listener perceiving this interpretation tree, as some chords/subtrees only have dissonant interpretations. Once the complexity of the entire subtree is computed, that complexity of the whole subtree/tree can be used to determine the likelihood of the subtree/root.
   3. Vice versa for low-complexity edges.
3. An edge with high dyadic tonicity alignment, i.e., the dyadic tonicity of parent-child shows higher tonicity for the parent, should:
   1. Increase the overall likelihood of this interpretation tree.
   2. Should not affect complexity.
   3. Vice versa for low dyadic tonicity alignment.
4. A node with high global tonicity alignment, i.e., parent with high global tonicity, or child with low global tonicity, should:
   1. Increase the overall likelihood of this interpretation tree.
   2. Vice versa for low global tonicity parents or high global tonicity children.
5. To obtain the complexity contribution from each child node, we consider a function of:
   1. The subtree complexity at that child (from 1. and 5.)
   2. The dyadic complexity of the edge connecting the parent to the child (from 2.)
   3. The subtree tonicity (the total probability of the subtree containing the root, which for now we conflate with the "strength" or "tonal gravity" of the subtree)
   4. The number of nodes in the subtree (to fairly compare between subtrees of different sizes and leaf children)
   5. We do not include the likelihood of the subtree (to discount the complexity contribution of unlikely subtrees), as that information is already propagated down to the root which weights the final dissonance contribution of the full tree.
   6. Unlike in the flawed model, the function which weights edge complexity over subtree complexity is now parametrized to allow control over whether and how much edge complexity should affect overall subtree complexity compared to subtree complexity. Ideally, edge complexity should still have more effect than subtree complexity, since edges close to the root of the interpretation tree are the first to be perceived (the model assumes that the pre-order traversal of the interpretation tree represents the order of note perception).
6. To obtain the likelihood contribution from each child node, we consider a function of:
   1. The likelihood of the subtree at that child (from 1. and 6.)
   2. The dyadic tonicity alignment of the edge connecting the parent to the child (from 3.)
   3. The global tonicity of the parent and child nodes. (from 4.)

This automatically solves the second weak spot: the subtree tonicity now has a clear purpose, and cross-interaction between dyadic tonicity and complexity is established through the likelihood computation.

The third weak spot is also resolved since tonicity is now consistently defined as the probability of being root for individual notes, and the probability of containing the root for subtrees. Tonicity is no longer conflated with likelihood, which is a separate metric.

# Dyadic complexity model update: Widening major vs. minor complexity score gap

In the first version, I used the dyadic roughness/complexity scoring system as per Base case: Dyadic complexity:

One of the key goals was to ensure major and minor triads did not have the same dissonance score, which is a common problem amongst dyad-based complexity/dissonance models. I've ascertained that minor and major triads are treated differently by this v2 model, but the gap between minor and major triad complexity in various inversions/voicings was abysmally small ( ), relative to the gap between the perfect fifth and fourth dyads. Initially I thought that there was an issue with the polyadic algorithm itself, but I later realized that the bottleneck was not the polyadic algorithm but the original dyadic base case we started with.

The first issue I realized was that, since I was performing most of the tests in 12edo, a lot of the dyadic roughness and tonicity values were not as distinct as I wanted, as the 12edo intervals stray far from the regions of consonance in the roughness curve in the image above, especially for the 5-limit major/minor thirds.

The second issue was that the smoothing model that I used (computing Sethares' roughness every 0.05 cents and averaging each 1-cent bucket) had two flaws:

1. It did not model how human interval perception has larger tolerances for simpler intervals
2. The uniform average of each 1-cent bucket did not do sufficient smoothing and the roughness curve had very tight tolerances for what it considered "close" to a consonant JI interval.

To solve these issues, I took a -cent sliding window weighted average of the half-cent resolution roughness curves (which were uniformly averaged from the 1/20th-cent resolution roughness curves). Each 1/2 cent in the sliding window was weighted by a Gaussian kernel with , multiplied by to increase the weight of lower-roughness intervals.

I also increased the smoothing on the dyadic tonicity curve so that dyadic tonicities more closely resemble human perception for tolerance of detunings around consonant JI intervals.

The resulting dyadic roughness and tonicity curves are as follows:

The final dyadic roughness calculation for intervals uses the blue line (scaled add) in Roughness vs Cents, while reducing the per-octave dropoff to 0.86 per octave instead of 0.5 as pictured in the graph (as in, every octave the roughness halves). Additionally, lower interval limit was emulated on a per-dyad basis in lower_interval_limit_penalty() in polyadic.rs.

The normalized dyadic roughness (green) in Roughness vs Cents is used for computing the raw dyad tonicity (blue line in Tonicity vs Cents). The final dyadic tonicity scores are retrieved from the smoothed tonicity (green line in Tonicity vs Cents).

The full 9 octaves of values can be found in sethares_roughness_31_5_0.95.csv and dyad_tonicity_19_5_0.95.csv.

# Information flow of revised model

# Computing values in revised model

The adjustable parameters below can be found and modified near the top of polyadic.rs, which implements this algorithm.

# Computing tree/subtree complexity

Tree/subtree complexity of the subtree at node is implemented in compute_child_subtree_complexity_contribution() and compute_child_leaf_complexity_contribution(). It is recursively computed in terms of the -th child's subtree weight ,dyadic complexity between its child and itself , and child's subtree complexity using:

The function is updated from the prior model to include a parameter EDGE_OVER_SUBTREE_COMPLEXITY_BIAS which controls how much more edge complexity should affect overall subtree complexity compared to subtree complexity. A value of means edge complexity has more precedence, i.e., comp(1, 0) = 0.75 but comp(0, 1) = 0.25, and means subtree complexity has more precedence. Since we model interpretation trees as the structure of how the listener perceives the chord from the root to leaves (in pre-order traversal where each node's children are sorted in ascending pitch order), edges closer to the root should have more effect on overall complexity, so EDGE_OVER_SUBTREE_COMPLEXITY_BIAS is initially set to 0.65:

This desmos graph visualizes how behaves for different values of .

At each subtree, the complexity is a value between which represents the minimum and maximum possible complexity attainable. The final complexity score of the tree is the model's dissonance score. See also: the distribution of dissonance scores for -note chords.

# Computing subtree weight

Subtree weight of the -th child of is computed in compute_subtree_weights() as such: For each child node, the child's subtree tonicity (sum of its nodes' tonicities) is computed. Each child has a local tonicity weight according to its subtree tonicity which is given by the softmax over all children's subtree tonicities, with temperature (inverse opinionatedness) parameter initially set to LOCAL_TONICITY_TEMP = 0.7. Let denote the softmax local tonicities of the -th child, such that .

The subtree weight now balances the softmax local tonicities with the proportion of nodes in the subtree at the -th child, so that large subtrees with the same subtree tonicity as small subtrees will still have higher weight. The importance of local tonicity vs. subtree size is controlled by TONICITY_BIAS initially set to 0.6: