CMF From Cone Fundamentals

Building a Color Space From Cone Fundamentals

Introduction

This tutorial shows you how to build a color space from the cone fundamentals by deriving the Color Matching Functions (CMFs) of a color space. We explore the linear transformation between the cone fundamentals and the CMFs, which allows us to understand why human vision is (mostly) trichromatic. We will get to appreciate the critical but subtle difference between absolute quantity of light (e.g., power) and relative units used in a particular color space, and the importance of the reference white in a color space. Once we build a color space, there are many cool insights we can get.

The tutorial is interactive, which means you get to build the CMFs step by step! For instance, you get to pick the primaries and the reference white, and can even build your own exotic LMS cone fundamentals to drive the rest of the tutorial. All the plots are interactive, so you have lots of freedom to explore things yourself. You will see some texts that are initially ${\boxed{??}}$: they are numbers that depend on your actions, e.g., the selection of the primaries, and will be dynamically updated as you go.

Caveats: this is not how the CIE 1931 RGB color space/CMFs are derived. We describe that process in this post. Among many tricky details, CIE 1931 RGB CMFs did not make use of the cone fundamentals; they started from earlier color matching experiments (by William David Wright and John Guild) that provided the chromaticity coordinates, adjusted the primaries and the reference white, and used the CIE 1924 photopic lumininous efficiency function to convert the chromaticity coordinates to the absolute primary units.

Modern-day research usually experimentally obtain both the cone fundamentals and the CMFs and calibrate the results with each other to derive a best-fit linear transformation, which is then applied to the CMFs to derive a "clean" set of cone fundamentals. In fact, the cone fundamentals that we will use in this tutorial are derived that way. See, for instance, Stockman et al. (1993) and Stockman and Sharpe (2000). By the way, the CIE 1931 XYZ CMFs are known to be erroneous. Fundamentally, the CIE 1931 XYZ CMFs were derived from CIE 1931 RGB CMFs, which were dependent on the CIE 1924 luminous efficiency function, which is known to severely under-estimate the contribution of the blue end of the spectrum.

Step 0: Understanding the LMS Cone Fundamentals

The chart below shows the spectral sensitivities of the three cone types (L, M, and S), which are also called the cone fundamentals. The sensitivity chart tells us the amount of cone responses (y-axis) stimulated from different spectral lights (x-axis) that have the same energy/power/irradiance/radiance. The exact choice of the radiometric metric varies by use-cases. We will use power for simplicity. The absolute values of the responses don't matter, and they are customarily normalized to peak at unity. By treating the three cone responses of a spectral light as 3D coordinates and then plotting the spectral lights in the 3D space, we get what's called the spectral locus, which is shown in the plot on the right.

The specific cone fundamentals we use are the 2-deg fundamentals by Stockman and Sharpe (2000), which is originally given between $390~nm$ and $830~nm$ at a $5~nm$ inverval. We clip anything beyond $780~nm$ and replicate the data at $390~nm$ for $380~nm$ and $385~nm$. You might sometimes see a different version of the cone sensitivities, where the y-axis is in log-scale and the x-axis is equal-quantal rather than equal-energy. "Equal quantal" means the cone responses are given assuming the spectral lights have the same amount of photons rather than the same energy.

Draw Custom Cone Fundamentals. Toggle this switch and start drawing your own cone fundamentals. By changing the LMS cone sensitivities, you are building an atypical kind of human being with unique color perception.

Step 1: Pick the Primaries

The first step in building a color space is to pick the primary lights. The primaries need not be spectral lights, but we limit ourselves to spectral lights in this tutorial for simplicity. We provide you two ways to do pick primaries.

Use Preset Primaries. This is a short-cut that automatically selects spectral lights $445~nm$, $540~nm$, and $590~nm$ as the primaries.
Select Spectral Primaries. This will allow you to select three spectral lights as the primaries. The LMS chart above dims. You can then point the mouse at a point on a curve and click it to pick a primary. When you click a point, the L, M, and S responses at the corresponding wavelength are highlighted, indicating that one primary has been selected. The chart will lock once three primaries are selected. We assume that the first primary selected is the blue primary, the second one is green, and the last one is red. Of course R, G, and B are just names, but if you want the colors of the charts later to match your intuition, select the primaries from left to right.

Once the primaries are selected, the following system of (three) linear equations are built. The system has the form $A_{3\times 3} \times M_{3\times n} = C_{3\times n}$, where $n$ is the number of spectral lights in the LMS chart, which is $81$ in our case, since the chart spans $380~nm$ to $780~nm$ at a $5~nm$ interval. This linear system lies at the heart of building RGB CMFs from the LMS cone fundamentals. Let's unpack it a bit more.

${}$ ${}$ ${}$ ${}$ ${}$

Matching Cone Responses

This linear system essentially asks the following question: how much Watts of each primary light do we need to match the color of 1 Watt of each spectral light? The answer is encoded in matrix $M$, which is what we want to solve for.

To answer this question, recall the key invariant in colorimetry: to match color between two lights their LMS cone responses must match. Therefore, we will first express the LMS cone responeses of each spectral light, which is what matrix $C$ encodes. Each column vector in matrix $C$ corresponds to a spectral light, and represents the L, M, and S cone responses under one Watt of that spectral light. Naturally, numbers in each column are exactly the same as the numbers in the tooltip as you hover the mouse over the LMS chart above.

Matrix $M$ is what we want to solve for. Each column vector of matrix $M$ corresponds to a spectral light, and represents the power of the three primary lights needed to match the color of that spectral light at one Watt. Without losing generality, let's take L cone and spectral light $500~nm$ as an example and ask: how many Watts of each primary lights do we need to match the L cone response (${}$) of the spectral light at $500~nm$?

We know that for the color to match the L cone response at $500~nm$ must match the total L cone responses contributed by the three primary lights. The L cone response contribution of a primary light is: the power of the primary light $\times$ the L cone response per Watt of that primary. The former is what we solve for, and the latter can be directly read from the LMS chart above and is exactly what matrix $A$ encodes. Matrix $A$ is a $3 \times 3$ matrix, where the first row represents the amount of L cone responses contributed by the three primaries per Watt; the second and third rows encode the same information for the M cone and the S cone, respectively. Naturally, matrix $A$ depends on what exactly the three primaries are. Before the primaries are selected, matrix $A$ is unknown. As you select your primaries, matrix $A$ will be dynamically updated.

Why Trichromacy?

Before we move on, take a look at the linear system above. It has the same number of unknowns and equations ($3 \times 81$), which, in general, leads to a unique solution, indicating that there is one unique way of mixing the primaries to match a target color. If we used more than 3, say 4, primaries, $A$ would be a $3 \times 4$ matrix, leading to an under-determined system ($A_{3\times 4} \times M_{4\times 81} = C_{3\times 81}$), meaning that there are, in general, infinite many ways of mixing the four primaries to match the color of an arbitrary light. In contrast, if we used fewer than 3, say 2, primaries, $A$ would be a $3 \times 2$ matrix, leading to an over-determined system ($A_{3\times 2} \times M_{2\times 81} = C_{3\times 81}$), meaning that that is no general way of mixing the two primaries to match the color of an arbitrary light.

Fundamentally, the trichromatic theory of color has to do with the fact that human has three types of cone cells. Certain species have four cone cells, leading to tetrachromacy. There are a small population of people who have only one or two types of cone cells; they are monochromatic and dichromatic, respectively. Monochromatic people are completely color blind. That's also the reason your color perception is very weak just before dawn or right after sunset, when your cone cells are inactive and only rods are activated; there is only type of rod cells, hence no color perception. Interestingly, actual LMS cone sensitivities can be measured from monochromatic and dichromatic people, because they, in theory, allow us to isolate the responses of different cone types. See, for instance, Stockman and Sharpe (2000).

Step 2: Solve the Linear System

Once the three primaries are selected, matrix $A$ will be populated. Click to solve the linear system and plot the resulting matrix $M$.

How to Interpret the Chart:

The figure above is not the CMFs yet. The triplets at each wavelength tells us how much primary lights, in Watts, we need to mix in order to match the color of one Watt of that spectral light. For instance, to match the color of $1 W$ of spectral light $500~nm$, we need to mix ${\boxed{??}}$ $W$ of the red primary, ${\boxed{??}}$ $W$ of the green primary, and ${\boxed{??}}$ $W$ of the blue primary. Hover your mouse over the chart above at $500~nm$ to confirm this.

Instead of referring to power, the CMFs tell us how many units of each primary light to mix. How is one unit of, say red primary, related to power? It could be arbitrary. Nothing prevents us from calling $1 W$ as one unit of red primary, or $19.5 W$ as one unit of green, or $5,000 W$ as one unit of blue. Instead of picking arbitrary numbers, we will define an intuitive unit system: equal units of the primaries light should match the color of "white". This is still somewhat arbitrary but certainly intuitive: we naturally expect white to be produced from mixing equal amount of primary lights.

Two steps remain before we can produce the final CMFs: 1) pick a light to represent white (Step 3), and 2) conver Watts to units based on the chosen white light (Step 4).

Step 3: Pick the White

What is White?

White, as common as it is, does not refer to a specific light. Instead, many different light spectra are perceived "white-ish" and can be used as the reference white. CIE publishes a set of so-called Standard Illuminants, which are commonly used as reference whites for color spaces. For instance, Standard Illuminant D65 resembles noon daylight and is used in the sRGB color space among others; Standard Illuminant A resembles incandescent light. A special Standard Illuminant E, a.k.a., Equal-Energy White, has a flat Spectral Power Distribution (SPD); it is used as the reference white in the CIE 1931 RGB color space and the CMFs.

Use the selection box to choose between D65, A, and E. You can also draw your own reference white by choosing "Custom" from the selection box. The original SPDs in the CIE standard are normalized to a value of 100 at the wavelength of $560~nm$. For simplicity, we normalize the SPDs such that the peak is 1.0. That is, the maximum power of each illuminant is $1 W$.

Step 4: Generate Color Matching Functions!

Given the specific white, we can now convert the power chart in Step 2 to the CMFs. Here is the process. We first calculate the power of each primary light needed to match the color of the given white. We can this by:

$\sum_{380}^{780}W(\lambda)R(\lambda)$ = ${\boxed{??}}$,

$\sum_{380}^{780}W(\lambda)G(\lambda)$ = ${\boxed{??}}$,

$\sum_{380}^{780}W(\lambda)B(\lambda)$ = ${\boxed{??}}$,

for $\lambda\in[380~nm, 780~nm]$ at a $5~nm$ interval; $W(\lambda)$ is the SPD of the white you choose before; and $R(\lambda)$, $G(\lambda)$, and $B(\lambda)$ are the spectral power curves plotted above.

Under the white and the primaries that you selected, the calculation shows that we need to mix ${\boxed{??}}$ $W$ of red primary, ${\boxed{??}}$ $W$ of green primary, and ${\boxed{??}}$ $W$ of blue primary, in order to match the color of $1 W$ of the chosen white.

As established earlier, the unit system is so defined that white color is produced by mixing equal units of the primary lights. Therefore, ${\boxed{??}}$ $W$ of the red primary, ${\boxed{??}}$ $W$ of the green primary, and ${\boxed{??}}$ $W$ of the blue primary must correspond to the same unit of the three primaries. Do they correspond to 1 unit each, 10 units each, or 1 million units each? It's up to you. Choosing a different number will scale up and down the CMF curves, but the primary quantity ratio won't change. Often in colorimetry it is the ratio that we care about. The derivation so far has already made a few normalizations such as the LMS cone sensitivities and the SPDs of the white.

For simplicity here we will just assume that $1 W$ reference white is matched with exactly 1 unit of the three primaries. Therefore, to convert from power to units, we would simply divide each red power value in the chart above by ${\boxed{??}}$, each green power value by ${\boxed{??}}$, and each blue power value by ${\boxed{??}}$. The results are the CMFs, which are usually denoted as $\bar{r}$, $\bar{g}$, and $\bar{b}$.

$\bar{r}(\lambda) = R(\lambda)/$ ${\boxed{??}},$ $~~~\bar{g}(\lambda) = G(\lambda)/$ ${\boxed{??}},$ $~~~\bar{b}(\lambda) = B(\lambda)/$ ${\boxed{??}}$ $,~~~\lambda \in [380~nm, 780~nm]$

Or in the matrix form: $\begin{bmatrix} \bar{r}(\lambda) \\ \bar{g}(\lambda) \\ \bar{b}(\lambda) \end{bmatrix} = $ ${\boxed{??}}$ $\times \begin{bmatrix} R(\lambda) \\ G(\lambda) \\ B(\lambda) \end{bmatrix}$

Let's . If you make changes to the cone fundamentals or the white light, click the calculate and the plot buttons again; the new CMFs will be updated.

Step 5: Interpret the Results

CMFs Give Us a Color Space

The triplets at each wavelength in the CMFs tells us how many units of the primary lights are needed to match the color of one Watt of that spectral light. We call the triplet the tristimulus values of that spectral light in a color space. Using the tristimulus values as the 3D coordinates we can plot all the spectral lights and get the spectral locus in your RGB color space. This is shown in the plot on the right.

We can get tristimulus values of non-spectral lights too. If a light has an SPD of $\Phi(\lambda)$, we calculate the amount of primaries needed to match its color by:

$ \begin{bmatrix} R \\ G \\ B \end{bmatrix} = \begin{bmatrix} \bar{r}(380) & \cdots \bar{r}(500) & \cdots & \bar{r}(780) \\ \bar{g}(380) & \cdots \bar{g}(500) & \cdots & \bar{g}(780) \\ \bar{b}(380) & \cdots \bar{b}(500) & \cdots & \bar{b}(780) \end{bmatrix} \times \begin{bmatrix} \Phi(380) \\ \cdots \\ \Phi(500) \\ \cdots \\ \Phi(780) \end{bmatrix} ,\label{a}\tag{1} $ where $\bar{r}(\lambda), \bar{g}(\lambda), \bar{b}(\lambda)$ are the CMFs.

Any light can be converted into the RGB values in this way. This is what a color space provides us. It allows us to precisely describe the color of any light using a triplet, which represents how many units of each primary is needed to match the color. Geometrically, the color of a light is a point in a 3D space.

One subtle thing that often gets overlooked is that the RGB value of a light depends on not only the primaries, but also the reference white. Change the reference white in Step 3 while keeping the primaries unchanged; see how the RGB values of the same light changes, too.

Here are two good question that you can use to test your understanding of CMFs and color space:

Where should the reference white be in your RGB space?
If you build a new RGB color space with a different set of primaries and a new reference white, will the RGB values of the reference white change?

Metamerism. Two lights could have different SPDs but have the same color. Colors that are matched in this way are called metamers. Why does metamerism exist? Look at the equation above. It's an under-determined system, in the sense that we have many wavelengths in a SPD ($\Phi$) but only three values in the RGB triplet. Therefore, given an RGB triplet there are generally infinitely many solutions, i.e., SPDs.

The Locus in the RGB Space

Primaries Sit on the Axes. Toggle the "Show Primaries" switch and confirm that the three primary lights sit on the axes, each on one axis. What this means is that, naturally, to match the color of a primary light you would not need any amount of the other two primaries. For instance, the "blue" primary you choose is ${\boxed{??}}$; the RGB tristimulus values of that light are ${\boxed{??}}$, ${\boxed{??}}$, and ${\boxed{??}}$ — the first two values being 0 (If you see something like $-0.000$, that's just a very small negative number where the negative sign sticks around after rounding). Note the value for the blue primary: it means you need ${\boxed{??}}$ units of ${\boxed{??}}$ to match $1 W$ of ${\boxed{??}}$.

You might think that the B value of the blue primary should be 1, indicating that the blue primary is made up of 100% of the blue primary and 0% of the other two primaries. Correct thinking but incorrect understanding of what the RGB values of a color mean. The RGB values refer to units, rather than the percentage, of the primaries. You could normalize the primary units such that they add up to 100%. In fact, there's a very good reason to do so, and that's what the chromaticity tutorial is going to explore. For now, as a teaser you can click the "Locus in Chromaticity" button to see how the locus would look like after such a normalization. Notice how the primaries are now $[0, 0, 1], [0, 1, 0], [1, 0, 0]$. The absolute units are replaced by relative percentages.

Since the primaries you choose are spectral lights, the primaries naturally sit on the spectral locus, too. But primaries don't have to be spectral lights. When non-spectral lights are used as primaries, the primaries, while are not on the spectral locus, still sit on the axes in the color space.

Negative Amount of Primaries. If you use the original cone fundamentals, you will see that all the spectral lights (except the primaries) require a negative amount of some primaries. Hover over the spectral locus to confirm this. If a spectral light $X$ requires $-0.2$ units of R, $1.1$ unit of G, and $0.4$ units of B ($X=-0.2R+1.1G+0.4B$), it just means if you mix $0.2$ units of R with that spectral light, you get the same color as mixing $1.1$ unit of G and $0.4$ units of B ($X+0.2R=1.1G+0.4B$). There is nothing wrong with that both mathematically and physically. It doesn't mean that that spectral light doesn't exist; it's a real light. However, it does mean that you can't match the color of that spectral light by mixing the particular primaries that you chose.

In fact, it's generally true that you won't be able to produce the colors of all the spectral lights at the same time no matter how you choose the primaries — even if they are not spectral lights! Try picking different primaries at the top and confirm this. We will provide an intuitive and semi-rigorous explanation to this in the next tutorial.

Off-Locus Points

We now know what points on the spectral locus mean; they represent colors of spectral lights. What about an arbitrary point in the RGB space? Do they mean anything? Let's say we have a point with the coordinates $[0.1, 0.2, 0.3]$; given how we define the RGB space, it should naturally follow that that point is a color that can be generated by mixing $0.1$ unit of R, $0.2$ unit of G, and $0.3$ unit of B. That's a real color that can be physically produced, and the RGB tristimulus values tell you how to produce the color.

Now consider a point $[-0.1, -0.2, -0.3]$. Does this point represent a real color? We know that if there existed a light $X$ that could generate this color, then mixing $X$ with $0.1$ unit of $R$, $0.2$ unit of $G$, and $0.3$ unit of $B$ would let the color disappear, i.e., zero color or the "color of nothing". Is that possible? How about another point $[0.1, 0.2, -0.4]$. If there exists a light $Y$ that does generate this color, then mixing $Y$ with $0.4$ unit of $B$ would match the color of mixing $0.1$ unit of $R$ and $0.2$ unit of $G$. Does light $Y$ exist in the physical world? How do we reason about this? What are real lights anyways?

It turns out not all RGB points in a color space correspond to a real color. We call all real colors that humans can see the color gamut of human visual system. We could, however, construct a physically-unrealizable light to generate an imaginary color for an out-of-gamut RGB point. This is the topic of the next tutorial.

Linear Transformation Between Color Spaces

Every light has a set of LMS tristimulus values and so is a point in the LMS space; it has another set of RGB tristimulus values and is a point in the RGB space (that you just built). How do points in the LMS space and points in the RGB space relate? The math we have done so far dictates that the relationship must be a linear transformation. In particular, the LMS value of a light and the RGB value of the light (in your RGB space) are related by:

$\begin{bmatrix} L \\ M \\ S \end{bmatrix} = T \times \begin{bmatrix} R \\ G \\ B \end{bmatrix}$, where $T = $ $\boxed{??}$

The linear transformation is formed by a combination of two intermediate transformations, the matrix $M$ in Step 1 and the scaling matrix in Step 4.

We keep emphasizing "your RGB space", because there is no one single RGB space. By picking different primaries you will get different CMFs and thus different RGB spaces. The exact linear transformation from LMS to RGB depends on the specific RGB space. But regardless of what RGB space you build, the transformation from/to LMS is necessarily linear. How are different RGB color spaces related? They must also be related by just one single linear transformation. If a color space $RGB_1$ is transformed from the LMS space by $T_1$ and another color space $RGB_2$ is transformed from the LMS space by $T_2$, then $RGB_1$ can be transformed from $RGB_2$ by $T_1^{-1}T_2$.

Metamers Are Always Metamers! Since different color spaces are related by a linear transformation, metamers in one color space are metamers in another color space. If two colors have the same LMS responses, after a linear transformation their RGB values are the same too. Similarly, if two colors have different LMS responses, they won't have the same RGB values after a linear transformation.