5 Nov
Introduction
A color seems complex to understand because sometimes we use CMYK values to talk about it or RGB or even Lab, XYZ, Lch, ... All those color spaces could define the same color depending of the use.
In fact the purest way to speak about a color is through its spectral data. Those data are measured directly through a spectrophotometer. Though we can have different spectral data for the same color depending of the measurement condition : without illuminant every color is black, so let's use an illuminant but which one ? Do we use the D50 normalized illuminant or the D65 normalized illuminant, should we use a filter or a polarizer (to prevent gloss interfering)? It does not seems to have one unique way to save colors.
There is a long way before a true digitalization of a color in the real world...
Why does accurate color digitalization is that important ?
There are two main goals of the color digitalization: saving and sharing.
Saving
As a picture would, by saving a color you can create a fixed point in the time of your color's perception (because the measured subject can change through the time by growing old, burning at the sun, ...). You can use it as a reference like Pantone does with its inks or keeping it to create a history and check the evolution over time.
Sharing
You might have laboratories that measure and work with the color within the same building but colors workflow often have many places all around the world. In this last case the manufacturer may never see the color reference he is about to reproduce, so he does need a way to know what he is supposed to do and what does it looks like.
There are others cases but let's keep it simple for today.
Current saving system
There are many way to save color information, indeed some well known companies such as X-Rite or Data Color though about it and made their own formats.
But all of them are very different from each other and make complicated the compatibility between software. This issue was a starting point of the Coraye software : creating a bridge between software by implementing import and export of any kind of files (e.g.: allowing to transform a QTX file to a CxF one). To achieve this, we needed to create a way to become compliant to any kind of file format and keeping data untouched (no rounding or transformation process) by structuring data.
Our first though was to choose one existing file format as a reference for all other one but we did not found the perfect one because our requirement for the Coraye Software are too specifics. Moreover we choose to work with web technologies and encoding such as JSON format whereas standardized files are often proprietary oriented and complicated to parse.
The QTX format
Created by DataColor, the QTX file store data in a way that each color is independent from each other. So you will find informations such as the spectrophotometer used or the measured date for each sample.
You do have the possibility to insert batches colors. Those are repeated measure in different conditions (e.g.: with a different spectrophotometer).
You can even set different custom keys with custom value without worry. This would allow the file system to grow without requiring a v2 format that would not be compatible with previous versions.
[STANDARD_DATA 0]
STD_NAME=11-0605 TCX
STD_GUID=3B5F4330-4B43-715F-4146-BB2B4146BB2B
STD_DATETIME=1304360181,
STD_REFLPOINTS=36,
STD_REFLINTERVAL=10,
STD_REFLLOW=400,
STD_VIEWING=%R LAV SCI UV 400
STD_X=76.039398,
STD_Y=79.696068,
STD_Z=79.106171,
STD_R=68.957001,71.864998,72.008003,72.107002,72.460999,73.196999,74.137001,75.147003,75.987000,76.930000,77.723000,78.228996,78.500999,78.888000,79.075996,78.955002,78.995003,79.988998,81.500000,82.607002,83.119003,83.142998,83.023003,83.170998,83.910004,85.121002,86.570999,87.936996,89.249001,90.633003,91.738998,92.385002,92.737000,92.898003,92.920998,92.829002,
STD_MEASDLL_PARAMS=Manufacture:unknown,Model:CE7000A,Serial number:unknown,FirmwareVersion:unknown,Geometry:d/8,Specular component:I,MeasSpot:N/A,MeasArea:LAV ,UV-Filter%:UVEXC ,UV-Cutoff:None,MeasurementType:R,MeasurementSource:Instrument,Number of bands:36,Start band:400,Bandwidth:10,Correlation:Off,
STD_INST_TYPE=CE7000A
DCC_JOB_COMMENT=
Vendor Name=
Vendor Number=
Mill=
Dyehouse=
Vendor Email=
Retailer Name=
Brand=
Division=
Department=
Season=
Delivery Number=
Retailer Email=
Color Name=
Color Number=
Fiber=
Fabric Style=
PID=
Submit Number=
Labdip_Number=
Date Submitted=
User Name=
PRG_PALETTE_PROGRAM=
PRG_COLOR_PROGRAM=
PRG_COORD_PROGRAM=
PRG_DATE_DUE=
PRG_DATE_SUBMIT=
PRG_DATE_REVIEW=
PRG_ITEM_STATUS=
PRG_ITEM_TYPE=
PRG_JOB_IDENTIFIER=
PRG_FORMAT_FILE=Pro:
- You can mix colors in a same file
- You can customize the content
- Human readable
Cons:
- Not standards format (Require custom interpreter)
- Repetitive content
- Custom key mean specific code to interpret (data is lost if the software that import the file is not able to work with it because it doesn't understand what PRG_DATE_REVIEW=XXXXX means)
- Full of unused empty keys (more difficult to read even for humans) that make file heavier that it should be
- Difficulties to use with basic tools such as XCEL
The Cxf format
Created by XRite, the Cxf file is a complexe format that allow you to store various information from Lab to spectral, with metadata. It's a complete format that allow a full customization. But what seems to be a key functionality could also be a dangerous one : the over-customization tend to make difficult to read what's essential.
More over the XML format also have a lot of repetitive content. You can escape this by settings common parameter but it does only make the file even more complicated to read.
<?xml version="1.0" encoding="utf-8"?>
<cc:CxF xmlns:cc="http://colorexchangeformat.com/CxF3-core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<cc:FileInformation>
<cc:Creator>Coraye</cc:Creator>
<cc:CreationDate>2018-07-16T19:30:53.225Z</cc:CreationDate>
<cc:Description/>
</cc:FileInformation>
<cc:Resources>
<cc:ObjectCollection>
<cc:Object ObjectType="Standard" Name="Orange" Id="C0">
<cc:ColorValues>
<cc:ColorCIELab ColorSpecification="S0">
<cc:L>90</cc:L>
<cc:A>78</cc:A>
<cc:B>125</cc:B>
</cc:ColorCIELab>
<cc:ColorSRGB ColorSpecification="S0">
<cc:R>255</cc:R>
<cc:G>151</cc:G>
<cc:B>0</cc:B>
</cc:ColorSRGB>
</cc:ColorValues>
<cc:PhysicalAttributes>
<cc:Quantity Units="TINT PERCENT">100</cc:Quantity>
<cc:Opacity>100</cc:Opacity>
</cc:PhysicalAttributes>
</cc:Object>
</cc:ObjectCollection>
<cc:ColorSpecificationCollection>
<cc:ColorSpecification Id="S0">
<cc:MeasurementSpec>
<cc:MeasurementType>Colorimetric_Reflectance</cc:MeasurementType>
<cc:GeometryChoice>
<cc:SingleAngle>
<cc:SingleAngleConfiguration>Annular</cc:SingleAngleConfiguration>
<cc:IlluminationAngle>45</cc:IlluminationAngle>
<cc:MeasurementAngle>0.0</cc:MeasurementAngle>
</cc:SingleAngle>
</cc:GeometryChoice>
</cc:MeasurementSpec>
</cc:ColorSpecification>
</cc:ColorSpecificationCollection>
</cc:Resources>
</cc:CxF>
Pro:
- Certified file
- Standardized way to communicate color
- Possibility of customization
Cons:
- Does have a versioning that require a program to be up to date with the latest rules
- Chaotic over-customization
- The data's path is not always intuitive (where should I write spectral data, Resources>ObjectCollection>Object>ColorValues>ColorCIELab to get Lab values)
The CGATS format
This format is not used as a color table format, it's mainly a file used for printing targets but it can be considered as it (since strictly speaking it does contain similar information than you can find in a QTX or a Cxf format).
The format is quite convenient for a list of color patches that share the same condition (same support, same spectrophotometer info, same color index value). It is easy to read for a human or a computer and is even adapted for a copy paste in an excel sheet. You can add custom keys and there aren't that much repetitive content.
Unfortunately it isn't convenient for color table saving since all colors must share a common context and color values (XYZ & LAB & SPECTRAL), if one were missing, the full file would collapse.
CGATS.17
ORIGINATOR "Fiery Printer Profiler5.1.1.16"
FILE_DESCRIPTOR "Output Characterisation"
CREATED "2019-11-04 15:48:00"
INSTRUMENTATION "X-Rite i1iO/i1iO2"
KEYWORD INSTRUMENTATION_SN
INSTRUMENTATION_SN ""
KEYWORD INSTRUMENT_FILTER
INSTRUMENT_FILTER "M2 - UV cut"
KEYWORD TRACKING_ID
TRACKING_ID "7865-78T6"
KEYWORD LAYOUT
LAYOUT "1847 patches"
KEYWORD MEASURED
MEASURED "2019-11-04 16:12:53"
BEGIN_DATA_FORMAT
SAMPLE_ID CMYK_C CMYK_M CMYK_Y CMYK_K XYZ_X XYZ_Y XYZ_Z SPECTRAL_380 SPECTRAL_390 SPECTRAL_400 SPECTRAL_410 SPECTRAL_420 SPECTRAL_430 SPECTRAL_440 SPECTRAL_450 SPECTRAL_460 SPECTRAL_470 SPECTRAL_480 SPECTRAL_490 SPECTRAL_500 SPECTRAL_510 SPECTRAL_520 SPECTRAL_530 SPECTRAL_540 SPECTRAL_550 SPECTRAL_560 SPECTRAL_570 SPECTRAL_580 SPECTRAL_590 SPECTRAL_600 SPECTRAL_610 SPECTRAL_620 SPECTRAL_630 SPECTRAL_640 SPECTRAL_650 SPECTRAL_660 SPECTRAL_670 SPECTRAL_680 SPECTRAL_690 SPECTRAL_700 SPECTRAL_710 SPECTRAL_720 SPECTRAL_730
END_DATA_FORMAT
BEGIN_DATA
1 60.00 0.00 50.00 0.00 41.64 60.43 43.78 0.14773 0.17724 0.19816 0.23544 0.30003 0.36096 0.41560 0.47331 0.54287 0.62900 0.70874 0.75830 0.78469 0.79415 0.79218 0.78167 0.76566 0.73601 0.68953 0.63560 0.56756 0.48263 0.37687 0.27670 0.20821 0.16947 0.14420 0.12414 0.11660 0.12573 0.14979 0.18434 0.22296 0.24888 0.25621 0.26340
2 10.00 100.00 20.00 0.00 35.70 18.06 20.42 0.22300 0.27595 0.31391 0.34810 0.38639 0.38871 0.35439 0.29652 0.23688 0.17549 0.11434 0.07549 0.05133 0.02626 0.01216 0.01064 0.01226 0.01059 0.01216 0.04120 0.16407 0.36062 0.51761 0.59821 0.62715 0.64484 0.67501 0.71278 0.74878 0.77579 0.79784 0.81395 0.82720 0.83588 0.84080 0.84354Pro:
- Easily readable
- No repetitive content
- Can store spectral data with custom wavelength index
- Easy to export to tool such as XCEL
Cons:
- Colors must share a common context
- Each color have the same data structure
- Text format is not convenient for machine parsing
Other formats
There are also other file formats that are made for color saving but those aren't efficient because they unfortunately do not support spectral data. It can mainly be explained by the fact that those file format are made to save spot colors only and are more focused on graphic art than printing.
Examples of other formats : Ase, Acb, Aco, Csv, Coreldraw, ...
Let's start structuring color data !
Now that we've seen other formats and understand what are expectation among all of them, let's build a way to save our colors.
Has previously said, we will use the JSON format, in that way it will be easier to parse for an online program or an online database.
Here is the start of our color file :
{}Color requirements
If we want to ensure compatibility with QTX files, we need equivalent for following keys :
- STD_NAME (Name of the color)
- STD_DATETIME (Created/Measured date)
- STD_REFLPOINTS + STD_REFLINTERVAL + STD_REFLLOW + STD_R (Spectral values)
If we want to add the CxF compatibility let's add equivalent for:
- Name (Name of the color - Common with QTX one)
- ID (Identifier within the file)
- ObjectType (Standard, Trial, Target, Substrate, Colorant, etc.)
- GUID (Identifier unique worldwide)
- Creation Date (Created/Measured date - Common with QTX one)
{
"id": "CN1",
"guid": "CORAYE-CN1",
"name": "My Color Name 1",
"created_date": "Thu May 07 2020 17:47:46 GMT+0200",
"color_preview": [0, 0, 0], // Always RGB
"type": "spot", // light, display
"header": {
"filter": "M0",
"specular_component_mode": "SCE"
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
},
"cie": {
"lab": [0, 0, 0],
"xyz": [0, 0, 0],
"observer": 2,
"white_reference": {
"name": "My Custom Illuminant", // Or reserved name "D50", "D65", "A", "B", "C", "E", ...
"color_preview": [255, 255, 255],
"type": "light",
"cie": {
"lab": [100, 0, 0],
"xyz": [1, 1, 1]
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
}
}
}
}From this we do support QTX, CxF and CGATS format. They would work but we would still loss any other kind of information. The way we store spectral data give us every information we need : we could eventually store spectral data with irregular step.
ArgyllCMS does provide spectral curve with a 3-3-4 steps such as 380, 383, 386, 390, 393, 396, 400, ...
Adding color spaces
Every software does not include a spectral converter to get Lab values, so it would be nice to save them as well BUT to get Lab values (or XYZ ones) we require a white reference and an observer angle value. The perfect thing would be to include full white reference object but that's a little overkill for a true use of the color. But let at least give the possibility to it by specifying the white reference with a reserved name (among standards).
{
"white_reference": {
"name": "D50"
}
}Or with a detailed white reference (tip: that is the exact same structure than a color but with a light type):
{
...
"white_reference": {
"name": "My Custom Illuminant",
"color_preview": [255, 255, 255],
"type": "light",
"cie": {
"lab": [100, 0, 0],
"xyz": [1, 1, 1]
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
}
}
}Let's merge the white reference with our main color object to get a more specific way to save it
{
"id": "CN1",
"guid": "CORAYE-CN1",
"name": "My Color Name 1",
"created_date": "Thu May 07 2020 17:47:46 GMT+0200",
"color_preview": [0, 0, 0], // Always RGB
"type": "spot", // light, display
"header": {
"filter": "M0",
"specular_component_mode": "SCE"
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
},
"cie": {
"lab": [0, 0, 0],
"xyz": [0, 0, 0],
"observer": 2,
"white_reference": {
"name": "My Custom Illuminant",
"color_preview": [255, 255, 255],
"type": "light",
"cie": {
"lab": [100, 0, 0],
"xyz": [1, 1, 1]
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
}
}
}
}From this point we do have a color with enough information for a RIP or many other softwares but you might want to get more color space values of your measured color. Let's add them inside a colorspace field.
{
...
"colorspace": {
"rgb": [0, 0, 0],
"cmyn": [0, 0, 0, 0],
"6clr": [0, 0, 0, 0, 0, 0]
}
}Color encoding ?
How should we encode our color values ? This is a complicated question since RGB is from 0 to 255, XYZ is from 0 to 1, Lab is 0 to 100 for L value, -125 to 125 for a and b values but every thing else is from 0 to 100. We may even need to encode a CMYKOG (O: Orange, G: Green) values.
The best is still to keep the data untouched and so saving data colors "as it": no rounding or mathematical formula required.
{
"colorspace": {
"rgb": [255, 255, 255]
}
}Basically, everything that is not RGB (or Lab, XYZ, Lch ...) is CMYK so it should be encoded from 0 to 100.
Where is the profile info used for conversion ?
Since the profile has already been used to get the RGB, CMYK or NCLR (N -> 1, 2, 3, ... color channel) color value, we do not need it anymore so it is not require to specify it again. But we can still add the information if you want into a metadata field. This field is made to store whatever information you may use to include context.
{
"metadata": { // Whatever you want, this is a custom field
"comment": "",
"rgb_profile": "ADOBE98",
"spectrophotometer": "MYIRO",
"manufacturer": "Konica Minolta",
"serial_number": null,
"firmware_version": null,
"area": "LAV",
"source": "instrument",
"correlation": false,
"physical_attributes": {
"dimensions": null,
"quantity": null,
"weight": null,
"thickness": null,
"gloss": null,
"finish": null, // coated, matte, glossy, ...
"substrate": null, // wood, paper, textile, ...
"image": null, // binary or URI type
"opacity": null,
"attributes": null
}
}
}Batches colors
The spectral data is the best way to communicate color without data loss, because you get Lab, XYZ and every other color space value from it. But the same color could have different spectral data (and so another Lab, XYZ, etc values) if you change your spectrophotometer configuration (Filter and specular component mode). That's what batches field is for : saving measurements of the same color but in another configuration mode.
{
...
"batches": [
{
"color_preview": [0, 0, 0],
"header": {
"filter": "M1",
"specular_component_mode": "SCE"
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
},
"cie": {
"lab": [0, 0, 0],
"xyz": [0, 0, 0],
"observer": 2,
"white_reference": {
"name": "D50"
}
}
}
]
}The batch data should follow the same structure than the main color but should not include another batches field into it, only the main color should have one. Everything included into the batch data should be impacted data by the filter change, every thing else is inherited from the main color.
Final example
If we include every thing that has been explained until here, we should have a file that looks like this:
{
"id": "CN1",
"guid": "CORAYE-CN1",
"name": "My Color Name 1",
"created_date": "Thu May 07 2020 17:47:46 GMT+0200",
"color_preview": [0, 0, 0],
"type": "spot",
"header": {
"filter": "M0",
"specular_component_mode": "SCE"
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
},
"cie": {
"lab": [0, 0, 0],
"xyz": [0, 0, 0],
"observer": 2,
"white_reference": {
"name": "My Custom Illuminant",
"color_preview": [255, 255, 255],
"type": "light",
"cie": {
"lab": [100, 0, 0],
"xyz": [1, 1, 1]
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
}
}
},
"colorspace": {
"rgb": [0, 0, 0],
"cmyn": [0, 0, 0, 0],
"6clr": [0, 0, 0, 0, 0, 0]
},
"metadata": {
"comment": "",
"spectrophotometer": "MYIRO",
"manufacturer": "Konica Minolta",
"serial_number": null,
"firmware_version": null,
"area": "LAV",
"source": "instrument",
"correlation": false,
"physical_attributes": {
"dimensions": null,
"quantity": null,
"weight": null,
"thickness": null,
"gloss": null,
"finish": null,
"substrate": null,
"image": null,
"opacity": null,
"attributes": null
}
},
"batches": [
{
"color_preview": [0, 0, 0],
"header": {
"filter": "M1",
"specular_component_mode": "SCE"
},
"spectrum": {
"380": 0,
"385": 0,
"390": 0,
"385": 0,
...
},
"cie": {
"lab": [0, 0, 0],
"xyz": [0, 0, 0],
"observer": 2,
"white_reference": {
"name": "D50"
}
}
}
]
}This file structure might not be perfect but tend to be as complete as possible. It mixe the use that can be made of data and takes into account colorimetric issues of the printing and graphic world. It is web focused and the JSON format is among one of the most used in the world because it is easily readable for both humans and computers.
