Technology Archives - DxO https://www.dxo.com/news/category/technology/ Simply Better Images Thu, 23 Apr 2026 14:03:51 +0000 en-US hourly 1 https://wordpress.org/?v=6.6.2 Automatic dust correction in digital photographs https://www.dxo.com/news/automatic-dust-correction/ Wed, 18 Mar 2026 14:47:35 +0000 https://www.dxo.com/?p=170882 How DxO PureRAW 6 uses deep learning to find and remove dust spots.

The post Automatic dust correction in digital photographs appeared first on DxO.

]]>
pureraw-logo-en@2x.png

Automatic dust correction in digital photographs: How DxO PureRAW 6 uses deep learning to find and remove dust spots

DxO PureRAW 6 introduces automatic dust detection and removal: a single click identifies dust spots across the entire image and erases them, automating a tedious manual process. The feature combines a state-of-the-art object detection neural network with DxO's proven inpainting engine.

Key benefits for users

  • Fully automatic workflow. Dust detection and removal is a single checkbox. Batch-process an entire shoot, and every image comes out clean.
  • Adjustable sensitivity. A slider lets the user balance between catching every possible spot (high sensitivity) and avoiding the risk of false positives (low sensitivity).

(That said, we still recommend cleaning your gear from time to time. 😉)

The problem

Interchangeable-lens cameras tend to accumulate dust on their sensor or lenses. These particles cast small, soft shadows in your images — most visible in smooth, uniform areas such as skies and studio backdrops.

Photographers have long dealt with this in post-processing, using repair, heal, and retouch brushes. On heavily affected images or when processing a high volume of images, this quickly becomes tedious.

DxO PureRAW 6 automates this process. A detection algorithm scans the image for dust spots, and an inpainting algorithm erases each one automatically.

Why dust detection is challenging

At first glance, sensor dust seems easy to describe: small, dark, roughly circular blobs. But the apparent simplicity is deceptive. Several properties make robust detection surprisingly difficult.

Extreme subtlety. Most dust spots attenuate only a small fraction of the incoming light — often just 5 to 20 percent. They are faint stains, not opaque blots, and their visibility depends heavily on the underlying image content.

Tiny spatial extent. At full resolution, a typical dust spot spans only a handful of pixels — small enough that general-purpose object detectors, which are optimized for people or cars, struggle to register them.

No rich structure. Unlike the objects that mainstream detectors excel at — a face with eyes, nose, and mouth; a car with wheels and windows — a dust spot offers almost nothing for a neural network to latch onto. It is, in essence, a faint dark smudge.

Enormous variability. The appearance of a dust spot depends on the size and shape of the particle, its distance from the sensor surface, the lens aperture, and the color and brightness of the underlying scene. Some spots are sharp-edged circles; others are soft, diffuse halos. Some appear nearly black against a bright sky; others are barely distinguishable from noise. The diversity is far greater than a casual glance would suggest. Dependency on aperture and scene means that the same physical particle can look quite different from one photograph to the next.

The detection model: RF-DETR

The heart of the feature is RF-DETR, a transformer-based object detection architecture. We evaluated several detection architectures, including multiple generations of CNN-based models. RF-DETR was selected for a combination of reasons:

State-of-the-art accuracy. RF-DETR achieves top scores on standard object detection benchmarks, outperforming many well-known alternatives.

Multiple model sizes. Nano, Small, Medium, Large, and XL variants allow us to choose the best trade-off between accuracy and computational cost. We selected the Medium variant (33 million parameters).

Resolution-agnostic architecture. RF-DETR contains no fully connected layers that would fix the input resolution. This flexibility is important for our tiled inference strategy: the image is divided into overlapping 512×512 pixel patches, and the detection model runs independently on each patch. Results are then merged across the full image.

In standard benchmarks, RF-DETR detects dozens of object categories — people, vehicles, animals, furniture. For our use case, we retrained the model to recognize a single class: dust spot. The challenge lies not in classification but in detection — finding tiny, low-contrast features in a vast image.

Training data

Training a reliable dust detector requires exposing the network to a very large number of examples covering every conceivable combination of dust shape, opacity, blur, and background.

We started by collecting thousands of real photographs with genuine dust spots, all carefully labelled by hand. This real-world dataset already covers a great diversity of dust shapes, sizes, opacity, blurriness, and backgrounds, but we wanted to go further.

With its expertise in image and signal processing, our research team developed a dust synthesizer: a compact algorithm that generates a dust spot — indistinguishable from a real one — and composites it onto a random photographic or synthetic background. The synthesizer models the key physical properties of real dust: the irregular blob shape, the per-channel light attenuation in linear space, the blur that softens the edges, and the optional directional shading that some particles exhibit. Every parameter is randomized within carefully calibrated ranges derived from statistical analysis of real dust spots.

This synthetic approach ensures even distribution of dust characteristics and backgrounds throughout the training set, avoiding the biases that inevitably arise in any manually collected dataset. It guarantees, for example, that the network sees enough very faint spots, enough very small spots, and enough unusual backgrounds — combinations that would be underrepresented in a purely real-world collection.

In total, our dust detection network has seen approximately one million dust spots — a mix of real and synthetic — during its training.


The post Automatic dust correction in digital photographs appeared first on DxO.

]]>
DeepPRIME XD3: Fourth-generation AI denoising and demosaicing https://www.dxo.com/news/deepprime-xd3-fourth-generation/ Wed, 18 Mar 2026 14:40:47 +0000 https://www.dxo.com/?p=170830 DxO PureRAW 6 introduces DeepPRIME XD3 for Bayer sensors, the latest generation of DxO's deep-learning engine for raw image processing.

The post DeepPRIME XD3: Fourth-generation AI denoising and demosaicing appeared first on DxO.

]]>
pureraw-logo-en@2x.png

DeepPRIME XD3: Fourth-generation AI denoising and demosaicing

DxO PureRAW6 introduces DeepPRIME XD3 for Bayer sensors, the latest generation of DxO's deep-learning engine for raw image processing. A single neural network now performs three tasks simultaneously — denoising, demosaicing, and chromatic aberration correction — delivering images with even finer detail than its predecessor.

The technology rests on three pillars: a new multi-task formulation that adds chromatic aberration correction to the network's responsibilities, an optimized convolutional architecture discovered through extensive research, and a significantly improved training pipeline that closes the gap between synthetic training data and real-world RAW images.

Key benefits

  • Better image quality. Cleaner color reconstruction, finer detail, and fewer artifacts, especially on high-frequency textures and edges, and particularly on recent sensors without an optical anti-aliasing filter.
  • Same processing speed. Despite a substantially more capable network, DeepPRIME XD3 runs as fast as DeepPRIME XD2s on consumer hardware.
  • Broad compatibility. DeepPRIME XD3 unites all our recent advances in RAW image processing, and it now supports all sensor types.

A six-year journey

Raw conversion — the process of turning a camera sensor's mosaic of noisy single-color samples into a full-color photograph — has been at the heart of DxO's expertise for over two decades. In 2020, DxO introduced DeepPRIME, the first commercially available neural network to perform denoising and demosaicing jointly in a single pass.

Since then, we have worked relentlessly to push quality further. Deep learning and this holistic approach were also what allowed us to finally support X-Trans sensors, a variant found in part of Fujifilm's camera lineup. These sensors had never been supported by our classical denoisers. In 2022, we introduced the "XD" (eXtreme Detail) family — a second tier of DeepPRIME engines that reach for the highest possible image quality, at the cost of significantly heavier computation that demands a powerful GPU — or a good measure of patience.

2020 — DxO PhotoLab4
DeepPRIME. Joint denoising and demosaicing in a single deep neural network (Bayer only).

2022 — DxO PureRAW 2
DeepPRIME extends to X-Trans sensors.

2022 — DxO PhotoLab6
DeepPRIME XD ("eXtreme Detail"). More capable architecture and perceptual loss function, encouraging finer detail (Bayer only).

2023 — DxO PureRAW 3
DeepPRIME XD extends to X-Trans sensors.

2024 — DxO PureRAW 4
DeepPRIME XD2. Adversarial discriminator loss for a more natural rendering (Bayer only).

2024 — DxO PhotoLab8
DeepPRIME XD2s. Improved noise calibration for selected camera bodies.

2025 — DxO PureRAW 5
DeepPRIME 3. Three joint tasks: denoising, demosaicing, and chromatic aberration correction (Bayer and X-Trans).

2025 — DxO PhotoLab9
DeepPRIME XD3. More capable architecture and two-phase training (X-Trans only).

2026 — DxO PureRAW 6
DeepPRIME XD3 extends to Bayer sensors.

Focusing on X-Trans first during the development of DeepPRIME XD3 was a natural choice: the X-Trans version of DeepPRIME XD was older and easier to surpass than DeepPRIME XD2s, which Bayer users already enjoyed. But it led to a somewhat complex situation for the latter. On most images, DeepPRIME XD2s delivered the highest quality, yet on certain low-ISO images affected by chromatic aberrations, DeepPRIME 3 could actually yield better results. The release of DeepPRIME XD3 for Bayer sensors finally brings us back to a simplicity we had not enjoyed since 2023: whatever camera you use, you can choose between two RAW conversion networks — one that strikes a balance between speed and image quality, and one that reaches for the utmost in image quality.

The RAW image restoration challenge

Every digital image captured by a CMOS sensor contains three fundamental defects, all introduced before any software touches the pixels:

Color mosaic. The sensor does not capture full color at each pixel. Instead, a grid of tiny color filters lets each photosite record only one of three colors (red, green, or blue). Reconstructing the two missing color values at every pixel is the task of demosaicing. Two filter patterns are common in digital photography: Bayer, used by approximately 95% of all digital cameras, and X-Trans, found in the remaining 5%.

Sensor noise. Each photosite collects a random number of photons. The resulting shot noise is an inescapable property of light itself, and electronic read noise compounds it further. At high ISO sensitivities, noise can obscure fine detail entirely.

Chromatic aberrations. Most lenses don't focus all wavelengths of light to exactly the same point. The result is small lateral shifts between the red, green, and blue channels, visible as colored fringes along high-contrast edges.

Traditional RAW processing treats these three problems independently: a demosaicing algorithm interpolates the missing colors, a separate denoiser suppresses noise, and a third module corrects chromatic aberrations. Each module works in isolation, unaware of the others' decisions, and each can introduce its own artifacts that complicate the next step. DxO's approach, starting with DeepPRIME in 2020, has always been to solve multiple problems jointly in a single neural network. With DeepPRIME XD3, that principle now extends to all three defects.

Three defects, one network

The case for solving denoising, demosaicing, and chromatic aberration correction jointly is a matter of fundamental interdependence.

Consider what happens when these tasks are separated. Denoising a RAW image requires some understanding of how the mosaic pattern relates to the underlying scene — essentially, an implicit demosaicing on the fly. Conversely, demosaicing a noisy image requires the ability to see structure through the noise — essentially, an implicit denoising — because distinguishing a real edge from a noise fluctuation is critical for correct color interpolation. And demosaicing an image affected by chromatic aberrations is very nearly the same problem as correcting those aberrations: if the red, green, and blue channels are laterally shifted relative to one another, then reconstructing the correct color at each pixel requires imagining what the image would look like if the channels were aligned.

Splitting these three tasks across three separate networks — even networks trained to cope with the artifacts produced by the previous stage — would require more weights and more computation globally, because each network would need to internally replicate part of the intelligence of the others. The result would be longer processing times for equivalent quality, or lower quality for equivalent speed.

A single network, by contrast, can share internal representations across all three tasks. The features it learns to detect edges for demosaicing also help it distinguish signal from noise and identify lateral chromatic shifts.

Synthetic training data

A neural network is only as good as the data it learns from. For DeepPRIME XD3, the quality and realism of the training data are every bit as important as the architecture of the network itself.

The training data problem

When research on DeepPRIME began at DxO in 2018, a fundamental question was: how do we obtain the training examples that a supervised neural network needs — pairs of degraded input images and their corresponding perfect originals?

All options were on the table. Taking pairs of real photographs — a clean, low-ISO shot alongside a noisy, high-ISO shot of the same scene — seemed natural, but proved impractical: the two exposures never align perfectly, moving subjects cause inconsistencies, and the approach would have to be repeated for every camera body and every ISO sensitivity DxO supports. The noise-to-noise approach, which substitutes burst sequences for clean references, suffers from similar scaling limitations. And classical labeling — the backbone of most supervised learning — is simply impossible here: no human can look at a noisy mosaic of single-channel pixel values and propose the correct full-color, noise-free output for billions of pixels.

That left synthetic data generation: starting from pristine, high-quality photographs and simulating the defects that a real camera sensor would introduce. Each training example is thus a pair: a synthetically degraded image, and the original pristine version serving as ground truth. On paper, this is the most scalable solution by far. DxO supports over 600 camera bodies across roughly 20 ISO settings each, creating over 12,000 possible configurations. And this figure accounts only for noise: chromatic aberrations depend on the lens, the aperture, the zoom setting, and the focusing distance. If we wanted to capture real image pairs for every camera–ISO–lens combination, the number of configurations would explode into the millions. Synthetic generation can cover all of them from the same pool of ground-truth images.

The distribution gap

The challenge with synthetic data is a phenomenon known as the distribution gap: the statistical difference between the simulated training images and the real RAW files the network will encounter in production.

A naïve simulation — shifting color channels slightly to mimic chromatic aberrations, removing two color values out of three to simulate the Bayer mosaic, then adding white Gaussian noise — is enough to generate the above illustrations for this white paper. It is not enough to train a neural network. A network trained on such idealized data would perform well on synthetic images drawn from the same simulation, including images it has never seen during its training, but it would fail on real RAW files from real cameras.

Real RAW images differ from a naĂŻve simulation in countless ways:

Noise is not purely white Gaussian. Photon shot noise is indeed white and signal-dependent, guaranteed by the physics of light. But real sensor data is a mixture of photonic and electronic noise. Electronic noise — read noise, dark current, banding — can exhibit spatial correlations, non-Gaussian tails, and fixed patterns that vary from one sensor design to the next.

Chromatic aberrations vary across the field. Lateral color shifts are not uniform — they change in magnitude and direction from the center of the image to the corners, following the optical properties of each specific lens.

"RAW" files are not truly RAW. Before the data is written to the memory card, the camera applies a series of in-camera processing steps that alter the signal: black level correction, fixed-pattern noise subtraction, static defective pixel correction, focus pixel interpolation. Some manufacturers go further and apply lossy compression or even noise reduction to what they label as RAW data.

Sensor behavior changes with usage. Noise characteristics can shift depending on sensor temperature, shutter mode (mechanical vs. electronic), and other operating conditions. All of this varies across manufacturers and across camera generations. Manufacturers do not document their internal processing. We must infer what they do based on careful observation.

Closing the gap

Since 2018, DxO has leveraged everything at its disposal to minimize the distribution gap: two decades of expertise in image signal processing and, crucially, a proprietary calibration database that has no equivalent in the industry. For every supported camera body, at every ISO setting, DxO's laboratory has captured and analyzed calibration images — both photographic content and dark frames — to characterize not just the standard deviation of the noise, but its full statistical profile: its distribution, any spatial correlations introduced by in-camera processing, and how these properties change across the sensor and across operating conditions. This database, originally built to feed DxO's classical denoising algorithms, turned out to be an invaluable foundation for training neural networks.

Sometimes, however, some cameras reveal gaps that the existing simulation does not cover. A recent example illustrates the challenge: Fujifilm's X-Trans sensors of the 4th and 5th generations, where something changed relative to the first three generations. Despite extensive efforts, our DeepPRIME XD2 training pipeline never managed to produce satisfactory results for these sensors, which is why DeepPRIME XD2 and XD2s were released as Bayer-only.

For DeepPRIME XD3, properly supporting these sensors was a top priority. Over months of investigation, the team dissected how the newer X-Trans sensors differed from their predecessors and systematically adjusted the training data synthesis until the distribution gap became small enough for the network to generalize well to real images from these cameras.

Finding the best architecture

Adding a third task and demanding better demosaicing quality required a more capable network. The team began with a broad exploration. Transformer architectures, which dominate many fields of deep learning today, were tested alongside multiple convolutional neural network (CNN) designs. For this particular task — recovering fine, local image detail from noisy and incomplete data — CNNs proved more effective. Their inherent local bias, which focuses on small spatial neighborhoods, naturally encourages the smoothing of noise without hallucinating structure that is not there. Transformers, which model long-range dependencies, tended to let noise through rather than suppress it. For a denoiser, the CNN's bias toward local regularity is a feature, not a limitation.

An early prototype of DeepPRIME XD3 achieved the desired quality, but ran three times slower than DeepPRIME XD2s — too slow for a production tool used on thousands of images. The challenge, then, was to find an architecture that could be just as intelligent while fitting within the same computational budget. The team explored different convolutional block designs, separable convolutions in place of the full 3D convolutions used in earlier generations, different activation functions, and varying amounts of computation allocated to each scale of the U-Net.

Each candidate architecture was trained for approximately three weeks on an Nvidia H100 GPU. Around 50 configurations were evaluated in total, amounting to roughly three years of cumulative H100 GPU time dedicated solely to architecture exploration.

This entire process was carried out twice: first for X-Trans, then for Bayer. This is the principal reason why the Bayer version arrives only now in DxO PureRAW 6, while the X-Trans version was already released six months earlier in DxO PhotoLab9.

The outcome is a network with significantly more parameters than DeepPRIME XD2s, arranged in a way that keeps inference time essentially the same on consumer hardware. More weights, more intelligence, but no significant penalty in processing speed.

Renoising, rethought

Almost twenty years ago, DxO's researchers made an observation that still holds today: it is very difficult to make a denoiser remove only part of the noise. Denoisers — from the earliest wavelet and non-local means filters to modern neural networks — generally perform best when asked to remove all noise. Attempting partial removal tends to produce artifacts. The better the denoiser, the more detail it preserves in the process, but even the best denoisers inevitably erase some fine structure along with the noise.

To avoid the "plastic" look that results from fully denoised images, our researchers devised a simple but effective technique: let the denoiser do its job completely, then add a small fraction of the removed noise back to the image. Reintroducing part of the original noise, rather than synthetic white noise, has a crucial advantage — it also reintroduces part of the fine detail that was lost in the process. The first product to feature this technique was DxO OpticsPro 5, released in 2008. Even though DeepPRIME XD3 is vastly more capable than the denoising and demosaicing algorithms of that era, the principle remains as valid as ever.

For DxO PureRAW 6, we reworked how this noise reintroduction interacts with our lens corrections, specifically with vignetting and distortion correction. Both corrections are now applied before adding the residual noise back to the image, which allows us to treat the main signal and the noise component differently.

Vignetting. The noise level in RAW images depends on the signal level in a nonlinear way. With a lens that exhibits strong vignetting, the signal-to-noise ratio decreases significantly in the corners. When we amplify the corners to produce a uniformly bright image, we also amplify the noise, leaving it visibly stronger than in the center. The solution is to use the noise model — the known relationship between signal level and noise level — to derive a correction factor that produces homogeneous noise across the frame, and to apply this factor to the noise before adding it back.

Distortion. Distortion correction requires geometric interpolation of the pixel grid. When applied to white noise, interpolation introduces two unwanted effects: it creates spurious structure in the noise, and it causes periodically varying noise levels. At positions where the interpolated coordinate coincides with a real pixel, the noise is preserved as-is, while at positions that fall between pixels, the noise is smoothed and its level drops. In DxO PureRAW 6, we address this by applying a specialized interpolation algorithm to the noise component separately, ensuring that its level remains uniform after distortion correction.

Both effects are most visible at high ISO settings, where the residual noise — even though it is only a fraction of the original — is clearly perceptible.

This improved renoising pipeline applies to both DeepPRIME 3 and DeepPRIME XD3. It is a good example of how much we care about the details: our ambition is not "only" to build the world's best denoiser, but the world's best RAW conversion engine.

The results

The practical effect of all these advances depends on image content and shooting parameters. Compared to DeepPRIME XD, which DeepPRIME XD3 replaces for X-Trans sensors, the new engine generally yields cleaner, more natural results. Compared to DeepPRIME 3, it almost always produces images that are both cleaner and more detailed, at all ISO sensitivities. The difference with DeepPRIME XD2s is more subtle: DeepPRIME XD3 shows its advantage most clearly on images with fine textures, sharp lenses, sensors without an optical anti-aliasing filter, and lenses exhibiting chromatic aberration. Improvements in demosaicing and chromatic aberration correction are best visible at low ISO, while improved detail preservation is most apparent at intermediate to high ISO settings.


The post DeepPRIME XD3: Fourth-generation AI denoising and demosaicing appeared first on DxO.

]]>
How DxO’s pioneering approach makes DNG files four times smaller without impacting quality https://www.dxo.com/news/dng-compression/ Wed, 18 Mar 2026 13:00:35 +0000 https://www.dxo.com/?p=170537 DxO PureRAW 6 introduces a new high-fidelity compression option for the DNG format, reducing file sizes by approximately 4x compared to the current lossless compression, while preserving full perceptual image quality.

The post How DxO’s pioneering approach makes DNG files four times smaller without impacting quality appeared first on DxO.

]]>
pureraw-logo-en@2x.png

How DxO’s pioneering approach makes DNG files four times smaller without impacting quality

DxO PureRAW 6 introduces a new high-fidelity compression option for the DNG format, reducing file sizes by approximately 4x compared to the current lossless compression, while preserving full perceptual image quality.

DxO’s new High-Fidelity Compression technology combines two complementary techniques: Dynamic Range Compression and the JPEG XL image codec.

Key benefits

  • 4x smaller files — A 50 MP camera's Linear DNG drops from ~200 MB to ~50 MB, making Linear DNG practical for everyday use and high-volume workflows. Smaller files mean faster imports, faster cloud syncs, and less disk usage.
  • High fidelity — The compression is perceptually transparent, even under aggressive editing.
  • Compatibility — The output remains a standard DNG file. Any DNG-compatible application (Adobe Lightroom, Capture One, etc.) can open and edit these files normally.

Why compress more?

Linear DNG is DxO's recommended output format for DxO PureRAW because it preserves maximum editing latitude while being universally compatible with third-party RAW processors. However, even with the lossless compression built into the DNG specification, a typical Linear DNG weighs in at approximately 4 MB per megapixel. For a 50 MP camera, that is 200 MB per image.

Clearly, there is a strong motivation to compress these files more aggressively.
But how far can we go without compromising quality?

From lossless to perceptually lossless

Lossless compression is the most reassuring approach for both developers and users alike, since it guarantees that the decompressed file is mathematically identical to the original, bit for bit. However, this class of algorithms is inherently limited in efficiency, especially when the signal being compressed contains information that, from a perceptual standpoint, is useless.

For DxO PureRAW 6, our image scientists have developed a compression scheme that targets this useless information, removing it before compression and thereby achieving much better compression ratios. The result is what is known as perceptually lossless compression: the mathematical loss it introduces is not perceivable by a human observer under typical viewing and editing conditions.

We identified two types of perceptually irrelevant information in Linear DNG files:

1. Excess pixel precision. Digital camera RAW files are typically encoded at 12 or 14 bits per pixel; the output of our DeepPRIME pipeline uses 16 bits. However, images always retain some residual noise, intentionally left in place to prevent the unnatural “plastic” appearance caused by complete denoising. As we explain below, the more noise a signal contains, the less its full numerical precision is relevant. Removing the unused precision is the role of Dynamic Range Compression (DRC).

2. Exact texture and grain shape. In practice, slight differences in the exact shape of noise grain or fine texture are imperceptible. Simplifying these micro-details is a classic principle in image and video compression, and is the domain of the JPEG XL codec.

Both techniques require standard DNG mechanisms so that any compatible software can open the resulting files transparently. DRC is encoded via the DNG Linearization Table tag, and JPEG XL is a compression mode introduced in DNG specification version 1.7. Both are supported by common RAW processing applications.

Dynamic Range Compression

Dynamic Range Compression (DRC) is a well-known technique in audio signal processing. A compressor reduces the dynamic range of a signal by applying a non-linear transfer function: in audio terms, loud parts are attenuated, and quiet parts are boosted so that the signal fits more efficiently within a given bit budget. The same principle turns out to be remarkably well-suited to RAW digital images.

Why DRC works for RAW images

Digital images are affected by photonic (shot) noise, a fundamental property of light itself. The standard deviation of this noise grows with the square root of the signal intensity.
This has a profound consequence for compression of linear images:

  • In dark regions, noise is very low, and the signal is finely structured. Every bit of precision can carry genuinely useful information — 14 or even 16 bits may be needed.
  • In bright regions, noise is comparatively large. The useful signal precision is far lower than what 14 or 16 bits represent. Those extra bits encode noise more precisely than anyone would ever need or could ever see.

It is precisely these perceptually useless high-precision samples in the highlights that make lossless compression less efficient: the compressor must faithfully encode bits that carry no meaningful information.

  • DRC addresses this by applying a companding function — concretely, a curve close to the square root — to the linear pixel values before compression. This is conceptually related to a variance-stabilizing transform: after the square root, the noise standard deviation becomes approximately constant across the entire tonal range. Precision is thereby allocated where it matters — many levels in the shadows, fewer in the highlights — without discarding any information that was perceptually meaningful to begin with.

At decompression time, the inverse function (stored in the DNG Linearization Table) restores the original linear encoding, exactly as the DNG specification intends. The process is fully transparent to any downstream application.

The number of quantization levels was chosen conservatively and validated against worst-case editing scenarios such as large exposure pushes combined with extreme shadow recovery to ensure that quantization artifacts remain invisible in all practical uses.

JPEG XL compression

After DRC, the conditioned image is compressed using JPEG XL, the next-generation image codec standardized by the JPEG committee.

What makes JPEG XL better than legacy JPEG?

Legacy JPEG dates from 1992 and relies on a fixed 8x8 block transform with relatively simple entropy coding. While groundbreaking in its time, this approach leaves significant compression performance on the table by today's standards. JPEG XL incorporates over two decades of advances in image compression research:

Variable-size transforms — As small as 2x2 and up to 256x256, these allow the encoder to use large, efficient blocks in smooth regions and small, precise ones near edges, adapting to local image content rather than forcing a one-size-fits-all grid.

Perceptually optimized color space — JPEG XL's internal color representation is modeled on the human visual system, enabling smarter allocation of bits to the aspects of the image that matter most to perception.

Advanced entropy coding — Modern and significantly more efficient coding techniques extract more redundancy from the data than legacy approaches could.

Sophisticated prediction and context modeling — The encoder builds a statistical model of the image as it goes, capturing fine-grained local structure and reducing the amount of truly unpredictable information that must be stored.

Native high bit-depth support — unlike legacy JPEG, JPEG XL is designed from the ground up for high bit-depth content, making it an ideal compression layer for RAW imaging pipelines.

We apply JPEG XL with a near-lossless quality setting, meaning the mathematical loss introduced by the codec is negligible — far below the noise floor of any real-world image. The combination with prior DRC is what makes the compression so effective: by removing perceptually irrelevant precision before handing the data to JPEG XL, we give the codec a signal that is inherently easier to compress, without asking it to make any quality-damaging decisions.


The post How DxO’s pioneering approach makes DNG files four times smaller without impacting quality appeared first on DxO.

]]>
What are Linear DNG files? How do you use them? https://www.dxo.com/news/linear-dng/ Thu, 02 Feb 2023 11:00:00 +0000 https://www.dxo.com/news/linear-dng/ Discover how linear DNG files can deliver the ultimate in image quality without overhauling your entire workflow.

The post What are Linear DNG files? How do you use them? appeared first on DxO.

]]>

What are Linear DNG files and how should you use them in your photo-editing workflow?

Photo editing software such as Lightroom and Capture One might give you creative freedom and flexibility, but that doesn’t necessarily mean that you’re getting the best possible image quality from your RAW files. Linear DNG files are part of a solution that allows you to combine different software to get ultimate image quality without having to overhaul your entire workflow.

Linear DNG files are RAW files that have been partially developed, having undergone some complex mathematical processing to lock in demosaicing. Depending on its intended purpose, a Linear DNG might also complete other parts of the RAW conversion process such as denoising and lens corrections. You can then take this optimized file into your photo-editing software of choice to complete the creative process — everything from white balance and exposure through to tone curves and color grading — with the knowledge that your final image will be the highest possible quality. As you edit your files, you might even find that you have greater flexibility than with the original RAW file.

By taking a RAW file and converting it to a Linear DNG file with one application and then editing it in another, you get to combine the strengths — one mathematical, one creative — of two different pieces of software.

This image was edited as a RAW file using Adobe Lightroom. In the centre, you can see the results of Noise Reduction in Lightroom, and on the right is the optimization as a result of using DxO PureRAW  3.

The basics: RAW conversion, linearity, and DNG files

Every image sensor in every camera produces RAW files that need to be converted into a format that is capable of being displayed on-screen or printed on paper. It’s worth remembering that a camera does not “see” an image; instead, it captures information that must be transformed using sophisticated digital image processing. Software — either in the camera or on your computer — takes data that has been measured by each pixel and converts it into a color image. Unprocessed information is called “RAW data” and the process of transforming it into something visible is called “RAW conversion.” If you shoot in RAW, you’re saving the unprocessed data from the sensor and saving it to your memory card, ready to be processed by software such as DxO PhotoLab or Adobe Lightroom.

In some ways, a RAW image can be compared to a frame of negative film in analog photography. It contains all of the information that was captured when taking the photograph but more processing is needed before an image can be revealed.

Why should you shoot RAW?

Why would you capture data that still needs to be processed rather than JPEG files that are ready to be viewed? There are two reasons.

1. The first is image quality. As mentioned above, RAW conversion is complex. Doing it outside of the camera allows you to use more sophisticated algorithms, yielding better images.

2. The second reason is that it gives you control over certain aspects of the RAW conversion, such as exposure, color, and contrast.

Again, comparing it to analog photography is a useful illustration: JPEG files are a bit like Polaroid instant film. You get immediate results and that’s a huge advantage. However, the image quality cannot match that of negative film. Furthermore, with negatives, you can adjust exposure and color while developing your prints in the darkroom.

Of course, you can change the exposure and color of JPEGS, but RAW files contain a lot more information. There’s more dynamic range, a greater array of colors, and more levels of gray that allow you to preserve smooth tone transitions

RAW data is linear

RAW data is linear — but what does this mean? In mathematics, a linear system is one where multiplying an input by a certain amount means that the output will be multiplied by the same amount. Going back to your algebra at school, consider this example: In a linear system where 2x = 5y, then 4x = 10y, and 6x = 15y. Double one factor and you double the other, triple one factor and you triple the other, etc. Mathematicians would describe a linear system as one where scaling an input by a factor means that the output is always scaled the same factor: in other words, f(αx) = α f(x) for all α.

A photographic lens is a linear system. If you double the intensity of your studio lighting or double the exposure time while keeping the aperture unchanged, the amount of light reaching the image sensor in the camera will also double.

Just like the lens, the camera’s sensor is also linear. As a result, the data in the RAW file is a linear function of the amount of light in the scene. Notably, the first few steps of RAW conversion are linear. However, some non-linear steps soon take place — most notably, color rendering and gamma correction (i.e., the process of making data fit more closely to human eye’s understanding of how brightness works). To be able to retain quality, you want your exposure adjustments to be applied before these non-linear functions occur. This is where a JPEG file falls short as it doesn’t allow you to do that.

By contrast, when editing a RAW file, all of these changes are made before color rendering and gamma correction take place. Exposure, white balance, highlight recovery, boosting shadows — all of this can be achieved before irreversible steps are imposed and the editing process remains non-destructive.

An example can be seen in this photograph of a color checker card, captured with a white balance that had not been set correctly:

If you take the RAW file from the camera, you can use the eyedropper in your photo-editing software on a part of the image that is neutral gray to allow it to calculate the correct white balance:

The result is accurate (taking into account the limitations of the camera). Adjusting white balance on your camera before the shot or as part of the editing process achieves the same result because, just like the camera, the software applies this change in the linear domain. However, if you take the JPEG produced by the camera and attempt to correct the image in the same way, the results are noticeably different:

In reality, all of the grays in this scene are completely neutral. However, in this JPEG, each of the grays has a slight color cast. In addition, the individual color tiles are both quite far from their true hues and have lost some of their intensity. Recovering an accurate reproduction of the colors is impossible because the linearity of this information was given up when the RAW data from the sensor was converted into a JPEG.

What is DNG?

DNG stands for Digital Negative and is a RAW image file format designed by Adobe. Adobe’s goal was to create an open standard for storing RAW data as an alternative to the proprietary formats used by camera manufacturers — ARW from Sony, NEF from Nikon, CR3 from Canon, etc. All of these files can be converted to DNG simply by changing how the pixel values are stored. The values themselves remain unchanged.

The DNG format also provides a standardized means of storing metadata. Metadata — called XMP (Extensible Metadata Platform) — might include information such as shooting parameters, copyright, and star ratings, but they also contain settings for Adobe’s RAW conversion algorithms.

So what is Linear DNG?

Typically, DNG files store unprocessed RAW data. By contrast, Linear DNG files store some intermediate results of RAW conversion. Some initial processing steps have been applied — but only linear ones. This means that none of the data is lost to one-way, non-linear processes. As a result, Linear DNGs have exactly the same flexibility as RAW files when it comes to editing aspects such as brightness, color, and contrast.

Linear DNG files become useful when you want more flexibility in your workflow. If your preferred photo-editing software does a poor job of performing those initial processing steps, you might want to have another piece of software do it for you instead as a means of ensuring better image quality. With Linear DNG files, you can combine the strengths of different tools, squeezing more from your RAW files in terms of quality before switching to different software that gives your preferred mode of creativity.

Objective and subjective steps when processing RAW files

RAW conversion consists of many steps, both linear and non-linear. At DxO, we regard each step as either restoration or rendering. Here’s how they differ.

Restoration is where you are trying to achieve a perfect image from imperfect RAW data. There is an objective truth — the scene in front of your camera — that you are trying to recreate in a photograph as accurately as possible. Two fundamentals of this process are demosaicing — the conversion of values from individual pixels for red, green, and blue into accurate colors — and denoising — the removal of inaccurate pixels created as a result of interference and heat inside your camera, both of which become amplified when ISO is increased. Other important steps are the removal of vignetting, geometric distortions, and chromatic aberrations, and the correction of a lack of sharpness in the corners.

Restoration is considered an objective task in that there is a ground truth that you are trying to achieve as closely as possible. For example, with demosaicing, you want to create accurate colors as though they were seen with the human eye rather than the millions of individual values for red, green, and blue, as captured by the pixels on the camera’s sensor. In denoising, you want to restore the pixel values as though the image had been captured under better lighting conditions and using a lower ISO. With both of these tasks, computers are far better than humans.

Rendering, on the other hand, is fundamentally subjective. Even exposure and white balance are subjective. Trade-offs have to be made between highlights and shadows, and for many, the goal is not to faithfully reproduce the scene as it appeared in front of their camera. Instead, you might want to create a certain atmosphere or emotion. Even though computers might become better at guessing what sort of rendering humans might appreciate, many photographers prefer to have control over this part of the process.

90% of the sliders in your RAW conversion software — whether it’s DxO PhotoLab, Adobe Lightroom, or Capture One — are dedicated to rendering. Only 10% of the sliders relate to restoration, even though the software spends 90% of its computing time on restoration tasks. Furthermore, this is in spite of the fact that the restoration can determine how much flexibility a photographer has when choosing how to edit a photograph, and can have an effect on other aspects such as the potential to crop without compromising too much detail, or how large a print can be achieved.

Splitting your RAW conversion into rendering and restoration

A photographer’s choice of RAW processing software is highly subjective, and most are guided by the software they find the most intuitive in terms of rendering their images — even though the choice for restoration should be a completely objective decision. Unfortunately, the best tool for one is not necessarily the best tool for the other.

For restoration, those evaluating cameras and RAW conversion software — whether they are photo industry journalists or keen amateurs — have been running analyses using test scenes and evaluation targets for decades. These assessments have long confirmed that DxO’s optical corrections are the most accurate and that our DeepPRIME technology is better at removing noise while preserving more detail than any other RAW converter on the market. In terms of restoration, DxO DeepPRIME should be the obvious choice for any photographer seeking to maximize the quality of their images.

However, when it comes to rendering, we appreciate that the choice is more personal. Not only are there preferences for certain colors or levels of contrast, there is also habit. Your photographic style might be tied up in the tools that are most familiar to you, and we understand that. Changing your RAW converter would require learning a completely new system and potentially losing what makes your images unique in terms of look and feel. Switching away from Adobe Lightroom or Capture One and over to DxO PhotoLab might seem too much of a hurdle, just as switching between Canon and Nikon would also mean a big commitment.

With this in mind, we decided that photographers should be able to get the best of both worlds. What if you could take advantage of DxO’s renowned restoration algorithms while keeping your existing workflow? You could get superior image quality and files with greater flexibility — all without having to switch to new software. This is where Linear DNG files and DxO PureRAW come into play.


DxO PureRAW  5

Supercharge your cameras and lenses

DxO PhotoLab 8

RAW photo editing at its finest

The post What are Linear DNG files? How do you use them? appeared first on DxO.

]]>
Support arrives for Fujifilm X-Trans sensors https://www.dxo.com/news/unique-sensor/ Thu, 02 Feb 2023 11:00:00 +0000 https://www.dxo.com/news/unique-sensor/ DxO’s latest software brings exciting news for Fujifilm photographers: both DxO PhotoLab 6 and DxO PureRAW  3 now process files from X-Trans sensors, producing remarkable levels of detail.

The post Support arrives for Fujifilm X-Trans sensors appeared first on DxO.

]]>

A unique sensor now matched by next-generation processing: DxO DeepPRIME and DeepPRIME XD now support Fujifilm X-Trans

DxO’s latest software brings exciting news for Fujifilm photographers: both DxO PhotoLab 6 and DxO PureRAW  3 now process files from X-Trans sensors, producing remarkable levels of detail.

What is it about X-Trans cameras that make them different to other cameras on the market, and how is machine learning revolutionizing the way that raw files are processed? Head Scientist Wolf Hauser discusses the pros and cons of X-Trans and how DxO’s approach to processing them leads to significant advances in image quality.

Never a company afraid to try something different, Fujifilm introduced the X-Trans sensor in 2012. Given that the rest of the camera industry almost exclusively uses Bayer sensors, this was a bold move and the last ten years have seen many heated debates about whether X-Trans brings genuine benefits to photographers or is little more than an elaborate marketing trick. As will be explored below, there are certainly advantages and disadvantages to X-Trans and the algorithms used to interpret the raw data from this sensor are critical for getting good results. Fujifilm enthusiasts have long searched for the best software to process their images and DxO PhotoLab 6 and DxO PureRAW  3 now support for X-Trans raw files – in both DeepPRIME and DeepPRIME XD modes – offering clean images from Fujifilm cameras with fantastic detail rendition.

Before we can understand what makes X-Trans different from Bayer, it’s useful to remind ourselves how sensors capture light, how moirĂ© comes about, and how the raw data from a sensor is turned into the images that we see on our screens.

How to Make a Camera Sensor See in Color

The pixels on a camera’s sensor, whether it’s the smartphone in your pocket or a medium format body, only capture the intensity of light. The solid-state photosites count photons, but they have no means of understanding the wavelength — and thus the color — of the light that they are receiving. To solve this problem, manufacturers creating the earliest digital cameras invented the Color Filter Array (CFA). This mosaic of red, green, and blue sits in front of the sensor and allows the camera to observe different colors through different pixels.

In order to create an image, the next step is to interpolate this data through a process called demosaicing. This uses sophisticated algorithms to calculate the missing red, green, and blue values for each individual pixel based on the surrounding pixels.

This design was inspired by nature: the human eye also has red, green, and blue receptors, although a critical difference is that these receptors are spread completely randomly across the retina. Our brains process this stream of continuously shifting data at incredible speed, expertly filling in any blanks using experience and assumptions — none of which is easily replicated in a camera. Instead, the typical camera sensor uses a uniform grid named after its inventor, Bryce Bayer, who came up with a beautifully simple design back in 1974.

The foundation of the Bayer pattern is a block of four pixels with one red and one blue pixel sitting diagonally opposite one another, with the two remaining pixels both green. In effect, the sensor has been split into three: one-half of the pixels sees only green, one-quarter sees only red, and the remaining quarter sees only blue. As a result, the camera has to guess twice as many red values and twice as many blue values as it does green.

Moiré Explained

Capturing light through the means of a uniform grid of pixels can produce some strange visual effects. Moiré is an interference phenomenon that can occur when two grids interact, with patterns often appearing as waves or ripples. In the real world, we tend to see them most often when one dense, wire mesh fence sits behind another.

Cameras are particularly prone to creating these patterns for the simple reason that moirĂ© is the result of two regular grids interacting, and one already exists in the form of the neatly arranged rows of pixels that make up the camera’s sensor. When the scene contains a regular pattern that is as finely detailed as the pixel grid, moirĂ© may appear.

The diagram below simplifies the phenomenon by showing a single row of pixels. The sensor carves up the incoming texture, averaging its intensity within each pixel. Signal processing engineers call this process “sampling”: converting a continuous signal into (spatially) discrete values. In the first instance, the sensor can accurately understand the scene, despite the simplification that occurs. Difficulties arise if the details of the pattern become finer than the pixel grid. As can be seen in the second instance, the high frequency of the signal does not match with the lower frequency of the pixels, and the pattern breaks down. Instead of the original high frequency, we observe a lower frequency that never was part of the scene, albeit with strongly reduced amplitude.

Things get even worse if there are gaps between pixels. Suppose that we take out all pixels at odd positions and observe the signal only through pixels at even positions. As before, the incoming signal intensity is averaged over even pixels — but whatever hits the odd pixels is lost completely.

On the two previously shown textures, this will have no decisive impact. However, there is a range of frequencies that could be captured well without gaps but which transmute into moiré patterns when we introduce gaps. Again, we observe a lower frequency that was not in the scene, and this time its amplitude is almost unchanged compared to the original signal.

You might wonder “why would we introduce gaps?” This is exactly what a color filter array does. Our so-called RGB sensors are actually a red sensor with many gaps, a green sensor with fewer gaps, and a blue sensor, also with many gaps. Having more gaps means a greater chance of creating moirĂ© patterns. More gaps in red and blue are the reason why you frequently observe color moirĂ©, i.e. false hue patterns.

How to Avoid Moiré

As we have seen above, moiré arises when there is a mismatch between the resolving power of the lens and the resolution of the sensor, with the lens trying to force too much fine detail into too few megapixels. With sensor resolutions steadily increasing, and the pixel grid becoming finer, this is becoming less of a problem over time.

But most cameras don’t yet have high enough resolution to mitigate this issue, so manufacturers have had to find other means of dealing with moirĂ©. One radical solution would be to remove the color filter array completely, creating a monochrome camera that is capable of capturing extra detail by virtue of the fact that the light reaching the sensor is no longer being filtered through three different colors. Obviously, for those of us who want color photographs, this is not ideal. Manufacturers typically use a slightly less brutal solution: blurring the image so that dense, fine patterns are eliminated before the light hits the sensor — effectively reducing the resolution of the image. This is achieved by placing an anti-aliasing (AA) filter (also called an optical low pass filter) in front of the Bayer filter, sacrificing a little detail in order to avoid moirĂ©. Of course, placing an AA filter behind an ultra-sharp lens that cost a lot of money is not ideal, either. Some manufacturers now produce alternative versions of their high-end, high-resolution cameras that do not feature the AA filter for photographers who wish to capture as much detail as possible, even if that means risking moirĂ©.

2012 saw Fujifilm make a bold design move. For several of its new cameras, it chose to get rid of this AA filter, claiming to have found an ingenious solution to the moiré problem: X-Trans.

As opposed to the two-by-two repeating pattern of the Bayer filter, the X-Trans filter is far more complex: a repeating pattern of six-by-six. This is still a long way from the random arrangement of the receptors in the human eye, but the increased periodicity means that the patterns which trigger moiré in Bayer filter cameras are less of a problem for X-Trans.

The more random-like pixel arrangement becomes more obvious when we look at, for instance, only the red pixels. If we hide the green and blue pixels, the gaps between the red pixels become more apparent. Bayer on the left, X-Trans on the right.

Note that with X-Trans, any given row or column is capable of “seeing” all three colors. By contrast, an individual row or column on a Bayer filter is always missing either a red or blue pixel:

Every row and every column on the X-Trans sensor (left) contains all three colors — even on the rows that have the fewest red and blue pixels. By contrast, on the Bayer sensor, every row and every column is always missing either red or blue pixels.

For regular patterns consisting of either only horizontal or only vertical high frequencies, this gives X-Trans a real advantage. In this instance, it actually avoids the gaps in the red and blue channels. We have seen above how these gaps can cause moiré, and X-Trans is definitely less prone to color moiré than Bayer as a result.

However, there is no such thing as a free lunch and the advantages of X-Trans bring with them certain disadvantages. Across its six-by-six base, only 8 pixels are red and only 8 are blue, while the same sized area of a Bayer sensor would have 9 of each. As a result, the X-Trans sensor is about 11% less sensitive to both red and blue. More of a factor, however, is that pixels of the same color can be further apart on the X-Trans filter compared to Bayer. On a Bayer filter, a red pixel is never more than one pixel’s width away from another red pixel. By contrast, on an X-Trans filter, it can be twice that.

On irregular textures and details as they typically occur in nature (foliage, flowers, faces, etc), the regular grid of the Bayer pattern does no harm, and X-Trans brings no benefits. By contrast, an X-Trans sensor’s lower density of red and blue pixels means that it will observe slightly less color detail. If a small color spot happens to fall in that fairly huge block of four green pixels, it is simply not observed at all.

Bayer Versus X-Trans, Pepsi Versus Coke?

As you may have noticed, discussing the finer details of camera technology can trigger some strong reactions, and debates over the advantages and disadvantages of X-Trans have played out on message boards and social media networks since it was launched. However, the color filter array is only one feature out of many that define a camera and few customers would have it foremost in mind when buying a camera. Fujifilm photographers appreciate the distinctive look and feel of their cameras, enjoying the ergonomics and ease of use, not to mention the results that can be achieved straight out of camera. Fujifilm’s engineers are experts when it comes to color following more than 70 years of experience in creating color photographs, and devotees love Fujifilm’s film simulations which draw on the company’s rich history of producing film stock. Names such as Astia and Velvia give their cameras a sense of authenticity.

The Bayer versus X-Trans argument has parallels with the ongoing battle between ARM processors and those designed by Intel. Apple’s marketing department will claim their iPad is better because it features their new ARM chip, while Microsoft wants you to believe that Surface is better because it uses the latest Intel chip. This allows aficionados of both brands to spend their nights in heated forum discussions about ARM vs Intel, RISC vs CISC — but 99% of all users don’t actually care. They chose their tablet because they prefer the look and feel of one over the other, the user experience, and ultimately, the branding.

Beyond some technical differences deep inside the system, for most users, the Fujifilm versus Canon/Nikon/Sony/etc decision might be not so different from choosing between Pepsi and Coke.

The Consequences of Complexity

Whatever the criteria for choosing a camera, squeezing the best possible performance out of its sensor is still important, and it’s useful to see what happens when you zoom into images at 100% and more.

It’s worth remembering that the final image is not solely the result of the sensor itself; whether it’s in-camera or through software, a variety of processes take place, notably demosaicing, the algorithm which fills in the gaps between the red, green, and blue channels. It’s a combination of these algorithms with the color filter array that determines the result.

One of the reasons that the Bayer filter has proven so tenacious is that engineers are used to processing its data. Finding the best recipe for demosaicing Bayer sensors has kept researchers busy for four decades, and the highly sophisticated algorithms developed over the years have allowed them to mitigate many of the limitations fundamental to its design. Even fairly simple algorithms, such as those embedded in the first digital cameras, yield fairly good results.

The increased complexity of the X-Trans pattern, on the other hand, entails a demosaicing process that is far more elaborate. Fujifilm’s engineers are said to have spent five years waiting for their cameras’ processing power to catch up before introducing X-Trans in the X-Pro1 in 2012. At the same time, the research community has published far fewer papers about X-Trans demosaicing than they have for Bayer; not only is it a more complex problem but less research effort has been spent solving it. It seems fair to assume that current X-Trans demosaicing algorithms are still some distance away from achieving a theoretically optimal solution. This is one reason that Fujifilm enthusiasts often find themselves jumping between software packages in search of a solution that delivers the best results.

Overcoming Complexity Through Machine Learning

At DxO, we have attempted to create better X-Trans processing in the past, but we were never quite satisfied with the outcome. The main challenge was that our processing traditionally took a different route to every other software: while most raw processors demosaic a RAW file before denoising, DxO always did it the other way around — one of the reasons that our software often produces cleaner results. As a consequence, adapting our RAW processes to cater for X-Trans sensors would not only have required a new demosaicing process, but also a new denoising process. Applying RAW denoising to X-Trans was again incredibly complicated compared to Bayer and rebuilding the process never yielded truly satisfactory results.

Today, image processing is being revolutionized by machine learning, particularly by a technology called convolutional neural networks. Within a few years, this new class of algorithms — no longer hand-crafted by researchers and engineers, but learned empirically by a computer from looking at millions of training examples — has made decades of research effort obsolete. With Bayer demosaicing, for instance, neural networks now easily beat the very best algorithms designed by humans.

While certainly frustrating for researchers who spent their life on demosaicing algorithms, this revolution is actually a huge opportunity. Not only are the results better, it also boosts productivity: computers find a state-of-the-art demosaicing algorithm in days or weeks rather than in years or decades. Machine learning is particularly well adapted to problems that have clearly defined inputs and expected outputs but where the mapping between them is too complex to be formulated as a classical algorithm. Image and speech recognition were the first examples but machine learning turns out to be such a powerful tool that it has proven useful in domains where decent classical algorithms already existed — such as demosaicing.

X-Trans demosaicing is a great candidate for machine learning. Being more complex than Bayer demosaicing, the advantage of machine learning over traditional engineering should be even greater than that achieved with Bayer demosaicing. Our counterparts at Adobe demonstrated exactly this when they introduced their machine learning-powered “Enhance Details” feature in early 2020. Reviewers concluded that, while the difference for Bayer images was rather subtle, it was a significant improvement for X-Trans images.

At DxO, we leveraged machine learning in DxO PhotoLab to solve another highly complex task: our RAW conversion technology — DxO DeepPRIME and DeepPRIME XD — uses a single, huge convolutional neural network to apply demosaicing and denoising at the same time. After 10 days of intensive work, our computer developed a highly sophisticated algorithm that outperforms our traditional demosaicing at low ISO, and both our traditional demosaicing and denoising at high ISO.

DxO PhotoLab 6 and DxO PureRAW  3 Feature DxO DeepPRIME and DeepPRIME XD for X-Trans

Of course, those 10 days of training our neural network were preceded by years of research. We had to define the “shape” of the network manually while the computer only determined its millions of parameters. But the most challenging part, on which we spent 80% of our effort, was ensuring that the problem was being described to the deep convolutional neural network as accurately as possible through the use of very precise training data.

Once work on Bayer sensor images was complete, it became clear that making changes to accommodate X-Trans raw files was no longer such a daunting task because the procedure for generating training data could be reused with very few modifications. There were still obstacles to overcome as we had to fundamentally change the network shape to accommodate for the complex X-Trans pattern, but it was conceivable and proved to be an exciting challenge. The results are exciting, too. Let’s have a look at two examples.

This landscape shot was taken with a Fujifilm X-T2 at ISO 200. The image looks reasonably good in terms of color and exposure, even without post-processing (top). However, when zooming in and examining the details — details that become important when making large prints — we notice that DxO DeepPRIME (using DxO PhotoLab 6, bottom right) does a much better job at preserving color details than the camera itself (bottom left). The camera fails to distinguish the varying hues of human skin, the wooden fences, and the grass; all end up being more or less uniformly greenish. By contrast, DxO DeepPRIME manages to distinguish these features, producing a more natural image. It also preserves the texture of the grass in the foreground more effectively. As a result, the image appears to be at a higher resolution and will reproduce better when printed in a large format.

This low light indoor action shot was taken with a Fujifilm X-T3 at ISO 6400. The original photo was underexposed so we pushed it by two stops during post-processing — the equivalent of ISO 25600 (top). Such heavy exposure adjustment cannot be performed on JPEG images, so the comparison here is not with the camera but against a well known raw converter: Adobe Lightroom with Enhance Details (bottom left). When we look at the faces more closely, we can see that DxO DeepPRIME (using DxO PhotoLab, bottom right) yields a significantly cleaner result. Because it uses a neural network to run demosaicing and denoising at the same time, DeepPRIME does a better job at removing noise while at the same time preserving more detail in terms of both luminance and color.

More than just machine learning

Obviously, it takes more than DxO DeepPRIME on its own to fully support X-Trans in software as sophisticated as DxO PhotoLab and DxO PureRAW  3. Many internal tools used by our lab to calibrate the color and noise model of each camera body had to be adapted. Several other processing blocks had to be designed from scratch, such as the demosaicing algorithm used to display a preview while the user makes adjustments.

Ready for your photos

After an intense period of research and development, both DxO PhotoLab 6 and DxO PureRAW  3 are now ready to bring dramatic improvements to your RAW files. We believe that photographers will appreciate how our DxO DeepPRIME and DeepPRIME XD technology can draw out color detail that was previously missing, bring new life to old photographs, and transform high ISO images. Download a free trial and discover what DxO DeepPRIME can do for your photos.


The post Support arrives for Fujifilm X-Trans sensors appeared first on DxO.

]]>
DxO’s new ultra-wide color space is unique https://www.dxo.com/news/white-paper-wide-gamut/ Thu, 02 Feb 2023 11:00:00 +0000 https://www.dxo.com/news/white-paper-wide-gamut/ For DxO PhotoLab 6, we created our most versatile and flexible color space yet. Here’s how it works and why it’s better for your photos.

The post DxO’s new ultra-wide color space is unique appeared first on DxO.

]]>

How the new DxO Wide Gamut helps PhotoLab 6 deliver truer colors and better images

For DxO PhotoLab 6, we created our most versatile and flexible color space yet. Here’s how it works and why it’s better for your photos.

Color is one of the most important aspects of any picture and the capacity to reproduce and adjust color to match the photographer's intent is fundamental to any image-editing software. Essentially, accurate color is the foundation on which you edit.

To make better adjustments and provide more intense and lifelike results, DxO PhotoLab 6 brings three major advances in terms of the way it handles color.

  • At its heart is a new working color space providing a much wider gamut for accurate reproduction of saturated colors and more headroom for color adjustments like hue, saturation, and luminance.
  • The algorithms that manage colors as they pass through your image-editing workflow have been completely reengineered so that they deliver better results in conjunction with the new working color space.
  • Finally, a new Soft Proofing mode ensures that the colors and tones that you see on your monitor match what comes out of your printer, or what is displayed on your other devices.

Together, not only do these advances give extra headroom for boosting colors and let you make the most of the latest generation of monitors, but they also allow you to accurately reproduce images throughout your entire workflow — from the second that you press the shutter button to the moment you hang your print on the wall.

From capture, through editing, to printing and sharing, PhotoLab 6 is a revolution in color.

Get to grips with color science lingo

To understand why we created a new wide gamut color space – and why it’s so effective – it’s important to have a working knowledge of color science. Below are some of the main components. If you’re comfortable with color science and just want to know more about how PhotoLab 6 handles color, just click here to skip to that section. Link to below

Color

Color is a sensation that the human eye perceives as a result of light being reflected by an object or emitted by a source. More precisely, this perception occurs as various parts of the spectrum of light interact with the three different types of cone photoreceptor cells on our retina. This stimulation is converted into an electrical signal which we interpret as color.

If all wavelengths present similar levels of intensity, we perceive achromatic colors such as gray or white, but when certain parts of the spectrum dominate, it is perceived as color. When more energy is concentrated in a small part of the spectrum, a color will appear more saturated and intense.

Surface colors

As we’ve already set out, color is not a literal property of objects – it’s a sensation we perceive, resulting from the interaction between reflected light and the cone cells in our retinas. Additionally, the colors we perceive depend on both the incident light (illumination) and the physical characteristics of the objects (reflectivity, fluorescence, etc) we’re viewing. Nevertheless, it makes sense to characterize objects in terms of the color we perceive when looking at them under standard conditions (e.g. bright daylight). We call those surface colors.

Surface colors are only a subset of all the colors that humans can perceive — e.g., they don't include spectral colors. For example, the pure green and blue of the monitor on which you are reading this article cannot be observed on real-world objects. But surface colors are typically what we care about in photography, because we want to reproduce those ‘real world’ colors in prints or on screens.


Spectral colors

If the light emitted from a source consists only of a single wavelength — i.e., a single spectral value — it is perceived as one, highly intense color. These spectral colors do not exist naturally but some can be produced by lasers.


Color space

Color space is the term we use for the organization of colors. That organization results in a defined range and uses a mathematical model to represent colors. Color spaces typically organize colors along three axes, which ties in with the human eye’s three different types of cones (those which are responsive to red, green, and blue light).

The most common family of color spaces is therefore RGB where colors are created through a blend of three primary colors: red, green, and blue. But these primaries are different for each device. For example, different screens use different chemical components in their color filters. To counter this, there are standard RGB color spaces such as sRGB, Adobe RGB, and Display P3, each of which defines its own distinct set of primaries.


XYZ and CIE color spaces

There are also color spaces that are defined independently of any device, such as the XYZ space designed in 1931 by the International Commission on Illumination (abbreviated to CIE from its French name, Commission Internationale de l'Éclairage). This space attempts to model color perception along three axes: X, Y, and Z.

The illustration below shows the spectral power distribution of the light reflected by two different objects — a piece of paper and a tomato – when the objects are illuminated under daylight. Each graph also indicates the corresponding surface color expressed as XYZ values. To the human eye, the paper is perceived as white, while the tomato is perceived as red.

Fig 1: The light reflected by two objects, shown in graph form.

Gamut

A gamut is the range of colors that can be reproduced by a given monitor or printer, or the colors that can be represented within a certain color space. For printing, gamut depends on both the inks and the paper being used. The larger the gamut, the more colors a device can reproduce.

If you are soft proofing an image using a color profile for a printer, your software might notify you that certain colors in your image simply cannot be reproduced by the printer — they are “out of gamut.” See below for more on soft proofing and why it’s important.

Wide gamut

Wide gamut is a loose term that describes any device or color space that can reproduce more colors than average. Since most of today’s monitors use the relatively small sRGB color space, any color space exceeding the gamut of sRGB tends to be called “wide gamut” by the relevant marketing team. Not all wide gamut devices are equal, however; the term can describe color spaces that are slightly larger than sRGB through to those that are far, far bigger.

Chromaticity

Chromaticity is a measurement that distinguishes different colors of equal luminance. It’s therefore commonly used in charts that illustrate color space. Several color spaces organize color so that one axis represents luminance, and the other two axes represent chromaticity, for instance in the form of hue and colorfulness.

The CIE has derived a chromaticity diagram from their XYZ color space where colors are represented along two axes, x and y. This diagram is popular for comparing color gamuts, mainly because it can be presented more easily than a three-dimensional chart.

Fig 2: Comparison of the AdobeRGB and Display P3 color spaces in the CIE XYZ color space (left) and CIE chromaticity diagram (right). Notice how, although AdobeRGB and Display P3 are of similar size on the chart, neither is a subset of the other – AdobeRGB contains more cyan, while Display P3 contains more red and yellow.

Color profile / ICC profile

A color profile – also called an ICC profile because the format was standardized by the International Color Consortium – characterizes the color space of a specific device, typically a monitor or a printer. They are normally supplied by the vendor or can be created by the user by measuring the device, for instance by using a colorimeter.

Color management

Color management is the process of converting colors from one color space to another while preserving the perception. Typically, when displaying an image on your calibrated screen that has been saved in a standard color space such as AdobeRGB, the color management system will convert the colors from AdobeRGB into XYZ (called the “profile connection space”) and then from there to the color space defined by the color profile of your screen.

Sensor native color

Sensor native color is what results from the interaction between light and the sensor of a digital camera. Strictly speaking, it’s not “color” because color is a human perception, but since camera sensors used in photography are designed to perceive color in a similar way to the human eye, the digital values they produce resemble those colors.

RAW files contain pixel values in sensor native color, so converting RAW files into finished photographs involves converting the former into actual color. The parameters required for this conversion are typically provided either by the image file’s metadata or directly by the RAW converter itself.

Because the sensor only approximates human photoreceptor cells, this conversion is also an approximation. Its precision can be improved by taking the light source into account. That is why at DxO we calibrate these parameters for every camera for both daylight and tungsten lighting.


Working color space

The working color space is internal to software, and not something you’ll experience visually when you edit an image. It’s the space within which the processing takes place. To be displayed on a monitor, the image needs to be converted from the working color space to the color space of the monitor.

In DxO PhotoLab, denoising and lens corrections are performed within sensor native color in order to achieve the best results. However, the image is then converted into our working color space, where red, green, and blue are defined according to human perception, rather than the color filters of the sensor. This ensures that editing tools such as HSL sliders, ColorWheel, or ClearView Plus are consistent, regardless of which camera has been used to take an image.

Output color space

Finally, the output color space defines how you save your images and is typically governed by what monitors can display. The most common is sRGB, and this is the standard for the World Wide Web, which uses the primaries as defined by the television standard Rec. 709. Other typical output color spaces include AdobeRGB, which is widely used in the prepress community, and Display P3, which is used for the displays in Apple’s recent desktop and notebook computers.

Fig 3: The different color spaces used in DxO’s RAW conversion pipeline

Why DxO PhotoLab 6 has moved to a wide gamut

Wide gamut monitors can display more vivid color than those with a standard gamut like sRGB. Whether this is useful depends on the content of an image, because under normal lighting conditions, even objects that we perceive as very colorful – for example red tomatoes or a blue sky — fit within sRGB.

However, there are a lot of colors that do not fit into sRGB. These are usually encountered on artificial objects such as brightly colored sportswear or from artificial lighting such as laser stage lights. On a wide-gamut monitor, these colors can be reproduced more accurately than on a regular monitor.

Fig 4: Comparison between wide gamut and sRGB monitors. Notice how, due to some of the values being out of gamut on the sRGB monitor, detail in the poppies’ petals is lost.

To fully exploit the capacity of a monitor, photo editing software should use a working color space with a gamut which at least matches that of the monitor. When we created our first RAW converter almost two decades ago, it was safe to assume that monitors would be either sRGB or – for the high-end, color-critical models – AdobeRGB. Choosing AdobeRGB as our working color space seemed to cover all needs, so that is what we did.

Since then, technology has evolved and monitors have improved. With Display P3 monitors used in recent Apple computers, their native red is “redder” than the “reddest red” that DxO PhotoLab 5 could produce. In order to simulate pure AdobeRGB red on such a monitor, the color management system must dilute it slightly and make it less intense by adding a small amount of blue. The much wider working color space of DxO PhotoLab 6 — which comprises both AdobeRGB and Display P3 — solves this and can produce pure, native color on such a display.

The same applies to printing. Certain printers and printing services can produce colors that are outside of AdobeRGB, and DxO PhotoLab 6 allows you to harness their full potential.

Fig 5: The gamut of WhiteWall’s Ultra HD photo prints (in color) in different RGB color spaces (in gray): sRGB (left), AdobeRGB (center) and DxO’s new working color space (right).

At the other end of the imaging workflow is the camera. Camera sensors do not actually have a gamut. Instead, they’re sensitive to every wavelength in the visible part of the spectrum and high-end sensors only differ from low-end models in that they better approximate the spectral sensitivity of the human eye. Thus, every color in a scene can be observed and recorded in the sensor native color space.

However, when converting from sensor native color into a working color space, as you do when developing RAW files, it may happen that a color cannot be represented. Essentially it has fallen outside of the working color space’s gamut. Having a working color space with a wider gamut therefore allows us to preserve more colors, just as they were recorded by a camera’s sensor. In combination with a wide gamut monitor and printer, the scene can then be captured, processed, and reproduced without losing its original intensity.

Finally, working in a wider color space gives photographers more headroom for adjusting the color in their images. For example, PhotoLab’s ClearView Plus tool can produce certain colors that do not fit within AdobeRGB. But with DxO Wide Gamut they are preserved. You can therefore use the ColorWheel or a Control Point to desaturate these colors, and bring them back into gamut.

The problem of ‘clamping’ out-of-gamut colors

What does falling outside the color gamut mean precisely? Let’s start by going back to the idea of color values.

The simplest way to describe out-of-gamut colors and how they are managed is to think in terms of 8-bit images. In an 8-bit image, each of the red, green, and blue pixel values can range from 0 to 255. 255/0/0 would be the reddest possible red, while 128/128/128 is mid-gray.

Mathematically, a color would be out of gamut if at least one of the three RGB components had negative values. But obviously, this wouldn’t make sense as a monitor cannot emit negative light. A color could also be out of gamut if some of the values exceed the maximum. That, again, is not technically possible as a monitor cannot display values brighter than its limit.

One way of handling out-of-gamut colors is to simply clamp them to the closest allowed values, for example, setting them to 0 if they’re below the low limit, or to 255 if they’re above it. This is what many color management systems do, but they can produce unwanted results.

What do we mean by unwanted results? This ‘clamping’, whereby one of the RGB components is altered while keeping the others unchanged, means altering the hue. A more sophisticated method involves preserving the hue while accepting a reduction in saturation, and this generally yields better results. Unfortunately, even this approach can cause some problems. For instance, textures flatten as the contrasting color within those areas falls completely out of gamut.

How DxO’s reimagined color processing fixes the problem

For DxO PhotoLab 6, we’ve worked to ensure that all of the luminance details captured by the sensor are maintained throughout your workflow. For the best possible quality, our reengineered algorithm is designed to act in two stages: first when converting from sensor native color to working color, and then when converting from working color to output color.

As the image moves from sensor native color to working color, in order to avoid losing any of the details originally captured, the algorithm smartly analyzes the colors in each image and then desaturates – only if necessary – highly saturated colors by a small amount. This applies even to those inside the gamut, and is done in order to make headroom for those outside the gamut. Thanks to this algorithm, we can therefore produce images that contain all luminance details that were captured by the sensor — and although they appear less colorful than in the original scene, all of the tonality and detail is maintained.

The first stage (Protect saturated colors in the Color Rendering palette) has been reworked and improved compared to PhotoLab 5, the second stage (Protect color details’ in the Soft Proofing palette) is entirely new.

Fig 6: Comparison when converting to sRGB from original image

How DxO PhotoLab 6’s Soft Proofing mode keeps colors consistent for output

Most of the time, photographers use wide-gamut monitors which, in combination with software such as DxO PhotoLab 6, allow accurate reproduction of most of the colors contained in images. But when it comes to sharing images, either online or as physical prints, these output media have different gamuts that are typically a lot smaller.

A smaller gamut means that colors can look different between what you see on your monitor and what you get in print, or after exporting to other devices. Those changes in color also mean that delicate textures can be lost. Wouldn’t it be better to take that output gamut into account during editing? This is where soft proofing comes into play.

Soft proofing allows photographers to get an on-screen simulation of what an image will look like when displayed or printed on a certain device. It gives an overview of the outcome by emulating the less saturated primaries of a standard screen, or the inks of the printer and the way they physically react with paper.

The conversion properties are embedded in specific color profiles created for each combination of printers/inks/papers and are usually provided by printing services, device manufacturers, or are created for personal printers.

Once downloaded and installed, users can select a specific profile to be used as a soft proofing base, and after activating the option in their application, can adapt their color adjustments according to the displayed results in order to achieve the desired image. This can include adjusting color casts, or contrast and luminance issues in areas such as shadows or highlights.

Though it cannot completely replace a hardcopy proof, soft proofing is crucial for saving time and money that would otherwise be wasted in the trial and error of getting a print acceptably close to the original image.

However, soft proofing isn’t a free pass to perfect output. It's important to remember that soft proofing mode, as with any settings dedicated to color accuracy, requires editing on a calibrated monitor and in a consistent viewing environment.

Fig 7: DxO Wide Gamut color space and out-of-gamut colors

How we designed our new working color space

As explained, the working color space used by a RAW converter determines the colors it can produce. The more, the better — correct?

At the same time, as its name suggests, the working color space should allow you to work with color and manipulate it in an intuitive way. For example, turning up the blue saturation in the DxO ColorWheel should produce some color that you perceive as very blue.

If it was only about reproducing color in the scene, the best working color space would be XYZ, as it contains every color that can be perceived by the human eye. However, while this color space was designed to express the color stimuli generated on our retina, it does not follow our perception of color as effectively as when using RGB color spaces. Deploying XYZ as a working color space would make our color adjustment tools behave strangely and make photo editing very challenging.

Among the RGB color spaces used today, the one that stands out is ProPhoto RGB. It covers a very large percentage of colors, and certainly all that are useful in practice. It achieves this through a trick: it uses imaginary colors for its blue and green primaries — points in the XYZ space that lie beyond spectral colors. Mathematically, this trick allows ProPhoto RGB to obtain certain spectral colors as a blend of the ProPhoto RGB primaries. The downside is that pure blue or pure green in this color space do not correspond to anything that physically exists or that any human has ever perceived.

When experimenting with our new algorithms for DxO PhotoLab 6 we found that even when you turn all saturation sliders up to 11, it is more intuitive to obtain results that tend toward spectral colors — and not beyond.

The diagram below shows a comparison of the hue and saturation between sRGB and ProPhoto RGB. The circles show all of the RGB values contained in each color space. The looping line symbolizes spectral colors, from a wavelength of 380 nanometers through to 750 nanometers — the most saturated colors that exist.

The gray dots represent all of the surface colors that can be observed in the real world. Since there are no surfaces that reflect light at a single wavelength, these surface colors are far less saturated than spectral colors. The sRGB color space on the left does not contain any spectral colors and you can see that some surface colors also lie outside of this color space.

The ProPhoto RGB color space on the right is far larger and easily contains all of the surface colors. However, while every RGB value in sRGB corresponds to some color, part of ProPhoto RGB lies outside of the spectral colors and corresponds to something that doesn’t exist. While fully saturated magenta, red, yellow, and cyan correspond to actual colors, fully saturated green and blue correspond to imaginary colors. This can make ProPhotoRGB counterintuitive when it comes to editing photographs.

For this reason we decided to design an RGB color space with the widest possible gamut that can be achieved utilizing spectral colors as primaries. The result is a color space that includes close to every color that can be reproduced on the best monitors and printers available today, and encompasses all of Pointer’s Gamut, the 4089 real-world surface color samples collected by scientist Doctor Michael R. Pointer at Kodak Research in 1980. link to https://onlinelibrary.wiley.com/doi/abs/10.1002/col.5080050308

Fig 9: DxO Wide Gamut (the green triangle) encompasses every possible color that a photographer might encounter in nature

The DxO PhotoLab 6 working color space uses spectral colors as its primaries. It is big enough to contain all real-world surface colors, and it achieves this without imaginary colors — i.e., every combination of R, G, and B in this color space represents an actual color.

Fig 10: DxO Wide Gamut (DPL6) vs AdobeRGB (DPL5 and older) vs sRGB, DisplayP3, and ProPhoto

DxO Wide Gamut: An intelligent compromise

We believe that this color space, which is quite similar to the television standard Rec. 2020, provides the best possible trade-off between preserving as much color as needed and allowing users to manipulate color in a way that feels natural and intuitive. Combined with our gamut-squeezing algorithm and soft proofing tools, it allows photographers to reproduce any color they may encounter, as closely as possible to the original, without ever losing details.


DxO PhotoLab 9

RAW photo editing at its finest

The post DxO’s new ultra-wide color space is unique appeared first on DxO.

]]>