[AI] Fix color shift in AI denoise on wide-gamut working profiles

[andriiryzhkov]

Problem

When AI denoise was applied to images in a wide-gamut working profile (Rec.2020, ProPhoto, etc.), the output had visible hue shifts – most noticeable as pink backgrounds shifting to teal/green. The denoise model was clearly working, but the colors came out wrong.

Root cause: the denoise model is trained on sRGB-primaried image data, but the pipeline was feeding it the working-profile RGB numbers directly with only a gamma-curve conversion. A pixel that means "saturated pink" in Rec.2020 primaries is interpreted as a different color when treated as sRGB. The model denoises that wrong color, and we convert the result back to Rec.2020 – producing a shifted hue.

A secondary issue: pixels outside the sRGB gamut (negative channel values, or values > 1.0 – common in wide-gamut workflows) were being clamped by the model to [0, 1], losing wide-gamut headroom and out-of-gamut color information entirely.

Fix

Two complementary changes in dt_restore_run_patch():

1. Primary conversion: when a working profile is set, convert pixels from working-profile linear RGB to sRGB linear via a 3×3 matrix before applying the sRGB gamma. After inference, invert the gamma and apply the inverse matrix to bring the result back to the working profile. This eliminates the hue shift for in-gamut pixels.

2. Per-pixel pass-through: during input conversion, record which pixels fall inside the sRGB [0, 1] range (cached as a 1-byte-per-pixel mask). On output, in-gamut pixels use the model's denoised result; out-of-gamut pixels pass through unchanged from the original working-profile input. Wide-gamut colors are preserved exactly – they're just not denoised.

The matrices are composed once per image at context setup using darktable's existing primaries/whitepoint helpers, so per-pixel cost is one matrix multiply on input plus one (conditional) on output.

Additional UI changes

• New profile combo box in the "output parameters" section, mirroring the standard export module. Lets the user pick an output color profile for the denoised TIFF, defaulting to "image settings" (working profile).
• Inline info note on the preview area when a wide-gamut output is selected:
  • Denoise: wide-gamut preserved, not denoised
  • Upscale: wide-gamut clipped (upscale has no pixel correspondence so pass-through doesn't apply)

Notes & trade-offs

On the test images I tried, no significant color shift remains, though slight differences are still possible since the model touches every in-gamut pixel.

On images with a large wide-gamut region, noise in those areas may pass through unchanged – they're preserved exactly, but not denoised. That's the explicit trade-off of pass-through.

There are no publicly available denoise models that handle wide-gamut input natively, so this is the best we can do with an sRGB-trained model in the loop.

[KarlMagnusLarsson]

I get specs using the PR that I do not get in git master in the denoised output. EDIT: in the example attached it is like green brown blobs at 100%.

I suppose this is expected with the new algorithm, and the dark brown/green background is technically out of sRGB (Rec.709) gamut, but a lot of ordinary dark and bright color hues are. You can go outside sRGB gamut by lowering exposure as well. Could this not be an issue?

The picture used is not extreme in any case, it is quite muted and when it comes to color.

I am using this raw file Canon CR3: https://www.dpreview.com/sample-galleries/3976759896/canon-eos-r5-mark-ii-sample-gallery/5840436413 [dpreview sample gallery]

Test: Import CR3 raw file into darktable. Run neural restore -> denoise at 50%

This PR

Without PR (git master)

Gamut check using sRGB (Web safe) as soft proof profile

[andriiryzhkov]

@KarlMagnusLarsson :

Could this not be an issue?

What issue?

[KarlMagnusLarsson]

@KarlMagnusLarsson :

Could this not be an issue?

What issue?

The fact that many ordinary looking, somewhat underexposed dark pictures might have a large fraction of out of sRGB pixels. These areas will not be denoised. This might come across as unexpected, intuitively. The example picture I used does not come across as out of sRGB gamut, when it comes to the forest background, but it is.

Besides that, I have tested on more pictures and I find that the PR handles out of sRGB gamut well. I can not detect any color shifts.

Logically the handling in this PR is the best one can do. I do not have a better answer at this stage.

What can be done to the case I am pointing out, of somewhat dark muted color areas that are actually out of sRGB and therefore will not be denosed? A tradeoff, or choice, or parameter, between color fidelity and denoising of the entire frame?

[andriiryzhkov]

I think this decisions to be made per image. I added notification about wide-gamut in preview. I think the only thing we can do to reduce out of sRGB pixels is to denoise in the sRGB. But that would be extreme case.

[da-phil]

I'm not sure if this issue is also related to the gamut handling, but I'm seeing some interesting colors going on in noisy areas, in this case a noisy sky image with stars behind clouds.
This is with the default settings for plugins/lighttable/neural_restore/detail_recovery_bands and a detail recovery of 0%. Detail recovery of 100% gives slightly better results.

Original image area:

After denoising:

Another area.

Original:

Denoised:

Notice also the higher contrast. The darker shadow areas almost dropped to black.
Is it possible that the posterization is caused by numeric effects in the model, due to using small floating point representations which are prone to underflow/overflow saturation effects?

[andriiryzhkov]

Is it possible that the posterization is caused by numeric effects in the model, due to using small floating point representations

What "small floating point representations" mean? Denoise model is full precision F32.

[da-phil]

Is it possible that the posterization is caused by numeric effects in the model, due to using small floating point representations

What "small floating point representations" mean? Denoise model is full precision F32.

Sorry, this was just a desperate attempt to find an easy explanation for this issue 😅

[KarlMagnusLarsson]

The algorithm in this PR leaves a lot of noise in out of sRGB gamut color areas, because it is designed to do that.

I got good results with many of my raw files using the git master algorithm, even if it has all the color fidelity issues listed in the problem statement of this PR. I could manage the color issues.

Can we select algorithms in the module UI or similar between this PR and current GIT master?

New algo in this PR

Git master:

[andriiryzhkov]

A few updates since the last round of feedback:

• Per-image opt-out: added a "preserve wide-gamut" checkbox in the output
  parameters (commit 4b6503e). Off = denoise everything (wide-gamut clipped to
  sRGB, matches master's behavior for affected pixels). On = pass-through
  preserves wide-gamut colors. @KarlMagnusLarsson's it addresses your ask for a
  per-image color-fidelity vs. denoise-coverage knob.

• Tighter gamut margin (commit 04faead): pixels within ~1% of the sRGB
  cube still go through the model instead of being passed through. Shrinks the
  pass-through region enough that far fewer borderline-noise pixels survive as
  specks.

• Luminance-only smoothing of pass-through specks (commit 6dd154c): for
  each pass-through pixel, a 5×5 window of denoised in-gamut neighbors is used
  to compute a local luminance delta, and that delta is added uniformly to R,
  G, B. Chroma is preserved exactly (wide-gamut colors unchanged), but the
  single-pixel brightness specks that were visible against the denoised
  background are gone.

• NAFNet as an alternative denoise model (commit ce53f27): added NAFNet
  alongside NIND so users can compare and pick what works better for their
  images. Getting NAFNet usable required fixing its tendency to crush dark
  areas into posterized black – a sqrt→square shadow boost now wraps inference
  for models that opt in via a new per-model attributes facility in the
  registry, and it's gated per image (only kicks in when a meaningful fraction
  of pixels are in deep shadow).

  Note: NAFNet requires the matching model-side change in
  Enable NAFNet denoise model darktable-ai#19 – @TurboGit could you take a look when you
  have a moment?

[da-phil]

I gave the latest changes a try, but the images still have the same issues as shown in my earlier post.
I'm curious on trying out the NAFNet model as soon as I can, I assume the darktable-ai PR needs to be merged first?

@andriiryzhkov if you cannot reproduce such an issue on images which are available to you, I'm also happy to share an image for you to reproduce.

[andriiryzhkov]

I assume the darktable-ai PR needs to be merged first?

Yes

if you cannot reproduce such an issue on images which are available to you, I'm also happy to share an image for you to reproduce

Slight color and contract changes are possible. I don't see any major problems in my tests, at least nothing that I can not fix after denosing. You can share your RAW images, if you want me to specifically analyse your situation.

[TurboGit]

Thanks, I'll merge this to ensure we can do more field testing about the noise in some part of the image.

[da-phil]

I assume the darktable-ai PR needs to be merged first?

Yes

if you cannot reproduce such an issue on images which are available to you, I'm also happy to share an image for you to reproduce

Slight color and contract changes are possible. I don't see any major problems in my test, at list nothing that I can not fix after denosing. You can share your RAW images, if you want me to specifically analyze your situation.

My bad, maybe I didn't read carefully enough, the problem with the posterization effect and artifacts are due to the fact that the denoised image is saved with a linear color profile (Linear Rec2020 RGB), which all my image viewers (don't ask why I used them to asses the images in the first place...) failed to interpret correctly, and only when opening one of those denoised files in gimp I realized that:

The denoised images also looks fine within darktable, except for some minor contrast & hue shifts.

Here are some examples (left: original image, right: denoised image):

So I guess such a result is acceptable due to the current constraints of the model, only being trained on sRGB data.

[KarlMagnusLarsson]

Hello @andriiryzhkov

Thank you for the updates of the PR. I have tested it. EDIT: All using NIND-model.

• Preserve wide-gamut colors (default) works for me, but now I have to use 100% detail recovery to get good results on Mallard bird feather details. Before ("last week-to yesterday") 50% was enough.
• Preserve wide-gamut colors (default) does not produce same amount of blobs/noise as before.
• If I remove Preserve wide-gamut colors tick mark = all pixels denoised then I get a cap to sRGB gamut (or very nearly). The darkroom toggle gamut check detects some out of gamut pixels (relative to sRGB), but I can not see any difference with or without sRGB softproof in darkroom.
• (I sometimes get black frames as denoised output. If I retry it works. I can not yet reproduce this in a reliable way.)

I tested with this raw file (Canon CR3): https://drive.google.com/file/d/1JU9srElIri_l19l7eQ6DvqjtuFQDlbNw/view?usp=drive_link

Default darktable import -> add to library + color balance RGB -> Basic colorfilnes -> standard (to push the green color of the Mallard out of sRGB).

The first series shows the impact of detail recovery. The 100% render of the head of the Mallard goes from crisp (rawfile, no denoise) to patchy at 0% detail recovery and get progressively better to 100%. Previously I used 50%. This is no longer enough.

Original RAW 100% (no denoise, ISO = 1600)

Preserve wide-gamut colors (default) 0% detail recovery [Suddenly very patchy rendering?]

Preserve wide-gamut colors (default) 50% detail recovery [I used to use 50%, no longer enough]

Preserve wide-gamut colors (default) 100% detail recovery [I like this result the best]

The second series shows that the Preserve wide-gamut colors = OFF caps gamut to sRGB (the last frame in series).

Original RAW 100% (no denoise, ISO = 1600) Gamut Check Toggle = on for sRGB. The Mallard's head is out of sRGB gamut.

Preserve wide-gamut colors (default) 100% detail recovery Gamut Check Toggle = on for sRGB. The out of sRGB gammut is still there.

Preserve wide-gamut colors = OFF 100% detail recovery Gamut Check Toggle = on. The gamut check shows some out of gamut relative to sRGB, but I can not detect it when I toggle sRGB soft proof on/off.

Something has changed with 'original git master algorithm', that is the algorithm before this PR. I used to get all wide gamut info through. Now I can not. I have not stepped backed or bisected though.