Fuji X-Trans files washed out colors in 2021

I am using the release verson 0.20.2 in my lib (librawfx.org). Now I wanted to update to the newest snapshot version 202110. The problem I have now is that all x-trans files have now washed out colors.

Must I call something special for fuji x-trans files to get the right colors ?

With 0.20.2 all is working as expected but with the latest snapshot version not. You can find the example file under https://github.com/lanthale/LibRawFX/blob/main/LibRawFX/src/main/ressour...

Code I am using (sorry for the Java language but basically I am using the C API):

MemoryAddress iprc = org.libraw.linuxosx.libraw_h.libraw_init(0);
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Memory dddress native lib was: " + iprc.toRawLongValue());
                MemorySegment datasegment = org.libraw.linuxosx.libraw_data_t.ofAddress(iprc, scope);
                MemorySegment params$slice = org.libraw.linuxosx.libraw_data_t.params$slice(datasegment);
                org.libraw.linuxosx.libraw_output_params_t.use_camera_wb$set(params$slice, 0);
                org.libraw.linuxosx.libraw_output_params_t.use_auto_wb$set(params$slice, 0);
                org.libraw.linuxosx.libraw_output_params_t.output_tiff$set(params$slice, 0);
                org.libraw.linuxosx.libraw_output_params_t.half_size$set(params$slice, 0);
                org.libraw.linuxosx.libraw_output_params_t.user_qual$set(params$slice, 0);
                //libraw_output_params_t.output_bps$set(params$slice, 8);
                //libraw_output_params_t.output_color$set(params$slice, 0);        
 
                MemorySegment inputStreamBytes = MemorySegment.ofArray(sourceFileAsByteArray);
                MemorySegment allocateNative = SegmentAllocator.ofScope(scope).allocateArray(C_CHAR, sourceFileAsByteArray);
                int k = org.libraw.linuxosx.libraw_h.libraw_open_buffer(iprc, allocateNative, inputStreamBytes.byteSize());
                if (k > 0) {
                    Logger.getLogger(LibrawImage.class.getName()).log(Level.SEVERE, null, "Cannot open stream, return value was: " + k);
                    throw new IOException("Cannot open file stream!");
                }
 
                org.libraw.linuxosx.libraw_h.libraw_unpack(iprc);
 
                MemoryAddress iParams = org.libraw.linuxosx.libraw_h.libraw_get_iparams(iprc);
                MemorySegment iParamsRestricted = org.libraw.linuxosx.libraw_iparams_t.ofAddress(iParams, scope);
                MemorySegment modelSlice = org.libraw.linuxosx.libraw_iparams_t.model$slice(iParamsRestricted);
                cameraModel = new String(modelSlice.toByteArray(), StandardCharsets.US_ASCII);
                MemoryAddress image_other_data = org.libraw.linuxosx.libraw_h.libraw_get_imgother(iprc);
                MemorySegment imageOtherRestricted = org.libraw.linuxosx.libraw_imgother_t.ofAddress(image_other_data, scope);
                long timestamp = org.libraw.linuxosx.libraw_imgother_t.timestamp$get(imageOtherRestricted);
                shootingDateTime = LocalDateTime.ofInstant(Instant.ofEpochSecond(timestamp), TimeZone.getDefault().toZoneId());
 
                org.libraw.linuxosx.libraw_h.libraw_dcraw_process(iprc);
                ByteArrayOutputStream bo = new ByteArrayOutputStream();
 
                MemorySegment errorCode = SegmentAllocator.ofScope(scope).allocate(C_INT.byteSize());
                MemoryAddress mem_image_adr = org.libraw.linuxosx.libraw_h.libraw_dcraw_make_mem_image(iprc, errorCode.address());
                MemorySegment imageMemSegment = org.libraw.linuxosx.libraw_processed_image_t.ofAddress(mem_image_adr, scope);
                MemorySegment data$slice = org.libraw.linuxosx.libraw_processed_image_t.data$slice(imageMemSegment);
                imageWidth = org.libraw.linuxosx.libraw_processed_image_t.width$get(imageMemSegment);
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Native width: " + imageWidth);
                imageHeight = org.libraw.linuxosx.libraw_processed_image_t.height$get(imageMemSegment);
                imageBits = org.libraw.linuxosx.libraw_processed_image_t.bits$get(imageMemSegment);
                imageColors = org.libraw.linuxosx.libraw_processed_image_t.colors$get(imageMemSegment);
                stride = imageWidth * imageColors * (imageBits / 8);
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Start reading image from native memory...");
                byte[] line = new byte[stride];
                for (var i = 0; i < imageHeight; i++) {
                    loader.updateImageProgress(i, imageHeight);
                    MemoryAddress addOffset = data$slice.address().addOffset(stride * i);
                    MemorySegment asSegmentRestricted = addOffset.asSegment(stride, scope);
                    line = asSegmentRestricted.toByteArray();
                    try {
                        bo.write(line);
                    } catch (IOException ex) {
                        Logger.getLogger(LibrawImage.class.getName()).log(Level.SEVERE, "Cannot retrieve image from native memory and write into byte array!", ex);
                        return null;
                    }
                }
                bo.flush();
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Start reading image from native memory...finished");
                byte[] data = bo.toByteArray();
                bo.close();
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Freeing of native memory...");
                org.libraw.linuxosx.libraw_h.libraw_dcraw_clear_mem(mem_image_adr);
                mem_image_adr = null;
                org.libraw.linuxosx.libraw_h.libraw_close(iprc);
                iprc = null;
                Logger.getLogger(LibrawImage.class.getName()).log(Level.FINEST, null, "Freeing of native memory...finished");
                return data;

Forums: 

Tested with dcraw_emu -T -w,

Tested with dcraw_emu -T -w, for both 0.20-stable branch (from github) and for master branch (also from github).
I do not see great color difference here: https://www.dropbox.com/s/vdlbhfyd6e53pu0/Screenshot%202022-01-11%2021.4...

Files produced are also (temp.) stored on dropbox for your inspection: https://www.dropbox.com/sh/ek4rezjhju4xmmx/AACVcj-2X4G1EvI6cpkbpA3xa?dl=0

Please try to reproduce the problem using LibRaw's sample application (dcraw_emu or simple_dcraw). If these applications produces correct results, the problem source is probably in your code or in Java wrapper you use.

-- Alex Tutubalin @LibRaw LLC

I try's the sample

I try's the sample applications and you are right that all is working.

The problem is that the sample applications are not setting any params. As soon as you add my param list the output colors are also washed out.

I am using the following params to speed up decoding of x-trans files (For Canon/Nikon it doesn't matter). Without those parameters the encoding is much slower. But if I set only one parameter the color are washed out independently which parameter I set:
use_camera_wb$set(params$slice, 0);
use_auto_wb$set(params$slice, 0);
output_tiff$set(params$slice, 0);
half_size$set(params$slice, 0);
user_qual$set(params$slice, 0);

I have now found the issue

I have now found the issue which parameter is causing the problem. If one of those parameters below are set the colors are washed out. In 0.20.2 this is not happening.

libraw_output_params_t.output_tiff$set(params$slice, 0);
libraw_output_params_t.half_size$set(params$slice, 0);
libraw_output_params_t.user_qual$set(params$slice, 0);

I do not see color change due

I do not see color change due to user_qual=0 (output_tiff=0, half_size=0 are the default value).

BTW, you set output_color=0 too. This turns off camera color to sRGB (or other known colorspace) conversion and will result in washed colors.

-- Alex Tutubalin @LibRaw LLC

Thank you for your help.

Thank you for your help.
Than I there is another issue related to the Java wrapper - Sorry for that. The output_color=0 was already commented out therefore was never active (leading slashes).

I have now removed the options and use the lib with the default options. After JDK18 I have to generate the sources from the header file again and will see if that helps.

Nevertheless under windows I had troubles with ljpeg-8d which is not compiling. For now I have removed libjpeg from all builds because it is not so important for the users.