Thanks, Alex!
this is really helpful. I will implement your suggestions as soon as possible.
Are these LIBRAW_RAWOPTIONS_DNG_STAGE* flags documented somewhere? I have found them in the source code and see where you apply them, and I have some rough idea about opcodes in DNG files, but I would never have thought that I have to apply them or which of them.
In most cases, the DNG files I have to handle have large-enough embedded previews and that's sufficient for my use-case.
But when it comes to actual processing of RAW data into a usable image, things get a lot more difficult and finding parameters which work across a wide range of RAW files is tough.
The image is linearized via OpcodeList2 tag value; For similar images use of OpcodeList3 is also possible (probably depends on Adobe software version used).
By default, only OpcodeList1 tag is processed by LibRaw
To enable OpcodeList2/3 processing LIBRAW_RAWOPTIONS_DNG_STAGE2_IFPRESENT/LIBRAW_RAWOPTIONS_DNG_STAGE3_IFPRESENT flags should be added to imgdata.rawparams.options
This translates into numeric value of (1<<20)|(1<<21) so 3145728
This is not perfect solution if one do not want to apply OpcodeList2/3 for all DNG files containing these tags.
To improve it: query decoder used via LibRaw::get_decoder_info() and set the flags only for jxl_dng_load_raw_placeholder() and/or lossy_dng_load_raw() decoders.
Please let me know which LibRaw setting or steps I need to perform to get rid of the color cast, if possible.
Or maybe I need to configure the Adobe DNG SDK host instance specifically?
All this is new to me and any help is appreciated.
As mentioned on the this site homepage/LibRaw project description:
Additionally, the LibRaw library offers some basic RAW conversion, intended for cases when such conversion is not the main function of the LibRaw-using application (for example: a viewer for 500+ graphic file formats, including RAW). These methods are inherited from the Dave Coffin’s dcraw.c utility (see below the “Project history” section); their further development is not currently planned
LibRaw RAW postprocessing (sample code) do not apply any custom aspect ratio change, although you may implement your own on processed image.
I have sent an email to the owner of the image and asked if I can provide it to you as an example.
I will post here when I have the permission and provide a download link.
It seems to use the JPXL compression, that's all I can tell so far.
auto balance is based on the idea of a "average gray" world, which is not true for global colored scenes.
If you take a photo of a green grass, then the result of automatic white balance will not satisfy you too.
I didn't know FastRawViewer has it's own processing. That's interesting...
I too was able to achieve that image, but only after i manually switched the wb method.
Anyway, since i cant invest in a whole new post-processing infra at the moment, and considering your comment:
"I do not think that auto-balance is well suited for such images with overall warm tint."
can you suggest a good way to detect said warm tints, so as to switch from auto to camera wb automatically, or is this something only human eye can detect and decern?
Separate user here. To clarify, if I’m interested in having the ProfileGainTableMap processed (so images don’t come out underexposed), would building with the DNG SDK and setting these flags accomplish this? I’m currently trying (and failing) to get a build for this, so it’d be helpful to know before I spend more time getting it to build.
I tried to extract the full image and the left image from an dual pixel cr3 RAW file. I have used dcraw_emu.exe -r 1 1 1 1 -M -o 0 -4 -T -s 0 "example.cr3". With changing the -s from 0 to 1 i get the second image. But the imagesis already debayered. Is it possible to get the non debayered images?
ARW files are really TIFF(like)-containers, so all ARW-specific code is specific tag values parsing/assignment to internal data structures.
The main analysis is common to all tiffs.
>> 1st: R6 Mark II is not supported yet
I know and I mentioned in my original post
I do not doubt that libraw will support R6 Mark II sooner or later that is why I am ready to provide sample files and take part in testing if needed.
>> 2nd: your 2298 file is damaged
Probably it is and there are many other files from my R6m2 damanged in same way.
I mention "probably" because it looks strange:
* other software (DXO PureRaw, Adobe Photoshop, FastStone Image Viewer) works with this file fine
* I took and processed thousands of images from this camera already and I got no issues until I tried to take astro images and load in into Siril
* I have many more samples of random files from the camera that are "damaged" (and could be processed fine with other software).
Sure, these flags are documented in actual documentation provided with LibRaw 202403-snapshot: https://github.com/LibRaw/LibRaw/blob/master/doc/API-datastruct.html#L12...
This site documentation section contains release docs.
We'll probably make these flags (or similar ones) set by default in the next snapshot/release because JPEG-XL compressed DNGs becomes very common.
Thanks, Alex!
this is really helpful. I will implement your suggestions as soon as possible.
Are these LIBRAW_RAWOPTIONS_DNG_STAGE* flags documented somewhere? I have found them in the source code and see where you apply them, and I have some rough idea about opcodes in DNG files, but I would never have thought that I have to apply them or which of them.
In most cases, the DNG files I have to handle have large-enough embedded previews and that's sufficient for my use-case.
But when it comes to actual processing of RAW data into a usable image, things get a lot more difficult and finding parameters which work across a wide range of RAW files is tough.
Thank you for the file sample.
Here is processing results: https://www.dropbox.com/scl/fi/ooqcaseziap0vwo5ario4/2023-11-11-003-Pano...
dcraw_emu command line used for that:
dcraw_emu -dngsdk -R 3145728 -T 2023-11-11 003-Pano.dngTechnical details:
LIBRAW_RAWOPTIONS_DNG_STAGE2_IFPRESENT/LIBRAW_RAWOPTIONS_DNG_STAGE3_IFPRESENTflags should be added toimgdata.rawparams.optionsThis is not perfect solution if one do not want to apply OpcodeList2/3 for all DNG files containing these tags.
To improve it: query decoder used via LibRaw::get_decoder_info() and set the flags only for jxl_dng_load_raw_placeholder() and/or lossy_dng_load_raw() decoders.
I have permission from the image owner and I have uploaded a sample image in a ZIP here:
Sample Image download (12 MB)
Please let me know which LibRaw setting or steps I need to perform to get rid of the color cast, if possible.
Or maybe I need to configure the Adobe DNG SDK host instance specifically?
All this is new to me and any help is appreciated.
You (probably) may receive tag values you want via exif callback: https://www.libraw.org/docs/API-datastruct-eng.html#libraw_callbacks_t and https://www.libraw.org/docs/API-CXX.html#exif
As mentioned on the this site homepage/LibRaw project description:
LibRaw RAW postprocessing (sample code) do not apply any custom aspect ratio change, although you may implement your own on processed image.
I have sent an email to the owner of the image and asked if I can provide it to you as an example.
I will post here when I have the permission and provide a download link.
It seems to use the JPXL compression, that's all I can tell so far.
It is usually pointless to discuss problems in processing a file without having the file itself
It will in some future public update.
You can start with our free preview extractor:
https://www.fastrawviewer.com/RawPreviewExtractor
Forgive me for my ignorance, but can this snapshot be used in Windows to preview RAW files in the folder? How then?
Thanks
ILCE-7RM5 (A7-R5) support is provided by the latest snapshot: https://www.libraw.org/news/libraw-202403-snapshot
auto balance is based on the idea of a "average gray" world, which is not true for global colored scenes.
If you take a photo of a green grass, then the result of automatic white balance will not satisfy you too.
Than you so much for the quick response.
I didn't know FastRawViewer has it's own processing. That's interesting...
I too was able to achieve that image, but only after i manually switched the wb method.
Anyway, since i cant invest in a whole new post-processing infra at the moment, and considering your comment:
can you suggest a good way to detect said warm tints, so as to switch from auto to camera wb automatically, or is this something only human eye can detect and decern?
Also: here is your file processed with dcraw_emu -w -T : https://www.dropbox.com/scl/fi/6gl21svjvs5wy1wzlb4fv/Y-Sigma24-70A-5D3-1...
I do not think that auto-balance is well suited for such images with overall warm tint.
FastRawViewer uses own postprocessing, not LibRaw's
Hello Alex,
sometimes I can't see the wood for the trees - sorry for the obvious mistake on my side :)
Many thanks for pointing it out,
Michael
You need to add LibRaw source files added in last public snapshot to your build.
(src/decoders/sonycc.cpp and src/decoders/pana8.cpp)
DNG SDK Applies ProfileGainTableMap in dng_renderer (RAW to RGB conversion), not in stage3 opcodes.
Separate user here. To clarify, if I’m interested in having the ProfileGainTableMap processed (so images don’t come out underexposed), would building with the DNG SDK and setting these flags accomplish this? I’m currently trying (and failing) to get a build for this, so it’d be helpful to know before I spend more time getting it to build.
To extract unprocessed data use unprocessed_raw sample
I tried to extract the full image and the left image from an dual pixel cr3 RAW file. I have used dcraw_emu.exe -r 1 1 1 1 -M -o 0 -4 -T -s 0 "example.cr3". With changing the -s from 0 to 1 i get the second image. But the imagesis already debayered. Is it possible to get the non debayered images?
What exactly are you doing and how and what are you failing at?
ARW files are really TIFF(like)-containers, so all ARW-specific code is specific tag values parsing/assignment to internal data structures.
The main analysis is common to all tiffs.
Please check with Adobe DNG Converter and select files that processed OK by this program, but damaged in LibRaw.
>> 1st: R6 Mark II is not supported yet
I know and I mentioned in my original post
I do not doubt that libraw will support R6 Mark II sooner or later that is why I am ready to provide sample files and take part in testing if needed.
>> 2nd: your 2298 file is damaged
Probably it is and there are many other files from my R6m2 damanged in same way.
I mention "probably" because it looks strange:
* other software (DXO PureRaw, Adobe Photoshop, FastStone Image Viewer) works with this file fine
* I took and processed thousands of images from this camera already and I got no issues until I tried to take astro images and load in into Siril
* I have many more samples of random files from the camera that are "damaged" (and could be processed fine with other software).
So, let me know if you need file samples.
Pages