Hi Alex,
Thank you for the quick reply. I tried it with -c 0. I had the brightness set to 2.3 (leaving it at 1 made everything too dark). I did not see any difference using -c 0. Some images came out darker than the result from DCRaw 9.27, some came out lighter and one came out very similar. Obviously, changing the brightness level would change which was most similar. For most files, the version from DCRaw 9.27 (with brightness set to 1) was pretty consistent with the brightness level of the embedded JPEG from the RAW files (the version from DCRaw_emu was not).
Any other thoughts? I'd really like to use your version because of its other benefits but this complicates it.
NOTE: I did not see - c mentioned in your documentation. In the DCRAW 9.27 documentation, it says that -c is used to: "Write decoded images or thumbnails to standard output."
Yes, this is bug in open_bayer() code, the standard tone curve (color.curve[] array) was not initialized to linear curve, but it is used in 8-bit decoder.
Just add these lines:
for (int i = 0; i < 0x10000; i++)
curve[i] = i;
To void CLASS initdata() (anywhere in the function body). This call is in
* dcraw/dcraw.c if you're using snapshot (and re-generate script/makefile)
* internal/dcraw_common.cpp (if you're do not regenerate this file from LibRaw's dcraw.c).
(you may patch both source files if you're not sure, if re-generate script is used, than dcraw_common.cpp to be generated from dcraw.c if needed)
I found a workaround to make LibRaw accept the file (and other 8bit files from the same camera)... I store each 8bit pixel in memory as if it were a 16bit pixel, then have LibRaw output 8bits. I cannot keep using this because it is exactly twice the amount of IO that should be done!
Hope it's an easy fix
P.S. might be helpful -
This one does not have 255-level pixels at the beginning, they are quite dark in the upper left.
It is in the "RGGB" RGRGRG.../GBGBGB.../ Bayer pattern
LibRaw contains 'automatic adjust maximum' code to not get 'pink clouds' problem when maximum data level reported by camera is higher than real data maximum (this is common problem with canon files esp. if intermediate iso are used).
To disable this feature use -c 0 option of dcraw_emu (imgdata.params.adjust_maximum_thr = 0 if you use LibRaw directly from your code, not dcraw_emu.exe)
We do not have Sinar 4-shot anfd 16-shot samples, so we cannot test this support (and hope dcraw-derived code works right).
Could you please provide several 4-shot and 16-shot samples for testing (use any file sharing service and send link to info@libraw.org or post link in comments here)
Sorry for delayed publishing of this topic, summer is vacation time even for libraw.org moderator :)
Could you please provide image sample to test it with. open_bayer should work with 8-bit images, but it looks like it needs additional testing.
version 0.19 expected to be released this year or in Q1-2019 (we definitely will publish 1-2 snapshots before that). Snapshots are used in our applications (RawDigger, FastRawViewer) and stable for production use.
To get the same Temp/Tint as in ACR you you need to do:
1. CameraNeutral[3] = {G1/R,1,G1/B}; // R, G1, B are WB mul
2. use NeutralToXY from dng_sdk_1_4\dng_sdk\source\dng_color_spec.cpp
3. use Set_xy_coord from dng_sdk_1_4\dng_sdk\source\dng_temperature.cpp to get Temp/Tint
I rewrote the code and got the same result as in ACR. But there is a trouble. I have to get CameraCalibration matrixf for RAW. Another matrix I goto from Adobe dcp files (from DNG Converter).
You may disable automatic brightening and use manual exposure shift (exp_correc, exp_shift, exp_preser parameters). The exact amount of shift will differ on different cameras/different ISO values (see, for example, this article: https://photographylife.com/adobes-silent-exposure-compensation )
'not exposed normally' means that graypoint value is below 18% by some amount.
Using no_auto_bright=0 made the resulting image brighter than how the CR2 looks.
The histogram of the image changes. It's generally the same shape, but shifted. What parameters of libraw can i modify to try to keep the histogram in the same place? would modifying the tone curves that libraw uses acheive this?
What do you men by "not exposed normally"? I used 1/10 second exposure time, fNumber 5.6, ISO 160.
I downloaded both packages again today (0.18.2 and rawspeed-master) and recompiled them from scratch in two separate folder with and without rawspeed. The tiff generated by dcraw_emu.exe with RawSpeed shows a blue sunflower, while the one without RawSpeed shows a proper yellow sunflower.
I did not change anything in the configurations except whatever was needed to get it to compile.
If you use no_auto_bright=1, than any brightening will be disabled, resulting in dark image if your shot is not exposed normally.
With auto_bright, LibRaw will put 1% of pixels (with default options) to saturation.
Today cameras gray point is usually at 12-10-even 7-8% of maximum (so, up to 4 stops from saturation). Raw converters (DPP, ACR, etc) usually use custom tone curves to compress highlights and get gray point back to 18% (so 0.7-1.5EV gray point move is usual)
I originally wrote Rawshack in 2011 but have recently been updating it with new features. It's GPL v3 and runs on Windows, OSX, and Linux. Here's the home page:
Yes, FastRawViewer uses LibRaw (+RawSpeed +Adobe DNG SDK) to decode raw data and its own (post)processing code (written in SSE3/AVX2 assembly for speed, that's why FRV is Fast)
RawDigger uses LibRaw (+libs listed above) for decoding and dcraw_process() (with bilinear 'demosaic', fast but far from perfect), because RGB rendering is only for user reference (we plan to change demosaic to own code in future releases to provide better rendering, because many users use RawDigger as raw processor :).
I'm not familiar with Apple Core Image. It is very likely it will accept (cropped?) imgdata.rawdata.rawimage[] as input to CIRawFilter (this is flat array of unprocessed raw values decoded from camera).
Thanks Alex. Given your reply, am I correct in assuming that applications like FastRawViewer are not using dcraw for processing and are instead using their own algorithms?
On a related note, in Apple's Core Image framework for macOS, they have a filter called CIRawFilter that can take RAW data as input. The header file states that the following formats are supported:
If I want to try and use Core Image for post-processing, is there a way to get the unprocessed image data from LibRaw in any of the formats listed above?
dcraw_process is derived (without any significant changes) from dcraw.c code
It is slow and image processing options are limited.
You may play with dcraw_emu sample and different command line options to get output that suits your needs. Each dcraw_emu command-line parameter translates into LibRaw.imgdata.params value.
-с is, indeed, documented: https://www.libraw.org/docs/Samples-LibRaw.html
-c float-value
This key sets params.adjust_maximum_thr parameter.
Use -c 0 to completely disable automatic maximum calculation.
Default value: 0.75
Could you please provide some samples to test against dcraw/dcraw_emu. With -c 0 it should be the same, at least with default settings.
Hi Alex,
Thank you for the quick reply. I tried it with -c 0. I had the brightness set to 2.3 (leaving it at 1 made everything too dark). I did not see any difference using -c 0. Some images came out darker than the result from DCRaw 9.27, some came out lighter and one came out very similar. Obviously, changing the brightness level would change which was most similar. For most files, the version from DCRaw 9.27 (with brightness set to 1) was pretty consistent with the brightness level of the embedded JPEG from the RAW files (the version from DCRaw_emu was not).
Any other thoughts? I'd really like to use your version because of its other benefits but this complicates it.
NOTE: I did not see - c mentioned in your documentation. In the DCRAW 9.27 documentation, it says that -c is used to: "Write decoded images or thumbnails to standard output."
Thanks for the sample.
Yes, this is bug in open_bayer() code, the standard tone curve (color.curve[] array) was not initialized to linear curve, but it is used in 8-bit decoder.
Just add these lines:
for (int i = 0; i < 0x10000; i++) curve[i] = i;To
void CLASS initdata()(anywhere in the function body). This call is in* dcraw/dcraw.c if you're using snapshot (and re-generate script/makefile)
* internal/dcraw_common.cpp (if you're do not regenerate this file from LibRaw's dcraw.c).
(you may patch both source files if you're not sure, if re-generate script is used, than dcraw_common.cpp to be generated from dcraw.c if needed)
Link to full patch: https://www.dropbox.com/s/xqi4ox7ui4r5cqg/8bit-openbayer.patch?dl=0
(sorry, our forum does not allow attachments to topic comment, it is easier to put it on dropbox than re-tune our forum).
This full patch also patches openbayer_sample.cpp to get correct parameters to this call.
Hi Alex, no worries! Hope it was good vacation time.
Here is my 8bit raw file on Google Drive:
https://drive.google.com/file/d/0B41mEzFmIzubcmxKMnVEcE14MEE/view?usp=sh...
Here it is demosaiced to a .tif using other software (with worse algorithms!):
https://drive.google.com/file/d/0B41mEzFmIzubZW0wMXE3YWo3eFk/view?usp=sh...
I found a workaround to make LibRaw accept the file (and other 8bit files from the same camera)... I store each 8bit pixel in memory as if it were a 16bit pixel, then have LibRaw output 8bits. I cannot keep using this because it is exactly twice the amount of IO that should be done!
Hope it's an easy fix
P.S. might be helpful -
This one does not have 255-level pixels at the beginning, they are quite dark in the upper left.
It is in the "RGGB" RGRGRG.../GBGBGB.../ Bayer pattern
LibRaw contains 'automatic adjust maximum' code to not get 'pink clouds' problem when maximum data level reported by camera is higher than real data maximum (this is common problem with canon files esp. if intermediate iso are used).
To disable this feature use -c 0 option of dcraw_emu (imgdata.params.adjust_maximum_thr = 0 if you use LibRaw directly from your code, not dcraw_emu.exe)
Thank you for your request!!
We do not have Sinar 4-shot anfd 16-shot samples, so we cannot test this support (and hope dcraw-derived code works right).
Could you please provide several 4-shot and 16-shot samples for testing (use any file sharing service and send link to info@libraw.org or post link in comments here)
The file you've shared is only 772415 bytes size (unless I downloaded it wrong, please check too).
The original RAW_CANON_40D_SRAW_V103.CR2 from Rawsamples.ch (same preview, so I'm sure it is the same file) is 6800865 bytes.
Looks like your sample is truncated.,
Sorry for delayed publishing of this topic, summer is vacation time even for libraw.org moderator :)
Could you please provide image sample to test it with. open_bayer should work with 8-bit images, but it looks like it needs additional testing.
version 0.19 expected to be released this year or in Q1-2019 (we definitely will publish 1-2 snapshots before that). Snapshots are used in our applications (RawDigger, FastRawViewer) and stable for production use.
Indeed: https://www.libraw.org/news/libraw-0-18-released
I agree, aligned scanlines are good.
We're open to patches/contribution.
To get the same Temp/Tint as in ACR you you need to do:
1. CameraNeutral[3] = {G1/R,1,G1/B}; // R, G1, B are WB mul
2. use NeutralToXY from dng_sdk_1_4\dng_sdk\source\dng_color_spec.cpp
3. use Set_xy_coord from dng_sdk_1_4\dng_sdk\source\dng_temperature.cpp to get Temp/Tint
I rewrote the code and got the same result as in ACR. But there is a trouble. I have to get CameraCalibration matrixf for RAW. Another matrix I goto from Adobe dcp files (from DNG Converter).
You may disable automatic brightening and use manual exposure shift (exp_correc, exp_shift, exp_preser parameters). The exact amount of shift will differ on different cameras/different ISO values (see, for example, this article: https://photographylife.com/adobes-silent-exposure-compensation )
'not exposed normally' means that graypoint value is below 18% by some amount.
Using no_auto_bright=0 made the resulting image brighter than how the CR2 looks.
The histogram of the image changes. It's generally the same shape, but shifted. What parameters of libraw can i modify to try to keep the histogram in the same place? would modifying the tone curves that libraw uses acheive this?
What do you men by "not exposed normally"? I used 1/10 second exposure time, fNumber 5.6, ISO 160.
I downloaded both packages again today (0.18.2 and rawspeed-master) and recompiled them from scratch in two separate folder with and without rawspeed. The tiff generated by dcraw_emu.exe with RawSpeed shows a blue sunflower, while the one without RawSpeed shows a proper yellow sunflower.
I did not change anything in the configurations except whatever was needed to get it to compile.
Here is a comparison image, from left to right original, with rawspeed and without.
https://www.dropbox.com/s/wlfwna06nf67ktn/rawspeed.JPG?dl=0
I could zip up my working folder and upload it if that would help.
So how should this be fixed? I presume it'll involved detecting if this is a SuperCCD camera and setting a different white level?
If you use no_auto_bright=1, than any brightening will be disabled, resulting in dark image if your shot is not exposed normally.
With auto_bright, LibRaw will put 1% of pixels (with default options) to saturation.
Today cameras gray point is usually at 12-10-even 7-8% of maximum (so, up to 4 stops from saturation). Raw converters (DPP, ACR, etc) usually use custom tone curves to compress highlights and get gray point back to 18% (so 0.7-1.5EV gray point move is usual)
I do not see any problems with RawSpeed and RAW_SAMSUNG_NX300.SRW sample.
What RawSpeed version do you use?
Thank you for the problem report.
This code line should work for old Fuji cameras (SuperCCD), but not for current line (X-Trans and current bayer line)
Looks like ./configure has missed -lc++ library. Try to add it manually to linker options,
I've also tried converting the .CR2 to .TIF using Digital Photo Professional and Irfan View, and the files these create don't lose any brightness.
I originally wrote Rawshack in 2011 but have recently been updating it with new features. It's GPL v3 and runs on Windows, OSX, and Linux. Here's the home page:
http://testcams.com/rawshack/
Thanks for the additional information and openness Alex, much appreciated.
Yes, FastRawViewer uses LibRaw (+RawSpeed +Adobe DNG SDK) to decode raw data and its own (post)processing code (written in SSE3/AVX2 assembly for speed, that's why FRV is Fast)
RawDigger uses LibRaw (+libs listed above) for decoding and dcraw_process() (with bilinear 'demosaic', fast but far from perfect), because RGB rendering is only for user reference (we plan to change demosaic to own code in future releases to provide better rendering, because many users use RawDigger as raw processor :).
I'm not familiar with Apple Core Image. It is very likely it will accept (cropped?) imgdata.rawdata.rawimage[] as input to CIRawFilter (this is flat array of unprocessed raw values decoded from camera).
Thanks Alex. Given your reply, am I correct in assuming that applications like FastRawViewer are not using dcraw for processing and are instead using their own algorithms?
On a related note, in Apple's Core Image framework for macOS, they have a filter called
CIRawFilterthat can take RAW data as input. The header file states that the following formats are supported:If I want to try and use Core Image for post-processing, is there a way to get the unprocessed image data from LibRaw in any of the formats listed above?
dcraw_process is derived (without any significant changes) from dcraw.c code
It is slow and image processing options are limited.
You may play with dcraw_emu sample and different command line options to get output that suits your needs. Each dcraw_emu command-line parameter translates into LibRaw.imgdata.params value.
Pages