I finally found the culprit of the crash: the thread executing LibRaw ran out of stack space! I observed LibRaw crashing internally when I single step into the actual decoder for DNG.
By replacing "LibRaw rawProcessor;" with "LibRaw *rawProcessor = new LibRaw;", the crash went away :)
output_color sets output color space (0 - no conversion, 1 - convert to sRGB, etc)
If you use same camera color profile (e.g built-in into LibRaw, or use_camera_matrix and camera matrix does not change between shots), it is safe to use some conversion.
What about "imgdata.params.output_color" ? I noticed it is producing different outputs if I set it to 0 (RAW) or 1 (sRGB). I would like the output to be linear as I am already setting gamm[0] = gamm[1] = 1, but looks like this parameter is override some options..
This will result into full build infrastructure sharing:
- other libraries used (jpeg, xml2)
- complete build environment share to make sure your app compiled with same compiler/same settings to ensure same offsets in libraw structures
I don't think it is the file, because when use_rawspeed = 0 , LibRaw has no problem loading this file.
Would you be so kindly post links to the libraw_r.a and librawspeed.a compiled for macOS so I can download them to try out? PM me if it is not convenient to do it in the forum post.
camera matrix should be the same unless changed by camera based on temperature or self-calibration (may be some high-end digital backs?)
camera WB will change from shot to shot, indeed.
So, you may get camera multiplicators for the first shot in sequence and re-use it for next shots (user_mul).
Same with black level: it may change from shot to shot (e.g. Panasonic or Canon cameras), so get it for first shot then reuse (user_black, user_cblack)
But you have successfully used RawSpeed in FastRawViewer under the Mac, right? I'm a registered user of FastRawViewer, and my DNG file doesn't crash it.
After half a day of playing around with it, I still get a crash whenever a DNG file is decoded.
Below is how I compile librawspeed and libraw (this is under macOS):
For RawSpeed:
- Unpack the rawspeed-master.zip file
- Copy it to the same folder that stores LibRaw-x.xx.x
- Rename it to RawSpeed
- Copy the patch files from LibRaw/RawSpeed to RawSpeed/RawSpeed directory
- Run patch < rawspeed.cpucount-unix.patch
- Edit Common.cpp and cut and paste the the function int rawspeed_get_number_of_processor_cores() to the end of the file, outside of the #elif
- Run patch < rawspeed.qmake-pro-files.patch
- Run qmake -makefile rawspeed-lib.pro
- Edit DngDecoderSlices, find the line with jpeg_read_header(&info, TRUE). Cast TRUE with (boolean)
- Edit RawDecoderThread.h, change 'uint32 taskNo' to 'int32 taskNo'
- Edit the Makefile. Change -mmacosx-version-min= to the version you want (such as 10.9)
- Run make
For LibRaw:
- Edit Makefile.dist
- Uncomment the two lines under '# RawSpeed Support'
- Add -mmacosx-version-min=xx.x to CFLAGS+=
- Uncomment the lines 'Demosaic Pack GPL2 and GPL3'
- Run make -f Makefile.dist
There were some warnings when compiling RawSpeed. The most notable one is bit shifting a negative number. This is undefined by the gcc compiler. This warning is in LJpegDecompressor.cpp. I tried changing -32768 to 0b1000000000000000, but the crash persists.
I found that sometime I can decode the DNG file and have it displayed on the screen. But my app crashes just by sitting there idle for a few seconds. Other times, my app crashes right at the line "rawProcessor.unpack()". It seems that the code in RawSpeed in decoding DNG file is corrupting some memory locations, and when some background garbage collection occurs, the app crashed.
As soon as I set 'imgdata.params.use_rawspeed = 0', DNG files are decoded correctly without any crash at all.
Perhaps you have fixed something in the master branch of RawSpeed to allow successful decoding of DNG files?
what 'iteration 6' mean in this context?
Hi, I'm getting the following warning from gcc-6.4.0 in libraw_cpp.cxx:
* /tmp/portage/portage/media-libs/libraw-0.18.5/work/LibRaw-0.18.5/src/libraw_cxx.cpp:1784:64: warning: iteration 6 invokes undefined behavior [-Waggressive-loop-optimizations]
CFLAGS="-O2 -march=native -ftree-vectorize -mprefer-avx128 -mvzeroupper -fstack-protector -ftree-loop-distribution -ftree-loop-distribute-patterns -pipe"
Code:
static inline void unpack28bytesto16x16ns(unsigned char *src, unsigned short *dest)
{
dest[0] = (src[3] << 6) | (src[2] >> 2);
dest[1] = ((src[2] & 0x3) << 12) | (src[1] << 4) | (src[0] >> 4);
dest[2] = (src[0] & 0xf) << 10 | (src[7] << 2) | (src[6] >> 6);
dest[3] = ((src[6] & 0x3f) << 8) | src[5];
dest[4] = (src[4] << 6) | (src[11] >> 2);
dest[5] = ((src[11] & 0x3) << 12) | (src[10] << 4) | (src[9] >> 4);
dest[6] = (src[9] & 0xf) << 10 | (src[8] << 2) | (src[15] >> 6);
dest[7] = ((src[15] & 0x3f) << 8) | src[14];
dest[8] = (src[13] << 6) | (src[12] >> 2);
dest[9] = ((src[12] & 0x3) << 12) | (src[19] << 4) | (src[18] >> 4);
dest[10] = (src[18] & 0xf) << 10 | (src[17] << 2) | (src[16] >> 6);
dest[11] = ((src[16] & 0x3f) << 8) | src[23]; ## line 1784
dest[12] = (src[22] << 6) | (src[21] >> 2);
dest[13] = ((src[21] & 0x3) << 12) | (src[20] << 4) | (src[27] >> 4);
dest[14] = (src[27] & 0xf) << 10 | (src[26] << 2) | (src[25] >> 6);
dest[15] = ((src[25] & 0x3f) << 8) | src[24];
}
use samples/unprocessed_raw.cpp as a starting point
Thanks for the quick response.
I found the answer to my question. It's very simple. For L *, use gamm [0] = 1/3 and gamm [1] = 9.033.
With best wishes, Konstantin
Output curve is used only on output phase (dcraw_make_mem_image or dcraw_ppm_tiff_writer)
So, you may modify source and replace calls to gamma_curve(...) by own output curve creation.
No direct way in LibRaw to do that, we'll consider to add some interface for user-defined curve in 0.19 release.
all postprocessing is done at dcraw_process() step.
Unpack only unpacks (uncompress for compressed formats) raw data and stores it without any modification
You mean demosaic the X-Trans CFA is done at the dcraw_process() step?
unpack only unpacks raw data ('as is')
interpolation is performed on dcraw_process() step
To use bilinear set
half_size to 0
user_qual to 0
So how do I specify bilinear filter?
When I execute unpack(), does it perform demosaic of the X-Trans CFA automatically?
Use bilinear. It is fast enough (but not perfect too)
Like the output of FastRawViewer?
What is criteria for 'best'?
Glad to hear.
Yes, LibRaw object is large
I finally found the culprit of the crash: the thread executing LibRaw ran out of stack space! I observed LibRaw crashing internally when I single step into the actual decoder for DNG.
By replacing "LibRaw rawProcessor;" with "LibRaw *rawProcessor = new LibRaw;", the crash went away :)
output_color sets output color space (0 - no conversion, 1 - convert to sRGB, etc)
If you use same camera color profile (e.g built-in into LibRaw, or use_camera_matrix and camera matrix does not change between shots), it is safe to use some conversion.
Thank you Alex,
What about "imgdata.params.output_color" ? I noticed it is producing different outputs if I set it to 0 (RAW) or 1 (sRGB). I would like the output to be linear as I am already setting gamm[0] = gamm[1] = 1, but looks like this parameter is override some options..
This will result into full build infrastructure sharing:
- other libraries used (jpeg, xml2)
- complete build environment share to make sure your app compiled with same compiler/same settings to ensure same offsets in libraw structures
Looks impossible.
I don't think it is the file, because when use_rawspeed = 0 , LibRaw has no problem loading this file.
Would you be so kindly post links to the libraw_r.a and librawspeed.a compiled for macOS so I can download them to try out? PM me if it is not convenient to do it in the forum post.
Thanks!
camera matrix should be the same unless changed by camera based on temperature or self-calibration (may be some high-end digital backs?)
camera WB will change from shot to shot, indeed.
So, you may get camera multiplicators for the first shot in sequence and re-use it for next shots (user_mul).
Same with black level: it may change from shot to shot (e.g. Panasonic or Canon cameras), so get it for first shot then reuse (user_black, user_cblack)
Also, set adjust_maximum_thr to 0.f
I do not see any problems with this file. Tested with RawDigger under both Windows and OS X.
But you have successfully used RawSpeed in FastRawViewer under the Mac, right? I'm a registered user of FastRawViewer, and my DNG file doesn't crash it.
But anyway, here is the link to the DNG file:
https://www.dropbox.com/s/zg6785bm9err9gf/DJI_0329.DNG?dl=0
Thanks.
LibRaw team is not responsible for RawSpeed bugs.
Meanwhile, if you provide sample file that crashes RawSpeed I'll take a look (under Windows/MSVC 2010 because it my main development platform).
After half a day of playing around with it, I still get a crash whenever a DNG file is decoded.
Below is how I compile librawspeed and libraw (this is under macOS):
For RawSpeed:
- Unpack the rawspeed-master.zip file
- Copy it to the same folder that stores LibRaw-x.xx.x
- Rename it to RawSpeed
- Copy the patch files from LibRaw/RawSpeed to RawSpeed/RawSpeed directory
- Run patch < rawspeed.cpucount-unix.patch
- Edit Common.cpp and cut and paste the the function int rawspeed_get_number_of_processor_cores() to the end of the file, outside of the #elif
- Run patch < rawspeed.qmake-pro-files.patch
- Run qmake -makefile rawspeed-lib.pro
- Edit DngDecoderSlices, find the line with jpeg_read_header(&info, TRUE). Cast TRUE with (boolean)
- Edit RawDecoderThread.h, change 'uint32 taskNo' to 'int32 taskNo'
- Edit the Makefile. Change -mmacosx-version-min= to the version you want (such as 10.9)
- Run make
For LibRaw:
- Edit Makefile.dist
- Uncomment the two lines under '# RawSpeed Support'
- Add -mmacosx-version-min=xx.x to CFLAGS+=
- Uncomment the lines 'Demosaic Pack GPL2 and GPL3'
- Run make -f Makefile.dist
There were some warnings when compiling RawSpeed. The most notable one is bit shifting a negative number. This is undefined by the gcc compiler. This warning is in LJpegDecompressor.cpp. I tried changing -32768 to 0b1000000000000000, but the crash persists.
I found that sometime I can decode the DNG file and have it displayed on the screen. But my app crashes just by sitting there idle for a few seconds. Other times, my app crashes right at the line "rawProcessor.unpack()". It seems that the code in RawSpeed in decoding DNG file is corrupting some memory locations, and when some background garbage collection occurs, the app crashed.
As soon as I set 'imgdata.params.use_rawspeed = 0', DNG files are decoded correctly without any crash at all.
Perhaps you have fixed something in the master branch of RawSpeed to allow successful decoding of DNG files?
I reverted the patch for rawspeed.uncompressed-color-dng.patch, then RawSpeed no longer crashes on DNG files.
But now it crashes on decoding Olympus ORF files. So something is definitely wrong...
I just experienced crashes when I decoded DNG and Olympus ORF files with RawSpeed turned on.
I've installed the rawspeed.uncompressed-color-dng.patch. So what else am I missing?
Pages