I've tested my software on ubuntu 16.04 (with libraw 0.17.1) with no errors.
But in testing the crash seems to occur only with VNG interpolation. (not AHD)
This change affects only Sony files processing, but your sample looks like it canon file (.CR2)
So, two questions:
1)what version of gcc compiler do you use (have you switched to gcc6?)
2) do you really use 0.17.2 or 0.18-stable from github (look in libraw\libraw_version.h)
Sorry,
the crash is repeatable yes. It only occurs with 0.17.2 because I have no problem with 0.17.1. I use the same code for at least one year with no problem or crash. So this is the reason I think it could be a regression.
The backtrace:
#0 0x00007ffff58d8158 in LibRaw::vng_interpolate() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#1 0x00007ffff590f738 in LibRaw::dcraw_process() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#2 0x0000000000432b75 in readraw (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 ) at io/image_formats_libraries.c:747
#3 0x00000000004334d4 in open_raw_files (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 , type=0)
at io/image_formats_libraries.c:900
#4 0x000000000042cb13 in any_to_fits (imagetype=TYPERAW, source=0x1be3850 "/home/lock/Bureau/test.CR2", dest=0x703680 ) at io/conversion.c:797
#5 0x0000000000441cc3 in read_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2", dest=0x703680 , realname_out=0x7fffffffd3e8)
at io/single_image.c:120
#6 0x0000000000441d6e in open_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2") at io/single_image.c:145
#7 0x0000000000447cc7 in opendial () at gui/callbacks.c:1684
#8 0x0000000000447de2 in on_open1_activate (menuitem=0x13f3f80, user_data=0x0) at gui/callbacks.c:1721
#9 0x00007ffff5e91fa5 in g_closure_invoke () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#10 0x00007ffff5ea3fc1 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#11 0x00007ffff5eacd5c in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#12 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#13 0x00007ffff785b0be in gtk_widget_activate () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#14 0x00007ffff7732376 in gtk_menu_shell_activate_item () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#15 0x00007ffff77326ab in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#16 0x00007ffff7716991 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#17 0x00007ffff5e921d4 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#18 0x00007ffff5eac4b8 in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#19 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#20 0x00007ffff7858cbc in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#21 0x00007ffff7713abe in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#22 0x00007ffff7715a42 in gtk_main_do_event () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#23 0x00007ffff724b795 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#24 0x00007ffff7278762 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#25 0x00007ffff5bbb1a7 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#26 0x00007ffff5bbb400 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#27 0x00007ffff5bbb722 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#28 0x00007ffff7714b75 in gtk_main () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#29 0x000000000041d909 in main (argc=1, argv=0x7fffffffe198) at main.c:351
It is Bayer and imgdata.rawdata.raw_image[] only returns nonzero. How can I copy this data to Mat object (OpenCV) Could you provide me an assistant, Please!!
after unpack() call, decoded raw data are placed in
imgdata.rawdata.raw_image[] - for bayer image
imgdata.rawdata.color3_image[3][] - for 3-color images
imgdata.rawdata.color4_image[4][] - for 4-color images
(only one of these pointers is non-zero after unpack() call).
So, you may access unprocessed raw values directly.
Also, if you port code from dcraw.c (or old LibRaw versions), you may use LibRaw::raw2image() call to copy data from above-mentioned arrays to imgdata.image[4][] array
See samples/unprocessed_raw.cpp as a sample of raw_image[] array access.
libraw do not contain any color->BW conversion code.
So, you may choose from two options
* convert interpolated (by libraw) rgb data to BW using your preferred formula
* or implement own raw to BW postprocessing instead of LibRaw::dcraw_process()
If you're processing data from 'converted' camera (initially color, but CFA removed, such as maxmax.com conversion), you may try to set imgdata.idata.colors to 1 after unpack() phase
In first assumption, you're right: there is ADC with fixed bit count, so data range should fit into this range.
But things are more complicated:
1. Some cameras use full pixel capacity at base (lowest) ISO, so data maximum is lower.
See this article: http://www.rawdigger.com/howtouse/rawdigger-histograms-overexposure-shapes
and inspect Panasonic histograms at low iso
(You may also find RawDigger software very useable for your work, to see raw data 'as is')
2. Some cameras alter RAW data in some way (Sony lossy compression mentioned above and many other formats with highlights compression tone curve), so data range is not same as ADC range
2b. Some cameras may subtract 'black level' (bias) before raw values recording, thus resulting in lower maximum values.
2c. Some cameras may clip data below ADC maximum value to avoid ADC non-linearity (many Canon cameras).
hi, Alex
thanks for reply. i am a ISP algorithm engineer. i use libraw for read dng and cr2 files for algorithm tunning.
usually sensor will output raw data in some bit width, such as 10bit, 12bit 14bit in mipi interface. which bit information will be used in my c-model.
as you mentioned, Libraw provides maximum value in imgdata.color.maximum. for 14bit sensor output, i see the maximum value will be 0x3f60, which is less than 0x3fff. so may i need write some codes to detect the highest non-zero bit so that i can know the real bits (14 in this case)in 16 bit data.
anyway, thanks for you reply. libraw is really helpful for me. thanks verymuch
When I run the program through command prompt (*.exe located in the bin folder) it seems to work fine, I can for example load my Raw file and get 4 seperate tiff files using 4channels.exe
The target .exe does exist, I had to change a few properties in the linker so Visual Studio was able to find it. Now when I debug the dcraw_emu.cpp a blank window briefly flashes and I get the following message:
"The program '[10976] dcraw_emu.exe' has exited with code 1 (0x1)."
In Visual Studio Solution Explorer:
- right click on project you want to run on debug (dcraw_emu, etc)
- select 'Set as startup project' in pull-down menu
When calling dcraw_make_mem_image (or even copy_mem_image) the libraw_processed_image_t.data order is [R (0), G (0), B (0), R (1), G (1), B (1)] where (n) is the nth pixel, right? Does this change when the output is 16 bits since a char may only be 8 bits on some platforms? If so, what does the order look like?
Hello.
That is strange indeed.
I use the libraw in debian testing repository : https://tracker.debian.org/pkg/libraw, 0.17.2-4
I've tested my software on ubuntu 16.04 (with libraw 0.17.1) with no errors.
But in testing the crash seems to occur only with VNG interpolation. (not AHD)
My gcc is gcc-6 indeed but same error with gcc-5
Best regards,
Cyril
The only difference between 0.17.1 and 0.17.2 is strncpy usage: https://github.com/LibRaw/LibRaw/commit/c4551c868d6986477071315e9e847de5...
This change affects only Sony files processing, but your sample looks like it canon file (.CR2)
So, two questions:
1)what version of gcc compiler do you use (have you switched to gcc6?)
2) do you really use 0.17.2 or 0.18-stable from github (look in libraw\libraw_version.h)
Sorry,
the crash is repeatable yes. It only occurs with 0.17.2 because I have no problem with 0.17.1. I use the same code for at least one year with no problem or crash. So this is the reason I think it could be a regression.
The file I tested is here: https://www.dropbox.com/s/nfpdqt9cqvmlfu4/test.CR2?dl=0
The backtrace:
#0 0x00007ffff58d8158 in LibRaw::vng_interpolate() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#1 0x00007ffff590f738 in LibRaw::dcraw_process() () from /usr/lib/x86_64-linux-gnu/libraw.so.15
#2 0x0000000000432b75 in readraw (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 ) at io/image_formats_libraries.c:747
#3 0x00000000004334d4 in open_raw_files (name=0x1be3850 "/home/lock/Bureau/test.CR2", fit=0x703680 , type=0)
at io/image_formats_libraries.c:900
#4 0x000000000042cb13 in any_to_fits (imagetype=TYPERAW, source=0x1be3850 "/home/lock/Bureau/test.CR2", dest=0x703680 ) at io/conversion.c:797
#5 0x0000000000441cc3 in read_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2", dest=0x703680 , realname_out=0x7fffffffd3e8)
at io/single_image.c:120
#6 0x0000000000441d6e in open_single_image (filename=0x1c0eaf0 "/home/lock/Bureau/test.CR2") at io/single_image.c:145
#7 0x0000000000447cc7 in opendial () at gui/callbacks.c:1684
#8 0x0000000000447de2 in on_open1_activate (menuitem=0x13f3f80, user_data=0x0) at gui/callbacks.c:1721
#9 0x00007ffff5e91fa5 in g_closure_invoke () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#10 0x00007ffff5ea3fc1 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#11 0x00007ffff5eacd5c in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#12 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#13 0x00007ffff785b0be in gtk_widget_activate () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#14 0x00007ffff7732376 in gtk_menu_shell_activate_item () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#15 0x00007ffff77326ab in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#16 0x00007ffff7716991 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#17 0x00007ffff5e921d4 in ?? () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#18 0x00007ffff5eac4b8 in g_signal_emit_valist () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#19 0x00007ffff5ead08f in g_signal_emit () from /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
#20 0x00007ffff7858cbc in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#21 0x00007ffff7713abe in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#22 0x00007ffff7715a42 in gtk_main_do_event () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#23 0x00007ffff724b795 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#24 0x00007ffff7278762 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#25 0x00007ffff5bbb1a7 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#26 0x00007ffff5bbb400 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#27 0x00007ffff5bbb722 in g_main_loop_run () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#28 0x00007ffff7714b75 in gtk_main () from /usr/lib/x86_64-linux-gnu/libgtk-3.so.0
#29 0x000000000041d909 in main (argc=1, argv=0x7fffffffe198) at main.c:351
Your backtrace is not informative, sorry.
Is the crash repeatable? Is it occurs with libraw own examples too? If both 'yes', could you please share problematic raw file?
Sorry, know nothing about OpenCV
Look into unprocessed_raw.cpp sample
It is Bayer and imgdata.rawdata.raw_image[] only returns nonzero. How can I copy this data to Mat object (OpenCV) Could you provide me an assistant, Please!!
It is implemented in different way:
after unpack() call, decoded raw data are placed in
imgdata.rawdata.raw_image[] - for bayer image
imgdata.rawdata.color3_image[3][] - for 3-color images
imgdata.rawdata.color4_image[4][] - for 4-color images
(only one of these pointers is non-zero after unpack() call).
So, you may access unprocessed raw values directly.
Also, if you port code from dcraw.c (or old LibRaw versions), you may use LibRaw::raw2image() call to copy data from above-mentioned arrays to imgdata.image[4][] array
See samples/unprocessed_raw.cpp as a sample of raw_image[] array access.
Thank you!!
So, libraw doesn't perform -d Document Mode (no color, no interpolation) like dcraw.
Thanks!
libraw do not contain any color->BW conversion code.
So, you may choose from two options
* convert interpolated (by libraw) rgb data to BW using your preferred formula
* or implement own raw to BW postprocessing instead of LibRaw::dcraw_process()
If you're processing data from 'converted' camera (initially color, but CFA removed, such as maxmax.com conversion), you may try to set imgdata.idata.colors to 1 after unpack() phase
For DNG files, linearization curve is applied during unpack(), but no other processing
In first assumption, you're right: there is ADC with fixed bit count, so data range should fit into this range.
But things are more complicated:
1. Some cameras use full pixel capacity at base (lowest) ISO, so data maximum is lower.
See this article: http://www.rawdigger.com/howtouse/rawdigger-histograms-overexposure-shapes
and inspect Panasonic histograms at low iso
(You may also find RawDigger software very useable for your work, to see raw data 'as is')
2. Some cameras alter RAW data in some way (Sony lossy compression mentioned above and many other formats with highlights compression tone curve), so data range is not same as ADC range
2b. Some cameras may subtract 'black level' (bias) before raw values recording, thus resulting in lower maximum values.
2c. Some cameras may clip data below ADC maximum value to avoid ADC non-linearity (many Canon cameras).
Using wrong maximum value in processing will result into false colored highlights: http://blog.lexa.ru/2010/03/28/taina_rozovykh_oblakov.html (sorry, it is in russian, but google translate will do the trick).
hi, Alex
thanks for reply. i am a ISP algorithm engineer. i use libraw for read dng and cr2 files for algorithm tunning.
usually sensor will output raw data in some bit width, such as 10bit, 12bit 14bit in mipi interface. which bit information will be used in my c-model.
as you mentioned, Libraw provides maximum value in imgdata.color.maximum. for 14bit sensor output, i see the maximum value will be 0x3f60, which is less than 0x3fff. so may i need write some codes to detect the highest non-zero bit so that i can know the real bits (14 in this case)in 16 bit data.
anyway, thanks for you reply. libraw is really helpful for me. thanks verymuch
When I run the program through command prompt (*.exe located in the bin folder) it seems to work fine, I can for example load my Raw file and get 4 seperate tiff files using 4channels.exe
Is the program run outside of debugger?
(use CMD window to run it to see 'command line help' output if you run it without raw file on command line)
I've added the breakpoint at the beginning of main() inside dcraw_emu.cpp but I get the same result as before.
try to add breakpoint at beginning of main()
The target .exe does exist, I had to change a few properties in the linker so Visual Studio was able to find it. Now when I debug the dcraw_emu.cpp a blank window briefly flashes and I get the following message:
"The program '[10976] dcraw_emu.exe' has exited with code 1 (0x1)."
Please make sure that target .exe is really exists after build stage.
Yes, that's what I've been trying to do previously but I receive an error that a specified file cannot be found, here's a 30s video showing the error:
https://youtu.be/_zGPEnCnJxc
I really appreciate your help
In Visual Studio Solution Explorer:
- right click on project you want to run on debug (dcraw_emu, etc)
- select 'Set as startup project' in pull-down menu
Yes I understand, but how would I go about running the examples (i.e. 4channels.exe) in Visual Studio after installing the library?
DLL file is a (shared) library, it cannot be 'started'
There is no such things as a 'bit depth':
Imagine Sony ARW2 format (described in details here: http://www.rawdigger.com/howtouse/sony-craw-arw2-posterization-detection ):
- local 7-bit lossy storage
- 11-bit non-linear after lossy (de)compression
- 0..17204 data range after linearization curve (so, 'more than 14bits'
Also, advertized as 14-bit ADC (Sony A7 series), but data gapped even in shadows).
So, what bitdepth?
LibRaw provides data range (maximum value) in imgdata.color.maximum
copy_mem_image() is called from dcraw_make_mem_image() with bgr parameter set to 0.
So, the order is always RGB
When calling dcraw_make_mem_image (or even copy_mem_image) the libraw_processed_image_t.data order is [R (0), G (0), B (0), R (1), G (1), B (1)] where (n) is the nth pixel, right? Does this change when the output is 16 bits since a char may only be 8 bits on some platforms? If so, what does the order look like?
Pages