This patch supports Canon and Olympus cameras, more is on the way.
Camera/Sensor themperatures are in (added) imgdata.other.*Temperature fields (in Celsius)
These fields are inited to -1000 at open_file() start, so check values against >-1000 (or >= -273 :) before use.
Yes, RawSpeed is LGPL2 licensed, that means (at least, in common interpretation) that you need it as separate DLL (or LibRaw+RawSpeed as single DLL).
lossless_jpeg_load_raw() in LibRaw/dcraw is completely different from RawSpeeds. It handles all ljpeg formats at once, while RawSpeed has separate code (subclassing/using own common decoder) for different vendors. LibRaw+RawSpeed combo works perfect now (although we've blacklisted some cameras in LibRaw::unpack()), so why do backport?
'RawSpeed2' testing is in progress now (just managed to get it working under Windows), so I expect support in next snapshot/release.
Sensor temp: could you please provide your patch to backport into LibRaw (my E-mail: lexa@lexa.ru)
Looks like I would need to include RawSpeed as a DLL, right? Has anybody ever tried to replace the DCRaw CR2 decompressor (lossless_jpeg_load_raw(), I guess) by the one from RawSpeed?
Another thing: I noticed that I couldn't readout the Canon sensor temperature (which is important for long exposure dark subtraction) by use of set_exifparser_handler(), because the sensor temp appears in parse_makernote(), where the EXIF callback is not called. For the moment I solved this by inserting this callback in parse_makernote() in dcraw_common. May I suggest to either do so too in future versions, or to readout the sensor temperature to imgdata.makernotes.canon?
We've reworked imgdata.thumbnails.tcolors field
- correct handling for bitmap (based on SubIFD SamplesPerPixel field)
- if compiled with JPEG, jpeg header parsed for color count too (so for BW JPEG previews this field is correct).
I have now ported my code to work with LibRaw and most things worked fine instantly.
Up to now there have been only 2 issues I'd like to report:
1.) I had to
#define _WINSOCK2API_
before
#include "libraw\libraw.h"
in order to prevent winsock.h and winsock2.h being included both. (Looks like winsock.h is included somewhere else in one of my headers, but I couldn't locate where.)
2.) On calling convert_to_rgb() right after scale_colors() and pre_interpolate() an exception was thrown because the histogram buffer was not allocated.
I'm no going to build LibRaw_static.lib (with VS 2017). Thereby, I would like to include RawSpeed, since I often work with Canon CR2 Raws. Is there a recipe how to do this on Windows?
Agree, it would be better to initialize this field into correct value, at least if preview IFD contains SamplesPerPixel tag.
I need to collect more samples (at least color DNG + BW preview) to implement it in correct way (unpack_thumb assumes 3 colors for PPM bitmaps, this may be not correct and will result into read beyond EOF).
Although most functions in LibRaw/internal/dcraw_common.cpp looks like untaltered dcraw.c code, this is not completely true.
Data layout in LibRaw is different and old (dcraw's) variables are #define-d in internal/var_defines.h
You may try this approach:
1) Subclass LibRaw:
class MyLibRaw: public LibRaw....
2) Implement own dcraw_process() (with different function name) and build it using dcraw/libraw functions (pre_interpolate(), scale_colors(), etc)
Thank you for the answer.
The size of the photo is correct - it has low resolution. I found that FastStone Image Viewer and IrfanView can open this file. Why LibRaw cannot do it?
Thanks Alex. Looking forward to playing around with this as I'm working on a project that needed support for filtering images by the usual metadata fields. Much appreciated.
You do not need adjust_sizes....() if you plan (or use) unpack()/dcraw_process() calls.
These calls (generally, dcraw_process()) will adjust image sizes correctly.
adjust_sizes_info_only() call is made specifically for metadata-parsing applications (take look into LibRaw/samples/raw-identify.cpp sample, this sample prints A LOT of metadata if called with -v switch)
Yeah, the OpenImageIO implementation doesn't offer too much flexibility, but that's understandable. Maybe for RAW files it makes sense for me to just call libraw directly.
This code path is being using while importing a large number of images into a database, so it will only ever need the metadata. Later on, when the image needs to be decoded and presented to the user, an entirely different code path is taking so the issues with adjust_sizes_info_only() shouldn't really be a problem. I'll make that call after open_file() when loading metadata but I'll make it after unpack() when preparing the image for presentation.
OpenImageIO::RawInput::open() do all things at one time: open, unpack, than process.
Looks like you do not need this, if you need metadata only.
LibRaw::open_file() will read all metadata, including image sizes (also, EXIF, makernotes, all).
It is really fast (fraction of second or less on SSDs)
But:
image sizes may not be completely correct if aspect ratio is not 1.0
To corect this, use asjust_sizes_info_only() call.
The bad thing that adjust_sizes_info_only() may prevent unpack()/dcraw_process() from working correctly (because sizes to be adjusted another time).
So, if you plan to call unpack() later, you'll need open_file() again.
Thanks for the quick reply. If you take a look at the implementation in OpenImageIO you'll see that they call unpack() immediately after calling open_file():
From your comments above, it sounds like I could start reading the metadata immediately after open_file() and not call unpack() at all. Lower down, on line 285, there's a comment that indicates where the metadata reading code begins:
Is all of that code "safe" to call after only calling open_file()? Will valid values, if present, be correctly read? The last call to read_tiff_metadata() looks like it's just using standard OpenImageIO routines to read a TIFF header, I assume that would be OK as well?
I do not know what OpenMP is. My project didn't use it :)
Thank you for quick fix!
Konrad
Thank you for sharing your analysis results.
Could you please note:
- was OpenMP used, or not?
- if yes, what is computer CPU/core count?
Here is the patch: https://www.dropbox.com/s/6ku6scc9ar33nr0/canon-oly-temperature.patch?dl=0
(against 0.19-20170812 snapshot, https://www.libraw.org/news/libraw-0-19-snapshot-20170812)
This patch supports Canon and Olympus cameras, more is on the way.
Camera/Sensor themperatures are in (added) imgdata.other.*Temperature fields (in Celsius)
These fields are inited to -1000 at open_file() start, so check values against >-1000 (or >= -273 :) before use.
RawSpeed2 support in next snapshot/release sounds good - I'll wait for it.
Many thanks, Erwin
Yes, RawSpeed is LGPL2 licensed, that means (at least, in common interpretation) that you need it as separate DLL (or LibRaw+RawSpeed as single DLL).
lossless_jpeg_load_raw() in LibRaw/dcraw is completely different from RawSpeeds. It handles all ljpeg formats at once, while RawSpeed has separate code (subclassing/using own common decoder) for different vendors. LibRaw+RawSpeed combo works perfect now (although we've blacklisted some cameras in LibRaw::unpack()), so why do backport?
'RawSpeed2' testing is in progress now (just managed to get it working under Windows), so I expect support in next snapshot/release.
Sensor temp: could you please provide your patch to backport into LibRaw (my E-mail: lexa@lexa.ru)
Looks like I would need to include RawSpeed as a DLL, right? Has anybody ever tried to replace the DCRaw CR2 decompressor (lossless_jpeg_load_raw(), I guess) by the one from RawSpeed?
Another thing: I noticed that I couldn't readout the Canon sensor temperature (which is important for long exposure dark subtraction) by use of set_exifparser_handler(), because the sensor temp appears in parse_makernote(), where the EXIF callback is not called. For the moment I solved this by inserting this callback in parse_makernote() in dcraw_common. May I suggest to either do so too in future versions, or to readout the sensor temperature to imgdata.makernotes.canon?
LibRaw 0.19-20170812 just published: https://www.libraw.org/news/libraw-0-19-snapshot-20170812
We've reworked imgdata.thumbnails.tcolors field
- correct handling for bitmap (based on SubIFD SamplesPerPixel field)
- if compiled with JPEG, jpeg header parsed for color count too (so for BW JPEG previews this field is correct).
2: yes, histogram is dynamically allocated (in dcraw_process()), to reduce size of LibRaw class sizeof()
RawSpeed:
- use 'master' (very old!) RawSpeed from https://github.com/klauspost/rawspeed/tree/master
(not the current one from darktable, current version does not compile with MSVC)
- Working recipe described in https://github.com/LibRaw/LibRaw/blob/master/README.RawSpeed.txt
I have now ported my code to work with LibRaw and most things worked fine instantly.
Up to now there have been only 2 issues I'd like to report:
1.) I had to
#define _WINSOCK2API_
before
#include "libraw\libraw.h"
in order to prevent winsock.h and winsock2.h being included both. (Looks like winsock.h is included somewhere else in one of my headers, but I couldn't locate where.)
2.) On calling convert_to_rgb() right after scale_colors() and pre_interpolate() an exception was thrown because the histogram buffer was not allocated.
I'm no going to build LibRaw_static.lib (with VS 2017). Thereby, I would like to include RawSpeed, since I often work with Canon CR2 Raws. Is there a recipe how to do this on Windows?
Erwin
Agree, it would be better to initialize this field into correct value, at least if preview IFD contains SamplesPerPixel tag.
I need to collect more samples (at least color DNG + BW preview) to implement it in correct way (unpack_thumb assumes 3 colors for PPM bitmaps, this may be not correct and will result into read beyond EOF).
Expect fix in 0.19-snapshot coming soon.
Currently, no plans to change present data layout.
You can re-use internal/var_defines.h with your code, so it will remain in agreement with new versions even if data fields will change.
Ok, "internal/var_defines.h" exactly is what I was looking for.
Can I presume that the meaning of the defined variables is unchanged?
Thanks a lot, Erwin
To be precise, this is data for last (largest) preview, which is incomplete too.
raw section is completely not present in file (because of truncation)
Thank you for the detailed answer.
Konrad
Although most functions in LibRaw/internal/dcraw_common.cpp looks like untaltered dcraw.c code, this is not completely true.
Data layout in LibRaw is different and old (dcraw's) variables are #define-d in internal/var_defines.h
You may try this approach:
1) Subclass LibRaw:
2) Implement own dcraw_process() (with different function name) and build it using dcraw/libraw functions (pre_interpolate(), scale_colors(), etc)
Followup:
metadata for this file shows:
raw data offset: 531196
raw data size: 944784
Looks strange for file only 700kb in size, so I'm sure the file was truncated.
Both FastStone and IrfanView shows embedded JPEG file. This part of image is correct.
Meanwhile, the file is exactly the same in contents (of JPEG preview) as file at rawsamples.ch. Why do you think this file is correct?
Thank you for the answer.
The size of the photo is correct - it has low resolution. I found that FastStone Image Viewer and IrfanView can open this file. Why LibRaw cannot do it?
Konrad
Understood.
Thanks Alex. Looking forward to playing around with this as I'm working on a project that needed support for filtering images by the usual metadata fields. Much appreciated.
You do not need adjust_sizes....() if you plan (or use) unpack()/dcraw_process() calls.
These calls (generally, dcraw_process()) will adjust image sizes correctly.
adjust_sizes_info_only() call is made specifically for metadata-parsing applications (take look into LibRaw/samples/raw-identify.cpp sample, this sample prints A LOT of metadata if called with -v switch)
Yeah, the OpenImageIO implementation doesn't offer too much flexibility, but that's understandable. Maybe for RAW files it makes sense for me to just call libraw directly.
This code path is being using while importing a large number of images into a database, so it will only ever need the metadata. Later on, when the image needs to be decoded and presented to the user, an entirely different code path is taking so the issues with adjust_sizes_info_only() shouldn't really be a problem. I'll make that call after open_file() when loading metadata but I'll make it after unpack() when preparing the image for presentation.
This is fantastic, thank you.
OpenImageIO::RawInput::open() do all things at one time: open, unpack, than process.
Looks like you do not need this, if you need metadata only.
LibRaw::open_file() will read all metadata, including image sizes (also, EXIF, makernotes, all).
It is really fast (fraction of second or less on SSDs)
But:
image sizes may not be completely correct if aspect ratio is not 1.0
To corect this, use asjust_sizes_info_only() call.
The bad thing that adjust_sizes_info_only() may prevent unpack()/dcraw_process() from working correctly (because sizes to be adjusted another time).
So, if you plan to call unpack() later, you'll need open_file() again.
Alex,
Thanks for the quick reply. If you take a look at the implementation in OpenImageIO you'll see that they call unpack() immediately after calling open_file():
https://github.com/OpenImageIO/oiio/blob/master/src/raw.imageio/rawinput...
From your comments above, it sounds like I could start reading the metadata immediately after open_file() and not call unpack() at all. Lower down, on line 285, there's a comment that indicates where the metadata reading code begins:
https://github.com/OpenImageIO/oiio/blob/master/src/raw.imageio/rawinput...
Is all of that code "safe" to call after only calling open_file()? Will valid values, if present, be correctly read? The last call to read_tiff_metadata() looks like it's just using standard OpenImageIO routines to read a TIFF header, I assume that would be OK as well?
Thanks again!
Followup2:
for non-square-pixel files (very old kodaks an nikons and several other cameras) you need to call adjust_sizes_info_only() call after open_file().
For most current cameras this call can be skipped.
Pages