I've got 3 more images from the photographer. They were all reported as being corrupted and after the patch you sent, libraw works with all of them. Thanks!
Thanks for your email too. That would be useful for the future because some photographers do not want to post their images publicly.
It seems that for rotated images (.imgdata.sizes.flip&4) you need to use height for stride calculation. And keep in mind that you will receive rotated image with width and height swapped.
The only real recommendation is RawSpeed (if you use it): LibRaw is tested with RawSpeed/master branch, not with develop branch.
Other libs are used in very standard way, so you need jpeg8+ (or any replacement with jpeg_mem_src), latest DNG SDK (if you use it), any modern version of zlib
Thanks for the response (apologies if you got my request twice due to me thinking the forum one got lost)
That reminds me, whilst looking at the current changelog in github, I wondered if you have a recommended set of supporting library versions to build against ?
If you'll find another problem(s), please share the files.
You may send it to info@libraw.org (or to personal E-mail lexa@lexa.ru) if you do not wish to expose these files to public.
Looks like we can safely ignore 'not enough data problem' for last line(s) of last block (it contains masked pixels, not needed for data display).
It may be Firmware issue(??)
Here is the patch (for last line of last block only):
diff --git a/src/libraw_xtrans_compressed.cpp b/src/libraw_xtrans_compressed.cpp
index b6555d3..ec6dbb0 100644
--- a/src/libraw_xtrans_compressed.cpp
+++ b/src/libraw_xtrans_compressed.cpp
@@ -46,6 +46,7 @@ struct xtrans_block {
unsigned max_read_size; // Amount of data to be read
int cur_buf_size; // buffer size
uchar *cur_buf; // currently read block
+ bool lastlast;
LibRaw_abstract_datastream *input;
struct int_pair grad_even[3][41]; // tables of gradients
struct int_pair grad_odd[3][41];
@@ -143,7 +144,7 @@ static inline void fuji_fill_buffer(struct xtrans_block *info)
#ifndef LIBRAW_USE_OPENMP
info->input->unlock();
#endif
- if(info->cur_buf_size<1) // nothing read
+ if(info->cur_buf_size<1 && info->max_read_size && !info->lastlast) // nothing read
throw LIBRAW_EXCEPTION_IO_EOF;
info->max_read_size -= info->cur_buf_size;
@@ -159,6 +160,7 @@ void LibRaw::init_xtrans_block(struct xtrans_block* info, const struct xtrans_pa
INT64 fsize = libraw_internal_data.internal_data.input->size();
info->max_read_size = _min(unsigned(fsize-raw_offset),dsize+16); // Data size may be incorrect?
+ info->lastlast = false;
info->input = libraw_internal_data.internal_data.input;
info->linebuf[_R0] = info->linealloc;
for(int i = _R1; i<=_B4;i++)
@@ -668,6 +670,7 @@ void LibRaw::xtrans_decode_strip(const struct xtrans_params* info_common, int cu
ztable[3]= {{_R2,3},{_G2,6},{_B2,3}};
for (cur_line = 0; cur_line < libraw_internal_data.unpacker_data.fuji_total_lines; cur_line++)
{
+ info.lastlast = (cur_block == libraw_internal_data.unpacker_data.fuji_total_blocks-1) && (cur_line ==libraw_internal_data.unpacker_data.fuji_total_lines-1);
xtrans_decode_block(&info, info_common, cur_line);
// copy data from line buffers and advance
I have only one sample. But the photographer that sent me that sample showed me a case where out of 36 images he had, 4 of them appeared corrupted. I'll try to get more samples.
Thank you for sharing the sample.
In this sample
data offset (of raw data) 805904
data size 25251744
sum is 26057648
but file size is 26057632, so 16 bytes missing somewhere
Do you have only one sample, or some camera produces many such files?
If only one, this looks like some data error (camera, card, transfer). If this is systematic error it should be handled in software (e.g. ignore last bytes without error raise)
1) We want to drop this license, so please select from CDDL/LGPL2 (btw, LibRaw license do not cover updates, so in general you need to re-sign when you go from LibRaw 0.17 to 0.18).
2-3) CDDL is very permissive, so yes for both questions
Thanks for claryfying, Alex :). I have a few follow-up questions though.
1.) Is it possible to use the (own) libraw-license for 0.17?
2.) Which licenses allow static linking without disclosing the complete source?
3.) Which licenses allow including of the source-code without disclosing the complete source?
LibRaw is licensed in triple-licensing form:
- LGPL2
- or CDDL
- or LibRaw own license (to be dropped in 0.18-release, because all signed agreements has ended).
You (app developer) is free to choose license (from LGPL/CDDL if you wish to use LibRaw 0.18 when we'll release it) that suits you best.
In short:
LGPL2 allows use in Commercial App if you use Library (LibRaw in this case) as shared-library and has published in open-source (under LGPL2) any changes you made to Library source. If you use LibRaw unmodified, you only need to use it in shared-library form (.DLL/.so/.dylib)
CDDL is more permissive, you may just use the library and do not forced to publish your changes to it.
Attribution is needed in both cases. Written/signed form is not needed.
I just used
dcraw_emu -T sample.DNG
to get sample.DNG.tiff
and tiff is OK.
Have not tried python bindings, i'm not a python programmer (even at small fraction)
I used 0.17 and 0.18-beta, both provide the same (wrong) output.
When you run your test, did you also use the Python interface?
I've got 3 more images from the photographer. They were all reported as being corrupted and after the patch you sent, libraw works with all of them. Thanks!
Thanks for your email too. That would be useful for the future because some photographers do not want to post their images publicly.
Current LibRaw (0.18-beta1) works fine with this image.
What version do you use?
Thanks!
To be included in beta-2
Hi, compilation fails when openmp is enabled. My fix below:
--- a/src/libraw_xtrans_compressed.cpp
+++ b/src/libraw_xtrans_compressed.cpp
@@ -738,10 +738,11 @@ void LibRaw::xtrans_compressed_load_raw()
void LibRaw::xtrans_decode_loop(const struct xtrans_params* common_info, int count, INT64* raw_block_offsets, unsigned *block_sizes)
{
+ int cur_block;
#ifdef LIBRAW_USE_OPENMP
#pragma omp parallel for private(cur_block)
#endif
- for (int cur_block = 0; cur_block < count ; cur_block++)
+ for (cur_block = 0; cur_block < count ; cur_block++)
{
xtrans_decode_strip(common_info, cur_block, raw_block_offsets[cur_block], block_sizes[cur_block]);
}
And here it is: http://www.libraw.org/news/libraw-0-18-beta1
dcraw_make_mem_image() returns correct width/height,
It seems that for rotated images (.imgdata.sizes.flip&4) you need to use height for stride calculation. And keep in mind that you will receive rotated image with width and height swapped.
The only real recommendation is RawSpeed (if you use it): LibRaw is tested with RawSpeed/master branch, not with develop branch.
Other libs are used in very standard way, so you need jpeg8+ (or any replacement with jpeg_mem_src), latest DNG SDK (if you use it), any modern version of zlib
Thanks for the response (apologies if you got my request twice due to me thinking the forum one got lost)
That reminds me, whilst looking at the current changelog in github, I wondered if you have a recommended set of supporting library versions to build against ?
Thanks
Kevin
If you'll find another problem(s), please share the files.
You may send it to info@libraw.org (or to personal E-mail lexa@lexa.ru) if you do not wish to expose these files to public.
We plan to release 0.18-beta in a few days. Only one item in TODO remains: changelog
Thanks for the patch, Alex. I'm waiting for more images so we can properly test the patch. I'll send another update when I've done that.
diff --git a/src/libraw_xtrans_compressed.cpp b/src/libraw_xtrans_compressed.cpp index b6555d3..ec6dbb0 100644 --- a/src/libraw_xtrans_compressed.cpp +++ b/src/libraw_xtrans_compressed.cpp @@ -46,6 +46,7 @@ struct xtrans_block { unsigned max_read_size; // Amount of data to be read int cur_buf_size; // buffer size uchar *cur_buf; // currently read block + bool lastlast; LibRaw_abstract_datastream *input; struct int_pair grad_even[3][41]; // tables of gradients struct int_pair grad_odd[3][41]; @@ -143,7 +144,7 @@ static inline void fuji_fill_buffer(struct xtrans_block *info) #ifndef LIBRAW_USE_OPENMP info->input->unlock(); #endif - if(info->cur_buf_size<1) // nothing read + if(info->cur_buf_size<1 && info->max_read_size && !info->lastlast) // nothing read throw LIBRAW_EXCEPTION_IO_EOF; info->max_read_size -= info->cur_buf_size; @@ -159,6 +160,7 @@ void LibRaw::init_xtrans_block(struct xtrans_block* info, const struct xtrans_pa INT64 fsize = libraw_internal_data.internal_data.input->size(); info->max_read_size = _min(unsigned(fsize-raw_offset),dsize+16); // Data size may be incorrect? + info->lastlast = false; info->input = libraw_internal_data.internal_data.input; info->linebuf[_R0] = info->linealloc; for(int i = _R1; i<=_B4;i++) @@ -668,6 +670,7 @@ void LibRaw::xtrans_decode_strip(const struct xtrans_params* info_common, int cu ztable[3]= {{_R2,3},{_G2,6},{_B2,3}}; for (cur_line = 0; cur_line < libraw_internal_data.unpacker_data.fuji_total_lines; cur_line++) { + info.lastlast = (cur_block == libraw_internal_data.unpacker_data.fuji_total_blocks-1) && (cur_line ==libraw_internal_data.unpacker_data.fuji_total_lines-1); xtrans_decode_block(&info, info_common, cur_line); // copy data from line buffers and advanceBTW, ignoring this error for the last bytes of last line looks OK.
I've digged this data in depth: decoder really wants to see these extra bytes for the last line, EOF error looks correct for this data.
I have only one sample. But the photographer that sent me that sample showed me a case where out of 36 images he had, 4 of them appeared corrupted. I'll try to get more samples.
Thank you for sharing the sample.
In this sample
data offset (of raw data) 805904
data size 25251744
sum is 26057648
but file size is 26057632, so 16 bytes missing somewhere
Do you have only one sample, or some camera produces many such files?
If only one, this looks like some data error (camera, card, transfer). If this is systematic error it should be handled in software (e.g. ignore last bytes without error raise)
CDDL will be the weapon of my choice then. Thanks for your instant support. :)
1) We want to drop this license, so please select from CDDL/LGPL2 (btw, LibRaw license do not cover updates, so in general you need to re-sign when you go from LibRaw 0.17 to 0.18).
2-3) CDDL is very permissive, so yes for both questions
Thanks for claryfying, Alex :). I have a few follow-up questions though.
1.) Is it possible to use the (own) libraw-license for 0.17?
2.) Which licenses allow static linking without disclosing the complete source?
3.) Which licenses allow including of the source-code without disclosing the complete source?
Cheers, Brian
LibRaw is licensed in triple-licensing form:
- LGPL2
- or CDDL
- or LibRaw own license (to be dropped in 0.18-release, because all signed agreements has ended).
You (app developer) is free to choose license (from LGPL/CDDL if you wish to use LibRaw 0.18 when we'll release it) that suits you best.
In short:
LGPL2 allows use in Commercial App if you use Library (LibRaw in this case) as shared-library and has published in open-source (under LGPL2) any changes you made to Library source. If you use LibRaw unmodified, you only need to use it in shared-library form (.DLL/.so/.dylib)
CDDL is more permissive, you may just use the library and do not forced to publish your changes to it.
Attribution is needed in both cases. Written/signed form is not needed.
For EOS 6D different color profile is used because dcraw's is not good enough.
Also, LibRaw has changed auto-brighness method, so disable auto-brightness for comparison
Thanks, I look forward to the next snapshot.
Pages