Fuji X Pro 2 image that libraw cannot read

Hi,

I'm having trouble with reading a Fuji X Pro 2 raw image. You can download the raw file using this link (26MB).

https://drive.google.com/file/d/0BzcmCgBZCb5WOTlfN0hGcFhfVTg/view?usp=sh...

Adobe Photoshop is able to open this image, but FastRawViewer and a number of other products are not able to.

I have the version of libraw that includes compressed raw support. It is not able to read this image either. dcraw_emu reports:

FujiXPro2_CannotUnpack.RAF: Unexpected end of file
Cannot unpack FujiXPro2_CannotUnpack.RAF: Input/output error

Do you know what could be the problem with this image?

Thanks.

Forums: 

Thank you for sharing the

Thank you for sharing the sample.
In this sample
data offset (of raw data) 805904
data size 25251744
sum is 26057648
but file size is 26057632, so 16 bytes missing somewhere

Do you have only one sample, or some camera produces many such files?
If only one, this looks like some data error (camera, card, transfer). If this is systematic error it should be handled in software (e.g. ignore last bytes without error raise)

-- Alex Tutubalin

I have only one sample. But

I have only one sample. But the photographer that sent me that sample showed me a case where out of 36 images he had, 4 of them appeared corrupted. I'll try to get more samples.

I've digged this data in

I've digged this data in depth: decoder really wants to see these extra bytes for the last line, EOF error looks correct for this data.

-- Alex Tutubalin

BTW, ignoring this error for

BTW, ignoring this error for the last bytes of last line looks OK.

-- Alex Tutubalin

Looks like we can safely

Looks like we can safely ignore 'not enough data problem' for last line(s) of last block (it contains masked pixels, not needed for data display). It may be Firmware issue(??) Here is the patch (for last line of last block only):
diff --git a/src/libraw_xtrans_compressed.cpp b/src/libraw_xtrans_compressed.cpp
index b6555d3..ec6dbb0 100644
--- a/src/libraw_xtrans_compressed.cpp
+++ b/src/libraw_xtrans_compressed.cpp
@@ -46,6 +46,7 @@ struct xtrans_block {
 	unsigned	max_read_size;	 // Amount of data to be read
 	int         cur_buf_size;    // buffer size
 	uchar       *cur_buf;        // currently read block
+	bool		lastlast;
 	LibRaw_abstract_datastream *input;
 	struct int_pair grad_even[3][41];    // tables of gradients
 	struct int_pair grad_odd[3][41];
@@ -143,7 +144,7 @@ static inline void fuji_fill_buffer(struct xtrans_block *info)
 #ifndef LIBRAW_USE_OPENMP
 			info->input->unlock();
 #endif
-			if(info->cur_buf_size<1) // nothing read
+			if(info->cur_buf_size<1 && info->max_read_size && !info->lastlast) // nothing read
 				throw LIBRAW_EXCEPTION_IO_EOF;
 			info->max_read_size -= info->cur_buf_size;
 
@@ -159,6 +160,7 @@ void LibRaw::init_xtrans_block(struct xtrans_block* info, const struct xtrans_pa
 	INT64 fsize = libraw_internal_data.internal_data.input->size();
 	info->max_read_size = _min(unsigned(fsize-raw_offset),dsize+16); // Data size may be incorrect?
 
+	info->lastlast = false;
 	info->input = libraw_internal_data.internal_data.input;
 	info->linebuf[_R0] = info->linealloc;
 	for(int i = _R1; i<=_B4;i++)
@@ -668,6 +670,7 @@ void LibRaw::xtrans_decode_strip(const struct xtrans_params* info_common, int cu
 		ztable[3]= {{_R2,3},{_G2,6},{_B2,3}};
 	for  (cur_line = 0; cur_line < libraw_internal_data.unpacker_data.fuji_total_lines; cur_line++)
 	{
+		info.lastlast = (cur_block == libraw_internal_data.unpacker_data.fuji_total_blocks-1) && (cur_line ==libraw_internal_data.unpacker_data.fuji_total_lines-1);
 		xtrans_decode_block(&info, info_common, cur_line);
 
 		// copy data from line buffers and advance

-- Alex Tutubalin

Thanks for the patch, Alex. I

Thanks for the patch, Alex. I'm waiting for more images so we can properly test the patch. I'll send another update when I've done that.

If you'll find another

If you'll find another problem(s), please share the files.
You may send it to info@libraw.org (or to personal E-mail lexa@lexa.ru) if you do not wish to expose these files to public.

-- Alex Tutubalin

I've got 3 more images from

I've got 3 more images from the photographer. They were all reported as being corrupted and after the patch you sent, libraw works with all of them. Thanks!

Thanks for your email too. That would be useful for the future because some photographers do not want to post their images publicly.