After much experiment, I tried the same code on an Ubuntu 9.04 box, and it works perfectly. Backporting the same code back to OS X gives me the original problem.
Finally, I just tried dcraw_emu from the samples directory, and that suffers the
same problem, so I've eliminated my application code.
There are 2 items I would like to see enhanced in the current Raw conversion libraries out there:
1. I would like extra output 32-bit float instead of kist 16-bit integer.
2. The ability to quickly extract a section of an image. Lets say I want to create a zoom tool that zooms into a section of a Raw file I would like to have a fast Raw library that only process the pixels involved in the area you are zooming in. I'm aware that raw is a stream of data but at leas a fast way of walking through the stream instead of interpolating all the pixels on its way.
The typical shape of the curve is lighter in the midtone and darker in the three-quartertone. However, it would be easy to incorporate some presets that would permit users to choose the type of color enhancement.
*Decoding* process is very simple and limited to file (disk) bandwidth.
Postprocessing (demosaicing, color conversion, noise suppression etc) is another story. GPU may help a lot. On the other side, only very good quality postprocessing code should be accelerated, there is no need to very fast and very bad postprocessing. We think, that current LibRaw postprocessing implementation is bad enough....
because the #line directives in various files direct to it. Unfortunately downloading dcraw.c doesn't help, because it changes all the time and is therefore out of sync with libraw.
Yes, for '99%' of cameras (bayer pattern ones) half_size is lossless. For most RGB-bayer patterns there is four different pixel positions in 2x2 block, so if you want to recover exact pixel position you need to examine "filter" field which describes exact layout.
There is some 'special cases': CMYG cameras, Leaf cameras with 8x2 blocks, but most cases are simple.
Also, we'll sometimes refactor this place in LibRaw. Having multiple planes will make simpler to deal with multi-planes cameras: Fuji SuperCCD (2 images in one file), Sinar 4-shot and 16-shot.....
I understand the philosophy of "one size fits all". The DNG-SDK persues the same strategy, but uses more than one plane for the remaining 1% of sensors, i.e. instead of using four values for one pixels it uses up to four pixel planes. But I certainly understand that you have to follow the design of dcraw.c closely. That will allow you to incorporate new cameras very easily into libraw.
Does 'half-size' processing mean that the top left corner, which may look as follows:
RG
GB
is merged into one pixel with four values? In other words, is 'half-size' processing lossless?
dcraw's (and LibRaw's) internal structures are 'one size fits all': image[][4] can fit all 'life types' of RAWs (1-color bayer, 3-color foveon, 4-color Sinar 4-shot). Sure, ~99% of RAW images are simply bayer-pattern RGB, but we should not ignore remaining 1%.
If you want to save memory, you can use 'half-size' processing on unpack() stage. In this case each 4 pixels of raw data are packed into one image[][4] item. You should take in account the exact bayer pattern layout to restore original pixel position.
I remember FC(row,col) from the times, when I tried to dig through dcraw.c. I didn't succeed, because after a pause a had to start all over again. I'm very grateful for your work here. Being frustrated by dcraw.c I turned to the DNG-SDK, which also has the ability to write raw files, which is crucial for my RawImageClearer ( http://www.RawImageClearer.de.tf ). The DNG-SDK has lots of pitfalls and is very bloated, I have to say.
I would like to implement a new kind of demosaicing with libraw. Rather than creating an RGB 4:4:4 image it creates directly a YUV 4:2:0 image, which is exactly the color space used for JPEG compression. Thus I do not really fill in the remaining 2 or 3 values as it is done with the usual RGB demosaicing methods.
There is FC(row,col) function, which returns index (in 0..3) range of value within image[i][4].
Look in the samples/unprocessed_raw.cpp for usage example
You're right, that raw_image[row*col], than processed_image[row*col][3] may be better alternative. Sometimes in future, LibRaw may be refactored this way.
There is many reasons to have current memory allocation pattern:
* (main one) LibRaw derived from dcraw sources, so memory pattern is preserved
* there is some special cases (Foveon, Sinar 4-shot) where more than one component is needed, so 4-component image[] is more general and fits all cases.
* most LibRaw users uses post-processing stage too. With postprocessing there is no advantage in 2-phase allocation (one for raw data and one for postprocessing).
Thank you for your prompt information.
If only one of the four possible values is filled with data by unpack, then there must be either some information about the mosaic pattern or the unfilled items must be marked by a special value?
It would be nice to have a low level funtion, which just writes the sensor data (only 1 ushort per pixel) to a specified buffer:
unpack(unsigned short *buffer, int pitch)
Of course, the user has to assure that there ist enough memory allocated. This rather flexible function has also the advantage that the user can padd the data appropriately, which is often neccessary. Also the DNG-SDK unpacks in stage1 the raw data with just one 16 bit value per pixel, if it's not a Foveon sensor. At least internally such a function should precede libraw::unpack(). It would be nice to have such a function also for libraw users.
On unpack() sensors only one component is filled. There is two exceptions:
* if half_size parameter is set, then image is resized two-fold (in each dimension), and all 4 components if image[i] filled with data
* for foveon-based sensors three components is filled.
On postprocessing stage remaining components are interpolated 'in-place', without image[] reallocation
Could you please describe how to repeat the problem with dcraw_emu? What command-line switches used?
After much experiment, I tried the same code on an Ubuntu 9.04 box, and it works perfectly. Backporting the same code back to OS X gives me the original problem.
Finally, I just tried dcraw_emu from the samples directory, and that suffers the
same problem, so I've eliminated my application code.
Any further input welcome...
There are 2 items I would like to see enhanced in the current Raw conversion libraries out there:
1. I would like extra output 32-bit float instead of kist 16-bit integer.
2. The ability to quickly extract a section of an image. Lets say I want to create a zoom tool that zooms into a section of a Raw file I would like to have a fast Raw library that only process the pixels involved in the area you are zooming in. I'm aware that raw is a stream of data but at leas a fast way of walking through the stream instead of interpolating all the pixels on its way.
The typical shape of the curve is lighter in the midtone and darker in the three-quartertone. However, it would be easy to incorporate some presets that would permit users to choose the type of color enhancement.
erreauk
Unfortunately, I'm not an Matlab user, so cannot provide significant help.
I guess, Matlab must have some interface to external libraries. LibRaw is a library, you can compile it according to Matlab requirements and use.
Is there an easy way to use LibRAW to move the whole processing procedure into Matlab?
Thanks.
Thank you, much appreciated.
You should call open() on each iteration:
for(x=0; ; x++) {
RawProcessor.open(filename);
if(x>P1.raw_count) break;
RawProcessor.unpack();
process();
}
I should mention that the loop for getting multiple images
looks something like:
for(x=0; x < P1.raw_count; x++) {
OUT.shot_select = x;
RawProcessor.unpack();
RawProcessor.dcraw_process();
}
libopenraw is completely different project.
*Decoding* process is very simple and limited to file (disk) bandwidth.
Postprocessing (demosaicing, color conversion, noise suppression etc) is another story. GPU may help a lot. On the other side, only very good quality postprocessing code should be accelerated, there is no need to very fast and very bad postprocessing. We think, that current LibRaw postprocessing implementation is bad enough....
Thank you for your suggestion.
We'll rename identify sample to libraw_identify
Maybe you could rename the
identifysample program. Anybody who installs it will have a conflict with ImageMagick's command of the same name.You're right.
We'll include original dcraw.c in next release.
because the #line directives in various files direct to it. Unfortunately downloading dcraw.c doesn't help, because it changes all the time and is therefore out of sync with libraw.
http://www.jellofishi.com/apps/libraw/mingw/Makefile.mingw
unfortunately, file not found....
wget http://www.jellofishi.com/apps/libraw/mingw/Makefile
--2009-03-01 17:29:09-- http://www.jellofishi.com/apps/libraw/mingw/Makefile
Resolving www.jellofishi.com... 67.15.172.4
Connecting to www.jellofishi.com|67.15.172.4|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2009-03-01 17:29:10 ERROR 404: Not Found.
http://www.jellofishi.com/apps/libraw/mingw/Makefile
Yes, for '99%' of cameras (bayer pattern ones) half_size is lossless. For most RGB-bayer patterns there is four different pixel positions in 2x2 block, so if you want to recover exact pixel position you need to examine "filter" field which describes exact layout.
There is some 'special cases': CMYG cameras, Leaf cameras with 8x2 blocks, but most cases are simple.
Also, we'll sometimes refactor this place in LibRaw. Having multiple planes will make simpler to deal with multi-planes cameras: Fuji SuperCCD (2 images in one file), Sinar 4-shot and 16-shot.....
Sometimes on LibRaw 0.9
I understand the philosophy of "one size fits all". The DNG-SDK persues the same strategy, but uses more than one plane for the remaining 1% of sensors, i.e. instead of using four values for one pixels it uses up to four pixel planes. But I certainly understand that you have to follow the design of dcraw.c closely. That will allow you to incorporate new cameras very easily into libraw.
Does 'half-size' processing mean that the top left corner, which may look as follows:
RG
GB
is merged into one pixel with four values? In other words, is 'half-size' processing lossless?
dcraw's (and LibRaw's) internal structures are 'one size fits all': image[][4] can fit all 'life types' of RAWs (1-color bayer, 3-color foveon, 4-color Sinar 4-shot). Sure, ~99% of RAW images are simply bayer-pattern RGB, but we should not ignore remaining 1%.
If you want to save memory, you can use 'half-size' processing on unpack() stage. In this case each 4 pixels of raw data are packed into one image[][4] item. You should take in account the exact bayer pattern layout to restore original pixel position.
I remember FC(row,col) from the times, when I tried to dig through dcraw.c. I didn't succeed, because after a pause a had to start all over again. I'm very grateful for your work here. Being frustrated by dcraw.c I turned to the DNG-SDK, which also has the ability to write raw files, which is crucial for my RawImageClearer ( http://www.RawImageClearer.de.tf ). The DNG-SDK has lots of pitfalls and is very bloated, I have to say.
I would like to implement a new kind of demosaicing with libraw. Rather than creating an RGB 4:4:4 image it creates directly a YUV 4:2:0 image, which is exactly the color space used for JPEG compression. Thus I do not really fill in the remaining 2 or 3 values as it is done with the usual RGB demosaicing methods.
There is FC(row,col) function, which returns index (in 0..3) range of value within image[i][4].
Look in the samples/unprocessed_raw.cpp for usage example
You're right, that raw_image[row*col], than processed_image[row*col][3] may be better alternative. Sometimes in future, LibRaw may be refactored this way.
There is many reasons to have current memory allocation pattern:
* (main one) LibRaw derived from dcraw sources, so memory pattern is preserved
* there is some special cases (Foveon, Sinar 4-shot) where more than one component is needed, so 4-component image[] is more general and fits all cases.
* most LibRaw users uses post-processing stage too. With postprocessing there is no advantage in 2-phase allocation (one for raw data and one for postprocessing).
Thank you for your prompt information.
If only one of the four possible values is filled with data by unpack, then there must be either some information about the mosaic pattern or the unfilled items must be marked by a special value?
It would be nice to have a low level funtion, which just writes the sensor data (only 1 ushort per pixel) to a specified buffer:
unpack(unsigned short *buffer, int pitch)
Of course, the user has to assure that there ist enough memory allocated. This rather flexible function has also the advantage that the user can padd the data appropriately, which is often neccessary. Also the DNG-SDK unpacks in stage1 the raw data with just one 16 bit value per pixel, if it's not a Foveon sensor. At least internally such a function should precede libraw::unpack(). It would be nice to have such a function also for libraw users.
Each pixel is 4-component, 16 bit per component.
On unpack() sensors only one component is filled. There is two exceptions:
* if half_size parameter is set, then image is resized two-fold (in each dimension), and all 4 components if image[i] filled with data
* for foveon-based sensors three components is filled.
On postprocessing stage remaining components are interpolated 'in-place', without image[] reallocation
Pages