Document mode is not supported in our dcraw processing emulation (dcraw_process() call) because it makes things too complex.
Our unprocessed_raw.cpp and 4channels.cpp samples provides unprocessed raw data output (for bayer case), that is enough for most practical cases. It is also good starting point(s) for ones who want only unprocessed raw data access and do full processing.
Black subtraction: if LibRaw docs covering imgdata.color.black/imgdata.color.cblack[] is not enough, you may use LibRaw::subtract_black() source as an example. It operates with imgdata.image[] array, assuming that rawdata.raw_image[] values are already in place.
Also, LibRaw::raw2image_ex() do raw_image=>image population and black subtraction in single step.
AFAICT using the -4 flag means gamma coefficients both set to 1. So I *think* that means that gamma correction should make no changes. Is that correct?
Certainly -d and -D give quite different results - d is quite a bit brighter than the -D version - would that fit with black subtraction processing? Where in the code would I find the black subtraction (so I can do that processing myself on the RAW data?
Is it out of order to ask why the document mode code was removed?
AFAIK, -D is completely no processing, while -d looks like black subtraction is performed.
Also, both -D/-d creates gamma corrected image (if no other flags)
AFAIK that's exactly what dcraw -d -4 does (i.e. no processing at all apart from unpacking the raw image, extracting the image from the frame, and writing the PGM file). Have I misunderstood this?
So I'm having a bit of difficulty understanding why I'm seeing a difference?
Raw data pixels are in raw_image array.
These pixels are unaltered (in particular, black level/bias is not subtracted), unscaled, uncropped and no other processing is made.
So, if you need *unaltered* pixels: here it is. If you need some type of processing (black subtraction, white balance, scale, crop) you need to do it in your code.
I want to extract the raw image pixel data from the file loaded - I'm asking if that code I posted with a bug will extract the 16 bit raw image data without the frame that is width*height. Right now I'm not seeing what I expect when I look at RawData.raw_image as what DCRAW -t 0 -d -4 wrote to the PGM file including the header looks like (I've shown the binary data as hex):
P5
5202 3465
65535
04 fA 02 80 04 40
Correction to the bug
for (int row = 0; row < S.height; row++)
{
// Write raw pixel data into our private bitmap format
pFiller->Write(RAW(row+S.top_margin, S.left_margin), sizeof(ushort), S.width);
}
and I cannot find that data in the raw_image array at or near raw_image(row+S.top_margin, S.left_margin)
Here's a shortened version of the code I'm proposing to use to process the data at raw_image
#define RAW(row,col) \
Rawdata.raw_image[(row)*S.raw_width+(col)]
BOOL littleEndian = htons(0x55aa) != 0x55aa;
if (!m_bColorRAW)
{
// This is a regular RAW file, so we should have the "raw" 16-bit greyscale
// pixel array hung off RawData.raw_image.
ZASSERT(NULL != RawData.raw_image);
// stuff omitted
// Convert raw data to big-endian
if littleEndian
_swab(
(char*)(RawData.raw_image),
(char*)(RawData.raw_image),
S.raw_height*S.raw_pitch); // Use number of rows times row width in BYTES!!
for (int row = 0; row < S.height; row++)
{
// Write raw pixel data into our private bitmap format
pFiller->Write(RAW(raw, S.left_margin), sizeof(ushort), S.width);
}
Image rectangle may not be centered in frame, this is camera-specific, so one need to use left/top_margin. You may use LibRaw::copy_bayer() and it's calling code (raw2image) as an example (neat bit of code).
Note: it is safer to use imgdata.sizes.raw_pitch (in bytes!) to access rows. Usually it is just multiplication of raw_width and pixel size, but there are some exceptions (e.g. if LibRaw is compiled with RawSpeed library).
Got it - is the data in the ushort arrays in "network byte order" or do I need to do a byte-swap?
Thank you.
AFAICT, the image at (e.g.) raw_image is raw_width*raw_height.
How to actually skip the image frame/border - do I need to offset by top_margin/left margin? Do you have a neat bit of code to iterate through the rows and skip the margins (I don't want to reinvent the wheel) so I only extract the rectangle that is width*height and (I assume) centred in the frame?
Thumbnail is generated by camera, using own camera settings (color profile, contrast curve). There is no built-in way (in LibRaw) to replicate out-of-camera JPEG.
Unaltered raw pixels are stored in imgdata.rawdata:
raw_image: for bayer/x-trans/monochrome (single component per pixel)
color3_image, color4_image - for 3/4 full-color images.
Sure - what I want is the 16 bit greyscale "raw" data so I can do my own de-bayer processing on it (except for the case of "full colour" raw files where I want the RGB image data). That's what the code that reads:
if (!m_bColorRAW)
{
// Document mode (i.e. don't demosaic the image in libraw).
O.no_interpolation = 1;
};
was intended to achieve: m_bColorRaw is set thus:
m_bColorRAW = P1.is_foveon || !(P1.filters);
Would that be accomplished by grabbing the "raw image data" from libraw_raw_data after running unpack()?
If so I think (hope) I'll try using that based on the "unprocessed_raw.cpp sample.
OK I'm a bit disappointed about that. Looking at the source in dcraw_common.cpp it appears that the "document mode" stuff has been #ifdef-ed out of existence.
I had specified equivalent options to those I used with dcraw and captured the output file by sub-classing libraw and over-riding just dcraw_ppm_tiff_writer() and write_ppm_tiff() which worked a treat except that it didn't produce the 16 bit PGM greyscale data I hoped for but instead produced a 16 bit PPM RGB bitmap (which I confirmed using dcraw_emu).
Advice and guidance in more detail of how to get at the grayscale data will be greatly welcome
Might I ask why you explicitly disabled the "document mode" code?
Here's what my code looks like right now:
if (!m_bColorRAW)
{
// Document mode (i.e. don't demosaic the image in libraw).
O.no_interpolation = 1;
};
// Don't stretch or rotate raw pixels (equivalent to dcraw -j)
O.use_fuji_rotate = 0;
// Don't flip the image (equivalent to dcraw -t 0)
O.user_flip = 0;
// Output color space : raw-> sRGB (default)
/*
argv[argc] = _T("-o");
argc++;
argv[argc] = _T("0");
argc++;*/
workspace.GetValue(REGENTRY_BASEKEY_RAWSETTINGS, _T("BlackPointTo0"), bBlackPointTo0);
if (bBlackPointTo0)
{
// Set black point to 0
O.user_black = 0;
};
// Output is 16 bits (equivalent of dcraw flag -4)
O.gamm[0] = O.gamm[1] = O.no_auto_bright = 1;
O.output_bps = 16;
lstrcpy(g_szInputFileName, (LPCTSTR)m_strFileName);
g_Progress = pProgress;
if ((ret = rawProcessor.unpack()) != LIBRAW_SUCCESS)
{
bResult = FALSE;
ZTRACE_RUNTIME("Cannot unpack %s: %s", m_strFileName, libraw_strerror(ret));
}
if (!bResult) break;
if (LIBRAW_SUCCESS != (ret = rawProcessor.dcraw_process()))
{
ZTRACE_RUNTIME("Cannot do postprocessing on %s: %s", m_strFileName, libraw_strerror(ret));
if (LIBRAW_FATAL_ERROR(ret))
bResult = FALSE;
}
if (!bResult) break;
pFiller = new BitMapFiller(pBitmap, pProgress);
pFiller->SetWhiteBalance(fRedScale, fGreenScale, fBlueScale);
// Get the Colour Filter Array type and set into the bitmap filler
m_CFAType = GetCurrentCFAType();
pFiller->SetCFAType(m_CFAType);
// Set up the intercept code to write the image data to our bitmap instead of
// to an external file, and invoke the overridden dcraw_ppm_tiff_writer()
rawProcessor.setBitMapFiller(pFiller);
if (LIBRAW_SUCCESS != (ret = rawProcessor.dcraw_ppm_tiff_writer("")))
{
bResult = FALSE;
ZTRACE_RUNTIME("Cannot write image data to bitmap %s", libraw_strerror(ret));
}
As mentioned above, there is no direct equivalent for dcraw's -d option in LibRaw (esp. dcraw_emu sample).
If you need to get RAW data as is, you, most likely, need unprocessed_raw sample.
I guess the question I *should* ask is when is the approximate release date for the next version?
I know from experience designing and managing IC design projects you may not be able to give me even an approximate date for a variety of reasons. But if you can, I'd appreciate knowing. Thank you.
Thank you! A very helpful couple of posts
BTW, I've already mentioned copy_bayer() call in previous reply: https://www.libraw.org/comment/5246#comment-5246
Document mode is not supported in our dcraw processing emulation (dcraw_process() call) because it makes things too complex.
Our unprocessed_raw.cpp and 4channels.cpp samples provides unprocessed raw data output (for bayer case), that is enough for most practical cases. It is also good starting point(s) for ones who want only unprocessed raw data access and do full processing.
Black subtraction: if LibRaw docs covering imgdata.color.black/imgdata.color.cblack[] is not enough, you may use LibRaw::subtract_black() source as an example. It operates with imgdata.image[] array, assuming that rawdata.raw_image[] values are already in place.
Also, LibRaw::raw2image_ex() do raw_image=>image population and black subtraction in single step.
AFAICT using the -4 flag means gamma coefficients both set to 1. So I *think* that means that gamma correction should make no changes. Is that correct?
Certainly -d and -D give quite different results - d is quite a bit brighter than the -D version - would that fit with black subtraction processing? Where in the code would I find the black subtraction (so I can do that processing myself on the RAW data?
Is it out of order to ask why the document mode code was removed?
Thanks again
AFAIK, -D is completely no processing, while -d looks like black subtraction is performed.
Also, both -D/-d creates gamma corrected image (if no other flags)
AFAIK that's exactly what dcraw -d -4 does (i.e. no processing at all apart from unpacking the raw image, extracting the image from the frame, and writing the PGM file). Have I misunderstood this?
So I'm having a bit of difficulty understanding why I'm seeing a difference?
Cheers
Raw data pixels are in raw_image array.
These pixels are unaltered (in particular, black level/bias is not subtracted), unscaled, uncropped and no other processing is made.
So, if you need *unaltered* pixels: here it is. If you need some type of processing (black subtraction, white balance, scale, crop) you need to do it in your code.
I want to extract the raw image pixel data from the file loaded - I'm asking if that code I posted with a bug will extract the 16 bit raw image data without the frame that is width*height. Right now I'm not seeing what I expect when I look at RawData.raw_image as what DCRAW -t 0 -d -4 wrote to the PGM file including the header looks like (I've shown the binary data as hex):
P5
5202 3465
65535
04 fA 02 80 04 40
Correction to the bug
for (int row = 0; row < S.height; row++)
{
// Write raw pixel data into our private bitmap format
pFiller->Write(RAW(row+S.top_margin, S.left_margin), sizeof(ushort), S.width);
}
and I cannot find that data in the raw_image array at or near raw_image(row+S.top_margin, S.left_margin)
I still do not know what you want to achieve, so this may be correct, may be not.
Is this code does what you want or not?
Thanks yet again,
Here's a shortened version of the code I'm proposing to use to process the data at raw_image
Does that make sense? Is it correct?
Thanks
Data is in native byte order.
Image rectangle may not be centered in frame, this is camera-specific, so one need to use left/top_margin. You may use LibRaw::copy_bayer() and it's calling code (raw2image) as an example (neat bit of code).
Note: it is safer to use imgdata.sizes.raw_pitch (in bytes!) to access rows. Usually it is just multiplication of raw_width and pixel size, but there are some exceptions (e.g. if LibRaw is compiled with RawSpeed library).
Got it - is the data in the ushort arrays in "network byte order" or do I need to do a byte-swap?
Thank you.
AFAICT, the image at (e.g.) raw_image is raw_width*raw_height.
How to actually skip the image frame/border - do I need to offset by top_margin/left margin? Do you have a neat bit of code to iterate through the rows and skip the margins (I don't want to reinvent the wheel) so I only extract the rectangle that is width*height and (I assume) centred in the frame?
Thanks again
LIBRAW_LIBRARY_BUILD is already defined in dcraw_common.cpp and dcraw_fileio.cpp
Thumbnail is generated by camera, using own camera settings (color profile, contrast curve). There is no built-in way (in LibRaw) to replicate out-of-camera JPEG.
Unaltered raw pixels are stored in imgdata.rawdata:
raw_image: for bayer/x-trans/monochrome (single component per pixel)
color3_image, color4_image - for 3/4 full-color images.
Followup: only one of pointers mentioned above is non-NULL after unpack()
Sure - what I want is the 16 bit greyscale "raw" data so I can do my own de-bayer processing on it (except for the case of "full colour" raw files where I want the RGB image data). That's what the code that reads:
was intended to achieve: m_bColorRaw is set thus:
Would that be accomplished by grabbing the "raw image data" from libraw_raw_data after running unpack()?
If so I think (hope) I'll try using that based on the "unprocessed_raw.cpp sample.
Thanks again
Dear Sir:
Could you please specify what result you want to achiveve?
Is it unaltered pixel values, or something else?
OK I'm a bit disappointed about that. Looking at the source in dcraw_common.cpp it appears that the "document mode" stuff has been #ifdef-ed out of existence.
I had specified equivalent options to those I used with dcraw and captured the output file by sub-classing libraw and over-riding just dcraw_ppm_tiff_writer() and write_ppm_tiff() which worked a treat except that it didn't produce the 16 bit PGM greyscale data I hoped for but instead produced a 16 bit PPM RGB bitmap (which I confirmed using dcraw_emu).
Advice and guidance in more detail of how to get at the grayscale data will be greatly welcome
Might I ask why you explicitly disabled the "document mode" code?
Here's what my code looks like right now:
Thanks
As mentioned above, there is no direct equivalent for dcraw's -d option in LibRaw (esp. dcraw_emu sample).
If you need to get RAW data as is, you, most likely, need unprocessed_raw sample.
It would appear there's a possible bug with libraw in this area.
If I run dcraw -d M103_L_0048_ISO800_60s__16C.CR2, it creates a PGM output file:
29/05/2019 12:01 18,024,947 M103_L_0048_ISO800_60s__16C.pgm
if I run dcraw_emu -disinterp M103_L_0048_ISO800_60s__16C.CR2, it creates a PPM output file:
29/05/2019 12:02 54,074,807 M103_L_0048_ISO800_60s__16C.CR2.ppm
I see two problems:
1) This is the trivial issue that the output fileid is slightly different (which I'm happy to forgive).
2) The major problem is that the output file is NOT a single colour PGM greyscale bitmap as was produced by dcraw, but a 3 colour PPM bitmap.
Cheers
https://www.libraw.org/node/2467
I guess the question I *should* ask is when is the approximate release date for the next version?
I know from experience designing and managing IC design projects you may not be able to give me even an approximate date for a variety of reasons. But if you can, I'd appreciate knowing. Thank you.
We publish releases each 1-1.5 years. We publish snapshots every 5-7 months or so.
Most likely, this link will help: https://www.libraw.org/docs/API-CXX.html#memwrite
Pages