Pixels (in raw_image) are recorded 'as in raw file'.
Colors are depending on camera CFA pattern.
The LibRaw::COLOR(row,col) call will return color for given pixel (row,col) are in image[] coordinate space i.e. visible area (so counted from top_margin,left_margin point of raw_image)
Let me take one step back on raw_image(). I see the initialization and dimensioning of raw_image, set as a one dimension array (row, or column, that's just an abstraction here):
does that go to the next row where I'd see the BGBG associated to the same pixel ? you said 4 values per pixel, as expected for Bayer image, so just checking I understand how they are set in the raw files.
raw2images arranges pixel values (from raw_image array, 1 value per pixel) to image[][4] array (4 values per pixel).
So, only one of image[N][0..3] is non-zero after raw2image, all other 3 components are still zero.
This is 'prepare for interpolation' step.
Actual interpolation (and white balance, output color profile conversion, brightness adjustment) is made by dcraw_process()
Could not understand the question
pixels 16799706 .... 16799710
are all in the same row
So, only two colors.
Again
ROWn: RGRGRGRGRGRGR
ROWn+1: GBGBGBGBGBGB
So, do you mean that, above, pixInd 16799706 is telling me that R = 2175, and that
pixInd 16799707 is giving my G2 = 2207 ?
pixInd 16799708 is B = 2134
pixInd 16799709 is G = 2161
and then these 4 values are my RGBG for one non-demosaiced single colored pixel?
Looks normal for me (after raw2image, not dcraw_process):
Each row (or column )in bayer pattern contains only two colors:
RGRGRGRG
GBGBGBGB
(really G2 in second row)
int iwidth = rawProcess.imgdata.sizes.iwidth;
// somewhere in the middle of the picture, far from edges.
int row = 2898;
int col = 2898;
// Display just 5 pixels
int nPixels = 5;
int first_visible_pixel = rawProcess.imgdata.sizes.raw_width*rawProcess.imgdata.sizes.top_margin + rawProcess.imgdata.sizes.left_margin;
then looping over a pixel index with i from 0 to 4 in :
after raw2image(), imgdata.image[pixInd][0 to 3] gives, with the following printout arguments:
image[pixInd][0] | image[pixInd][1] | image[pixInd][2] | image[pixInd][3],
row and column are both in 1-st dimension.
The (row,col) pixel values are in
image[row*imgdata.sizes.iwidth + col] [0..3]
iwidth is equal to imgdata.sizes.width for normal processing and width/2 for 'half' interpolation (where imgdata.params.half_size is non-zero).
Border values (1st/last row or 1st/last column) in image may be not fully interpolated because there is no data for it.
Better use image[iwidth*2 + 2] for inspection (this is pixel at (2,2))
Which dimension is going across the R G B G values. At first I thought it was [0 to 3] (2nd dimensions, "columns") which was giving the R G B G (which I abusively call the "channels"). Is that the other way round? Are they, in fact, in the 1st dimension (rows)? I'm often confused on the dimension when I see tables of pointers such as: imgdata.image = (ushort (*)[4]) calloc(S.iwidth*S.iheight,sizeof(*imgdata.image));
I've been doing some testing. I've selected the 4 first pixels of the data (I ignored the hidden ones for calibration), for a .CR2 file (canon 5D), at 3 stages of processing: after unpack(), after raw2image(), after dcraw_processed(). See below (the 4 "channels" of the 2nd dimension are separated by the separator " | "):
Why aren't the raw2image() values in the same column and instead, dispatched alternatively in the 0th and 1st column? Is that just to prepare the data for later demosaicing in a way that's compatible with all possible sensors? The values are indeed identical to the raw values. So it is unclear what happens between unpack() raw_image[] values and raw2image() image[][] values
Are the dcraw_processed() values dispatched in the 1st three columns to be considered as the R | G | B (and 4th is unused) components, after demosaicing?
Initialization of rawdata buffer in unpack(), can't miss it, it's clear. Populated this raw data buffer, still can't see it within unpack(). I make here a distinction between initializing something, and assigning value (which you call "populate" i guess). That's that part I still don't grasp.
Consider my test (it's done in Qt framework, hence the qDebug() instead of printf() for printing out in debug mode):
LibRaw rawProcess;
rawProcess.open_file("/......./F36A7292.CR2");
rawProcess.unpack()
// printout raw values
int first_visible_pixel = rawProcess.imgdata.sizes.raw_width*rawProcess.imgdata.sizes.top_margin + rawProcess.imgdata.sizes.left_margin;
for( int i=0; i< 100; i++)
{
qDebug() << "raw_image["<< i << "] =" << rawProcess.imgdata.rawdata.raw_image[i+first_visible_pixel];
}
This is giving me my raw values.
And I still don't see how the raw_image values get assigned (populated) within unpack(). In what I pasted in the early post, i don't see the lines of codes where this happen, for the Canon case (Nikon case seems a bit different). All I see is initialized buffer, and freed buffer (for Bayer image, and non-Nikon)
Sorry for repeating the question, I must be missing something obvious. Maybe a slap in my face will make me see. (by Copy pasting that invisible line that I don't see?)
If you don't want to repeat yourself, I'll understand and will go back again to what you say and try to find what i missed.
I'm confused again. Maybe I missed something else.
In the code samples that I pasted, my trouble was that these were only memory allocation of buffers, and I failed to see where the data from the .CR2 file where going. Then imgdata.image seemed the only time when some non-zero data were passed. But I'm wrong since from what you say imgdata.image is not populated with any raw or processed data at that time: the imgdata.image gets populated only after "raw2image()" or "dcraw_process()". So the quoted codes I sent aren't all there is to see regarding how the raw_data gets populated from a Canon CR2 file , right? Again, the code I sent are only memory allocation, so am i not missing the part where the raw_image is populated with the actual raw data and is not just given the pointer to an initialized, non-populated buffer? That's basically what I'm missing: where in the code is that buffer, whose pointer is given to raw_image, populated with the raw data from the CR2 file.
Just to clarify something else. Can I assume i'm not going into the rawSpeed related blocks when using Bayer image from canon CR2? This is to make sure i'm not missing anything in the pipeline. It was not clear to me what rawSpeed was, and if having a canon DSLR (5d mark III) was of any concern to this.
imgdata.image is populated in
1) raw2image() call (use it for compatibility, if you need to use 4-component image[] in your code)
2) dcraw_process() calls raw2image_ex() call, which do populating and black level extraction in single pass.
raw_alloc is just a pointer to allocation (to be free()-ed at recycle() call)
raw_image, color3_image and color4_image are pointers to pixel buffer (allocated by LibRaw or by RawSpeed). Only one pointer is non-zero for given image and this is the only way to know exact image format (1-component bayer/BW, or 3-component LinearDNG /Canon sRAW extracted by RawSpeed, or 4-component 3/4 color image extracted by LibRaw /LinearDNG, Canon sRAW).
The last piece of code (with imagedata.image set to 0) is a way to handle non-bayer image extracted by LibRaw: LinearDNG, canon sraw and 4-shot sinar unpackers works with imgdata.image[] (the code is from dcraw), so after unpacking we need to assign color4_image pointer (and raw_alloc for correct release in recycle()), and clear imgdata.image pointer
Things are so complicated because so many RAW flavours exists :(
Thanks Lexa, I understand a bit better now, when I look at the unpack() source code.
I'm still missing maybe a last piece of the puzzle to understand the basic pipeline in my particular case:
Canon EOS 5D mark III, and a good old fashioned EOS 350D (although what follows applies to more kind of Bayer sensors).
In unpack(), I see (assuming I do not have/use rawSpeed):
else if(imgdata.idata.filters || P1.colors == 1) // Bayer image or single color -> decode to raw_image
{
imgdata.rawdata.raw_alloc = malloc(rwidth*(rheight+8)*sizeof(imgdata.rawdata.raw_image[0]));
imgdata.rawdata.raw_image = (ushort*) imgdata.rawdata.raw_alloc;
So you seem to populate the raw_alloc (and thus raw_image?) with imgdata.image (correct me if i'm wrong). But I still have a hard time tracking up where imgdata.image was populated on the first place.
Looking into libraw_cxx.cpp, I don't see much either. It would be nice to point me where imgdata.image gets the raw data from the .CR2 file. I see hasselblad_full_load_raw() that seems to do something with it, but I'm not sure if that's what i'm looking for. Maybe it happens within open_file and subsequent "stream" functions?
Pixels (in raw_image) are recorded 'as in raw file'.
Colors are depending on camera CFA pattern.
The LibRaw::COLOR(row,col) call will return color for given pixel (row,col) are in image[] coordinate space i.e. visible area (so counted from top_margin,left_margin point of raw_image)
Ok. I'm still processing your explanations:
Let me take one step back on raw_image(). I see the initialization and dimensioning of raw_image, set as a one dimension array (row, or column, that's just an abstraction here):
imgdata.rawdata.raw_alloc = malloc(rwidth*(rheight+8)*sizeof(imgdata.rawdata.raw_image[0])); imgdata.rawdata.raw_image = (ushort*) imgdata.rawdata.raw_alloc;When i display the raw values, if I printout the raw_image[] at 4 consecutive indices,
say:
unpack(): raw_image[ i ] = 2135
unpack(): raw_image[ i+1 ] = 2091
unpack(): raw_image[ i+2 ] = 2126
unpack(): raw_image[ i+3 ] = 2174
assuming i'm in row_N, say, at the beginning of the row, are these values corresponding to RGRG?
Then if I print at
unpack(): raw_image[ i + width]
unpack(): raw_image[ i+1 + width]
unpack(): raw_image[ i+2 + width]
unpack(): raw_image[ i+3 +width]
does that go to the next row where I'd see the BGBG associated to the same pixel ? you said 4 values per pixel, as expected for Bayer image, so just checking I understand how they are set in the raw files.
Thanks
raw2images arranges pixel values (from raw_image array, 1 value per pixel) to image[][4] array (4 values per pixel).
So, only one of image[N][0..3] is non-zero after raw2image, all other 3 components are still zero.
This is 'prepare for interpolation' step.
Actual interpolation (and white balance, output color profile conversion, brightness adjustment) is made by dcraw_process()
Yes, I'm aware of how a detector is structured. I'm confused on the software side.
The RGBG is just confusing me with how they get represented in the ushort (*)[4] array.
I saw your other reply. i'm gonna do some more tests and try to understand it better, displaying Row_N and Row_N+1
Thanks.
Have you noted, that in bayer-pattern cameras (all modern, except foveon sensors) each pixel is monochrome (R _or_ G _or_ B) ?
Could not understand the question
pixels 16799706 .... 16799710
are all in the same row
So, only two colors.
Again
ROWn: RGRGRGRGRGRGR
ROWn+1: GBGBGBGBGBGB
yes, that's what I did 2 replies above with
pixInd = iwidth*row + col + i(I iterated over i 5 times).
Please see my "in-between" reply above. It's really important that I get that cleared out.
Hmm... getting there...
raw2image(): imgdata.image[ 16799706 ][0 to 3] = | 2175 | 0 | 0 | 0
raw2image(): imgdata.image[ 16799707 ][0 to 3] = | 0 | 2207 | 0 | 0
raw2image(): imgdata.image[ 16799708 ][0 to 3] = | 2134 | 0 | 0 | 0
raw2image(): imgdata.image[ 16799709 ][0 to 3] = | 0 | 2161 | 0 | 0
raw2image(): imgdata.image[ 16799710 ][0 to 3] = | 2139 | 0 | 0 | 0
So, do you mean that, above, pixInd 16799706 is telling me that R = 2175, and that
pixInd 16799707 is giving my G2 = 2207 ?
pixInd 16799708 is B = 2134
pixInd 16799709 is G = 2161
and then these 4 values are my RGBG for one non-demosaiced single colored pixel?
Also, please note, that image[] do not contain invisible pixels.
So, raw_image[(row+top_margin)*raw_width + left_margin+col] becomes just image[row*iwidth+col]
Looks normal for me (after raw2image, not dcraw_process):
Each row (or column )in bayer pattern contains only two colors:
RGRGRGRG
GBGBGBGB
(really G2 in second row)
Away from edges, I did this:
int iwidth = rawProcess.imgdata.sizes.iwidth; // somewhere in the middle of the picture, far from edges. int row = 2898; int col = 2898; // Display just 5 pixels int nPixels = 5; int first_visible_pixel = rawProcess.imgdata.sizes.raw_width*rawProcess.imgdata.sizes.top_margin + rawProcess.imgdata.sizes.left_margin;then looping over a pixel index with i from 0 to 4 in :
pixInd = first_visible_pixel + iwidth*row + col+ i;rawProcess.imgdata.rawdata.raw_image[pixInd] gives:
unpack(): raw_image[ 17273428 ] = 2135
unpack(): raw_image[ 17273429 ] = 2091
unpack(): raw_image[ 17273430 ] = 2126
unpack(): raw_image[ 17273431 ] = 2174
unpack(): raw_image[ 17273432 ] = 2191
after raw2image(), imgdata.image[pixInd][0 to 3] gives, with the following printout arguments:
image[pixInd][0] | image[pixInd][1] | image[pixInd][2] | image[pixInd][3],
with now,
pixInd = iwidth*row + col + i;raw2image(): imgdata.image[ 16799706 ][0 to 3] = | 2175 | 0 | 0 | 0
raw2image(): imgdata.image[ 16799707 ][0 to 3] = | 0 | 2207 | 0 | 0
raw2image(): imgdata.image[ 16799708 ][0 to 3] = | 2134 | 0 | 0 | 0
raw2image(): imgdata.image[ 16799709 ][0 to 3] = | 0 | 2161 | 0 | 0
raw2image(): imgdata.image[ 16799710 ][0 to 3] = | 2139 | 0 | 0 | 0
I'd like to understand why the values in imgdata.image, after raw2image() are put alternatively in channel 0 and channel 1.
Currently, to me these values mean (for imgdata.image[][]):
pixel 16799706 has R = 2175 , G = 0, B = 0, G2 = 0
pixel 16799707 has R = 0 , G = 2207, B = 0, G2 = 0
etc...
Which i probably misunderstand as my image is clearly filled with all colors.
What am i missing this time?
Oh ok, indeed I was wondering about 0 values may be due to edges.
Ok, i'll try inspecting away from edges.
You cleared out my question on the dimensions, thanks a lot!
Raphael
row and column are both in 1-st dimension.
The (row,col) pixel values are in
image[row*imgdata.sizes.iwidth + col] [0..3]
iwidth is equal to imgdata.sizes.width for normal processing and width/2 for 'half' interpolation (where imgdata.params.half_size is non-zero).
Border values (1st/last row or 1st/last column) in image may be not fully interpolated because there is no data for it.
Better use image[iwidth*2 + 2] for inspection (this is pixel at (2,2))
Ok, I understand the Bayer pattern of RGBG. My question is just to be clear on how to read the dimensions.
By printing out:
raw2image(): imgdata.image[ 0 ][0 to 3] = | 2141 | 0 | 0 | 0
raw2image(): imgdata.image[ 1 ][0 to 3] = | 0 | 2098 | 0 | 0
raw2image(): imgdata.image[ 2 ][0 to 3] = | 2034 | 0 | 0 | 0
raw2image(): imgdata.image[ 3 ][0 to 3] = | 0 | 2084 | 0 | 0
Which dimension is going across the R G B G values. At first I thought it was [0 to 3] (2nd dimensions, "columns") which was giving the R G B G (which I abusively call the "channels"). Is that the other way round? Are they, in fact, in the 1st dimension (rows)? I'm often confused on the dimension when I see tables of pointers such as:
imgdata.image = (ushort (*)[4]) calloc(S.iwidth*S.iheight,sizeof(*imgdata.image));You didn't say anything about the other question:
dcraw_processed(): imgdata.image[ 0 ][0 to 3] = | 1576 | 0 | 688 | 0
Is the row here the demosaiced RGB values of a given pixel (from left to right column) ?
Thanks
raw2image() put pixel values according to color of this particular pixel (0 for Red, 1 and 3 for G/G2, 2 for Blue)
I've been doing some testing. I've selected the 4 first pixels of the data (I ignored the hidden ones for calibration), for a .CR2 file (canon 5D), at 3 stages of processing: after unpack(), after raw2image(), after dcraw_processed(). See below (the 4 "channels" of the 2nd dimension are separated by the separator " | "):
user_qual = -1
imgdata.idata.cdesc = RGBG
unpack(): raw_image[ 0 ] = 2141
unpack(): raw_image[ 1 ] = 2098
unpack(): raw_image[ 2 ] = 2034
unpack(): raw_image[ 3 ] = 2084
raw2image(): imgdata.image[ 0 ][0 to 3] = | 2141 | 0 | 0 | 0
raw2image(): imgdata.image[ 1 ][0 to 3] = | 0 | 2098 | 0 | 0
raw2image(): imgdata.image[ 2 ][0 to 3] = | 2034 | 0 | 0 | 0
raw2image(): imgdata.image[ 3 ][0 to 3] = | 0 | 2084 | 0 | 0
dcraw_processed(): imgdata.image[ 0 ][0 to 3] = | 1576 | 0 | 688 | 0
dcraw_processed(): imgdata.image[ 1 ][0 to 3] = | 616 | 136 | 626 | 0
dcraw_processed(): imgdata.image[ 2 ][0 to 3] = | 0 | 132 | 371 | 0
dcraw_processed(): imgdata.image[ 3 ][0 to 3] = | 0 | 287 | 36 | 0
Why aren't the raw2image() values in the same column and instead, dispatched alternatively in the 0th and 1st column? Is that just to prepare the data for later demosaicing in a way that's compatible with all possible sensors? The values are indeed identical to the raw values. So it is unclear what happens between unpack() raw_image[] values and raw2image() image[][] values
Are the dcraw_processed() values dispatched in the 1st three columns to be considered as the R | G | B (and 4th is unused) components, after demosaicing?
Thanks
Oh dear... that explains all! Thanks a lot!!!!!
Raphael
raw_image values are read in *_load_raw() call (specific to image format) called by
(this->*load_raw)();
Initialization of rawdata buffer in unpack(), can't miss it, it's clear. Populated this raw data buffer, still can't see it within unpack(). I make here a distinction between initializing something, and assigning value (which you call "populate" i guess). That's that part I still don't grasp.
Consider my test (it's done in Qt framework, hence the qDebug() instead of printf() for printing out in debug mode):
LibRaw rawProcess; rawProcess.open_file("/......./F36A7292.CR2"); rawProcess.unpack() // printout raw values int first_visible_pixel = rawProcess.imgdata.sizes.raw_width*rawProcess.imgdata.sizes.top_margin + rawProcess.imgdata.sizes.left_margin; for( int i=0; i< 100; i++) { qDebug() << "raw_image["<< i << "] =" << rawProcess.imgdata.rawdata.raw_image[i+first_visible_pixel]; }This is giving me my raw values.
And I still don't see how the raw_image values get assigned (populated) within unpack(). In what I pasted in the early post, i don't see the lines of codes where this happen, for the Canon case (Nikon case seems a bit different). All I see is initialized buffer, and freed buffer (for Bayer image, and non-Nikon)
Sorry for repeating the question, I must be missing something obvious. Maybe a slap in my face will make me see. (by Copy pasting that invisible line that I don't see?)
If you don't want to repeat yourself, I'll understand and will go back again to what you say and try to find what i missed.
Thanks.
No-no-no
load_raw() uses preallocated buffer (unles OWNALLOC specified in decoder flags).
Again:
imgdata.image[] is allocated and populated in raw2image() or raw2image_ex() calls.
imgdata.rawdata.* pointers are initialized in unpack() call.
Is that in fact in
(this->*load_raw)()(within unpack())that the raw_data buffer gets populated?
imgdata.image appears temporarily within unpack() (before (*load_raw)() call) if needed, than hided again.
If you need imgdata.image[] in your code, call raw2image() after unpack() to get it.
I'm confused again. Maybe I missed something else.
In the code samples that I pasted, my trouble was that these were only memory allocation of buffers, and I failed to see where the data from the .CR2 file where going. Then imgdata.image seemed the only time when some non-zero data were passed. But I'm wrong since from what you say imgdata.image is not populated with any raw or processed data at that time: the imgdata.image gets populated only after "raw2image()" or "dcraw_process()". So the quoted codes I sent aren't all there is to see regarding how the raw_data gets populated from a Canon CR2 file , right? Again, the code I sent are only memory allocation, so am i not missing the part where the raw_image is populated with the actual raw data and is not just given the pointer to an initialized, non-populated buffer? That's basically what I'm missing: where in the code is that buffer, whose pointer is given to raw_image, populated with the raw data from the CR2 file.
Just to clarify something else. Can I assume i'm not going into the rawSpeed related blocks when using Bayer image from canon CR2? This is to make sure i'm not missing anything in the pipeline. It was not clear to me what rawSpeed was, and if having a canon DSLR (5d mark III) was of any concern to this.
Thanks (a lot!)
imgdata.image is populated in
1) raw2image() call (use it for compatibility, if you need to use 4-component image[] in your code)
2) dcraw_process() calls raw2image_ex() call, which do populating and black level extraction in single pass.
raw_alloc is just a pointer to allocation (to be free()-ed at recycle() call)
raw_image, color3_image and color4_image are pointers to pixel buffer (allocated by LibRaw or by RawSpeed). Only one pointer is non-zero for given image and this is the only way to know exact image format (1-component bayer/BW, or 3-component LinearDNG /Canon sRAW extracted by RawSpeed, or 4-component 3/4 color image extracted by LibRaw /LinearDNG, Canon sRAW).
The last piece of code (with imagedata.image set to 0) is a way to handle non-bayer image extracted by LibRaw: LinearDNG, canon sraw and 4-shot sinar unpackers works with imgdata.image[] (the code is from dcraw), so after unpacking we need to assign color4_image pointer (and raw_alloc for correct release in recycle()), and clear imgdata.image pointer
Things are so complicated because so many RAW flavours exists :(
Thanks Lexa, I understand a bit better now, when I look at the unpack() source code.
I'm still missing maybe a last piece of the puzzle to understand the basic pipeline in my particular case:
Canon EOS 5D mark III, and a good old fashioned EOS 350D (although what follows applies to more kind of Bayer sensors).
In unpack(), I see (assuming I do not have/use rawSpeed):
else if(imgdata.idata.filters || P1.colors == 1) // Bayer image or single color -> decode to raw_image { imgdata.rawdata.raw_alloc = malloc(rwidth*(rheight+8)*sizeof(imgdata.rawdata.raw_image[0])); imgdata.rawdata.raw_image = (ushort*) imgdata.rawdata.raw_alloc;(...)
(...)
// recover saved if( !(imgdata.idata.filters || P1.colors == 1) && !imgdata.rawdata.color4_image) { imgdata.image = 0; imgdata.rawdata.color4_image = (ushort (*)[4]) imgdata.rawdata.raw_alloc; }So you seem to populate the raw_alloc (and thus raw_image?) with imgdata.image (correct me if i'm wrong). But I still have a hard time tracking up where imgdata.image was populated on the first place.
Looking into libraw_cxx.cpp, I don't see much either. It would be nice to point me where imgdata.image gets the raw data from the .CR2 file. I see hasselblad_full_load_raw() that seems to do something with it, but I'm not sure if that's what i'm looking for. Maybe it happens within open_file and subsequent "stream" functions?
Thank you for your help
Raphael
Pages