Alex,
For the black-levels that are specified in cblack[0]->cblack[3], do those also need to be applied starting at the visible region. I am assuming this is the case because the subtract_black_internal operates on imgdata.image which only operates on the visible image.
Also, for the white-levels, if linear_max is all zeros, then do I just assume maximum applies to all the channels.
Also, if both maximum and linear_max value is present, which value should I use?
Alex,
Thanks for this clarification. So it looks like my best bet would be to call raw2image() and subtract_black() to leverage libraw to do the black-level correction for me. I still would like to return the CFA data as a rows x columns matrix instead of the output of raw2image. I guess I have to add some custom logic to do the copying.
Also, it appears that raw2image() only returns the visible portion of the image. It typically has only 1 out of the 4 channels as non-zero. I am assuming this does not hold true for Fovean sensors or DNG files that are created from Sigma cameras?
Can I use the COLOR function for non-Bayer sensors? I have a DNG file that appears to be created from a Sigma camera. The output of COLOR(0, 0), COLOR(0, 1) returned 6.
Yes, this camera uses E-M5MarkII string in EXIF 0x0110 tag (Model).
LibRaw 0.20 (Beta1 just published) provides 'E-M5 Mark II' string in normalized_model[]
BTW, LibRaw::cameraList() is 'user readable', it contains some notes and special cases (like 'lossless only' or 'CHDK hack'). It does not targeted to be matched against model/normalized_model.
Yes, black/cblack reflects different black level specifications in RAW metadata
black - single black level
cblack[0-3] per channel BL in channel numbering order (channel numbers are returned by COLOR(..))
cblack[4-5] defines pattern BL (e.g. 6x6 for X-Trans)
As a rule: black is base level and cblack is correction, so if black is non-zero and cblack[.. is non-zero (for given channel or row/col), resulting BL for pixel/channel is a sum of black + cblack[channel] + cblack[pattern]. In practice all three are rarely present.
white is either format maximum (defined by bits per pixel), or maximum value defined by metadata (DNG whitelevel).
linear_max is 'specular white' read from metadata (if present in file)
Yeah, the weird histogram is from the inverted data. I think I'm going to have to deeply familiarize myself with what scale_colors does, because I think something in that process is relying on values that were generated prior to inverting the raw data.
Alex,
So it looks like cmatrix and ccm are also for converting from camera space to sRGB. Correct?
The ccm appears to be only read from the file and not computed and it appears to be normalized by sum(imgdata.colors.ccm).
cmatrix also appears to be either read from the file or derived from camera properties.
So, if these are not available in the file, I can assume this to be all zeros?
Is it safe to assume that cam_xyz and rgb_cam are sufficient for most color space transformations?
* rgb_cam for direct conversion to sRGB?
* pseudoinverse(cam_xyz) to move to XYZ space and then apply any suitable transformation to move to a colorspace such as Rec.2020 or ProPhoto of my choice?
Also, a quick question about wb_coeffs. Are the coeffs in the order of the CFA pattern OR are they always R G1 B G2?
But I think the auto scaling is having some issues scaling the channels correctly. I think it's probably because the upper and lower bounds of each channel were calculated with the un-inverted negative?
Alex,
I am not clear if my earlier follow up was actually posted. So apologies for a repeated post.
The link you had sent me was really helpful. I think I now understand how the cam_xyz, rgb_cam, cam_mul and pre_mul matrices work.
However, the code referenced in the link appears to be older and hence does not contain any reference to ccm. Also, the link does not contain any information/comments about cmatrix.
I just noticed from the latest code that appears to populate cmatrix primarily through calls to cam_xyz_coeff.
Would appreciate any help about these two matrices (ccm and cmatrix)?
no_auto_scale=1 is very special use parameter. If set, it will instruct dcraw_process() to skip scale_colors() step (this step performs WB and data scaling). This is not intended to use with standard LibRaw's interpolation methods, this parameter is targeted if one implements own interpolation (demosaic) that wants to deal with unchanged (not scaled) source data.
It looks like we need to document no_auto_scale better in LibRaw docs :)
To manual control output brightness, use no_auto_bright=1 and change imgdata.params.bright from 1 to something.
I tried auto white balancing and it went a long way toward fixing the issues. I did have to manually reduce the brightness though. Is there a way to control the brightness adjustment in ppm_tiff_writer?
I'm also noticing that the red and blue channels appear very compressed in the output image relative to the green channel. I'm not sure what would cause that.
[edit] I set no_auto_bright = 1 and no_auto_scale = 1 and that seems to have helped. Not sure why one of those would have compressed one channel and not the others. I think I'll probably need to do some custom analysis of the negative to set the white balance and scale of the resulting image. Something about inverting the values seems to throw off the automatic processes.
Alex,
For the black-levels that are specified in cblack[0]->cblack[3], do those also need to be applied starting at the visible region. I am assuming this is the case because the subtract_black_internal operates on imgdata.image which only operates on the visible image.
Also, for the white-levels, if linear_max is all zeros, then do I just assume maximum applies to all the channels.
Also, if both maximum and linear_max value is present, which value should I use?
Regards,
Dinesh
This depends on beta feedback. Hope to finish Beta-RC-Release cycle in May.
Is there any schedule / target / roadmap currently for when the final release will drop?
Followup, 6 is intended value for non-bayer:
int COLOR(int row, int col) { if (!imgdata.idata.filters) return 6; /* Special value 0+1+2+3 */ ...COLOR() for non-bayer(x-trans) data is useless, check filters too before use.
Alex,
Thanks for this clarification. So it looks like my best bet would be to call raw2image() and subtract_black() to leverage libraw to do the black-level correction for me. I still would like to return the CFA data as a rows x columns matrix instead of the output of raw2image. I guess I have to add some custom logic to do the copying.
Also, it appears that raw2image() only returns the visible portion of the image. It typically has only 1 out of the 4 channels as non-zero. I am assuming this does not hold true for Fovean sensors or DNG files that are created from Sigma cameras?
Can I use the COLOR function for non-Bayer sensors? I have a DNG file that appears to be created from a Sigma camera. The output of COLOR(0, 0), COLOR(0, 1) returned 6.
Dinesh
Yes, you're the only person affected.
We do not log authentication data on our sites, so no way to recover the problem source. If you have your HTTP sessions dump it may help.
Hmmm.... That's what you said last time I reported this (about two years ago).
Oh well.
No plans for publishing binaries or source tarballs for beta.
Binaries and tarballs will be updated on 'release' status.
We use this site engine (drupal 7) for many sites for many years. Never seen such problem.
Hi!
Is it possible to try Windows binaries of v0.20 beta?
Or when binaries will be available?
Thanks.
Yes, this camera uses E-M5MarkII string in EXIF 0x0110 tag (Model).
LibRaw 0.20 (Beta1 just published) provides 'E-M5 Mark II' string in normalized_model[]
BTW, LibRaw::cameraList() is 'user readable', it contains some notes and special cases (like 'lossless only' or 'CHDK hack'). It does not targeted to be matched against model/normalized_model.
Yes, black/cblack reflects different black level specifications in RAW metadata
black - single black level
cblack[0-3] per channel BL in channel numbering order (channel numbers are returned by COLOR(..))
cblack[4-5] defines pattern BL (e.g. 6x6 for X-Trans)
As a rule: black is base level and cblack is correction, so if black is non-zero and cblack[.. is non-zero (for given channel or row/col), resulting BL for pixel/channel is a sum of black + cblack[channel] + cblack[pattern]. In practice all three are rarely present.
white is either format maximum (defined by bits per pixel), or maximum value defined by metadata (DNG whitelevel).
linear_max is 'specular white' read from metadata (if present in file)
Yeah, the weird histogram is from the inverted data. I think I'm going to have to deeply familiarize myself with what scale_colors does, because I think something in that process is relying on values that were generated prior to inverting the raw data.
wb_coeffs are in channel numbering order (as returned by COLOR(row,col))
Alex,
So it looks like cmatrix and ccm are also for converting from camera space to sRGB. Correct?
The ccm appears to be only read from the file and not computed and it appears to be normalized by sum(imgdata.colors.ccm).
cmatrix also appears to be either read from the file or derived from camera properties.
So, if these are not available in the file, I can assume this to be all zeros?
Is it safe to assume that cam_xyz and rgb_cam are sufficient for most color space transformations?
* rgb_cam for direct conversion to sRGB?
* pseudoinverse(cam_xyz) to move to XYZ space and then apply any suitable transformation to move to a colorspace such as Rec.2020 or ProPhoto of my choice?
Also, a quick question about wb_coeffs. Are the coeffs in the order of the CFA pattern OR are they always R G1 B G2?
Regards,
Dinesh
Is this histogram is from inverted data?
If so, I could not answer the question, I simply do not know what happens w/ processing if channels are inverted.
Here are histograms of your tether8387 file processed with
1) dcraw_emu -T -W -a -b 1 (Tiff, no_auto_bright,auto-wb, bright=1): https://www.dropbox.com/s/6l67ry7y03xlj7p/screenshot%202020-05-04%2009.0...
2) same, but -b 0.8: https://www.dropbox.com/s/7o1w581gamxy4nz/screenshot%202020-05-04%2009.0...
I do not see anything not expected here.
I
So I've made some progress by setting:
use_auto_wb = 1
no_auto_bright = 1
bright = 0.8
But I think the auto scaling is having some issues scaling the channels correctly. I think it's probably because the upper and lower bounds of each channel were calculated with the un-inverted negative?
This is the original histogram from RawDigger: http://ur.sine.com/temp/tether8387-Full-8000x5320.png
...and this is the histogram from the generated TIFF from Rawtherapee: http://ur.sine.com/temp/post-process-channels.png
Is there a good way to have libRaw recalculate the per-channel min and max, or alternatively, to set them manually?
Accidentally replied to the wrong thread!
Please see https://www.libraw.org/news/libraw-snapshot-201910
I hope you support Canon Eos RP raw format (cr3).
cmatrix is camera color data similar to rgb_cam, but calculated from in-RAW color data.
ccm is also 'camera color matrix' retireved from RAW in 'as is' (excl. normalizing) form.
Alex,
I am not clear if my earlier follow up was actually posted. So apologies for a repeated post.
The link you had sent me was really helpful. I think I now understand how the cam_xyz, rgb_cam, cam_mul and pre_mul matrices work.
However, the code referenced in the link appears to be older and hence does not contain any reference to ccm. Also, the link does not contain any information/comments about cmatrix.
I just noticed from the latest code that appears to populate cmatrix primarily through calls to cam_xyz_coeff.
Would appreciate any help about these two matrices (ccm and cmatrix)?
Dinesh
no_auto_scale=1 is very special use parameter. If set, it will instruct dcraw_process() to skip scale_colors() step (this step performs WB and data scaling). This is not intended to use with standard LibRaw's interpolation methods, this parameter is targeted if one implements own interpolation (demosaic) that wants to deal with unchanged (not scaled) source data.
It looks like we need to document no_auto_scale better in LibRaw docs :)
To manual control output brightness, use no_auto_bright=1 and change imgdata.params.bright from 1 to something.
I tried auto white balancing and it went a long way toward fixing the issues. I did have to manually reduce the brightness though. Is there a way to control the brightness adjustment in ppm_tiff_writer?
I'm also noticing that the red and blue channels appear very compressed in the output image relative to the green channel. I'm not sure what would cause that.
[edit] I set no_auto_bright = 1 and no_auto_scale = 1 and that seems to have helped. Not sure why one of those would have compressed one channel and not the others. I think I'll probably need to do some custom analysis of the negative to set the white balance and scale of the resulting image. Something about inverting the values seems to throw off the automatic processes.
Pages