Sure, I will send it. But actually, I know what the problem is, I just remembered. The camera is a Canon SD980 running CHDK, and I helped porting that CHDK version (at the time this camera was using a newer API set, so CHDK didn't work on it, it needed quite a bit of reverse engineering). And the port might have had some stuff missing.
The interesting thing is that all programs I tried worked fine with it, so I guess they had some detection for invalid Wb settings, and calculated their own.
So probably the DNG is not too useful for you, as only my camera is affected by it (all my other RAWs from other CHDK cameras are OK). But if you still want it, I can send it to you.
Use daylight or auto multipliers if camera WB looks wrong (as in this case). As a general rule, red/blue multipliers should be in 0.05..20.0 (or may be less) range after normalization (green one set to 1.0)
BTW, could you please share this DNG (e.g. send link to info@libraw.org if you do not want to share it public) for detailed examination?
I use the cam_mul for WB, and then do the debayering (manually) and after that I do the sRGB stuff.
You also have to do channel clipping before the sRGB conversion (and optionally highlight recovery).
A good place to start IMHO is "Image sensors and signal processingfor digital still cameras", edited by Junichi Nakamura and "Way Beyond Monochrome" by Ralph Lambrecht. One book covers the digital part, the other speaks of photographic process.
To get flatten output you may try samples/unprocessed_raw sample
B-channel pattern you show looks ok (one value in 2x2 square), while R-channel is not. Could you please upload raw somewhere and provide the link for in-depth analysis?
1) To disable auto-brightening of images:
- set params.no_auto_bright to 1
- set params.adjust_maximum_thr to 0.0 (in this case, for some cameras you may need to set params.user_sat /white point/ manually to avoid pink tint in highlights)
Afterh that, images (with same exposure) will be processed with same data maximum and no automatic brightening.
2) To adjust brightness (at output stage) use params.bright to some value above 1.0
Thank you so much! Is there any way that I can scale the colors manually, to make the above two images the same?
Also, what does auto_bright exactly do? You said in some post that it changes the histogram to make the image brighter. Is it histogram equalization on the output sRGB image?
I open one of my images in rawdigger and noticed that the min and max values for the colours are really different:
(values are min max avg and sigma)
R 2 2521 307 446
G 8 4045 716 1004
B 1 4046 683 1038
G2 5 4045 716 1004
This pef file can be found here: https://drive.google.com/file/d/1TcCcoIL-q7K85UAJC4FcwtpDdlihDuoP/view?u...
Is there anything I can do to correct this image (and about 50 other images I have with similar problem)? There are some wedding pictures among them and I'd like to get these fixed...
There is certain amount of flare, and possibly the exposure was too much, but I can't say for sure without the raw. You can download free trial of our RawDigger and check the exposure yourself of course: https://www.rawdigger.com/download
I have created a jpg from my raw file, if needed I can also share the raw file. The fact is that for all raw images from a certain day, this effect is present in the same are (left-hand side of each picture). Or is it then the intensity that I need to reduce for the raw image?
There is no problem with white balance on the image. Possibly you mean that the shot contains blown-out areas, but I can't say how much of those can be brought back as the image you've uploaded is in fact JPEG, not raw PEF.
It is unclear how do you mean saying "white balance set too high". Could you please upload a sample raw and provide a link to it, to better understand the issue?
for(int i = 0; i < 1000; i++)
gamma_curve (O.gamm[0], O.gamm[1], 2, (t_white << 3)/O.bright);
And run Canon 5Ds image though profiler (Intel VTune amplifier). Gamma curve (1000 repetitions) takes 0.619 seconds, so usual one-time run should be about than 0.6 msec
CPU: i7-7700k at base speed.
Other processing: 411ms for decode, 700ms for scale colors, 900ms for (fastest available) linear interpolation, so about 2sec total.
So, gamma_curve is less than 1/3000 of total processing time for such large files.
Either your profiler is wrong, or your math library (pow(), log(), or exp() functions) is incredibly slow.
Well, after some tests and thinking, it downed to me that I should just adjust the gamma curve, there is no point in multiplying everything with a constant multiplyer.
So my function is now like this, and it works (but not exactly how it does in RawTherapee)
I also want to pick a single gamma curve and apply those same remap value to hundreds or thousands of images. So having the ability to directly manipulate the libraw_colordata_t.curve[] data in lieu of defining gammas would be VERY nice. I find on 50mp canon cr2 images calculating the gamma curve remap per image is almost 15s per image on an ivy bridge xeon.
Hi Alex,
I'm the rawpy developer and just had a look at this. I'm using LibRaw::COLOR(row, col) to construct the color pattern matrix, which works fine for non-x-trans images. But for x-trans any value after the first row returns 0. so ::COLOR(1, 0) is 0, as well as ::COLOR(1,1) is 0. I'm using libraw 0.18.2 currently and noticed some x-trans related changes in newer versions: https://github.com/LibRaw/LibRaw/compare/0.18.2...0.18.5 But they seem to be related to security only.
My test image is: http://img.photographyblog.com/reviews/fujifilm_x_t1/photos/fujifilm_x_t...
Can you check whether the COLOR method works correctly here? I think there may be a problem.
Thanks!
Maik
Yes, we collect any 'strange' DNG/RAW files to make sure LibRaw works fine even in corner cases :)
Sure, I will send it. But actually, I know what the problem is, I just remembered. The camera is a Canon SD980 running CHDK, and I helped porting that CHDK version (at the time this camera was using a newer API set, so CHDK didn't work on it, it needed quite a bit of reverse engineering). And the port might have had some stuff missing.
The interesting thing is that all programs I tried worked fine with it, so I guess they had some detection for invalid Wb settings, and calculated their own.
So probably the DNG is not too useful for you, as only my camera is affected by it (all my other RAWs from other CHDK cameras are OK). But if you still want it, I can send it to you.
Use daylight or auto multipliers if camera WB looks wrong (as in this case). As a general rule, red/blue multipliers should be in 0.05..20.0 (or may be less) range after normalization (green one set to 1.0)
BTW, could you please share this DNG (e.g. send link to info@libraw.org if you do not want to share it public) for detailed examination?
I use the cam_mul for WB, and then do the debayering (manually) and after that I do the sRGB stuff.
You also have to do channel clipping before the sRGB conversion (and optionally highlight recovery).
Fantastic! Thanks for the recommendations!
A good place to start IMHO is "Image sensors and signal processingfor digital still cameras", edited by Junichi Nakamura and "Way Beyond Monochrome" by Ralph Lambrecht. One book covers the digital part, the other speaks of photographic process.
Got it. Thanks for taking the time to reply.
If you had any book recommendations that help explain how to do this postprocessing, I'd be very interested.
Thanks again, and thanks for making this amazing library available!
FRV does not use LibRaw for raw image postprocessing, only raw decode phase is used (LibRaw::open_file/LibRaw::unpack).
'Variable contrast' curves in FRV are based on real photographic film contrast curves.
To get flatten output you may try samples/unprocessed_raw sample
B-channel pattern you show looks ok (one value in 2x2 square), while R-channel is not. Could you please upload raw somewhere and provide the link for in-depth analysis?
1) To disable auto-brightening of images:
- set params.no_auto_bright to 1
- set params.adjust_maximum_thr to 0.0 (in this case, for some cameras you may need to set params.user_sat /white point/ manually to avoid pink tint in highlights)
Afterh that, images (with same exposure) will be processed with same data maximum and no automatic brightening.
2) To adjust brightness (at output stage) use params.bright to some value above 1.0
Thank you so much! Is there any way that I can scale the colors manually, to make the above two images the same?
Also, what does auto_bright exactly do? You said in some post that it changes the histogram to make the image brighter. Is it histogram equalization on the output sRGB image?
Thanks!
Hi Iliah,
Rawdigger gave me new insight :-)
I open one of my images in rawdigger and noticed that the min and max values for the colours are really different:
(values are min max avg and sigma)
R 2 2521 307 446
G 8 4045 716 1004
B 1 4046 683 1038
G2 5 4045 716 1004
This pef file can be found here:
https://drive.google.com/file/d/1TcCcoIL-q7K85UAJC4FcwtpDdlihDuoP/view?u...
Is there anything I can do to correct this image (and about 50 other images I have with similar problem)? There are some wedding pictures among them and I'd like to get these fixed...
Regards, Ronald.
Dear Sir:
There is certain amount of flare, and possibly the exposure was too much, but I can't say for sure without the raw. You can download free trial of our RawDigger and check the exposure yourself of course: https://www.rawdigger.com/download
Hi Iliah,
I have created a jpg from my raw file, if needed I can also share the raw file. The fact is that for all raw images from a certain day, this effect is present in the same are (left-hand side of each picture). Or is it then the intensity that I need to reduce for the raw image?
Regards, Ronald.
Dear Sir:
There is no problem with white balance on the image. Possibly you mean that the shot contains blown-out areas, but I can't say how much of those can be brought back as the image you've uploaded is in fact JPEG, not raw PEF.
Hi Iliah,
if you look at the following picture https://drive.google.com/file/d/1autVVPZejMNJjeltv9E4_o8JqW266qukEw/view... you can see the effect. Does this help to understand the issue with the pictures and what I'd like to accomplish?
Regards, Ronald.
It is unclear how do you mean saying "white balance set too high". Could you please upload a sample raw and provide a link to it, to better understand the issue?
Just changed (in copy_mem_image code)
From:
to
And run Canon 5Ds image though profiler (Intel VTune amplifier). Gamma curve (1000 repetitions) takes 0.619 seconds, so usual one-time run should be about than 0.6 msec
CPU: i7-7700k at base speed.
Other processing: 411ms for decode, 700ms for scale colors, 900ms for (fastest available) linear interpolation, so about 2sec total.
So, gamma_curve is less than 1/3000 of total processing time for such large files.
Either your profiler is wrong, or your math library (pow(), log(), or exp() functions) is incredibly slow.
Thank you.
Corrected on site (/documentation) and in current 0.19-branch (to be published on next update)
Yes, this will effectively map [0...max_value/exposure] to full output range.
Well, after some tests and thinking, it downed to me that I should just adjust the gamma curve, there is no point in multiplying everything with a constant multiplyer.
So my function is now like this, and it works (but not exactly how it does in RawTherapee)
void do_color_curve() { int i; double resulting_gamma; double max_color=original_exposure_max_channels*(1.0f/exposure); for(i=0;i<65535;i++) { double i_double=i; resulting_gamma=pow(i_double/(double)max_color,1.0f/gamma); if(resulting_gamma>1)resulting_gamma=1; color_curve[i]=resulting_gamma*255; } }I also want to pick a single gamma curve and apply those same remap value to hundreds or thousands of images. So having the ability to directly manipulate the libraw_colordata_t.curve[] data in lieu of defining gammas would be VERY nice. I find on 50mp canon cr2 images calculating the gamma curve remap per image is almost 15s per image on an ivy bridge xeon.
OK, looks like over-optimization in COLOR:
int COLOR(int row, int col) { return libraw_internal_data.internal_output_params.fuji_width ? FCF(row, col) : FC(row, col); } int FC(int row, int col) { return (imgdata.idata.filters >> (((row << 1 & 14) | (col & 1)) << 1) & 3); } int fcol(int row, int col);where FCF() is for 'fuji-rotated' images (S5 Pro, etc).
It should be
int COLOR(int row, int col) { return imgdata.idata.filters < 1000 ? fcol(row,col): (libraw_internal_data.internal_output_params.fuji_width ? FCF(row, col) : FC(row, col)); }To be fixed in 0.19 (0.18 is security-only-patches now)
Hi Alex,
I'm the rawpy developer and just had a look at this. I'm using LibRaw::COLOR(row, col) to construct the color pattern matrix, which works fine for non-x-trans images. But for x-trans any value after the first row returns 0. so ::COLOR(1, 0) is 0, as well as ::COLOR(1,1) is 0. I'm using libraw 0.18.2 currently and noticed some x-trans related changes in newer versions: https://github.com/LibRaw/LibRaw/compare/0.18.2...0.18.5 But they seem to be related to security only.
My test image is: http://img.photographyblog.com/reviews/fujifilm_x_t1/photos/fujifilm_x_t...
Can you check whether the COLOR method works correctly here? I think there may be a problem.
Thanks!
Maik
That was it, thank you very much!
I changed my code to:
libraw_raw2image(raw_struct);
libraw_subtract_black(raw_struct);
And it works very nicely :) (well, still have to do other things now, but the results are much better).
Pages