Both 2272 and 250 are too Both 2272 and 250 are too high values for gradient variable b/c table size is only 41. gradient is calculated by: #define fuji_quant_gradient(i, v1, v2) (9 * i->q_table[i->q_point[4] + (v1)] + i->q_table[i->q_point[4] + (v2)]) (at line 402: grad = fuji_quant_gradient(params, Rb - Rf, Rc - Rb); ) q_table contains values from -4 to +4, so maximum gradient value is 40. Could you please try to dump q_table values? And, also, values used to calc gradient: Rb - Rf, Rc - Rb -- Alex Tutubalin @LibRaw LLC reply
Both 2272 and 250 are too high values for gradient variable b/c table size is only 41.
gradient is calculated by:
q_table contains values from -4 to +4, so maximum gradient value is 40.
Could you please try to dump q_table values? And, also, values used to calc gradient: Rb - Rf, Rc - Rb