If we need to override data supplied in one of the DNG fields using hard-wired data or specific algorithms, all derived from original raw, - what is the place for DNG than? Each DNG interpreter will do the interpretation in its own way, just as it happens with original raw.
When you say "the DNG proper" are you referring to the XMP space? Or to the Adobe-generated tags? Isn't the EXIF data available to the third-party raw converter from the Adobe-created DNG? How specifically would you suggest modifying the spec, and/or modifying the behavior of the DNG converter?
Is there anything else you can put your finger on that is concrete? I hope we can agree that it is impossible for a live parametric rendering to be identical from program to program, and that the DNG presents the best current possibility for attaching universally accessible renderings to raw image data. I hope we can agree that the 1.2 extension made the format considerably more friendly to third parties. I also hope we can agree that full backwards compatibility of rendering engines is not going to happen. I hope we can agree that forward compatibility is the most important goal, and that there are some really imaginative solutions included in the DNG that are not found anywhere else.
I also hope we can agree that most of the other points you've identified are ones that are coming from the camera manufacturer side, such as the difficulty of parsing some encoded or encrypted data, and that camera manufacturers don't publish the spectral response of their cameras.
I'd say that if we are on the same page, a more appropriate title for the article, regarding DNG, would be "A few small suggestions for the next DNG spec" rather than "Path to Nowhere." I'll spend a lot of time undoing misconceptions that are spread with analysis like this one.
A converter can't rely on just the information as contained in DNG proper, like the black level in the example above. Interpretation of the DNG data in a raw converter often depends on the specific make/model and some other EXIF tags; and worse, converters that can render quality images from DNG need to ignore certain DNG data substituting it with hard-wired data of their own, derived from hacking the original raw format.
>> We would appreciate if instead of simply ruling out the results of our experiment
>> as purposely skewed you would do those yourself and post the results
>> proving your point.
> The premise of what you are asking for is impossible and/or undesirable.
In the first experiment, you say to alter the EXIF data, and then seem surprised that it changes the output. What is unexpected here?
In the second example, are you saying that Adobe changes an EXIF value or improperly calculates the Black Level Tag? If so, that sounds like an implementation issue, not one of the spec. (Unless you see something in the spec that indicates that the right data is being tossed out by design.
Mostly what I think you're saying is that if you pull stuff out form DNG's as Adobe makes them, they don't render predictably. Again, that does not seem to be a huge surprise.
What is the problem one would encounter if he were not trying to intentionally damage the file?
Peter
- take any .CR2 from Canon camera
- convert it to .DNG by Adobe (i.e. 'right') converter
- replace Make/Model fields by Exiftool:
exiftool -Make=xxx -Model=yyy canon_eos_50d_01-02.dng
- try to render this file with some Non-Adobe DNG-enabled applications. I've used ACDSee Pro 2.5 and shadows are _very_ different. I'm sure, that LightZone (example of another non-Adobe DNG-enabled app) will show the same problem.
Updated: ACDSee and Lightzone show *right* rendering for Make/model preserved and incorrect one if Make/Model changed in Exif.
> I have no idea what you're driving at here. Are you saying that some DNG-aware
> applications can make use of EXIF
Sure!
I've done simple experiment in two minutes
- image from Canon 50D converted to DNG with Adobe 5.2.0.65 converter
Results
* Black Level tag value: 1023 1023 1023 1023
* Black level repeat dim: 2 2
* White level tag: 13200 (i.e. data is unscaled, 14-bit preserved as is)
But the real Black Level for this image is 511, not 1023.
Right RAW processor should:
- think: "this is Canon! we should calculate black level by left side of black mask"
- ignore black level tag from DNG
- calculate black level by averaging black (masked pixels)
If you strip all vendor-specific metadata from this file you will get unacceptable nonlinearity in shadows.
>Please, in your own words, take some time to try and understand what the article is about.
I'm pretty sure I understand where you're going with it. I just think the fundamental underlying assumptions are flawed. On top of that, you don't clearly outline what is an issue with the format, and what is an implementation issue.
>In doing so you will find the answers to many of your questions right in the article, including specific suggestions and the archival value of allowing more proprietary tags like colour decoding profiles of un-specified structure.
Well, the DNG spec does currently provide for anyone to embed profiles that describe the color any way they wish. If a manufacturer wanted to create his own tags and embed, or make profiles available, it could do that, as I understand it.
I think the far more important and challenging issue is how to deal with tags for further decoding and output of the files. As imaging technology moves from simply trying to render the files "correctly" to making more creative interpretations from the captured data, the challenge becomes attaching the custom profiles that can control that output, and keeping them available as the file gets passed from user to user. In this regard , the new 2.5-D profiles that the DNG allows is a really ingenious solution. I suggest checking them out.
Charting the spectral response of the camera's chip is far less important than enabling the user to create a portable parametric rendering. I
Once again, all parametric renderings will be engine-specific, and are likely to change somewhat over time, as applications are improved. (And those are implementation decisions made by the application's creator, and not an issue of format.) In fact, the rendering does not even *exist* until the raw data is loaded into the imaging pipeline along with the instructions. That's why the ability to attach a fixed rendering is so important. It is the *only* possible solution to create a totally predictable rendering, unless we stop software from developing any further.
Likewise, *backwards* compatibility of parametric rendering engines simply cannot work the way you outline unless:
1. The engines are not allowed to change and improve. or
2. A software manufacturer is obligated to update all old versions with new capability whenever it's developed.
Neither one of these will happen.
>By the way, in your opinion, shall we consider that the history of DNG starts with v. 1.2 and ACR v. 4.5?
I'd say 1.1, rather than 1.2. That was when all image data,a swell as all private makernotes was first transferred. Prior to that, the format definitely left some stuff behind.
>In that case, it is not a bad start. But what to do with the users invested into DNG at v.1.0 stage? Were they misled?
Sure, if that's the way you want to look at it. The first implementation had problems. It would be great if all 1.0 implementations were problem-free.
>We would appreciate if instead of simply ruling out the results of our experiment as purposely skewed you would do those yourself and post the results proving your point.
Did you understand my criticisms? The premise of what you are asking for is impossible and/or undesirable.
>As one of the methods of verification archival properties, we suggest you make a DNG out of a RAW file, and then delete from a copy of that DNG all vendor-specific EXIF fields to evaluate the difference in processing results between the "pure" DNG stripped of those vendor-specific EXIF fields and more complete data.
I have no idea what you're driving at here. Are you saying that some DNG-aware applications can make use of EXIF?
When you talk about "archival" what are you really talking about? Are you talking about access to the metadata, access to a particular rendering, or the survivability of the format.
The first of these is really the province of the camera maker, who encode the data, and could make it more discoverable. The second one can *only* be addressed by an embedded rendering. And the third is an area that you glossed over entirely in your discussion of the spec. The data validation tags in DNG 1.2 are a huge leap forward in archive stability, as they offer a way to determine the file integrity automatically. I suggest you check out this part of the spec.
>We would also love to see your priorities being sorted out. You begin your second paragraph with "most important", and continue with "most importantly" in the paragraph next to your last one. Such phrasing indicates that you were rather sloppy, again, to use a word of yours, while writing your comment.
Okay, fine, it was sloppy. I should have said that all of those things are important. It's important to read and understand something before you send out a purportedly comprehensive discussion of it. And it's also important to understand what is the problem with the spec, and what is the problem with any specific implementation. And it's important to understand what's possible, and what's not.
>It is our feeling that the readers of your comment would benefit from a disclosure more detailed than just The DAM Book author; like mentioning your special relations with Adobe in the status of Adobe Alpha Tester.
Okay, yes, I am an unpaid alpha tester for Adobe, which gives me access to the development teams. And I helped Adobe improve the DNG spec, with particular emphasis on addressing the issues raised by other software vendors, such as Bibble (ask Eric - he brought specific problems to light, and they were addressed in 1.2). I also focused on the ability to facilitate predictable rendering, which can only be achieved by use of embedded rendered data. In addition, Adobe occasionally pays me (poorly) to speak. Adobe also paid me to write a white paper on this subject.
Please, in your own words, take some time to try and understand what the article is about. In doing so you will find the answers to many of your questions right in the article, including specific suggestions and the archival value of allowing more proprietary tags like colour decoding profiles of un-specified structure.
By the way, in your opinion, shall we consider that the history of DNG starts with v. 1.2 and ACR v. 4.5? In that case, it is not a bad start. But what to do with the users invested into DNG at v.1.0 stage? Were they misled?
We would appreciate if instead of simply ruling out the results of our experiment as purposely skewed you would do those yourself and post the results proving your point. As one of the methods of verification archival properties, we suggest you make a DNG out of a RAW file, and then delete from a copy of that DNG all vendor-specific EXIF fields to evaluate the difference in processing results between the "pure" DNG stripped of those vendor-specific EXIF fields and more complete data.
We would also love to see your priorities being sorted out. You begin your second paragraph with "most important", and continue with "most importantly" in the paragraph next to your last one. Such phrasing indicates that you were rather sloppy, again, to use a word of yours, while writing your comment.
It is our feeling that the readers of your comment would benefit from a disclosure more detailed than just The DAM Book author; like mentioning your special relations with Adobe in the status of Adobe Alpha Tester.
1. The most important is that the author does not seem to have looked at the DNG spec 1.2 very closely. There were several important changes incorporated in that document that make the format considerably more friendly to third parties. These include:
a. Provisions for any software to attach its color decoding profiles into the DNG, even if they are entirely different than those used by Adobe.
b. A tag was added to indicate which application built the embedded rendering. This was added specifically because users will have DNG files created by different applications, and need to send them for output and get predictable results.
c. The licensing terms for Adobe's color matrix was updated to address concerns of third-party software, and licensing terms were added to enable third-party software to protect or distribute its own rendering profiles, if they so choose.
2. The article makes two claims about the rendering differences produced by DNG files processed by Adobe software. In the first, the author uses "an old version of the DNG converter." It would be helpful to know which one. Is it the first implementation, or a later one? There were acknowledged problems with the very first version that have been rectified in later versions. And what does "amplified" mean, exactly?
In the second claim, the author states that the renderings produced by different versions of Camera Raw are, in fact, somewhat different. This is pretty unsurprising, since the current software has had further engineering, and refinements to the rendering engine. For the comparison to be of real value, the author should document how he has created and applied the parametric rendering instructions for the comparison of the files. It's quite likely that the first image is taking advantage of improvements in sharpening and shadow rendering that were built into the later version but not available to version 3.x (he does not document which versions he used to test with). The 5.x version is also highly likely to be using a different profile to render, although he could choose to use the old one if he wishes.
Either the author tested in the manner he did to purposely skew the results, or he does not understand rendering and forward compatibility very well. For instance, it would be much more relevant to know if a DNG made in ACR 3.6 looked the same when rendered out of 5.2, and what rendering instructions were present. The method he chose will clearly employ the improvements in the current version when rendering the file, including sharpening, vibrance, new profiles and other rendering tools. These will not be available in all past versions, just as some TIFF layer adjustments from Photoshop CS3 can't be understood by Photoshop CS, and so a different result will be produced. At minimum, this needs to be understood.
Most importantly, the author's criticism's here seems to be confused by the capabilities of the specification, and Adobe's implementation of the spec. He also does not seem to understand that the universality of DNG is that of a container, not a particular rendering. The *only* way to get a universal static rendering of the file it to embed a non engine-specific one: this has also been greatly enhanced by the tools in the 1.2 spec, which the author should take some time to try and understand. This kind of unclear thinking, along with sloppy research from the primary documents, does not move the discussion forward.
I would be interested in hearing specific suggestions regarding these issues, if they are informed by clear thinking about what is the job of the specification, and what is the job of the software, and how to handle forward compatibility in a parametric image editing environment. While there are surely a number of important issues to address - I can think of at least a half a dozen - very little in this article approaches that.
Peter Krogh
Author, The DAM Book, Digital Asset Management for Photographers
As you can see in RAW data histograms, Green and Red are balanced perfectly (without any filters) in incadescent bulb light. You don't want to use any filters in this low-light condition, anyway.
Also, I don't think that 'gamut' is a good word in describing digital camera. Camera is sensitive to any visible light (to infrared - too), so camera gamut is equal (or broader) to human gamut.
We should describe camera behavior in terms of metamerism: some colors cannot be separated (i.e. same RGB(G) response for different colors)
This is a real dilemma - they could make two sensor types, one with narrow-bandpass colour filters, one with broad spectrum RGB. You would end up with a low ISO sensor offering maximum colour discrimination and Delta-E accuracy (but of course the potential for greater Delta-E errors if software conversion was not precisely matched to the transmission curves); and a high ISO sensor where the overlapping curves could reduce colour discrimination, relying on software to restore it.
I always felt that the RGB filters used on the KM7D/5D were narrower in band than the filters used on the same sensor by Nikon (D70 etc), allowing the lower ISO 100 setting and also giving the distinctive colour of the 7D/5D. I have also always thought that Canon's (early) RGB filters must be weak, helping reduce noise but giving a colour quality I never liked as much. The wide gamut captured by Canon models compared to the 7D/5D also points to this - narrow cut filters will, of course, limit the overall gamut but improve discrimination and accuracy within that gamut.
Did you mean green and blue are perfectly balanced in tungsten light, not green and red?
I have a large stock of ex-Minolta 85B (daylight to tungsten) conversion filters here, never been able to sell them. I gave away 200 40.5mm ones last week (40.5mm is not much use!). I may try the 85B in daylight, using the camera's tungsten preset, to see if it has some benefits.
Sure, pure CC30-40M (for different cameras) balances Green with Blue. Red channel remains half-stop underexposed.
Combination of something like СС30M plus CC15-20R should result in better channel balance in daylight. The cost is extra filter with extra glare (and one more slot in filter holder).
I'll try this combination when Moscow weather become more sunny :)
Filters on sensor is result of some compromise:
* For shooting in tungsten light (espessially, in 'home' bulbs, not high-temperature halogen ones) you need to lower sensitivity of red. If not, then skin tones will be overexposed. Look at the last chapter of my 'White balance problems' article ( http://www.libraw.org/articles/white-balance-in-digital-cameras.html ): green and red (without filters) are near-perfectly balanced under incadescent bulb.
* Also, there is 'High ISO race' now: last Canon/Nikon models have ISO 25600 sensitivity. To lower the signal/noise level, manufactures need to widen filter response curves. And, surely, color characteristics on Canon 5D-2 or Nikon D3 is much worse than on previous generation.
Hope, that 'high color quality' race will start sometime later.....
I realise that, but the blue channel will not be proportionately affected. A mix of CCR and CCM would be needed to equalise the balance, just wondered in view of the serious effect of red channel noise on blue skies whether CCR might not be a better option.
Some time ago (2-3 years) I did some tests using 80B filters for tungsten, and found that between 1 and 1.5 stops of highlight headroom was recovered by shooting with 80B (based on ACR conversions of the time, KM 7D). Iliah's findings show that maybe ACR was pre-boosting the red channel anyway. A different filter (not 80B) might have enabled equal channel gain in raw conversion. I worked in the 1990s with Leaf and other scanning backs, researching hot mirror filters and light balance filters to recover better colour information.
It surprises me that this kind of work (I'm just a photographer, my wife is a colour scientist) does not seem to have influenced the design of RGB filters for Bayer-type sensors. The sensor itself, unfiltered, is highly sensitive to red. But gradation in the red channel is determined by the blue component more than any other. The work of Dr Andrew Stevens in the early 1990s has also been ignored - he showed very effectively how only two coloured filters were required to produce full colour, not three. Andrew was a colour scientist with Kodak but I don't know where he is now. He was able to produce pseudo-isocolour from a panchromatic film and an orange filter, plus an exposure level - nothing more needed (two exposures)!
LibRaw is for software developers. Many users of digiKam, Krita and some other RAW processing software are happy with these programs (which, in turn, uses LibRaw via libkdcraw).
Anyway, LibRaw distribution includes several command-line tools. Take a look to half_mt utility which is similar to 'dcraw -h', but much faster in batch processing on multi-CPU/multicore machines
A magenta filter will crosstalk more than a red filter, and CC40R or 30R would be easily obtained. Have you tried to make up a filter pack which will equalize the highlight clip position for all three channels in daylight?
Second question - to what extent do you think the IR-cut filters used are attenuating the far end of the red response, and might this be in part responsible for the low red sensitivity?
Yes, It works now with LX3 files. For first time I am seeing my true raw as it should be seen.
I will update you once I have a decent UI that uses libraw and it's up on sourceforge.
I've tried LX3 samples from photographyblog and process them without any problems on LibRaw 0.6.1.
Anyway, there is some strange problems when processing Panasonic files (in my tests - files from FZ28) on multi-threaded LibRaw version, so I'll check it.
Hi There,
Thanks for the pointers. I am able to compile the code in VC++ 2008. Samples are also running fine.
I noticed in the list of cameras it says panasonic lx3 but the tiff conversion results in a file with only vertical line.
The same program works fine for Pentax K10D.
Is it because of dcraw only recently supported the LX3.
If so, will it be possible to reimport DCRaw latest version. It will be a while before new cameras will come out and this step may not be needed very soon again.
If we need to override data supplied in one of the DNG fields using hard-wired data or specific algorithms, all derived from original raw, - what is the place for DNG than? Each DNG interpreter will do the interpretation in its own way, just as it happens with original raw.
When you say "the DNG proper" are you referring to the XMP space? Or to the Adobe-generated tags? Isn't the EXIF data available to the third-party raw converter from the Adobe-created DNG? How specifically would you suggest modifying the spec, and/or modifying the behavior of the DNG converter?
Is there anything else you can put your finger on that is concrete? I hope we can agree that it is impossible for a live parametric rendering to be identical from program to program, and that the DNG presents the best current possibility for attaching universally accessible renderings to raw image data. I hope we can agree that the 1.2 extension made the format considerably more friendly to third parties. I also hope we can agree that full backwards compatibility of rendering engines is not going to happen. I hope we can agree that forward compatibility is the most important goal, and that there are some really imaginative solutions included in the DNG that are not found anywhere else.
I also hope we can agree that most of the other points you've identified are ones that are coming from the camera manufacturer side, such as the difficulty of parsing some encoded or encrypted data, and that camera manufacturers don't publish the spectral response of their cameras.
I'd say that if we are on the same page, a more appropriate title for the article, regarding DNG, would be "A few small suggestions for the next DNG spec" rather than "Path to Nowhere." I'll spend a lot of time undoing misconceptions that are spread with analysis like this one.
Peter
> I'm still not sure what you're driving at.
A converter can't rely on just the information as contained in DNG proper, like the black level in the example above. Interpretation of the DNG data in a raw converter often depends on the specific make/model and some other EXIF tags; and worse, converters that can render quality images from DNG need to ignore certain DNG data substituting it with hard-wired data of their own, derived from hacking the original raw format.
>> We would appreciate if instead of simply ruling out the results of our experiment
>> as purposely skewed you would do those yourself and post the results
>> proving your point.
> The premise of what you are asking for is impossible and/or undesirable.
Please explain why.
I'm still not sure what you're driving at.
In the first experiment, you say to alter the EXIF data, and then seem surprised that it changes the output. What is unexpected here?
In the second example, are you saying that Adobe changes an EXIF value or improperly calculates the Black Level Tag? If so, that sounds like an implementation issue, not one of the spec. (Unless you see something in the spec that indicates that the right data is being tossed out by design.
Mostly what I think you're saying is that if you pull stuff out form DNG's as Adobe makes them, they don't render predictably. Again, that does not seem to be a huge surprise.
What is the problem one would encounter if he were not trying to intentionally damage the file?
Peter
- take any .CR2 from Canon camera
- convert it to .DNG by Adobe (i.e. 'right') converter
- replace Make/Model fields by Exiftool:
exiftool -Make=xxx -Model=yyy canon_eos_50d_01-02.dng
- try to render this file with some Non-Adobe DNG-enabled applications. I've used ACDSee Pro 2.5 and shadows are _very_ different. I'm sure, that LightZone (example of another non-Adobe DNG-enabled app) will show the same problem.
Updated: ACDSee and Lightzone show *right* rendering for Make/model preserved and incorrect one if Make/Model changed in Exif.
> I have no idea what you're driving at here. Are you saying that some DNG-aware
> applications can make use of EXIF
Sure!
I've done simple experiment in two minutes
- image from Canon 50D converted to DNG with Adobe 5.2.0.65 converter
Results
* Black Level tag value: 1023 1023 1023 1023
* Black level repeat dim: 2 2
* White level tag: 13200 (i.e. data is unscaled, 14-bit preserved as is)
But the real Black Level for this image is 511, not 1023.
Right RAW processor should:
- think: "this is Canon! we should calculate black level by left side of black mask"
- ignore black level tag from DNG
- calculate black level by averaging black (masked pixels)
If you strip all vendor-specific metadata from this file you will get unacceptable nonlinearity in shadows.
>Please, in your own words, take some time to try and understand what the article is about.
I'm pretty sure I understand where you're going with it. I just think the fundamental underlying assumptions are flawed. On top of that, you don't clearly outline what is an issue with the format, and what is an implementation issue.
>In doing so you will find the answers to many of your questions right in the article, including specific suggestions and the archival value of allowing more proprietary tags like colour decoding profiles of un-specified structure.
Well, the DNG spec does currently provide for anyone to embed profiles that describe the color any way they wish. If a manufacturer wanted to create his own tags and embed, or make profiles available, it could do that, as I understand it.
I think the far more important and challenging issue is how to deal with tags for further decoding and output of the files. As imaging technology moves from simply trying to render the files "correctly" to making more creative interpretations from the captured data, the challenge becomes attaching the custom profiles that can control that output, and keeping them available as the file gets passed from user to user. In this regard , the new 2.5-D profiles that the DNG allows is a really ingenious solution. I suggest checking them out.
Charting the spectral response of the camera's chip is far less important than enabling the user to create a portable parametric rendering. I
Once again, all parametric renderings will be engine-specific, and are likely to change somewhat over time, as applications are improved. (And those are implementation decisions made by the application's creator, and not an issue of format.) In fact, the rendering does not even *exist* until the raw data is loaded into the imaging pipeline along with the instructions. That's why the ability to attach a fixed rendering is so important. It is the *only* possible solution to create a totally predictable rendering, unless we stop software from developing any further.
Likewise, *backwards* compatibility of parametric rendering engines simply cannot work the way you outline unless:
1. The engines are not allowed to change and improve. or
2. A software manufacturer is obligated to update all old versions with new capability whenever it's developed.
Neither one of these will happen.
>By the way, in your opinion, shall we consider that the history of DNG starts with v. 1.2 and ACR v. 4.5?
I'd say 1.1, rather than 1.2. That was when all image data,a swell as all private makernotes was first transferred. Prior to that, the format definitely left some stuff behind.
>In that case, it is not a bad start. But what to do with the users invested into DNG at v.1.0 stage? Were they misled?
Sure, if that's the way you want to look at it. The first implementation had problems. It would be great if all 1.0 implementations were problem-free.
>We would appreciate if instead of simply ruling out the results of our experiment as purposely skewed you would do those yourself and post the results proving your point.
Did you understand my criticisms? The premise of what you are asking for is impossible and/or undesirable.
>As one of the methods of verification archival properties, we suggest you make a DNG out of a RAW file, and then delete from a copy of that DNG all vendor-specific EXIF fields to evaluate the difference in processing results between the "pure" DNG stripped of those vendor-specific EXIF fields and more complete data.
I have no idea what you're driving at here. Are you saying that some DNG-aware applications can make use of EXIF?
When you talk about "archival" what are you really talking about? Are you talking about access to the metadata, access to a particular rendering, or the survivability of the format.
The first of these is really the province of the camera maker, who encode the data, and could make it more discoverable. The second one can *only* be addressed by an embedded rendering. And the third is an area that you glossed over entirely in your discussion of the spec. The data validation tags in DNG 1.2 are a huge leap forward in archive stability, as they offer a way to determine the file integrity automatically. I suggest you check out this part of the spec.
>We would also love to see your priorities being sorted out. You begin your second paragraph with "most important", and continue with "most importantly" in the paragraph next to your last one. Such phrasing indicates that you were rather sloppy, again, to use a word of yours, while writing your comment.
Okay, fine, it was sloppy. I should have said that all of those things are important. It's important to read and understand something before you send out a purportedly comprehensive discussion of it. And it's also important to understand what is the problem with the spec, and what is the problem with any specific implementation. And it's important to understand what's possible, and what's not.
>It is our feeling that the readers of your comment would benefit from a disclosure more detailed than just The DAM Book author; like mentioning your special relations with Adobe in the status of Adobe Alpha Tester.
Okay, yes, I am an unpaid alpha tester for Adobe, which gives me access to the development teams. And I helped Adobe improve the DNG spec, with particular emphasis on addressing the issues raised by other software vendors, such as Bibble (ask Eric - he brought specific problems to light, and they were addressed in 1.2). I also focused on the ability to facilitate predictable rendering, which can only be achieved by use of embedded rendered data. In addition, Adobe occasionally pays me (poorly) to speak. Adobe also paid me to write a white paper on this subject.
http://www.adobe.com/digitalimag/ps_pro_primers.html
I have also licensed photographs to Adobe in my capacity as a professional photographer.
Peter Krogh
Dear Peter,
Please, in your own words, take some time to try and understand what the article is about. In doing so you will find the answers to many of your questions right in the article, including specific suggestions and the archival value of allowing more proprietary tags like colour decoding profiles of un-specified structure.
By the way, in your opinion, shall we consider that the history of DNG starts with v. 1.2 and ACR v. 4.5? In that case, it is not a bad start. But what to do with the users invested into DNG at v.1.0 stage? Were they misled?
We would appreciate if instead of simply ruling out the results of our experiment as purposely skewed you would do those yourself and post the results proving your point. As one of the methods of verification archival properties, we suggest you make a DNG out of a RAW file, and then delete from a copy of that DNG all vendor-specific EXIF fields to evaluate the difference in processing results between the "pure" DNG stripped of those vendor-specific EXIF fields and more complete data.
We would also love to see your priorities being sorted out. You begin your second paragraph with "most important", and continue with "most importantly" in the paragraph next to your last one. Such phrasing indicates that you were rather sloppy, again, to use a word of yours, while writing your comment.
It is our feeling that the readers of your comment would benefit from a disclosure more detailed than just The DAM Book author; like mentioning your special relations with Adobe in the status of Adobe Alpha Tester.
I see a few fundamentals issues with the article.
1. The most important is that the author does not seem to have looked at the DNG spec 1.2 very closely. There were several important changes incorporated in that document that make the format considerably more friendly to third parties. These include:
a. Provisions for any software to attach its color decoding profiles into the DNG, even if they are entirely different than those used by Adobe.
b. A tag was added to indicate which application built the embedded rendering. This was added specifically because users will have DNG files created by different applications, and need to send them for output and get predictable results.
c. The licensing terms for Adobe's color matrix was updated to address concerns of third-party software, and licensing terms were added to enable third-party software to protect or distribute its own rendering profiles, if they so choose.
2. The article makes two claims about the rendering differences produced by DNG files processed by Adobe software. In the first, the author uses "an old version of the DNG converter." It would be helpful to know which one. Is it the first implementation, or a later one? There were acknowledged problems with the very first version that have been rectified in later versions. And what does "amplified" mean, exactly?
In the second claim, the author states that the renderings produced by different versions of Camera Raw are, in fact, somewhat different. This is pretty unsurprising, since the current software has had further engineering, and refinements to the rendering engine. For the comparison to be of real value, the author should document how he has created and applied the parametric rendering instructions for the comparison of the files. It's quite likely that the first image is taking advantage of improvements in sharpening and shadow rendering that were built into the later version but not available to version 3.x (he does not document which versions he used to test with). The 5.x version is also highly likely to be using a different profile to render, although he could choose to use the old one if he wishes.
Either the author tested in the manner he did to purposely skew the results, or he does not understand rendering and forward compatibility very well. For instance, it would be much more relevant to know if a DNG made in ACR 3.6 looked the same when rendered out of 5.2, and what rendering instructions were present. The method he chose will clearly employ the improvements in the current version when rendering the file, including sharpening, vibrance, new profiles and other rendering tools. These will not be available in all past versions, just as some TIFF layer adjustments from Photoshop CS3 can't be understood by Photoshop CS, and so a different result will be produced. At minimum, this needs to be understood.
Most importantly, the author's criticism's here seems to be confused by the capabilities of the specification, and Adobe's implementation of the spec. He also does not seem to understand that the universality of DNG is that of a container, not a particular rendering. The *only* way to get a universal static rendering of the file it to embed a non engine-specific one: this has also been greatly enhanced by the tools in the 1.2 spec, which the author should take some time to try and understand. This kind of unclear thinking, along with sloppy research from the primary documents, does not move the discussion forward.
I would be interested in hearing specific suggestions regarding these issues, if they are informed by clear thinking about what is the job of the specification, and what is the job of the software, and how to handle forward compatibility in a parametric image editing environment. While there are surely a number of important issues to address - I can think of at least a half a dozen - very little in this article approaches that.
Peter Krogh
Author, The DAM Book, Digital Asset Management for Photographers
I now have a D90, which is supported, so can use rawanalyze without conversion.
A good and interesting article in general. Interesting comparisons of different processors.
Noise reduction before debayering is clearly the way to go, especially if read noise can be estimated and removed from masked pixels at the same time.
Constructing a noise profile, per camera model and per ISO setting, would be ideal. (Commercial programs like Noise Ninja do this, for example).
I notice once again that ETTL is being described as ETTR, but that doesn't impact the main point of the article.
As you can see in RAW data histograms, Green and Red are balanced perfectly (without any filters) in incadescent bulb light. You don't want to use any filters in this low-light condition, anyway.
Also, I don't think that 'gamut' is a good word in describing digital camera. Camera is sensitive to any visible light (to infrared - too), so camera gamut is equal (or broader) to human gamut.
We should describe camera behavior in terms of metamerism: some colors cannot be separated (i.e. same RGB(G) response for different colors)
This is a real dilemma - they could make two sensor types, one with narrow-bandpass colour filters, one with broad spectrum RGB. You would end up with a low ISO sensor offering maximum colour discrimination and Delta-E accuracy (but of course the potential for greater Delta-E errors if software conversion was not precisely matched to the transmission curves); and a high ISO sensor where the overlapping curves could reduce colour discrimination, relying on software to restore it.
I always felt that the RGB filters used on the KM7D/5D were narrower in band than the filters used on the same sensor by Nikon (D70 etc), allowing the lower ISO 100 setting and also giving the distinctive colour of the 7D/5D. I have also always thought that Canon's (early) RGB filters must be weak, helping reduce noise but giving a colour quality I never liked as much. The wide gamut captured by Canon models compared to the 7D/5D also points to this - narrow cut filters will, of course, limit the overall gamut but improve discrimination and accuracy within that gamut.
Did you mean green and blue are perfectly balanced in tungsten light, not green and red?
I have a large stock of ex-Minolta 85B (daylight to tungsten) conversion filters here, never been able to sell them. I gave away 200 40.5mm ones last week (40.5mm is not much use!). I may try the 85B in daylight, using the camera's tungsten preset, to see if it has some benefits.
David
Sure, pure CC30-40M (for different cameras) balances Green with Blue. Red channel remains half-stop underexposed.
Combination of something like СС30M plus CC15-20R should result in better channel balance in daylight. The cost is extra filter with extra glare (and one more slot in filter holder).
I'll try this combination when Moscow weather become more sunny :)
Filters on sensor is result of some compromise:
* For shooting in tungsten light (espessially, in 'home' bulbs, not high-temperature halogen ones) you need to lower sensitivity of red. If not, then skin tones will be overexposed. Look at the last chapter of my 'White balance problems' article ( http://www.libraw.org/articles/white-balance-in-digital-cameras.html ): green and red (without filters) are near-perfectly balanced under incadescent bulb.
* Also, there is 'High ISO race' now: last Canon/Nikon models have ISO 25600 sensitivity. To lower the signal/noise level, manufactures need to widen filter response curves. And, surely, color characteristics on Canon 5D-2 or Nikon D3 is much worse than on previous generation.
Hope, that 'high color quality' race will start sometime later.....
I realise that, but the blue channel will not be proportionately affected. A mix of CCR and CCM would be needed to equalise the balance, just wondered in view of the serious effect of red channel noise on blue skies whether CCR might not be a better option.
Some time ago (2-3 years) I did some tests using 80B filters for tungsten, and found that between 1 and 1.5 stops of highlight headroom was recovered by shooting with 80B (based on ACR conversions of the time, KM 7D). Iliah's findings show that maybe ACR was pre-boosting the red channel anyway. A different filter (not 80B) might have enabled equal channel gain in raw conversion. I worked in the 1990s with Leaf and other scanning backs, researching hot mirror filters and light balance filters to recover better colour information.
It surprises me that this kind of work (I'm just a photographer, my wife is a colour scientist) does not seem to have influenced the design of RGB filters for Bayer-type sensors. The sensor itself, unfiltered, is highly sensitive to red. But gradation in the red channel is determined by the blue component more than any other. The work of Dr Andrew Stevens in the early 1990s has also been ignored - he showed very effectively how only two coloured filters were required to produce full colour, not three. Andrew was a colour scientist with Kodak but I don't know where he is now. He was able to produce pseudo-isocolour from a panchromatic film and an orange filter, plus an exposure level - nothing more needed (two exposures)!
David
LibRaw is for software developers. Many users of digiKam, Krita and some other RAW processing software are happy with these programs (which, in turn, uses LibRaw via libkdcraw).
Anyway, LibRaw distribution includes several command-line tools. Take a look to half_mt utility which is similar to 'dcraw -h', but much faster in batch processing on multi-CPU/multicore machines
Magenta is 'Minus Green filter', but Red is 'Minus Green and Minus Blue'.
In daylight situation CC30M-CC40M produces near perfect sensitivity balance on my cameras.
A magenta filter will crosstalk more than a red filter, and CC40R or 30R would be easily obtained. Have you tried to make up a filter pack which will equalize the highlight clip position for all three channels in daylight?
Second question - to what extent do you think the IR-cut filters used are attenuating the far end of the red response, and might this be in part responsible for the low red sensitivity?
Yes, It works now with LX3 files. For first time I am seeing my true raw as it should be seen.
I will update you once I have a decent UI that uses libraw and it's up on sourceforge.
Thanks,
We've found and fixed bug in Panasonic .RW2 files processing. This bug affects only thread-safe version of library.
Could you please test LibRaw 0.6.2 on your files?
I've tried LX3 samples from photographyblog and process them without any problems on LibRaw 0.6.1.
Anyway, there is some strange problems when processing Panasonic files (in my tests - files from FZ28) on multi-threaded LibRaw version, so I'll check it.
You also may check new LibRaw release (sorry, source code only): http://www.libraw.org/blog/libraw-062-beta1.html
To test with any samples, you can use the raw files from here.
http://www.photographyblog.com/reviews_panasonic_lumix_dmc_lx3_3.php
If you scroll down there are bunch of RW2 files.
Thanks,
Hi There,
Thanks for the pointers. I am able to compile the code in VC++ 2008. Samples are also running fine.
I noticed in the list of cameras it says panasonic lx3 but the tiff conversion results in a file with only vertical line.
The same program works fine for Pentax K10D.
Is it because of dcraw only recently supported the LX3.
If so, will it be possible to reimport DCRaw latest version. It will be a while before new cameras will come out and this step may not be needed very soon again.
Thanks,
Unfortunately, I've no Mac with Mac OS X Tiger. Will try to find VMWare image to test.
When is the article describing how to set UniWB into the camera going to be published? This is of great interest to me.
Pages