Alex,
So is the old DNG processing behaviour a bug which has been fixed in the new code? Because the colours in the rendered sRGB image with 0.20 appear duller than 0.19.5.
But, I am surprised why the rgb_cam is being returned as an identity matrix. There was a bunch of changes made to DNG Frame Selection. I want to ensure that I do not have to add any additional code to ensure the correct frame and the properties corresponding to it are selected.
Is there a way to fallback on 0.19.5 behaviour? I see there is ifdef'ed code to fallback to use auto_ptr vs unique_ptr. Is there similar for old DNG processing?
1) Many smartphone-produced DNG files has 'unusual' order of color data, the 1st one is daylight matrix, the 2nd one is for incandescent.
The old code (LibRaw 0.19/dcraw.c) always uses the last colordata (ColorMatrix) seen in the file. This works fine for Adobe-DNG-SDK-produced files (daylight colormatrix is the last one), but not with files where ColorMatrix1/ColorMatrix2 order is reversed.
Also, if several different sub-images are present in the DNG file with separate (own) colordata, the old code (again) uses the last one ColorMatrix specified
In the LibRaw 0.20 correct color matrix selection is implemented in both cases.
2) Visible size (or margins size) has changed in 0.20 for many cameras:
- for some cameras, vendor specified size is used instead of hardcoded one
- for all Bayer cameras margins are rounded to multiple of 2
- for all X-Trans cameras margins are rounded to multiple of 6
The last two changes is because of ambiguity in previous version: since LibRaw 0.20 bayer(/X-Trans) pattern is the same for both possible coordinate systems (entire area/visible area)
> So, you are suggesting I create CFLAGS_ZLIB ....
There are multiple ways to change variables in makefile. I think, it is much faster to try and see what happens, than wait for approval....
> On a related note. Where do I specify USE_X3FTOOLS and USE_6BY9RPI
Again: there are multiple ways.The easiest one is to modify COPT=... line
>Do, I need download the sources
No, both X3F_TOOLS and RPI-related code is already in place. It is just not compiled until user explicitly selects it (via specific defines, so /D.... statements in msvc case)
I noticed there were a bunch of changes made to DNG processing. I am not sure if there are any changes I need to be making to the source code for this.
So, you are suggesting I create CFLAGS_ZLIB and LDFLAGS_ZLIB and add them to COPT and LINKLIB lines?
On a related note. Where do I specify USE_X3FTOOLS and USE_6BY9RPI on the Makefile.msvc? Can I update the COPT line to say /DUSE_X3FTOOLS?
Also, I am a little confused on the imported code policy? Do, I need download the sources from the suitable github repo in addition to the libraw source OR is it sufficient to specify the macros enable them as the sources are already present?
It is not possible to add getter(s) for every field in LibRaw structure (while every field may be needed in some specific case).
We provide very limited C-API that solves most common need(s): open RAW file, render it, provide the rendering.
We do not have any plans to extend it much for every need that may occur.
C-API is simple enough, so any not very experienced programmer may extend it using provided C-API code as a sample. Although, there is no guarantee that we'll not reorder internals in future.
Alex, what do you think, makes it sense to provide a getter method in C API only for this field (libraw_data_t->image)?
I just checked the definition of that field and when I am correct, it is a pointer to a whole 4 element array of ushorts.
I guess that array will be resized internally by whatever call. What is the final size then?
Unlike unix systems (Linux, macos, etc), there is no standard path to zlib (headers, library), also there is no standard name for the library, it may be called zlib(.lib,.dll) zlib.lib+zlib1.dll, zlib_static.lib, etc.
So, I'm unable to provide specific and precise instructions, only general direction:
Easiest way to re-build libraw under Windows is to call
nmake -f Makefile.msvc
(for example, In Visual Studio command prompt)
So, modify Makefile.msvc to add /DUSE_ZLIB, path to zlib includes in compiler flags, flags to link with zlib to link flags
Alternatively, you may use Visual Studio solution/project provided and add the same to project properties via property settings.
Hi Alex,
I am having trouble adding ZLIB support on Windows. I believe the Windows build using NMAKE. Not very clear on how to specify the USE_ZLIB and the supporting include and lib paths to nmake on Windows.
That's because I did not directly define the libraw_data_t structure (a record in pascal) in the wrapper. I am simply using a general void* (a Pointer type in pascal) instead. Means any pascal program using this wrapper has access only to the pointer itself (to the libraw_data_t structure) but not the structure fields directly.
Reason is simple: All subsequent calls after libraw_init simply accepts the pointer (to structure) and they just do their job.
There was no reason yet to directly define the libraw_data_t record in the wrapper because no direct access to any field was required.
I cannot understand how it happens:
- you're able to access libraw_processed_image_t fields (e.g. data)
- but not able to access libraw_data_t fields (e.g. imgdata.image)
....
I don't have a solution for it at the moment. I assume it has to do something with ushort but the current Pascal wrapper doesn't contain a reference to it. I can only hope Laheller has a solution for it.
ushort (*image)[4];
The memory area that contains the image pixels per se. It is filled when raw2image() or dcraw_process() is called.
make_mem_image() is not targeted to make copy of intermediate state if imgdata.image[] content.
it is targeted to create 3-component image (RGB) from 4-component imgdata.image (also, rotation, 16-8 bit conversion and gamma curve is applied).
make_mem_image() assumes that all processing is already done, so RGB image is contained in first 3 components of 4-component imgdata.image[][4] pixel.
If one need to access unmodified imgdata.image data it should be done directly.
Fujifilm X-Trans (6x6 pattern) files are 3-component, there is no 'second green'.
There is no way to get separate image planes via LibRaw:
- unprocessed data (assuming we're talking about bayer/xtrans camera) is single layer
- intermediate data(after LibRaw::raw2image) is a 4-component per pixel interleaved (4th component is not used in X-trans case)
Maybe Alex can clarify:
Why is the second green missing and how to fix? Having the raw packed in a three color image is not a problem but the missing second green is a problem.
Is this the LibRaw standard way for getting unprocessed raw: "getting three buffers: RED, GREEN, and BLUE. All buffers have the same sizes (size of the image) but with “holes” in the place of other colors pixels." or is a better way?
And what is also puzzling is in an other program setting libraw_set_output_color(handler, LibRaw_output_color.raw); doesn't do anything either.
I do not see that processing result is dull
Here is the file processed with LibRaw 0.19.5 (dcraw_emu -w -T): https://www.dropbox.com/s/efh2f1rkukunuif/screenshot%202020-10-13%2016.2...
Here is LibRaw 0.20 processing (with daylight WB): https://www.dropbox.com/s/6x7l71j1e9f90s1/screenshot%202020-10-13%2016.2...
Please note that 'camera matrix' is used for DNG files if use_camera_matrix is either default (1) or greater.
Alex,
So is the old DNG processing behaviour a bug which has been fixed in the new code? Because the colours in the rendered sRGB image with 0.20 appear duller than 0.19.5.
But, I am surprised why the rgb_cam is being returned as an identity matrix. There was a bunch of changes made to DNG Frame Selection. I want to ensure that I do not have to add any additional code to ensure the correct frame and the properties corresponding to it are selected.
Is there a way to fallback on 0.19.5 behaviour? I see there is ifdef'ed code to fallback to use auto_ptr vs unique_ptr. Is there similar for old DNG processing?
Regards,
Dinesh
I will give this a shot and get back if I have any trouble.
Dinesh
1) Many smartphone-produced DNG files has 'unusual' order of color data, the 1st one is daylight matrix, the 2nd one is for incandescent.
The old code (LibRaw 0.19/dcraw.c) always uses the last colordata (ColorMatrix) seen in the file. This works fine for Adobe-DNG-SDK-produced files (daylight colormatrix is the last one), but not with files where ColorMatrix1/ColorMatrix2 order is reversed.
Also, if several different sub-images are present in the DNG file with separate (own) colordata, the old code (again) uses the last one ColorMatrix specified
In the LibRaw 0.20 correct color matrix selection is implemented in both cases.
2) Visible size (or margins size) has changed in 0.20 for many cameras:
- for some cameras, vendor specified size is used instead of hardcoded one
- for all Bayer cameras margins are rounded to multiple of 2
- for all X-Trans cameras margins are rounded to multiple of 6
The last two changes is because of ambiguity in previous version: since LibRaw 0.20 bayer(/X-Trans) pattern is the same for both possible coordinate systems (entire area/visible area)
> So, you are suggesting I create CFLAGS_ZLIB ....
There are multiple ways to change variables in makefile. I think, it is much faster to try and see what happens, than wait for approval....
> On a related note. Where do I specify USE_X3FTOOLS and USE_6BY9RPI
Again: there are multiple ways.The easiest one is to modify COPT=... line
>Do, I need download the sources
No, both X3F_TOOLS and RPI-related code is already in place. It is just not compiled until user explicitly selects it (via specific defines, so /D.... statements in msvc case)
I noticed there were a bunch of changes made to DNG processing. I am not sure if there are any changes I need to be making to the source code for this.
So, you are suggesting I create CFLAGS_ZLIB and LDFLAGS_ZLIB and add them to COPT and LINKLIB lines?
On a related note. Where do I specify USE_X3FTOOLS and USE_6BY9RPI on the Makefile.msvc? Can I update the COPT line to say /DUSE_X3FTOOLS?
Also, I am a little confused on the imported code policy? Do, I need download the sources from the suitable github repo in addition to the libraw source OR is it sufficient to specify the macros enable them as the sources are already present?
Regards,
Dinesh
(and second answer)
imgdata.image array is allocated either in raw2image() call or in unpack() call.
array size is no less than imgata.sizes.iwidth * imgdata.sizes.iheight
It is not possible to add getter(s) for every field in LibRaw structure (while every field may be needed in some specific case).
We provide very limited C-API that solves most common need(s): open RAW file, render it, provide the rendering.
We do not have any plans to extend it much for every need that may occur.
C-API is simple enough, so any not very experienced programmer may extend it using provided C-API code as a sample. Although, there is no guarantee that we'll not reorder internals in future.
Alex, what do you think, makes it sense to provide a getter method in C API only for this field (libraw_data_t->image)?
I just checked the definition of that field and when I am correct, it is a pointer to a whole 4 element array of ushorts.
I guess that array will be resized internally by whatever call. What is the final size then?
Unlike unix systems (Linux, macos, etc), there is no standard path to zlib (headers, library), also there is no standard name for the library, it may be called zlib(.lib,.dll) zlib.lib+zlib1.dll, zlib_static.lib, etc.
So, I'm unable to provide specific and precise instructions, only general direction:
Easiest way to re-build libraw under Windows is to call
nmake -f Makefile.msvc
(for example, In Visual Studio command prompt)
So, modify Makefile.msvc to add /DUSE_ZLIB, path to zlib includes in compiler flags, flags to link with zlib to link flags
Alternatively, you may use Visual Studio solution/project provided and add the same to project properties via property settings.
Hi Alex,
I am having trouble adding ZLIB support on Windows. I believe the Windows build using NMAKE. Not very clear on how to specify the USE_ZLIB and the supporting include and lib paths to nmake on Windows.
Regards
Dinesh
It looks like Han needs this access for his code.
That's because I did not directly define the libraw_data_t structure (a record in pascal) in the wrapper. I am simply using a general void* (a Pointer type in pascal) instead. Means any pascal program using this wrapper has access only to the pointer itself (to the libraw_data_t structure) but not the structure fields directly.
Reason is simple: All subsequent calls after libraw_init simply accepts the pointer (to structure) and they just do their job.
There was no reason yet to directly define the libraw_data_t record in the wrapper because no direct access to any field was required.
I cannot understand how it happens:
- you're able to access libraw_processed_image_t fields (e.g. data)
- but not able to access libraw_data_t fields (e.g. imgdata.image)
....
I don't have a solution for it at the moment. I assume it has to do something with ushort but the current Pascal wrapper doesn't contain a reference to it. I can only hope Laheller has a solution for it.
ushort (*image)[4];
The memory area that contains the image pixels per se. It is filled when raw2image() or dcraw_process() is called.
??? Your code is calling raw2image, but not calling dcraw_process().....
Also, current processing stage is reflected in imgdata.progress_flags
Thanks.for the info. But how to detect where the image data is after raw2mem?
Understood the problem.
make_mem_image() is not targeted to make copy of intermediate state if imgdata.image[] content.
it is targeted to create 3-component image (RGB) from 4-component imgdata.image (also, rotation, 16-8 bit conversion and gamma curve is applied).
make_mem_image() assumes that all processing is already done, so RGB image is contained in first 3 components of 4-component imgdata.image[][4] pixel.
If one need to access unmodified imgdata.image data it should be done directly.
I have tried two camera images:
CR2, EOS 80D
CR3, EOS R
unprocessed_raw.exe writes normal raws. The 3 layer format is not a problem but then missing second green, G2 is essential
Han, what camera files do you test with?
Fujifilm X-Trans (6x6 pattern) files are 3-component, there is no 'second green'.
There is no way to get separate image planes via LibRaw:
- unprocessed data (assuming we're talking about bayer/xtrans camera) is single layer
- intermediate data(after LibRaw::raw2image) is a 4-component per pixel interleaved (4th component is not used in X-trans case)
Laheller, thanks for the wrapper. It works for colour images but at the moment not for extracting unprocessed raw.
The second green color is missing. I get RGB0 where 0 stand for zero value. See this screen shot.
https://ibb.co/d0TD6Lv
I do the following
libraw_open_file
libraw_unpack
libraw_raw2image
libraw_dcraw_make_mem_image
Maybe Alex can clarify:
Why is the second green missing and how to fix? Having the raw packed in a three color image is not a problem but the missing second green is a problem.
Is this the LibRaw standard way for getting unprocessed raw: "getting three buffers: RED, GREEN, and BLUE. All buffers have the same sizes (size of the image) but with “holes” in the place of other colors pixels." or is a better way?
And what is also puzzling is in an other program setting libraw_set_output_color(handler, LibRaw_output_color.raw); doesn't do anything either.
Han
You may need to recompile libraw samples using older version of XCode.
make -f Makefile.dist
should do that
Yes that fixes it. :)
>>What version of Mac OS X/macOS do you use?
High Sierra, 10.12
Pages