How to detecting charts in over-exposed HDRI images?

[ahemberger]

Hello! I'm trying to build a small pipeline for myself at home whereby I a) collect bracketed exposures using a pano head on a tripod, 2) stitch these into an equirectangular HDR image using ptGui, with the ultimate intent of 3) using the output as an IBL for vfx purposes.

I would love to take ptGUI's output and auto-detect a MacBeth chart in my panorama, then use this project to neutralize my panorama (and also use it to neutralize a plate that I've shot separately). The output of ptGUI doesn't seem to choose the "middle" exposure from a sequence as its exposure setting when exporting an HDR panorama, however, and this seems to cause this tool to fail to find the chart.

I'm curious what my most efficient path forward might be. I could bring the panorama into nuke or photoshop to stop it up until it appears more visually-reasonable before feeding it to colour-checker-detection, but I would love to skip this step if I can. Is there a way – either in this module or in colour itself – that I can pre-process my image to encourage the segmentation code to find the chart properly?

Thanks so much!
--a

[KelSolaar]

Hi @ahemberger,

What about normalising your image by its mean and 18% gray before trying to detect the colour checker:

>>> import numpy as np
>>> RGB = np.random.random([2048, 4096, 3])
>>> RGB *= 2**8
>>> np.mean(RGB)
127.98784742921099
>>> RGB *= 1 / np.mean(RGB) * 0.18
>>> np.mean(RGB)
0.17999999999999969

You could store the factor to reciprocally rescale after the detection?

[KelSolaar]

Hi @ahemberger,

The expectation is to have the ColorChecker Classic relatively straight, flat and prominent in the picture. This image from the doc is a good example:

You certainly want enough sample to have a meaningful result.

[ahemberger]

Hey sir;

Argh, ok thanks, that's a bummer. What I'm ultimately trying to do is neutralize an IBL as well as a plate I've captured on set, using as few intermediary steps/software as possible to keep things efficient. My image above is the bottom face of a cubic map representation of my IBL; I was hoping this module would be a sweet replacement for the Nuke/Natron setup I have going for this right now (ultimately all I need as a color matrix to correct my images, and having to get at that through a convoluted Nuke/Natron pipeline is a lot of complexity I'm interested in sidestepping).

I'm unsure how I'd make the colorChecker be larger in frame without needing to shoot an extra bracket with the checker mounted on a tripod or something...something that I think works against speed/portability for me. I suppose I could try to crop in on the chart in my image above, but that doesn't really offer me any advantages over just staying in Natron (I'm a solo artist, so Nuke's [very expensive] license fees are out of my reach personally, and even the Indie version is resolution-limited so unusable for IBL work). Either way I realize I'm skimping on samples, but I'm trying to do the best I can with what I have...so far I've managed to get results that look pretty nice to me.

In any case, thanks so very much for your very responsive help! I love this entire Colour universe and am very interested in using as much of it as I can!

[KelSolaar]

Hi @ahemberger,

This might be a relevant read for you: https://www.colour-science.org/posts/the-colorchecker-considered-mostly-harmless/

Cheers,

Thomas

[ahemberger]

Hi Thomas,

Ha, thanks; I was just reading through this Friday! It's very eloquent and interesting, to be sure.

I think my own situation is a bit different than your target audience – I'm just a hobbyist trying to make nice pictures, and can't bring to bear the full arsenal (read: financial weight) of a full VFX studio. I don't have spectral measurement equipment, etc. I don't have access to Manuka or any other spectral renderer, so I think my universe might well be less sophisticated than the one mandated by Colour.

What I'm interested in doing is shooting a plate and an IBL using a Canon 5dMkiii, and I'd like my IBL and my plate to match reasonably well so that a CG grey ball lit by a captured IBL composited over my plate looks good. The note about using a colorChecker to match white balance and general exposure is probably the best way to describe what I'm currently doing.

My current strategy of using numpy to least-squares fit a photographed chart to a digital color checker works marvelously for my needs; it's just got a lot of steps. I was drawn to this tool thinking it might be a way for me to cut out a few of those. Your helpful clues definitely give me some thoughts as to how I may or may not be able to do this with this code.

Thank you again so very much for all your kind help sir!

--a

[KelSolaar]

So, technically, your 5D is probably good-to-go and at this point, you would probably want to do something along those lines:

Assuming a lens and sensor that are well characterized, technically, you should white-balance to D65 or (~D60/ACES depending on your working space whitepoint). This way, when you put the HDRI into your renderer, it will be roughly in a device-independent colorimetric state. To take an example, if you shoot with low-colour temperature lighting. e.g. 3200K, you should probably still white-balance at the encoding white point, e.g. D65/6500k so that when you put the HDRI into your scene, it is very orange AND then you can use your physical camera at 3200K to get neutral.

It becomes rapidly a can of worm though because the cameras have different matrices optimized for different lighting conditions... So if you white balance at 6500 a scene that was lit with tungsten, you might introduce errors. When you have realised that, you are willing to get a spectral renderer ;)

This was actually paraphrasing a reply I made to an artist on Linkedin! :)