[BUG] Tuning file support not thread-safe?

[signag]

Hello,

Bug Description

Implementing tuning file support in raspi-cam-srv, I observed that the expected effect of different tuning files did not show up when streaming two cameras.
While the tuning files for the cameras differed significantly in white balance, the resulting videos and photos looked as if the same tuning file had been applied for both cameras.
Individual threads are used to stream each of the cameras simultaneously.

To analyze the issue, I have written a small test program which is attached to this issue (see To Reproduce).

The program generates 4 pairs of photos, taken with both cameras (see Console Output, Screenshots) :

• 01 and 02 use the default tuning files for both cameras

• 03 and 04 use prepared modified tuning files with extreme white balance shifts
  These photos are taken in the main thread

  The subsequent photos are taken in two parallel threads with a small delay between their start:

• 05 and 06 use the identical setup as 03 and 04

• 07 and 08 have the sequence of cameras reversed

It is seen that 06 shows the same white balance shift as 05 whereas it should be identical with 04.

In the last set, camera 1 (photo 07) was started before camera 0 (photo 08).
Here, camera 0 produces the same white balance shift as camera 1, whereas photo 08 was taken with the same tuning file as photo 03.

To Reproduce

To reproduce the issue, I have attached the following test set.

testPicamera2Tuning.zip

The zip archive includes:

• program testPicamera2Tuning.py
• A subdirectory 'tuningfiles' with tuning files to be used for testing.
  The base versions have been copied from /usr/share/libcamera/ipa/rpi/pisp
  Afterwards, two variants (*_a.json and *_b.json) have been created, in which the white balance has been changed significantly in an opposite direction.

To run the test:

• Use a Raspberry Pi 5
• Attach two cameras (same or different model)
• Unzip the archive to a suitable folder on the Pi5
• Run the Python program

As a result, a subfolder output_YYYYMMDD_HHMMSS will be created which will contain the 8 photos.

Error messages are printed if the program did not find 2 cameras or if a test tuning file for one of the cameras was not found.
In the latter case you can create an own set of two tuning files in the same manner as the available files have been created.

Expected behaviour

A camera, when initialized with a specific tuning file, should use this file in all cases, also when used in parallel threads.

Console Output, Screenshots

Console output of testPicamera2Tuning.py:

Photos resulting from a test run:

• 01_Camera_0_Tuning_Default.jpeg
  
  
• 02_Camera_1_Tuning_Default.jpeg
  
  
• 03_Camera_0_Tuning_imx708_a.json.jpeg
  
  
• 04_Camera_1_Tuning_imx708_wide_noir_b.json.jpeg
  
  
• 05_Camera_0_Tuning_imx708_a.json.jpeg
  
  
• 06_Camera_1_Tuning_imx708_wide_noir_b.json.jpeg
  
  
• 07_Camera_1_Tuning_imx708_wide_noir_b.json.jpeg
  
  
• 08_Camera_0_Tuning_imx708_a.json.jpeg
  
  

Hardware

RPi 5
Attached cameras:

• (0): module 3 camera ("Model": "imx708")
• (1): module 3 camera ("Model": "imx708_wide_noir")

OS: Bookworm
Debian version: 12.7
Kernel version: 6.6.47+rpt-rpi-2712

Additonal context

[davidplowman]

Hi, thanks for raising this.

Yes, I think there is a fundamental libcamera issue here. The tuning files get loaded for all camera types when the camera system starts. To force this to happen again, thereby loading a new tuning file, all currently open cameras have to be closed. This is why everything works in the same thread, because the only open camera gets closed before you open the next one.

Unfortunately, this means there's just no way to change the tuning for camera B if camera A is running. We are talking to the libcamera team to try and solve this problem.

In the meantime, if you want a workaround, I would suggest running each camera in a separate process. For example, you could use the Python multiprocessing module to create a sub-process for each camera and then send commands to those processes.

Can you say anything more about why you want to use different tuning files? I'm just wondering if there might be other alternatives.

[signag]

Hi, thanks for your response.

The request for explicit support of tuning files came from a user of raspiCamSrv:
"How can I include the settings for my V2.1 NoIR camera in order to get rid of the red tint?"
(See raspi-cam-srv Issue #26 - NoIR camera settings)

Although several people seem to use raspiCamSrv for streaming, I would not think that many of them have an issue with tuning files when streaming two cameras in parallel; and with one camera they can use the feature, if necessary.

Also, as all this is under the hood of a Flask web server, I doubt that there would not be any side effects with multiprocessing.

Therefore, I'll stay with the current state until there may be a solution on the libcamera side.

Thanks again for your support.