Use saturation visualizer as the default visualizer for cameras

[bruno-f-cruz]

Describe changes:

Added an operator that visualizes the saturation of an incoming image instead of the default bonsai visualizer

What issues or discussions does this update address?

@rachelstephlee can you add the context here?

Describe the expected change in behavior from the perspective of the experimenter

The visualizer should now highlight in red/blue over/under exposed pixels

Describe any manual update steps for task computers

Update the layout files. If needed consider setting the min/max from the settings file

Was this update tested in 446/447?

Does this update impact downstream processing by adding new saved files, or changing their format? If so, have you documented changes?

Nop

[bruno-f-cruz]

This PR replaces the previous visualizer for cameras by:

Each one of those combinator operators has 3 properties (max, min, alpha) that can be used to dynamically adjust what is getting shown.

[rachelstephlee]

Notes:

• layout file will need to be updated, or will crash
• alternatively, need to update latest bonsai version (Update Bonsai to version 2.9 #1625)

[alexpiet]

Please merge to develop not main

[micahwoodard]

@rachelstephlee how does the layout file need to change? I think that will be a faster implementation that waiting for the bonsai version to be updated but could be wrong about that

[rachelstephlee]

Micah, Bruno, Galen, Ella, and I had a long discussion on the desired design:

We will use this to replace the current camera visualizer.

The desired workflow (which likely should be documented in a different place, or captured by Ella for all RA's in 446/447) is that an RA would check the saturation overlay (with and without gamma) and check if the mouse body, mouse tongue isn't showing up as oversaturated or undersaturated.

If the RA can be transient changes easily, they should make those changes; otherwise, they should flag the video as problematic and we should be adding a flag in the QC for that particular session to notify the PI/Sci of the mouse that it's problematic.

This should either be done via the QC portal or Alex's email-- I prefer the former so the data would be tracked fully. Given that we're due to have this PR merged, which will allow us to get a bunch of QC check for free, we might want an adhoc notification system (RA message Sci) until that gets merged and we add the camera testing suite, after which we will have this warning in the QC portal.

[ellahiltonvano]

Here is the related SIPE ticket

[micahwoodard]

@bruno-f-cruz @rachelstephlee @ellahiltonvano it looks like the camera recording visualizer was replaced with the saturation visualizer. Would it make more sense for the camera preview window to be the saturation visualizer since that is what's used during set up?

[ellahiltonvano]

@micahwoodard most RAs that run FIP do not use the preview window outside of the first session. It is used session 1 to set lickspout location before starting, but after that we just use the recording window.

I suppose per this new saturation window, we would need to be checking before starting the session, is that right @rachelstephlee?

[micahwoodard]

@micahwoodard most RAs that run FIP do not use the preview window outside of the first session. It is used session 1 to set lickspout location before starting, but after that we just use the recording window.

Is the difference between recoding and preview window that the recording window runs during a session and, I'm guessing based on the name, actually recording video? So would you be able to use it before a session?

[ellahiltonvano]

@micahwoodard we can start recording before starting behavior, but its in the SOP to use autocontrol (where the recording starts 5sec after behavior).

Correct about preview and recording, we are currently only recording in FIP sessions (all other sessions use preview).

[micahwoodard]

@ellahiltonvano @rachelstephlee so should we add the saturation visualizer to both the preview and recording window? If it's not standard practice for fip users to preview videos before hand, does this make us want to revisit the implementation of this saturation window?

[ellahiltonvano]

@rachelstephlee I think I'm still concerned that this is adding an additional step to the RA workflow when the likelihood that Sci intervention will be done before the session is very low. It seems like this is something that would be better to live in the QC portal - scientists can check their mice daily, and find time to resolve the issue in that box. If scientists are checking their data every day, then we wouldn't have more than a few sessions with an issue - even if RAs identify the issue before a session, it doesn't seem like this is something that would be able to be resolved before the mouse is run given time constraints.

[micahwoodard]

decided not to implement visualizer check