Side effects when setting TriggerMode and TriggerSource

[huywork]

Describe the bug
Summary

1. With ARV_REGISTER_CACHE_POLICY_DEBUG, there are a lot of warnings about mismatches between cached and current values for RegTriggerMode and RegTriggerSource
2. With ARV_REGISTER_CACHE_POLICY_ENABLE, setting TriggerMode to On or Off can have side effects on TriggerSource and setting TriggerSource to Freerun or Line1 can also have side effects on TriggerMode.
3. With ARV_REGISTER_CACHE_POLICY_ENABLE, some of the side effects above are such that it it not possible to have the combination (TriggerMode, TriggerSource) = (On, Freerun).

The results we are getting are

=====================================================================
Test suite 1
Set cache policy to ARV_REGISTER_CACHE_POLICY_DEBUG
This shows that with ARV_REGISTER_CACHE_POLICY_DEBUG everything works
but there are mismatches between current & cached values
=====================================================================
Initialise (TriggerMode, TriggerSource) to (On, Freerun)
[15:34:50.902] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = On
TriggerSource = Freerun

1. Change (On, Freerun) to (On, Line1)
[15:34:51.304] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1

2. Change (On, Line1) to (On, Freerun)
[15:34:51.681] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = On
TriggerSource = Freerun

3. Change (On, Freerun) to (Off, Freerun)
[15:34:52.056] 🆆 policies> Current and cached value mismatch for 'RegTriggerSource'
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun

4. Change (Off, Freerun) to (On, Freerun)
[15:34:52.427] 🆆 policies> Current and cached value mismatch for 'RegTriggerSource'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = On
TriggerSource = Freerun

5. Re-initialise to (On, Line1)
[15:34:52.805] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1

6. Change (On, Line1) to (Off, Line1)
[15:34:53.214] 🆆 policies> Current and cached value mismatch for 'RegTriggerSource'
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Line1

7. Change (Off, Line1) to (On, Line1)
[15:34:53.621] 🆆 policies> Current and cached value mismatch for 'RegTriggerSource'
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1

8. Re-initilalise to (Off, Line1)
[15:34:54.022] 🆆 policies> Current and cached value mismatch for 'RegTriggerSource'
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Line1

9. Change (Off, Line1) to (Off, Freerun)
[15:34:54.438] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun

10. Change (Off, Freerun) to (Off, Line1)
[15:34:54.842] 🆆 policies> Current and cached value mismatch for 'RegTriggerMode'
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Line1

========================================================================
Test suite 2
Set cache policy to ARV_REGISTER_CACHE_POLICY_ENABLE
This shows that with ARV_REGISTER_CACHE_POLICY_ENABLE there are problems
-We cannot have (TriggerMode, TriggerSource) = (On, Freerun)
-There are side effects when setting TriggerMode and TriggerSource
========================================================================
Trying to set (TriggerMode, TriggerSource) to (On, Freerun)
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun
!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!
Changing this back to On to continue with test suite
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1
!!! PROBLEM: Setting TriggerMode to On has side effect of changing TriggerSource to Line1 !!!
Changing this back to Freerun to continue with test suite
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun
!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!
!!! PROBLEM: It seems we can't have (On, Freerun) !!!

1. Trying to set (On, Line1)
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Line1
!!! PROBLEM: Setting TriggerSource to Line1 has side effect of changing TriggerMode to Off !!!
Changing this back to On to continue with test suite
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1
So we can have (On, Line1), notwithsanding the side effect above

2. Trying to change (On, Line1) to (On, Freerun)
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun
!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!
Changing this back to On to continue with test suite
Read back with with arv_camera_get_string:
TriggerMode = On
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = On
TriggerSource = Line1
!!! PROBLEM: Setting TriggerMode to On has side effect of changing TriggerSource to Line1 !!!
!!! PROBLEM: It seems we can't have (On, Freerun) !!!

3. Trying to set (Off, Freerun)
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Freerun
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Freerun
So we can have (Off, Freerun)

4. Trying to set (Off, Line1)
Read back with with arv_camera_get_string:
TriggerMode = Off
TriggerSource = Line1
Read back with arv-tool:
TriggerMode = Off
TriggerSource = Line1
So we can have (off Line1)

To Reproduce
Run the reproducer script. This runs 2 test suites. The first test suite shows that with ARV_REGISTER_CACHE_POLICY_DEBUG there are mistmaches between current and cached values, but arv_camera_set_string works as expected. The second testsuite shows that with ARV_REGISTER_CACHE_POLICY_ENABLE, there are a lot of strange side effects, plus it's not possible to have (On, Freerun).

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <arv.h>

void checkError(GError *error){
	if (error != NULL) {
		printf ("Error: %s\n", error->message);
		exit(1);
	}
}

void printStatus(ArvCamera *camera, const char *expectedTriggerMode = NULL, const char *expectedTriggerSource = NULL){

	GError *error = NULL;
	const char *triggerMode = arv_camera_get_string(camera, "TriggerMode", &error);
	checkError(error);

	const char *triggerSource = arv_camera_get_string(camera, "TriggerSource", &error);
	checkError(error);	

	printf ("Read back with with arv_camera_get_string:\nTriggerMode = %s\nTriggerSource = %s\n",
		triggerMode,
		triggerSource
	);

	if (expectedTriggerMode != NULL && strcmp(expectedTriggerMode, triggerMode) != 0) {
		printf ("WARNING: TriggerMode is expected to be %s but it is %s\n", expectedTriggerMode, triggerMode);
	}
	if (expectedTriggerSource != NULL && strcmp(expectedTriggerSource, triggerSource) != 0 ){
		printf ("WARNING: TriggerSource is expected to be %s but it is %s\n", expectedTriggerSource, triggerSource);
	}	

	printf ("Read back with arv-tool:\n");
	int returnCode = system("arv-tool-0.8 -n 172.23.240.151 control 'TriggerMode' 'TriggerSource'");
}

void testSuite1(ArvCamera *camera) {
	GError *error = NULL;

	arv_camera_set_register_cache_policy(camera, ARV_REGISTER_CACHE_POLICY_DEBUG);

	printf ("=====================================================================\n");
	printf ("Test suite 1\n");
	printf ("Set cache policy to ARV_REGISTER_CACHE_POLICY_DEBUG\n");
	printf ("This shows that with ARV_REGISTER_CACHE_POLICY_DEBUG everything works\n");
	printf ("but there are mismatches between current & cached values\n");
	printf ("=====================================================================\n");

	printf ("Initialise (TriggerMode, TriggerSource) to (On, Freerun)\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf("\n");

	printf ("1. Change (On, Freerun) to (On, Line1)\n");
	arv_camera_set_string (camera, "TriggerSource", "Line1", &error);
	checkError(error);
	printStatus(camera, "On", "Line1");
	printf("\n");

	printf ("2. Change (On, Line1) to (On, Freerun)\n");
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf("\n");

	printf ("3. Change (On, Freerun) to (Off, Freerun)\n");
	arv_camera_set_string (camera, "TriggerMode", "Off", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("4. Change (Off, Freerun) to (On, Freerun)\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("5. Re-initialise to (On, Line1)\n");
	arv_camera_set_string (camera, "TriggerSource", "Line1", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("6. Change (On, Line1) to (Off, Line1)\n");
	arv_camera_set_string (camera, "TriggerMode", "Off", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("7. Change (Off, Line1) to (On, Line1)\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("8. Re-initilalise to (Off, Line1)\n");
	arv_camera_set_string (camera, "TriggerMode", "Off", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("9. Change (Off, Line1) to (Off, Freerun)\n");
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera);
	printf("\n");

	printf ("10. Change (Off, Freerun) to (Off, Line1)\n");
	arv_camera_set_string (camera, "TriggerSource", "Line1", &error);
	checkError(error);
	printStatus(camera);	
	printf("\n");

}

void testSuite2(ArvCamera *camera) {
	GError *error = NULL;

	arv_camera_set_register_cache_policy(camera, ARV_REGISTER_CACHE_POLICY_ENABLE);

	printf ("========================================================================\n");
	printf ("Test suite 2\n");
	printf ("Set cache policy to ARV_REGISTER_CACHE_POLICY_ENABLE\n");
	printf ("This shows that with ARV_REGISTER_CACHE_POLICY_ENABLE there are problems\n");
	printf ("-We cannot have (TriggerMode, TriggerSource) = (On, Freerun)\n");
	printf ("-There are side effects when setting TriggerMode and TriggerSource\n");
	printf ("========================================================================\n");

	printf ("Trying to set (TriggerMode, TriggerSource) to (On, Freerun)\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf ("!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!\n");
	printf ("Changing this back to On to continue with test suite\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf ("!!! PROBLEM: Setting TriggerMode to On has side effect of changing TriggerSource to Line1 !!!\n");
	printf ("Changing this back to Freerun to continue with test suite\n");
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf ("!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!\n");
	printf ("!!! PROBLEM: It seems we can't have (On, Freerun) !!!\n");
	printf("\n");

	printf ("1. Trying to set (On, Line1)\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	arv_camera_set_string (camera, "TriggerSource", "Line1", &error);
	checkError(error);
	printStatus(camera, "On", "Line1");
	printf ("!!! PROBLEM: Setting TriggerSource to Line1 has side effect of changing TriggerMode to Off !!!\n");
	printf ("Changing this back to On to continue with test suite\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	printStatus(camera, "On", "Line1");
	printf ("So we can have (On, Line1), notwithsanding the side effect above\n");
	printf("\n");

	printf ("2. Trying to change (On, Line1) to (On, Freerun)\n");
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf ("!!! PROBLEM: Setting TriggerSource to Freerun has side effect of changing TriggerMode to Off !!!\n");
	printf ("Changing this back to On to continue with test suite\n");
	arv_camera_set_string (camera, "TriggerMode", "On", &error);
	checkError(error);
	printStatus(camera, "On", "Freerun");
	printf ("!!! PROBLEM: Setting TriggerMode to On has side effect of changing TriggerSource to Line1 !!!\n");
	printf ("!!! PROBLEM: It seems we can't have (On, Freerun) !!!\n");
	printf("\n");

	printf ("3. Trying to set (Off, Freerun)\n");
	arv_camera_set_string (camera, "TriggerMode", "Off", &error);
	checkError(error);
	arv_camera_set_string (camera, "TriggerSource", "Freerun", &error);
	checkError(error);
	printStatus(camera);
	printf ("So we can have (Off, Freerun)\n");
	printf("\n");

	printf ("4. Trying to set (Off, Line1)\n");
	arv_camera_set_string (camera, "TriggerMode", "Off", &error);
	checkError(error);
	arv_camera_set_string (camera, "TriggerSource", "Line1", &error);
	checkError(error);
	printStatus(camera);
	printf ("So we can have (off Line1)\n");
	printf("\n");
}


int main (int argc, char **argv) {
	ArvCamera *camera;
	GError *error = NULL;

	camera = arv_camera_new ("172.23.240.151", &error);
	checkError(error);

	testSuite1(camera);
	testSuite2(camera);

	printf ("Ok\n");

	return 0;
}

Expected behavior
We expect that setting TriggerMode have no side effect on TriggerSource and vice versa even with ARV_REGISTER_CACHE_POLICY_ENABLE

Camera description:

• Manufacturer: Allied_Vision_Technologies
• Model: MAKO_A_SONY_CMOS
• Interface: GigE

Platform description:

• Aravis version: 0.8
• OS: RHEL 8
• Hardware: x86_64

Additional context
Add any other context about the problem here.

[huywork]

GeniCam.xml

[huywork]

Hi, we are still stuck with this problem. Would anyone be able to help?

[EmmanuelP]

Hi,

It will be difficult to help without having the device on hand.

You should try to understand if it is an aravis issue, or a problem in the register node description. For MaskedIntReg The default Cachable value is WriteThrough. That means the content of the cache is updated on register write. Add some debug output in arvgcregisternode.c to display the written and retrieved values.

It is possible the manufacturer has not taken care to describe correctly the caching for its camera. Genicam specification says: « Note that caching is an optional feature of any implementation. »

[huywork]

Hi Emmanuel,

Thank you for your reply.

I have done done further invesigation and I think the problem comes from the fact that the camera's TriggerMode and TriggerSource share the same 32-bit hardware register, with TriggerMode using bit 0 and TriggerSource using bits 15-31.

As far as I know, aravis creates two cache items, one for TriggerMode, one for TriggerSource, each item being masked 32 bits, so the shared 32-bit hardware register is duplicated in two items in the aravis cache.

When I update, say, TriggerMode, aravis updates bit 0 in the 32-bit cache item for TriggerMode, but it does not update bit 0 in the 32-bit cache item for TriggerSource, so the cache item for TriggerSource has stale data for TriggerMode. If subsequently I update TriggerSource, aravis will merge the stale bit 0 in the TriggerSouce cache item and will write that stale bit to the hardware. This corrupts the camera's TriggerMode. This is the same as the lost update problem in database applications.

[EmmanuelP]

Thanks for your debugging effort. You are pointing the root issue, which is the fact there is one cache instance per register node.

The more obvious fix to what you are seeing would be to move the cache to the genicam root node.

There would be still some possible issues caused by the fact the current cache implementation creates a cache entry for every register address. This also breaks if registers are overlapping (different address, but shared memory area).

Any chance you can propose a patch ? At least for moving the cache instance.

[huywork]

Can you explain in more details what you mean by moving the cache to the genicam root node?

How difficult would it be to change the caching so that each hardware register maps to one cache item even if it's used by multiple features like TriggerMode and TriggerSource?

[EmmanuelP]

Currently, the cache is a hashtable stored in the private data of ArvGcRegisterNode:

aravis/src/arvgcregisternode.c

Lines 46 to 95 in b1aba1f

	typedef struct {
	gint64 address;
	gint64 length;
	} ArvGcCacheKey;
	static gboolean
	arv_gc_cache_key_equal (gconstpointer v1, gconstpointer v2)
	{
	return ((((ArvGcCacheKey *) v1)->address == ((ArvGcCacheKey *) v2)->address) &&
	(((ArvGcCacheKey *) v1)->length == ((ArvGcCacheKey *) v2)->length));
	}
	static guint
	arv_gc_cache_key_hash (gconstpointer v)
	{
	return g_int64_hash (&((ArvGcCacheKey *) v)->address);
	}
	static ArvGcCacheKey *
	arv_gc_cache_key_new (gint64 address, gint64 length)
	{
	ArvGcCacheKey *key = g_new (ArvGcCacheKey, 1);
	key->address = address;
	key->length = length;
	return key;
	}
	typedef struct {
	GSList *addresses;
	GSList *swiss_knives;
	GSList *indexes;
	ArvGcPropertyNode *length;
	ArvGcPropertyNode *port;
	ArvGcPropertyNode *cachable;
	ArvGcPropertyNode *polling_time;
	ArvGcPropertyNode *access_mode;
	ArvGcPropertyNode *endianness;
	GSList *invalidators; /* #ArvGcPropertyNode */
	gboolean cached;
	GHashTable *caches;
	guint n_cache_hits;
	guint n_cache_misses;
	guint n_cache_errors;
	char v_string[G_ASCII_DTOSTR_BUF_SIZE];
	} ArvGcRegisterNodePrivate;

This should be moved to https://github.com/AravisProject/aravis/blob/main/src/arvgc.c, with the necessary API to push and pop data from the cache.