Improve IAMService user to CS conversion by maxnoe · Pull Request #8491 · DIRACGrid/DIRAC

maxnoe · 2026-03-25T19:03:52Z

Handle nested groups
Filter groups by vo name
Filter groups by having voms.role label
Add tests

BEGINRELEASENOTES

*Core
FIX: Improve VOMS2CSAgent to handle nested groups and only sync groups that have the voms.role label
ENDRELEASENOTES

Closes #8487

maxnoe · 2026-03-26T08:06:42Z

@chaen the CI failure here seems unrelated to me

aldbr · 2026-04-02T07:42:39Z

@maxnoe is that ready to be reviewed now?

maxnoe · 2026-04-02T08:08:12Z

I would be happy about a review, yes, but I left it on draft as I also wanted to test it in our integration test environment but didn't have a chance yet.

chaen · 2026-04-02T09:10:41Z

I've just tried it against our prod server, and I see discrepencies, but I did not yet dig down in it

maxnoe · 2026-04-02T09:12:20Z

I've just tried it against our prod server, and I see discrepencies, but I did not yet dig down in it

There is an additional filter now to only create entries for groups that are actually voms roles. I would expect that you now get less roles attached to users, but the roles that are no longer attached do not correspond to real voms roles in the iam server.

src/DIRAC/Core/Security/IAMService.py

chaen · 2026-04-02T09:16:58Z

There's definitely something off with the logic as it starts adding people without a single voms role to our default group

maxnoe · 2026-04-02T09:18:04Z

There's definitely something off with the logic as it starts adding people without a single voms role to our default group

ok, thanks, I will dig into that next week

src/DIRAC/Core/Security/IAMService.py

maxnoe · 2026-04-02T09:28:05Z

There's definitely something off with the logic as it starts adding people without a single voms role to our default group

Where should this filtering happen?

The IAMService class getUsers() method according to the test correctly returns a list of all users, with most of them having 0 voms roles.

Should getUsers() already filter out users with no roles? Or should that happen later?

chaen · 2026-04-02T12:15:50Z

oos sorry I pushed by mistake, I'll fix my mess

- Handle nested groups - Filter groups by vo name - Filter groups by having voms.role label - Add tests

maxnoe · 2026-04-09T14:52:33Z

I finally ran this in our test setup with groups mapped to the nested group voms roles.

One observation: the voms2cs script did not modify groups that didn't yet have any users. I needed to add an initial User = admin-user in the cs, only then the sync would add the IAM users.

[dirac@dpps-wms-dirac-voms2cs-67cc4f4d8b-skn55 ~]$ dirac-admin-voms-sync --useIAM -V ctao.dpps.test
Could not convert {'familyName': 'User', 'formatted': 'Admin User', 'givenName': 'Admin'} KeyError('certificates')
/CN=DPPS User ['/ctao.dpps.test/dpps/Role=user', '/ctao.dpps.test/dpps/dataquality/Role=user', '/ctao.dpps.test/Role=user', '/ctao.dpps.test/dpps/pipelines/Role=user', '/ctao.dpps.test/Role=manager', '/ctao.dpps.test/dpps/archive/Role=user']
Could not convert {'familyName': 'User', 'formatted': 'Admin User', 'givenName': 'Admin'} KeyError('certificates')
Could not convert {'familyName': 'User', 'formatted': 'DPPS Unprivileged User', 'givenName': 'DPPS Unprivileged'} KeyError('certificates')
/DIRAC Test User 1 ['/ctao.dpps.test/dpps/Role=user', '/ctao.dpps.test/dpps/pipelines/Role=user']
/DIRAC Test User 2 ['/ctao.dpps.test/dpps/pipelines/Role=manager', '/ctao.dpps.test/dpps/Role=user', '/ctao.dpps.test/dpps/Role=manager', '/ctao.dpps.test/dpps/pipelines/Role=user']
There were in total 3 errors
VOMS user entries There are 3 user entries in VOMS for VO ctao.dpps.test
Users already registered : there are 4 registered users in DIRAC VO ctao.dpps.test
Added user admin-user to group pipeline_users
Added user admin-user to group dpps_users
Modified user admin-user
Modified user admin-user: {'DN': '/CN=DPPS User', 'CA': '/CN=DPPS Development CA', 'Email': 'dpps@test.example', 'Groups': ['pipeline_users', 'dpps_users', 'dirac_user', 'dirac_admin', 'dpps_group', 'dpps_genpilot']}
Added user dirac-test-user1 to group dpps_users
Added user dirac-test-user1 to group pipeline_users
Modified user dirac-test-user1
Modified user dirac-test-user1: {'DN': '/DIRAC Test User 1, /DIRAC Test User 1, /DIRAC Test User 1, /DIRAC Test User 1, /DIRAC Test User 1, /DIRAC Test User 1, /DIRAC Test User 1', 'CA': '/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA', 'Email': 'albert.einstein@dpps.test', 'Groups': ['dpps_users', 'dirac_user', 'pipeline_users']}
Added user dirac-test-user2 to group pipeline_users
Added user dirac-test-user2 to group dpps_users
Added user dirac-test-user2 to group pipeline_managers
Modified user dirac-test-user2
Modified user dirac-test-user2: {'DN': '/DIRAC Test User 2, /DIRAC Test User 2, /DIRAC Test User 2, /DIRAC Test User 2, /DIRAC Test User 2, /DIRAC Test User 2, /DIRAC Test User 2', 'CA': '/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA', 'Email': 'albert.einstein@dpps.test', 'Groups': ['pipeline_users', 'dpps_users', 'dirac_user', 'pipeline_managers']}

User results: new 0, modified 3, deleted 0, new/suspended 0
There are 3 user entries in VOMS for VO ctao.dpps.test
There are 4 registered users in DIRAC for VO ctao.dpps.test

  Modified user admin-user:
    Added to group(s) dpps_users,pipeline_users


  Modified user dirac-test-user1:
    Added to group(s) dpps_users,pipeline_users


  Modified user dirac-test-user2:
    Added to group(s) pipeline_users,dpps_users,pipeline_managers

There are changes to Registry ready to commit, do you want to proceed ? [Y|n]:y
Registry changes committed for VO ctao.dpps.test

Group             Number of users VOMS Role
===========================================
dpps_group                      2 /ctao.dpps.test/Role=user
dpps_users                      3 /ctao.dpps.test/dpps/Role=user
pipeline_managers               1 /ctao.dpps.test/dpps/pipelines/Role=manager
pipeline_users                  3 /ctao.dpps.test/dpps/pipelines/Role=user

Users with multiple DNs:
  dirac-test-user1:
    /DIRAC Test User 1
    /DIRAC Test User 1
    /DIRAC Test User 1
    /DIRAC Test User 1
    /DIRAC Test User 1
    /DIRAC Test User 1
    /DIRAC Test User 1
  dirac-test-user2:
    /DIRAC Test User 2
    /DIRAC Test User 2
    /DIRAC Test User 2
    /DIRAC Test User 2
    /DIRAC Test User 2
    /DIRAC Test User 2
    /DIRAC Test User 2

I'm also a bit confused about the "Multi-DN" users. As far as I can see, there is a one-to-one mapping of users and dns here.

maxnoe · 2026-04-09T14:54:02Z

I think this might be because I messed up the certificate creation for the two test users, the dn is /DIRAC Test User 2 but should have been /CN=DIRAC TEst User 2

maxnoe · 2026-04-09T17:04:46Z

Confirmed: with correct CN=... DNs, I do not get the multiple DN warnings:

Could not convert {'familyName': 'User', 'formatted': 'Admin User', 'givenName': 'Admin'} KeyError('certificates')
Could not convert {'familyName': 'User', 'formatted': 'Admin User', 'givenName': 'Admin'} KeyError('certificates')
There were in total 2 errors
VOMS user entries There are 3 user entries in VOMS for VO ctao.dpps.test
Users already registered : there are 2 registered users in DIRAC VO ctao.dpps.test
Removed user admin-user from group dpps_pipeline_managers
Added user admin-user to group dirac_user
Modified user admin-user
Modified user admin-user: {'DN': '/CN=DPPS User', 'CA': '/CN=DPPS Development CA', 'Email': 'richart.feynman@dpps.test', 'Groups': ['dpps_pipeline_users', 'dpps_users', 'dpps_genpilot', 'dpps_group', 'dirac_user', 'dirac_admin']}
Setting DN property for user dirac-test-user1 to /CN=DIRAC Test User 1,/CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1
Setting CA property for user dirac-test-user1 to /CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA
Added user dirac-test-user1 to group dirac_user
Added user dirac-test-user1 to group dpps_pipeline_users
Added user dirac-test-user1 to group dpps_users
Modified user dirac-test-user1
Modified user dirac-test-user1: {'DN': '/CN=DIRAC Test User 1,/CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1', 'CA': '/CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA', 'Email': 'albert.einstein@dpps.test', 'Groups': ['dirac_user', 'dpps_pipeline_users', 'dpps_users']}
Setting DN property for user dirac-test-user2 to /CN=DIRAC Test User 2,/CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2
Setting CA property for user dirac-test-user2 to /CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA
Added user dirac-test-user2 to group dpps_pipeline_managers
Added user dirac-test-user2 to group dirac_user
Added user dirac-test-user2 to group dpps_pipeline_users
Added user dirac-test-user2 to group dpps_users
Modified user dirac-test-user2
Modified user dirac-test-user2: {'DN': '/CN=DIRAC Test User 2,/CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2', 'CA': '/CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA', 'Email': 'enrico.fermi@dpps.test', 'Groups': ['dpps_pipeline_managers', 'dirac_user', 'dpps_pipeline_users', 'dpps_users']}

User results: new 0, modified 3, deleted 0, new/suspended 0
There are 3 user entries in VOMS for VO ctao.dpps.test
There are 2 registered users in DIRAC for VO ctao.dpps.test

  Modified user admin-user:
    Added to group(s) dirac_user
    Removed from group(s) dpps_pipeline_managers


  Modified user dirac-test-user1:
    DN: /CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1 -> /CN=DIRAC Test User 1,/CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1, /CN=DIRAC Test User 1
    CA: /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA -> /CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA
    Added to group(s) dpps_users,dpps_pipeline_users,dirac_user


  Modified user dirac-test-user2:
    DN: /CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2 -> /CN=DIRAC Test User 2,/CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2, /CN=DIRAC Test User 2
    CA: /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA -> /CN=DPPS Development CA,/CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA, /CN=DPPS Development CA
    Added to group(s) dpps_pipeline_managers,dpps_pipeline_users,dirac_user,dpps_users

There are changes to Registry ready to commit, skipped because of dry run
Accumulated diff with Controller CS
-       #@@-host - /CN=dirac-voms2cs - 2026-04-09 17:00:53
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
-       #@@-host - /CN=dirac-voms2cs - 2026-04-09 17:00:53
+       DN += /CN=DIRAC Test User 1
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       CA += /CN=DPPS Development CA
-       #@@-host - /CN=dirac-voms2cs - 2026-04-09 17:00:53
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
-       #@@-host - /CN=dirac-voms2cs - 2026-04-09 17:00:53
+       DN += /CN=DIRAC Test User 2
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       CA += /CN=DPPS Development CA
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       Users += admin-user
+       Users += dirac-test-user1
+       Users += dirac-test-user2
-       Users = admin-user
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       Users = dirac-test-user2
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       Users += dirac-test-user1
+       Users += dirac-test-user2
+       #@@-dirac_admin - /CN=DPPS User - 2026-04-09 17:03:53
+       Users += dirac-test-user1
+       Users += dirac-test-user2

Group                  Number of users VOMS Role
================================================
dpps_group                           2 /ctao.dpps.test/Role=user
dpps_pipeline_managers               1 /ctao.dpps.test/dpps/pipelines/Role=manager
dpps_pipeline_users                  1 /ctao.dpps.test/dpps/pipelines/Role=user
dpps_users                           1 /ctao.dpps.test/dpps/Role=user

maxnoe force-pushed the voms2cs-nested-groups branch 3 times, most recently from 2d5e13c to b1fbfd8 Compare March 25, 2026 19:17

chaen reviewed Apr 2, 2026

View reviewed changes

src/DIRAC/Core/Security/IAMService.py Show resolved Hide resolved

maxnoe commented Apr 2, 2026

View reviewed changes

src/DIRAC/Core/Security/IAMService.py Outdated Show resolved Hide resolved

maxnoe force-pushed the voms2cs-nested-groups branch 2 times, most recently from 0bffc2e to 9d725c9 Compare April 2, 2026 10:41

chaen force-pushed the voms2cs-nested-groups branch from 73737a6 to 9d725c9 Compare April 2, 2026 12:16

fix(Core): Improve IAMService user to CS conversion

159b51e

- Handle nested groups - Filter groups by vo name - Filter groups by having voms.role label - Add tests

maxnoe force-pushed the voms2cs-nested-groups branch from 7c8f92c to 159b51e Compare April 7, 2026 12:30

maxnoe marked this pull request as ready for review April 9, 2026 14:50

maxnoe requested review from atsareg and fstagni as code owners April 9, 2026 14:50

Conversation

maxnoe commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maxnoe commented Mar 26, 2026

Uh oh!

aldbr commented Apr 2, 2026

Uh oh!

maxnoe commented Apr 2, 2026

Uh oh!

chaen commented Apr 2, 2026

Uh oh!

maxnoe commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

chaen commented Apr 2, 2026

Uh oh!

maxnoe commented Apr 2, 2026

Uh oh!

Uh oh!

maxnoe commented Apr 2, 2026

Uh oh!

chaen commented Apr 2, 2026

Uh oh!

maxnoe commented Apr 9, 2026

Uh oh!

maxnoe commented Apr 9, 2026

Uh oh!

maxnoe commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

maxnoe commented Mar 25, 2026 •

edited

Loading

maxnoe commented Apr 2, 2026 •

edited

Loading