Skip to content

scaled culler for multiple classes#34

Open
memalhot wants to merge 1 commit intoOCP-on-NERC:mainfrom
memalhot:cronjob
Open

scaled culler for multiple classes#34
memalhot wants to merge 1 commit intoOCP-on-NERC:mainfrom
memalhot:cronjob

Conversation

@memalhot
Copy link
Copy Markdown
Contributor

@memalhot memalhot commented Apr 1, 2026

As classes running in the academic cluster scale, we need a faster and more dynamic way to shut down notebooks for the classes, as previously everything was hard coded. This allows the admin to provide a json dict of the class, type of class, and the cutoff time. It then creates a user map, and chooses the longer time if a student is in multiple classes, and shuts down/deletes when necessary

Copy link
Copy Markdown
Collaborator

@DanNiESh DanNiESh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Just some minor issues

return set(group.get("users") or [])
except Exception as e:
LOG.error("Failed to get users for group %s: %s", group_name, e)
sys.exit(1)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If one group lookup fails, it will exit without processing other classes. Can you just log the error and skip it?

"annotations": {"kubeflow-resource-stopped": now_utc}
}
}
oc.invoke("patch", ["notebook", nb_name, "-n", namespace, "--type=merge", "-p", json.dumps(patch_obj)])
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add error handling around this instead of return true?

COPY requirements.txt ./requirements.txt
RUN pip3 install -r requirements.txt

# Install the group-sync application
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor issue: it should be install the scaled-nb-culler

@@ -0,0 +1 @@
openshift-client
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the specific openshift-client version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants