Moving forward with maintenance

[molpopgen]

We have a couple of action items that would be nice to knock off the plate.

First, #577 merged a bug fix but has not been released. We should do a release!

Second, it would be useful to streamline our dependency management prior to a release. In late 2024 (omg), I made separate "requirements" files for various elements of this project (packages/tests/docs/etc..). The resulting setup is far from ideal. For example, there are many automatically-opened security issues for this repo at the moment. They all (I think...) come from the docs. However, the requirements file for the docs is difficult to update piecemeal: we cannot update jupyter-book itself (docs will just break....) and updating the various insecure deps will lead to "dependency hell" because we also need to work out what deps of those deps also need bumping.

I propose that we drag ourselves into 2023 and move to using uv for managing dependencies for the CI and development environments. I did this for fwdpy11 a while ago and I think it has paid off with less time lost due to "ecosystem drift". Jerome also moved tskit/msprime/etc., over, and I suspect folks there will be happy?

Third, it would seem that #574, #575 are low-hanging fruit that would be straightforward to get ready before the next release.

Thoughts? I'm happy to take the lead on uv.

pinging: @grahamgower @apragsdale @jeromekelleher

[apragsdale]

Hi @molpopgen - yes, I agree we should release that bug fix (finally!).

I don't personally have much experience using uv yet, but I know you and others have adopted it and been happy with it. Do we imagine it will be supported longish term, enough to justify moving over ourselves (and not have it fall out of support in a year or two)? I know you and I have had a few of these conversations, and I see no reason not to move over to uv if everyone has been happy with it.

[grahamgower]

Hey @molpopgen. That all sounds reasonable. I haven't used uv at all, but I'm happy for you to take the lead and use it to help modernise the repo. Those "security issues" all look rather irrelevant to me because they are only lint or docs deps. But updating the jupyter-book/docs deps has always been a pain point, so any suggestions you have here to improve matters would be very welcome!

Note also that github workflows/actions related to release management tend to bitrot quite badly. Since it's been a while since the previous release, I expect these actions will be broken. Doing a beta release first is definitely recommended! We should probably mention this in the dev docs.

[molpopgen]

@apragsdale -- there is a lot of excitement about uv. Some of its pros are:

1. It (mostly? totally?) uses agreed-upon standards vis-a-vis pyproject. Other popular tools in this space, like poetry, do their own thing in some key aspects.
2. It is pretty intuitive how to separate direct dependencies of your tool from those of the test suite, docs, etc..
3. It runs very very quickly, and makes dependency caching on github unnecessary. (That is a positive b/c you get get rid of an entire set of workflow components that themselves change a bit over time.)

Some cons:

1. Build dependencies are not (currently) part of the lock setup. These are the things you may need to compile parts of a module -- Cython, pybind11, etc.. I have a hack for this over in fwdpy11 so that my CI is always built with the same versions of these things.
2. A given uv version only supports a specific range of Python versions. This is probably fine in practice, as the current uv supports most/all of the currently supported Python versions, which is >= the set that we care about.
3. We don't know how long it will be the "it" tool for this set of tasks.

[molpopgen]

Turns out that moving to uv helps in a few respects -- we have pinned versions of things incompatible w/our claimed range of supported Python versions!

[jeromekelleher]

I'm 100% behind moving to UV, I think it's great.