CheckDependencies: Picks the lowest version number of PIP packages

[fyrak1s]

Hi @sage-etcher thanks for the fix you have provided with #54!

Unfortunately (or fortunately? :P) this exposes a follow up bug in version resolution in case of PIP!

For example, I've added a dependency PIP3:meson to the Dependency file of recipe Tree 2.1.3.

Upon running Compile Tree or simply: CheckDependencies Tree it will prompt us to install meson 0.29.0.dev1 - which is the oldest one in the repo: https://pypi.org/project/meson/#history. If we specify > 0.29.0 it picks the next lowest version.

I have tested it with other PIP packages with the same results....

---

Interestingly in the case of LuaRocks (e.g. LuaSocket) the latest dependency is correctly picked. Also a dependency definition like LuaRocks:LuaSocket < 3.1.0 correctly picks 3.0.0.

Others like CPAN seems to work resonably as well (it at least picks the latest version), although I could not test most of them like Cabal, CTAN, RubyGems, etc since they are not even present on my machine or a default gobo installation at this time.

Therefore it could be that the bug lays within PIP sepecific code: https://github.com/gobolinux/Scripts/blob/master/bin/Alien-PIP, or maybe it lays somewhere else.

I do not consider the other package managers integrations that crucial right now, but PIP is an important one that needs to work correctly for our packaging purposes.

[fyrak1s]

I could reduce the testcase down to Alien-PIP3 --getinstallversion meson which outputs 0.29.0.dev1, rather than the latest 1.8.2.

So that points to somewhere between the lines of getinstallversion():

Scripts/bin/Alien-PIP

Lines 22 to 51 in 41260da

	getinstallversion() {
	prog="$1"
	versions=($(python${python_major} - << EOF
	import json
	try:
	from urllib2 import urlopen, Request
	except ImportError:
	from urllib.request import urlopen, Request
	from pkg_resources import parse_version
	def available_versions(prog):
	try:
	url = "https://pypi.python.org/pypi/%s/json" %prog
	data = json.load(urlopen(Request(url)))
	versions = list(data["releases"].keys())
	versions.sort(key=parse_version)
	return versions
	except:
	return ''
	print("\n".join(available_versions("$prog")))
	EOF
	))
	for V in ${versions[@]}
	do
	if In_Version_Range "$2" "$V" "$3"
	then
	echo $V
	return 0
	fi
	done
	}

[sage-etcher]

That is verry interesting, I will take a look! what version of python are you running?

[sage-etcher]

I believe I know the issue, the new sorting callback sorts in reverse order from what distutils did. since after its sorted it iterates through the versionlist until it finds a matching version for min max, if the array is oldest to newest it always gets the oldest possible. should be as easy as reversing the sort order, will implement tomorrow, in bed atm.

[fyrak1s]

I am on Python 3.8 and my plan is to switch to Python 3.9 soon (is required as minimum build dep by some packages), but I got stuck at "switching" our Alien subsystem to a new minor python revision, see https://github.com/orgs/gobolinux/discussions/31. Out of curiosity did you manage to get it work on your side?

Eventually I'll brute-force my way through it, but I think it won't be pretty... We don't seem to have a clear mechanism or workflow to do such a switch. I think something might have been attempted with #24, but it never got finished...

(FYI I am on purpose withholding a switch to anything higher than 3.9 atm, since on a quick test this broke some GoboLinux tools. They'd have to be adapted for newer python's.)

I believe I know the issue, the new sorting callback sorts in reverse order from what distutils did. since after its sorted it iterates through the versionlist until it finds a matching version for min max, if the array is oldest to newest it always gets the oldest possible.

Lol, that's kind of hilarious^^ Ok good that you have an idea already, sounds like it should be an easy enough fix 😸

[sage-etcher]

@nuc1eon it is fixed now, closing this issue

[fyrak1s]

@sage-etcher I'm happy to see your commit rights work alright... thanks for being so swift about it! :)