Soft deprecate urllib.parse.urlparse()

[serhiy-storchaka]

urlparse() is similar to urlsplit(), but additionally it splits path on path and params using the ";" separator. This is wrong. First, the corresponding RFCs (RFC 3986, RFC 8820) do not list "params" as a separate URI component. Second, the ";" separator is used to specify parameters for a path segment, not the whole path. And this is scheme-specific, most schemes don't use it or use "," for similar purpose. It is mentioned in RFC 3986, but not in RFC 8820.

So, it is better to use urlsplit() and then parse path if needed. I afraid that most users of urlparse() are not aware of this, and only use it because it have more attractive name than urlsplit().

We should clearly document urlparse() as obsolete.

cc @orsenthil

Linked PRs

• gh-144148: Update the urllib.parse documentation #144497
• [3.14] gh-144148: Update the urllib.parse documentation (GH-144497) #144507
• [3.13] gh-144148: Update the urllib.parse documentation (GH-144497) (GH-144507) #144509

[aisk]

I've used urlparse heavily in my previous jobs and seen a lot of code using it, and never noticed this and never used urlsplit😂

In my opinion, deprecating urlparse and remove it in the near future will have a huge impact on existing user code, so I will vote to just mention this in docs and recommend users to use urlsplit for now.

[orsenthil]

We should clearly document urlparse() as obsolete.

This can become controversial. I think, urlparse was first higher level parsing function from the Python2 module urlparse and after migration it was commonly used as from urllib.parse import urlparse. There are plenty of libraries, documentation examples, and now AI outputs that will recommend urlparse over urlsplit.

We will have to see if fixing the behavior is an option over deprecation.

[picnixz]

@VanshAgarwal24036 As you can just read from the previous comments, we are already discussing this. Can you avoid spitting out what the LLM is giving you? The comments you share are honestly not useful.

[serhiy-storchaka]

urlsplit() is the fixed urlparse().

urlparse() was based on obsolete RFC 1808 which has "params" as one of six URL components. It was updated by RFC 2396 and obsoleted by RFC 3986 which specify a four- and five-component URI, and "params" is not a separate component, but a part of "path". The latest RFC 8820 don't mention it at all.

Maybe the uriparse() name would be better, but it would cause more confusion.

Currently, urlparse() is documented first, as the main parsing function. urlsplit() is documented further, it refers to urlparse() (even if mention that urlsplit() is more preferable in most cases) and it can be easily missed. So, I suggest to move the urlsplit() documentation first, make it as full as the current urlparse() documentation, make the urlparse() documentation minimal, just a reference to urlsplit() with differences, and clearly stay that it is obsolete and based on obsolete RFCs. This way we can hope that at least new code will gradually start using urlsplit(), and that if a user has problems with urlparse(), they will likely replace it with urlsplit() in the existing code after looking in the docs.

This is a task not for LLM and not for beginners. It can involve moving large amount of text, so it can only be performed by a trustful core developer. As soon as we agree on a plan.

[ncoghlan]

I edited the title to make it clear this is proposing soft deprecation in the docs (as new code should be using urlsplit, but the docs definitely don't make that clear) rather than outright removal, or even programmatic deprecation (as that would break far too much code to be worthwhile).

Coincidentally, I just ran into #129745 (which is an artifact of the parameter parsing being a legacy quirk of the older RFCs that urlparse was initially based on) when reading the urlparse example code.

[orsenthil]

Maybe the uriparse() name would be better, but it would cause more confusion.

If parsing will become more consistent, and maintainability becomes easy, then a new function named uriparse might be a good idea. We had explored a bit in some discussions, but we didn't pursue it. As far as I know, we didn't come up with a concrete proposal.

[serhiy-storchaka]

My concrete proposal is in #144148 (comment):

move the urlsplit() documentation first, make it as full as the current urlparse() documentation, make the urlparse() documentation minimal, just a reference to urlsplit() with differences, and clearly stay that it is obsolete and based on obsolete RFCs.

[orsenthil]

@serhiy-storchaka , this idea sounds good to me.