Note that the www domain label is enough to change the equality of two URLs according to the Url-package. This is expected behaviour, as it does actually change semantics.
For scraping, some links may use the domain label and others may not, e.g. human-written links would probably omit the label while automated links would probably include it for completeness. This could result in a page with two seemingly different tasks, that are actually pointing to the same page.
Note that the
wwwdomain label is enough to change the equality of two URLs according to theUrl-package. This is expected behaviour, as it does actually change semantics.For scraping, some links may use the domain label and others may not, e.g. human-written links would probably omit the label while automated links would probably include it for completeness. This could result in a page with two seemingly different tasks, that are actually pointing to the same page.