Switch wget for wget-at for better WARC compliance

[pirate]

https://github.com/ArchiveTeam/wget-lua

It's the version that archive team warrior runs and is very well tested + ready to feed directly into archive.org.

[pirate]

TODO: implement this over on the new plugins repo http://github.com/archiveBox/abx-plugins

[devicefricker]

I don't really want to nag you (in another issue, you said you could only work on this on weekends), but can we transfer this to abx-plugins?

[pirate]

@claude can you take a crack at adding a new plugin wgetlua thats similar to wget, but it should install wget-at using binprovider overrides defined in config.json.

Install must succeed locally / in the test environment when running abx-dl install wgetlua (uv pip install -e ../abx-dl. auto-install in macos/linux at runtime is a core part of the user-facing surface area and must succeed in all environments including CI and sandboxes with no mocking or "faked" code paths in tests.

Make sure the plugin includes a live integration test that hits a real https://example.com url and confirms the warc and html saved actually contain the correct contents. Do not mock the url with a test server and do not mock the wget-at/wgetlua binary, it must succeed an install the real binary and hit a real url and produce real content in the output. Do not simply check that it runs without error, your tests must check that the content saved is actually correct.

[pirate]

Unlike all our other plugin binary dependencies, it seems wget-at offers no package distribution, prebuilt binaries, or install script. This makes it significantly harder to offer as a plugin, as users would have to install an entire C/C++ build system toolchain.

If that changes and they release a package manager distribution then I'm more likely to add it, otherwise this is very low on my list. I'm more likely to invest in adding a browsertrix/archivweb.page plugin which is great at producing WARC/WACZs.

[pirate]

Vibecoded approach but untested / not going to enable by default with this gnarly of a build step: #19

[devicefricker]

Eh, it's only a plugin, nobody will really care if it's vibecoded or not (as long as it doesn't delete the user's D drive!)