[Change]: Deduped crawl should 'just work' with QA/replay

[ikreymer]

Browsertrix Host

Self-Hosted

What change would you like to see?

Deduped crawls should automatically pull in their dependencies for replay, as we're tracking them within crawls and also within WACZ files.

The replay for deduped crawls should 'just work' as well as QA for deduped crawls, without letting user know that something might be missing.

This involves the following:

• Backend change to resolve all the dependent crawls and include them for replay.
• Replay fixes to load additional WACZ files for revisits as needed.
• Frontend changes: allow QA of deduped crawls

Some other options include:

• Warn users if dependent crawls are missing / been removed, in that case, the replay may be incorrect.
• Allow a 'no dependents' mode for replay (eg current behavior) where data not in this crawl will not be shown - is this useful?

Additional details

Note: Mongo $graphLookup makes looking up all dependencies of a crawl easy