Skip to content

[Task]: Make streaming downloads more resilient #3221

@ikreymer

Description

@ikreymer

Description

Currently, if the requests.get() from S3 fails or gets interrupted from a multi-file streaming download, the entire operation fails. This can be frustrating when downloading a multi-WACZ with many large files.

Instead, the backend should track how many bytes were downloaded and retry the requests.get() with the specified range, which should make large streaming downloads more resilient to errors or interruptions.

Context

No response

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

Status

Done!

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions