BiospexImageFetcher

Purpose

A "universal downloader" that pulls images from external sources into the project's S3 buckets. It ensures images are valid and resized before storage.

Workflow

1. Trigger: Receives messages from SQS.
2. Download: Pulls the image from the provided accessURI using axios.
3. Validation: Checks the image magic bytes to ensure it's a supported type (JPG, PNG).
4. Processing: Uses the sharp library to resize the image (default max 1500x1500px) and converts it to high-quality JPEG.
5. Upload: Saves the processed image to S3.
   • For ocr tasks: Includes metadata (queue-id, file-id, etc.) in the S3 object which triggers the next step in the pipeline.
6. Callback:
   • For export tasks: Sends a success status to the Laravel app via SQS.
   • For ocr tasks: Remains silent if successful (S3 event trigger handles the OCR start), but reports failures.

Inputs/Outputs

• Inputs (JSON):
  • taskType: Either export or ocr.
  • accessURI: Source URL of the image.
  • s3Bucket/s3Path: Destination in S3.
  • updatesQueueUrl: SQS URL for status reporting.
  • maxWidth/maxHeight: (Optional) Custom resizing constraints.
• Outputs:
  • S3: Processed JPEG image at s3Path.
  • SQS: Success/Failure notification to updatesQueueUrl.

Configuration (Environment Variables)

• JPEG_QUALITY: The quality level for the output JPEG (Default: 80).

Related Components

• Laravel Job: App\Jobs\ZooniverseExportImageUpdateJob (Processes fetcher updates for exports).