Skip to content

chore: add support for quantized versions of CV models CLIP, Style Transfer, EfficientNetV2, SSDLite#940

Open
barhanc wants to merge 11 commits intomainfrom
@bh/quantize-cv-models
Open

chore: add support for quantized versions of CV models CLIP, Style Transfer, EfficientNetV2, SSDLite#940
barhanc wants to merge 11 commits intomainfrom
@bh/quantize-cv-models

Conversation

@barhanc
Copy link
Contributor

@barhanc barhanc commented Mar 6, 2026

Description

Adds support for quantized versions of CV models CLIP, Style Transfer, EfficientNetV2, SSDLite and updates paths to non-quantized models exported with ExecuTorch v1.1.0.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

Testing instructions

  1. Run the Computer Vision example app:
    • Object detection with model set to:
      • SSDLITE_320_MOBILENET_V3_LARGE
    • Classification with model set to:
      • EFFICIENTNET_V2_S,
      • EFFICIENTNET_V2_S_QUANTIZED
    • Style transfer with model set to:
      • STYLE_TRANSFER_CANDY,
      • STYLE_TRANSFER_MOSAIC,
      • STYLE_TRANSFER_UDNIE,
      • STYLE_TRANSFER_RAIN_PRINCESS,
      • STYLE_TRANSFER_CANDY_QUANTIZED,
      • STYLE_TRANSFER_MOSAIC_QUANTIZED,
      • STYLE_TRANSFER_UDNIE_QUANTIZED,
      • STYLE_TRANSFER_RAIN_PRINCESS_QUANTIZED,
  2. Run the Text Embeddings example app:
    • CLIP embeddings with image model set to:
      • CLIP_VIT_BASE_PATCH32_IMAGE,
      • CLIP_VIT_BASE_PATCH32_IMAGE_QUANTIZED
  3. Check HF pages for updated models:

Screenshots

Related issues

Closes #719

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

@barhanc barhanc self-assigned this Mar 6, 2026
@barhanc barhanc added chore PRs that are chores model Issues related to exporting, improving, fixing ML models labels Mar 6, 2026
@NorbertKlockiewicz
Copy link
Contributor

I will run the new models later today to see if they work. I think you should also benchmark them and add the results to our docs.

You can ask @IgorSwat for the tips about benchmarking ;D

@IgorSwat
Copy link
Contributor

IgorSwat commented Mar 6, 2026

@barhanc Did you profile the added XNNPACK models following these instructions?

I guess they should be fine considering that those are only a quantized versions of already profiled models, but it's always nice to check if everything is alright with export.

@barhanc
Copy link
Contributor Author

barhanc commented Mar 8, 2026

I've added the profiling results to the corresponding READMEs in the internal exports gitlab. They all look fine to me (>80% delegated ops), but you can also take a look to make sure everything is correct.

Copy link
Contributor

@NorbertKlockiewicz NorbertKlockiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change examples in demo apps to use quantized models by default.

Comment on lines +394 to +396
export const EFFICIENTNET_V2_S_QUANTIZED = {
modelSource: EFFICIENTNET_V2_S_QUANTIZED_MODEL,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
export const EFFICIENTNET_V2_S_QUANTIZED = {
modelSource: EFFICIENTNET_V2_S_QUANTIZED_MODEL,
};
export const EFFICIENTNET_V2_S_QUANTIZED = {
modelName: "efficientnet-v2-s-quantized"
modelSource: EFFICIENTNET_V2_S_QUANTIZED_MODEL,
};

Please add modelName to each new model

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore PRs that are chores model Issues related to exporting, improving, fixing ML models

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Quantize CV models

3 participants