Question about image repository content summary

[grzleadams]

When I look at the content summary for the latest version of one of my push repositories, container.manifest shows a large number of manifests for a relatively small number of tags (as shown in container.tag).

$ pulp show --href $(pulp container repository -t push show --repository myimage/external-dev | jq -r '.latest_version_href') | jq -r '.content_summary.present'
{
  "container.blob": {
    "count": 8836,
    "href": "/pulp/api/v3/content/container/blobs/?repository_version=/pulp/api/v3/repositories/container/container-push/01901276-89b9-74c8-b90b-9fd7c8ea5b8a/versions/6515/"
  },
  "container.manifest": {
    "count": 4175,
    "href": "/pulp/api/v3/content/container/manifests/?repository_version=/pulp/api/v3/repositories/container/container-push/01901276-89b9-74c8-b90b-9fd7c8ea5b8a/versions/6515/"
  },
  "container.tag": {
    "count": 39,
    "href": "/pulp/api/v3/content/container/tags/?repository_version=/pulp/api/v3/repositories/container/container-push/01901276-89b9-74c8-b90b-9fd7c8ea5b8a/versions/6515/"
  }
}

Furthermore, if I recursively go through each tag in container.tag and gather tagged_manifest and listed_manifests for that manifest and all related manifests, that list is completely different from the list in container.manifest. So what exactly does container.manifest show if not manifests that are associated with tags associated with the repository?

[grzleadams]

For reference, this is related to the issue described in #1749. I believe I have a large number of manifests (specifically, cache manifests created by buildx) that are not related to any tag in the repository and can't be purged easily, but are related to a repository and so aren't cleaned by orphan cleanup.

[gerrod3]

container.manifest is counting all the manifest objects that have been pushed to that repository. For example you have image foo and you do a push podman push pulp.io/foo:latest. You should see one manifest added to the repository. If you rebuild foo with a new latest tag and redo the push you will now see two manifests, but still only one tag since the new latest tag collides on a repository uniqueness level with the old one and thus removes the old one.

Maybe we should change the behavior for push-repositories to remove the newly untagged manifests from the new repo-version on push. Note a similar behavior change would be needed for Blobs as well.

[grzleadams]

Maybe we should change the behavior for push-repositories to remove the newly untagged manifests from the new repo-version on push.

This is what I would advocate for; I don't see a downside to removing them, but a huge downside (and no real upside) to keeping them since they're effectively orphans but don't get cleaned as such. I ended up writing a script to recursively go through the tags and tagged manifests to see what manifests are actually used, and delete ones that aren't.

Note a similar behavior change would be needed for Blobs as well.

I'm not sure this is actually the case. Once the manifests are gone, the blobs are orphaned and it seems (I could be wrong) that cleanup handles them appropriately.

[mdellweg]

Maybe we should change the behavior for push-repositories to remove the newly untagged manifests from the new repo-version on push.

Maybe even turn them back into the pending state, as they might just be part of a pending push operation at that point in time. (Being a pending Manifest/Blob, they are direct subject to orphan cleanup. So nothing is lost on that part.)