--- title: "Install images by registry digest" description: "Pin Calico Open Source operator deployments to immutable image digests with an ImageSet resource so security teams can review and verify each image." product: "Calico Open Source" version: "3.32 (latest)" section: "Operations" canonical_url: "https://docs.tigera.io/calico/latest/operations/image-options/imageset" --- # Install images by registry digest ## Big picture Deploy images by container registry digest for operator installations. ## Value Some deployments have strict security requirements that require deploying images by immutable digest instead of tags. Once released, official Calico images and tags will not be modified. However using an immutable digest allows specific images to be reviewed and verified by security teams. ## Concepts ### Container registry A container registry provides access to container images referenced by tags or digest. ### Image tag Versioned container images are typically referenced by a tag which is appended to an image reference. Example: `/:`. Container image tags are typically not expected be changed or updated, but this is not required or enforced by most image registries, meaning it is possible to push new code to the same image tag. ### Image digest Container images, when added to a container registry, have a unique hash created that can be used to pull a specific version of an image that cannot be changed or updated. ## Before you begin **Required** - Calico managed by the operator - Docker client is configured to pull images from the container registries where images are stored - Kubernetes permissions to apply an ImageSet manifest to your cluster ## How to 1. [Update the operator deployment with a digest](#update-the-operator-deployment-with-a-digest) 2. [Create an ImageSet](#create-an-imageset) 3. [Verify the correct ImageSet is being used](#verify-the-correct-imageset-is-being-used) **Other tasks** - [Create new ImageSet when upgrading or downgrading](#create-new-imageset-when-upgrading-or-downgrading) **Troubleshooting** - [Why does the Installation resource status not include my ImageSet?](#why-does-the-installation-resource-status-not-include-my-imageset) - [How can I tell if there is a problem with my ImageSet?](#how-can-i-tell-if-there-is-a-problem-with-my-imageset) ### Update the operator deployment with a digest Before applying `tigera-operator.yaml`, modify the operator deployment to use the operator image digest. Use commands like the following to get the image digest (adjust the image in the commands if you are using a different operator image): ```bash docker pull quay.io/tigera/operator:v1.42.3 docker inspect quay.io/tigera/operator:v1.42.3 -f '{{range .RepoDigests}}{{printf "%s\n" .}}{{end}}' ``` If multiple digests are returned, select the one matching the registry you are using. Update the tigera-operator deployment: ```bash sed -ie "s|\(image: .*/operator\):.*|?\1@|" tigera-operator.yaml ``` ### Create an ImageSet Create an [ImageSet](https://docs.tigera.io/calico/latest/reference/installation/api.md#imageset) manifest file named `imageset.yaml` like the following: ```yaml apiVersion: operator.tigera.io/v1 kind: ImageSet metadata: name: calico-v3.32.1 spec: images: - image: 'calico/apiserver' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/cni' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/kube-controllers' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/node' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/typha' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/pod2daemon-flexvol' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'calico/node-windows' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'tigera/operator' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' - image: 'tigera/key-cert-provisioner' digest: 'sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' ``` You can create an ImageSet manifest manually or by script. **Tab: Manual** 1. Copy the above example into a file called `imageset.yaml` and edit that file in the steps below. 2. Set the name for your ImageSet to `calico-` (Example: `calico-v3.32.1`). The version can be obtained by running: ```text docker run quay.io/tigera/operator:v1.42.3 --version ``` 3. Add the correct digest for each image. If you are using a private registry, ensure you pull the image from the private registry and use the digest associated with the private registry. 1. If using the default images, get a list of them by running: ```text docker run quay.io/tigera/operator:v1.42.3 --print-images=list ``` > **SECONDARY:** If you are not using the default image registries or paths, you must create your own list of images (and the above command will not apply). > **SECONDARY:** The list will contain images for an Enterprise deployment but they do not need to be added to the ImageSet. 2. Get the needed digests by using the images returned from the above step in the following command: ```text docker pull && docker inspect -f '{{range .RepoDigests}}{{printf "%s\n" .}}{{end}}' ``` 3. Use the digest from the image that matches the repo/image you will use. If you are using a private registry or have specified an [imagePath](https://docs.tigera.io/calico/latest/reference/installation/api.md#installation) you will still use the "default" `/` in the `image` field, for example if you your node image is coming from `example.com/registry/imagepath/node` you will still use `calico/node` in the image field of the ImageSet. > **SECONDARY:** For image `quay.io/tigera/operator@sha256:d111db2f94546415a30eff868cb946d47e183faa804bd2e9a758fd9a8a4eaff1` copy everything after `@` and add it as the digest for the `tigera/operator` image. **Tab: Script** Copy the following script into a file, make it executable, and run the script. The script creates an `imageset.yaml` file in the directory it was run. > **SECONDARY:** This script will only work if using the default registries and image paths. ```text #!/bin/bash -e images=(calico/apiserver calico/cni calico/kube-controllers calico/node calico/typha calico/pod2daemon-flexvol calico/node-windows tigera/key-cert-provisioner tigera/operator) OPERATOR_IMAGE=quay.io/tigera/operator:v1.42.3 echo "Pulling $OPERATOR_IMAGE" echo docker pull $OPERATOR_IMAGE -q >/dev/null versions=$(docker run $OPERATOR_IMAGE --version) ver=$(echo -e "$versions" | grep 'Calico:') imagelist=($(docker run $OPERATOR_IMAGE --print-images=list)) cat > ./imageset.yaml <> ./imageset.yaml echo " digest: \"$digest\"" >> ./imageset.yaml fi done done ``` Apply the created `imageset.yaml` to your cluster. ### Verify the correct ImageSet is being used 1. Check tigerastatus for components that are Degraded with `kubectl get tigerastatus`. - If any components show Degraded, [investigate further](#how-can-i-tell-if-there-is-a-problem-with-my-imageset). 2. When tigerastatus for all components show Available True, the ImageSet has been applied. ```text NAME AVAILABLE PROGRESSING DEGRADED SINCE calico True False False 54s ``` 3. Verify that the correct ImageSet is being used. In Installation status, check that the `imageset` field is set to the ImageSet you created. Check the field by running the following command: ```bash kubectl get installation default -o yaml | grep imageSet ``` You should see output similar to: ```yaml imageSet: calico-v3.32.1 ``` ## Other tasks ### Create new ImageSet when upgrading or downgrading Before upgrading to a new release or downgrading, you must create a new [ImageSet](https://docs.tigera.io/calico/latest/reference/installation/api.md#imageset) with updated image references and names for the new release. This must be done prior to upgrading the cluster so when the new manifests are applied, the appropriate ImageSet is available. ## Troubleshooting ### Why does the Installation Resource status not include my ImageSet? The [status.imageset](https://docs.tigera.io/calico/latest/reference/installation/api.md#installationstatus) field of the Installation Resource will not be updated until the `calico` component has fully been deployed. `calico` is fully deployed when `kubectl get tigerastatus calico` reports Available True with Progressing and Degraded as False. ### How can I tell if there is a problem with my ImageSet? If you suspect an issue with your ImageSet, check tigerastatus with `kubectl get tigerastatus`. If any components are degraded, you can get additional information with `kubectl get tigerastatus -o yaml`. If the digest provided for an image is incorrect or unable to be pulled, the tigerastatus will not directly report that information, but you should see information that there is an issue rolling out a Deployment, Daemonset, or Job. If you suspect an issue with a resource rollout due to an issue with an image, you will need to `get` or `describe` a specific pod to see details about the problem.