Caching in scyld-modimg

To provide the best performance, the scyld-modimg command keeps a local cache of images within the ~/.scyldcw/workspace directory. This directory contains a manifest.json file that lists the cached images. An image is added to the list whenever a cluster administrator downloads it. When the scyld-modimg command executes, it checks the workspace directory for images that are older than an hour and identical to the same images, matched by UID and name, stored by the head node. Images matching these criteria are evicted from the cache.

When you run the scyld-modimg command, the tool checks the local cache for a copy of the image based on the identifier provided, usually the image name. If a match is found, that image is unpacked for modification (to avoid a fresh download from the head node). You can modify the image in stages without losing changes in between, even if you do not upload the image to the cluster at each step.

This behavior can cause unexpected conflicts in very specific circumstances. If you modify an image and then delete that image from the cluster using the scyld-imgctl or scyld-bootctl commands, the local cache still contains a copy. This means that the next time the scyld-modimg command executes it will see the locally cached image and, regardless of the age, not delete it because an identical image is no longer available from the head node.

If you then create a new image by the same name, perhaps through cloning an existing image using the scyld-imgctl command, and attempt to modify that image using scyld-modimg, you will actually be modifying the local cache due to the name match. When you attempt to upload this modified image, the upload will fail because of a UID mismatch. For example:

ERROR: No image found for ID=d67501a26509486ebaad00827d7fac23

The simplest way to resolve this problem is to delete the locally cached image:

scyld-modimg -i <IMAGENAME> --delete

Then re-run the scyld-modimg command to modify the image. Since the local cache no longer contains an image with a matching name, the tool will download a fresh copy from the head node and record the correct UID in manifest.json. This is almost always the desired solution since you have likely modified the wrong image without realizing it.

If you do want to keep the changes, the simplest approach is to identify the new image UID:

scyld-imgctl --show-uids ls <IMAGENAME>

Then replace the image UID in the ~/.scyldcw/workspace/manifest.json file, rename the image file itself to the new UID, and reattempt the upload:

scyld-modimg -i <IMAGENAME> --upload

Once this command completes, the old image content overwrites the new image content and the changes made previously are preserved.