Baking ML model weights into Docker images can be silently undone by one env variable
Developers building serverless GPU applications often embed model weights directly into container images at build time to avoid slow cold starts caused by network downloads. This is done by setting the HF_HOME environment variable to a fixed path inside the image during the Docker build process. However, if any runtime configuration — such as a mounted volume or a template variable — overrides HF_HOME to a different path, the model loader silently falls back to downloading weights from the network, negating the optimization entirely. The failure produces no errors, making it easy to mistakenly blame the hosting platform for slow cold starts. To prevent this, developers are advised to avoid attaching volumes, verify boot logs for unexpected download activity, and explicitly document the cache path as a shared contract between the Dockerfile and the runtime environment.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in