Amazon S3
Overview
Section titled “Overview”An S3 source is identified by a bucket, an object key, and an optional version. Terragrunt reads the object’s metadata to derive a cache key, preferring a real content checksum over the weaker ETag.
The cheap probe
Section titled “The cheap probe”The probe Terragrunt uses to attempt to avoid fetching S3 sources is a HeadObject call with checksum mode enabled. It reads the object’s stored checksum, preferring the strongest available in this order:
- SHA-256
- CRC64NVME
- SHA-1
- CRC32C
- CRC32
ETag(last resort)
Cache key & deduplication
Section titled “Cache key & deduplication”When any checksum is available, the key is content-addressed, so the same object bytes reached through different URLs or buckets deduplicate to one stored tree. When only an ETag is available, the key is scoped to the URL, because a multipart ETag (<md5>-<n>) is not a content hash. File blobs are content-addressed in every case.
Cache miss
Section titled “Cache miss”If the object has neither a checksum nor an ETag, or the probe errors, Terragrunt downloads the object and keys the tree by the content hash of what it fetched. A fresh probe key that is not yet stored also downloads, then stores the tree under that key so the next run hits.
Cache hit
Section titled “Cache hit”When the probe returns a key already present in the store, Terragrunt links the cached tree with no download.
Fallback behavior
Section titled “Fallback behavior”Any AWS or network error makes the probe yield no key, and Terragrunt falls back to downloading and content-hashing the object.