Built-In Caching
One of Dagger's most powerful features is its speed, by virtue of its ability to cache data across workflow runs.
Dagger caches two types of data:
- Layers: This refers to build instructions and the results of some API calls. This cache is implemented by Buildkit
- Volumes: This refers to the contents of a Dagger filesystem volume and is persisted across Dagger Engine sessions It is implemented by Dagger (distinct from Buildkit)
Layer caching
"Layers" are the step-wise instructions and arguments that go into building a container image, including the result of each step. As container images are built by Dagger, Dagger automatically caches the layer involved for future use.
When Dagger executes a function, it first checks if it already has the layers required by that function. If it does, these layers are automatically reused by Dagger if their inputs remain unchanged.
Volume caching
Volume caching involves caching specific parts of the filesystem and reusing them on subsequent function calls if they are unchanged. This is especially useful when dealing with package managers such as npm, maven, pip and similar. Since these dependencies are usually locked to specific versions in the application's manifest, re-downloading them on every session is inefficient and time-consuming.
For these tools to cache properly, they need their own cache data (usually a directory) to be persisted between sessions. By using a cache volume for this data, Dagger can reuse the cached contents across workflow runs and reduce execution time.