Skip to main content

Directory

Dagger Functions do not have access to the filesystem of the host you invoke the Dagger Function from (i.e. the host you execute a CLI command like dagger from). Instead, host files and directories need to be explicitly passed as command-line arguments to Dagger Functions.

There are two important reasons for this.

  • Reproducibility: By providing a call-time mechanism to define and control the files available to a Dagger Function, Dagger guards against creating hidden dependencies on ambient properties of the host filesystem that could change at any moment.
  • Security: By forcing you to explicitly specify which host files and directories a Dagger Function "sees" on every call, Dagger ensures that you're always 100% in control. This reduces the risk of third-party Dagger Functions gaining access to your data.

The Directory type represents the state of a directory. This could be either a local directory path or a remote Git reference.

Common operations

Some of the common operations available on the Directory type include:

FieldDescription
dockerBuildBuilds a new Docker container from the directory
entriesReturns a list of files and directories in the directory
exportWrites the contents of the directory to a path on the host
fileReturns a file at the given path as a File
withFile / withFilesReturns the directory plus the file(s) copied to the given path

Default paths

It is possible to assign a default path for a Directory or File argument in a Dagger Function. Dagger will automatically use this default path when no value is specified for the argument.

The Directory or File loaded in this manner is not merely a string, but the actual filesystem state of the directory or file.

tip

Default paths are only available for Directory and File arguments. They are commonly used to load constant filesystem locations, such as an application's source code directory. Additionally, when a value is explicitly passed for the argument, it always overrides the default path.

Here's an example:

The default path is set by adding a defaultPath pragma on the corresponding Dagger Function source argument.

package main

import (
"context"
"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) ReadDir(
ctx context.Context,
// +defaultPath="/"
source *dagger.Directory,
) ([]string, error) {
return source.Entries(ctx)
}

When determining how to resolve a default path, Dagger first identifies a "context directory", and then resolves the path starting from the context directory.

  • For Git repositories (defined by the presence of a .git sub-directory), the context directory is the repository root (for absolute paths), or the directory containing a dagger.json file (for relative paths).
  • For all other cases, the context directory is the directory containing a dagger.json file.
tip

For security reasons, it is not possible to retrieve files or directories outside the context directory.

The following sections contain examples of how a Directory argument is resolved for different default path values. The same rules are followed for File arguments.

Git repositories

Default pathContext directoryResolved path
/Repository root (/)/
/srcRepository root (/)/src
.Directory with dagger.json (/my-module)/my-module
..Directory with dagger.json (/my-module)/
  • If the default path is an absolute path / (or /src), the context directory is the repository root (/). The resolved path will then be / (or /src).
  • If the default path is the relative path ., the context directory is the directory containing dagger.json (say, /my-module). The resolved path will then be /my-module.
  • If the default path is the relative path .., the context directory is still the directory containing dagger.json. The resolved path will then be the parent of the context directory (/).

Non-Git repositories

Default pathContext directoryResolved path
/Directory with dagger.json (/my-module)/my-module
/srcDirectory with dagger.json (/my-module)/my-module/src
.Directory with dagger.json (/my-module)/my-module
..Directory with dagger.json (/my-module)Outside context directory; error
  • If the default path is an absolute path / (or /src), the context directory is the directory containing dagger.json (say, /my-module). The resolved path will then be /my-module (or /my-module/src).
  • If the default path is the relative path ., the context directory is still the directory containing dagger.json. The resolved path will then be /my-module.
  • If the default path is the relative path .., the context directory is still the directory containing dagger.json. The resolved path will then be the parent of the context directory. This will trigger an error, since Dagger does not permit access to paths outside the context directory.
tip

It's also possible to provide an ignore parameter to a contextual argument of type Directory to [automatically ignore or include files in the directory].

Directory evaluation

When relying on default paths in Dagger Shell, it's important to know that the source file or directory is re-evaluated on each command execution within the shell session. This differs from passing the source explicitly as an argument, where it's evaluated once and cached. This re-evaluation can lead to unintended behavior where changes to the source directory during the session (such as through exports or logs) invalidate the cache, causing the entire pipeline to re-execute.

For example, given the following Dagger Function:

func (m *MyModule) Test(
ctx context.Context,
// +optional
// +defaultPath="/"
// +ignore=["build"]
source *dagger.Directory,
) (*TestResult, error) {
// ...
}

The command below results in the directory being evaluated multiple times (due to a combination of default path usage and cache invalidation from the export):

dagger shell <<'EOM'
result=$(test)
$result | report | export "build/test-report"
.exit $($result | exit-code)
EOM

The command below evaluates the directory once (and then subsequently uses it from the cache, since the directory is explicitly passed):

dagger shell <<'EOM'
result=$(test --source=".")
$result | report | export "build/test-report"
.exit $($result | exit-code)
EOM

Filters

When you pass a directory to a Dagger Function as argument, Dagger uploads everything in that directory tree to the Dagger Engine. For large monorepos or directories containing large-sized files, this can significantly slow down your Dagger Function while filesystem contents are transferred. To mitigate this problem, Dagger lets you apply filters to control which files and directories are uploaded.

Dagger offers pre- and post-call filtering to mitigate this problem and optimize how your directories are handled.

Filtering improves the performance of your Dagger Functions in three ways:

  • It reduces the size of the files being transferred from the host to the Dagger Engine, allowing the upload step to complete faster.
  • It ensures that minor unrelated changes in the source directory don't invalidate Dagger's build cache.
  • It enables different use-cases, such as setting up component/feature/service-specific workflows for monorepos.

It is worth noting that Dagger already uses caching to optimize file uploads. Subsequent calls to a Dagger Function will only upload files that have changed since the preceding call. Filtering is an additional optimization that you can apply to improve the performance of your Dagger Function.

Pre-call filtering

Pre-call filtering means that a directory is filtered before it's uploaded to the Dagger Engine container. This is useful for:

  • Large monorepos. Typically your Dagger Function only operates on a subset of the monorepo, representing a specific component or feature. Uploading the entire worktree imposes a prohibitive cost.

  • Large files, such as audio/video files and other binary content. These files take time to upload. If they're not directly relevant, you'll usually want your Dagger Function to ignore them.

tip

The .git directory is a good example of both these cases. It contains a lot of data, including large binary objects, and for projects with a long version history, it can sometimes be larger than your actual source code.

  • Dependencies. If you're developing locally, you'll typically have your project dependencies installed locally: node_modules (Node.js), .venv (Python), vendor (PHP) and so on. When you call your Dagger Function locally, Dagger will upload all these installed dependencies as well. This is both bad practice and inefficient. Typically, you'll want your Dagger Function to ignore locally-installed dependencies and only operate on the project source code.
note

Dagger Functions are not aware of the host filesystem, so they cannot automatically read exclusion patterns from existing .dockerignore or .gitignore files. You need to manually implement the same patterns in your Dagger Function. At the time of writing, Dagger does not read exclusion patterns from existing .dockerignore/.gitignore files. If you already use these files, you'll need to manually implement the same patterns in your Dagger Function.

To implement a pre-call filter in your Dagger Function, add an ignore parameter to your Directory argument. The ignore parameter follows the .gitignore syntax. Some important points to keep in mind are:

  • The order of arguments is significant: the pattern "**", "!**" includes everything but "!**", "**" excludes everything.
  • Prefixing a path with ! negates a previous ignore: the pattern "!foo" has no effect, since nothing is previously ignored, while the pattern "**", "!foo" excludes everything except foo.

Here's an example of a Dagger Function that excludes everything in a given directory except Go source code files:

package main

import (
"context"
"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Foo(
ctx context.Context,
// +ignore=["*", "!**/*.go", "!go.mod", "!go.sum"]
source *dagger.Directory,
) (*dagger.Container, error) {
return dag.
Container().
From("alpine:latest").
WithDirectory("/src", source).
Sync(ctx)
}

Here are a few examples of useful patterns:

// exclude Go tests and test data
// +ignore=["**_test.go", "**/testdata/**"]

// exclude binaries
// +ignore=["bin"]

// exclude Python dependencies
// +ignore=["**/.venv", "**/__pycache__"]

// exclude Node.js dependencies
// +ignore=["**/node_modules"]

// exclude Git metadata
// +ignore=[".git", "**/.gitignore"]

You can also split them into multiple lines:

// +ignore=[
// "**_test.go",
// "**/testdata/**"
// ]

Post-call filtering

Post-call filtering means that a directory is filtered after it's uploaded to the Dagger Engine.

This is useful when working with directories that are modified "in place" by a Dagger Function. When building an application, your Dagger Function might modify the source directory during the build by adding new files to it. A post-call filter allows you to use that directory in another operation, only fetching the new files and ignoring the old ones.

A good example of this is a multi-stage build. Imagine a Dagger Function that reads and builds an application from source, placing the compiled binaries in a new sub-directory (stage 1). Instead of then transferring everything to the final container image for distribution (stage 2), you could use a post-call filter to transfer only the compiled files.

To implement a post-call filter in your Dagger Function, use the DirectoryWithDirectoryOpts or ContainerWithDirectoryOpts structs, which support Include and Exclude patterns for Directory objects. Here's an example:

package main

import (
"context"
"dagger/my-module/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) Foo(
ctx context.Context,
source *dagger.Directory,
) *dagger.Container {
builder := dag.
Container().
From("golang:latest").
WithDirectory("/src", source, dagger.ContainerWithDirectoryOpts{Exclude: []string{"*.git", "internal"}}).
WithWorkdir("/src/hello").
WithExec([]string{"go", "build", "-o", "hello.bin", "."})
return dag.
Container().
From("alpine:latest").
WithDirectory("/app", builder.Directory("/src/hello"), dagger.ContainerWithDirectoryOpts{Include: []string{"hello.bin"}}).
WithEntrypoint([]string{"/app/hello.bin"})
}

Here are a few examples of useful patterns:

// exclude all Markdown files
dirOpts := dagger.ContainerWithDirectoryOpts{
Exclude: "*.md*",
}

// include only the build output directory
dirOpts := dagger.ContainerWithDirectoryOpts{
Include: "build",
}

// include only ZIP files
dirOpts := dagger.DirectoryWithDirectoryOpts{
Include: "\*.zip",
}

// exclude Git metadata
dirOpts := dagger.DirectoryWithDirectoryOpts{
Exclude: "\*.git",
}

Mounts

When working with directories and files, you can choose whether to copy or mount them in the containers created by your Dagger Function. The Dagger API provides the following methods:

  • Container.withDirectory() returns a container plus a directory written at the given path
  • Container.withFile() returns a container plus a file written at the given path
  • Container.withMountedDirectory() returns a container plus a directory mounted at the given path
  • Container.withMountedFile() returns a container plus a file mounted at the given path

Mounts only take effect within your workflow invocation; they are not copied to, or included, in the final image. In addition, any changes to mounted files and/or directories will only be reflected in the target directory and not in the mount sources.

tip

Besides helping with the final image size, mounts are more performant and resource-efficient. The rule of thumb should be to always use mounts where possible.

Debugging

Using logs

Both Dagger Cloud and the Dagger TUI provide detailed information on the patterns Dagger uses to filter your directory uploads - look for the upload step in the TUI logs or Trace:

Dagger TUI

Dagger Cloud Trace

Inspecting directory contents

Another way to debug how directories are being filtered is to create a function that receives a Directory as input, and returns the same Directory:

func (m *MyModule) Debug(
ctx context.Context,
// +ignore=["*", "!analytics"]
source *dagger.Directory,
) *dagger.Directory {
return source
}

Calling the function will show you the directory’s digest and top level entries. The digest is content addressed, so it changes if there are changes in the contents of the directory. Looking at the entries field you may be able to spot an interloper: ` You can open the directory in an interactive terminal to inspect the filesystem:

You can export the filtered directory to your host and check it with local tools:

Examples

Clone a remote Git repository into a container

The following Dagger Function accepts a Git repository URL and a Git reference. It copies the repository at the specified reference to the /src path in a container and returns the modified container.

note

For examples of SSH-based cloning, including private or public Git repositories with an SSH reference format, select the SSH tabs below. This approach requires explicitly forwarding the host SSH_AUTH_SOCK to the Dagger Function. Learn more about this in Dagger's security model.

package main

import (
"context"
"dagger/my-module/internal/dagger"
)

// Demonstrates cloning a Git repository over HTTP(S).
//
// For SSH usage, see the SSH snippet (CloneWithSsh).
type MyModule struct{}

func (m *MyModule) Clone(ctx context.Context, repository string, ref string) *dagger.Container {
d := dag.Git(repository).Ref(ref).Tree()

return dag.Container().
From("alpine:latest").
WithDirectory("/src", d).
WithWorkdir("/src")
}

Examples

Clone the public dagger/dagger GitHub repository to /src in the container:

dagger -c 'clone https://github.com/dagger/dagger 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5'

Clone the public dagger/dagger GitHub repository at reference 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 to /src in the container and open an interactive terminal to inspect the container filesystem:

dagger <<EOF
clone https://github.com/dagger/dagger 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 |
terminal
EOF

Clone over SSH with socket forwarding:

dagger <<EOF
clone-with-ssh git@github.com:dagger/dagger.git 196f232a4d6b2d1d3db5f5e040cf20b6a76a76c5 $SSH_AUTH_SOCK |
terminal
EOF

Mount or copy a directory or remote repository to a container

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or from a remote Git repository. It mounts the specified directory to the /src path in a container and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

package main

import (
"context"

"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a mounted directory
func (m *MyModule) MountDirectory(
ctx context.Context,
// Source directory
source *dagger.Directory,
) *dagger.Container {
return dag.Container().
From("alpine:latest").
WithMountedDirectory("/src", source)
}

An alternative option is to copy the target directory in the container. The difference between these two approaches is that mounts only take effect within your workflow invocation; they are not copied to, or included, in the final image. In addition, any changes to mounted files and/or directories will only be reflected in the target directory and not in the mount sources.

tip

Besides helping with the final image size, mounts are more performant and resource-efficient. The rule of thumb should be to always use mounts where possible.

package main

import (
"context"

"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a specified directory
func (m *MyModule) CopyDirectory(
ctx context.Context,
// Source directory
source *dagger.Directory,
) *dagger.Container {
return dag.Container().
From("alpine:latest").
WithDirectory("/src", source)
}

Examples

  • Mount the /myapp host directory to /src in the container and return the modified container:

    dagger -c 'mount-directory ./myapp/'
  • Mount the public dagger/dagger GitHub repository to /src in the container and return the modified container:

    dagger -c 'mount-directory https://github.com/dagger/dagger#main'
  • Mount the private user/foo GitHub repository to /src in the container and return the modified container:

    dagger -c 'mount-directory ssh://git@github.com/user/foo#main'
  • Mount the public dagger/dagger GitHub repository to /src in the container and list the contents of the directory:

    dagger -c 'mount-directory https://github.com/dagger/dagger#main | directory /src | entries'
  • Copy the /myapp host directory to /src in the container and return the modified container:

    dagger -c 'copy-directory ./myapp/'
  • Copy the public dagger/dagger GitHub repository to /src in the container and return the modified container:

    dagger -c 'copy-directory https://github.com/dagger/dagger#main'
  • Copy the private user/foo GitHub repository to /src in the container and return the modified container:

    dagger -c 'copy-directory ssh://git@github.com/user/foo#main'
  • Copy the public dagger/dagger GitHub repository to /src in the container and list the contents of the directory:

    dagger -c 'copy-directory https://github.com/dagger/dagger#main | directory /src | entries'

Modify a copied directory or remote repository in a container

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or from a remote Git repository. It copies the specified directory to the /src path in a container, adds a file to it, and returns the modified container.

warning

When a host directory or file is copied or mounted to a container's filesystem, modifications made to it in the container do not automatically transfer back to the host.

Data flows only one way between Dagger operations, because they are connected in a DAG. To transfer modifications back to the local host, you must explicitly export the directory or file back to the host filesystem.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host. :::

package main

import (
"context"

"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a specified directory and an additional file
func (m *MyModule) CopyAndModifyDirectory(
ctx context.Context,
// Source directory
source *dagger.Directory,
) *dagger.Container {
return dag.Container().
From("alpine:latest").
WithDirectory("/src", source).
WithExec([]string{"/bin/sh", "-c", `echo foo > /src/foo`})
}

Examples

Copy the /myapp host directory to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory ./myapp/'

Copy the public dagger/dagger GitHub repository to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory github.com/dagger/dagger#main'

Copy the private user/foo GitHub repository to /src in the container, add a file to it, and return the modified container:

dagger -c 'copy-and-modify-directory ssh://git@github.com/user/foo#main'

Copy the public dagger/dagger GitHub repository to /src in the container, add a file to it, and list the contents of the directory:

dagger -c 'copy-and-modify-directory https://github.com/dagger/dagger#main | directory /src | entries'

Copy a subset of a directory or remote repository to a container using filters specified at run-time

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or a remote Git repository. It copies the specified directory to the /src path in a container, using a filter pattern specified at call-time to exclude specified sub-directories and files, and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

note

This is an example of post-call filtering with directory filters.

package main

import (
"context"

"dagger/my-module/internal/dagger"
)

type MyModule struct{}

// Return a container with a filtered directory
func (m *MyModule) CopyDirectoryWithExclusions(
ctx context.Context,
// Source directory
source *dagger.Directory,
// Exclusion pattern
// +optional
exclude []string,
) *dagger.Container {
return dag.Container().
From("alpine:latest").
WithDirectory("/src", source, dagger.ContainerWithDirectoryOpts{Exclude: exclude})
}

Examples

Copy the current host directory to /src in the container, excluding all sub-directories and files starting with dagger, and return the modified container:

dagger -c 'copy-directory-with-exclusions . --exclude=dagger*'

Copy the public dagger/dagger GitHub repository to /src in the container, excluding all Markdown files, and list the contents of the directory:

dagger <<EOF
copy-directory-with-exclusions https://github.com/dagger/dagger#main --exclude=*.md |
directory /src |
entries
EOF

Copy the private user/foo GitHub repository to /src in the container, excluding all Markdown files, and list the contents of the directory:

dagger <<EOF
copy-directory-with-exclusions ssh://git@github.com/user/foo#main --exclude=*.md |
directory /src |
entries
EOF

Copy a subset of a directory or remote repository to a container using pre-defined filters

The following Dagger Function accepts a Directory argument, which could reference either a directory from the local filesystem or a remote Git repository. It copies the specified directory to the /src path in a container, using pre-defined filter patterns to exclude specified sub-directories and files, and returns the modified container.

note

When working with private Git repositories, ensure that SSH authentication is properly configured on your Dagger host.

note

This is an example of pre-call filtering with directory filters.

package main

import (
"context"
"main/internal/dagger"
)

type MyModule struct{}

func (m *MyModule) CopyDirectoryWithExclusions(
ctx context.Context,
// +ignore=["*", "!**/*.md"]
source *dagger.Directory,
) (*dagger.Container, error) {
return dag.
Container().
From("alpine:latest").
WithDirectory("/src", source).
Sync(ctx)
}

Examples

Copy the specified host directory to /src in the container, excluding everything except Markdown files, and return the modified container:

dagger -c 'copy-directory-with-exclusions ../docs'

Copy the public dagger/dagger GitHub repository to /src in the container, excluding everything except Markdown files, and list the contents of the /src directory in the container:

dagger -c 'copy-directory-with-exclusions https://github.com/dagger/dagger#main | directory /src | entries'

Set a module-wide default path

The following Dagger module uses a constructor to set a module-wide Directory argument and point it to a default path on the host. This eliminates the need to pass a common Directory argument individually to each Dagger Function in the module. If required, the default path can be overridden by specifying a different Directory value at call time.

package main

import (
"context"
"dagger/my-module/internal/dagger"
)

type MyModule struct {
Source *dagger.Directory
}

func New(
// +defaultPath="."
source *dagger.Directory,
) *MyModule {
return &MyModule{
Source: source,
}
}

func (m *MyModule) Foo(ctx context.Context) ([]string, error) {
return dag.Container().
From("alpine:latest").
WithMountedDirectory("/app", m.Source).
Directory("/app").
Entries(ctx)
}

Examples

Call the Dagger Function without arguments. The default path (the current directory .) is used:

dagger -c foo

Call the Dagger Function with a constructor argument. The specified directory (/src/myapp) is used:

dagger -c 'my-module --source $(host | directory /src/myapp) | foo'

Export a directory or file to the host

The following Dagger Functions return a just-in-time directory and file. These outputs can be exported to the host with dagger call ... export ....

note

When a host directory or file is copied or mounted to a container's filesystem, modifications made to it in the container do not automatically transfer back to the host. Data flows only one way between Dagger operations, because they are connected in a DAG. To transfer modifications back to the local host, you must explicitly export the directory or file back to the host filesystem.

package main

import "dagger/my-module/internal/dagger"

type MyModule struct{}

// Return a directory
func (m *MyModule) GetDir() *dagger.Directory {
return m.Base().
Directory("/src")
}

// Return a file
func (m *MyModule) GetFile() *dagger.File {
return m.Base().
File("/src/foo")
}

// Return a base container
func (m *MyModule) Base() *dagger.Container {
return dag.Container().
From("alpine:latest").
WithExec([]string{"mkdir", "/src"}).
WithExec([]string{"touch", "/src/foo", "/src/bar"})
}

Examples

  • Export the directory returned by the Dagger Function to the /home/admin/export path on the host:

    dagger -c 'get-dir | export /home/admin/export'
  • Export the file returned by the Dagger Function to the /home/admin/myfile path on the host:

    dagger -c 'get-file | export /home/admin/myfile'