-- Leo's gemini proxy
-- Connecting to nox.im:1965...
-- Connected
-- Sending request
-- Meta line: 20 text/gemini; charset=utf-8
With Go 1.16 we can natively embed static files into binaries. The common alternative to this day was using community alternatives, of which one of the most commonly used ones is gobuffalo/packr[1]. There are issues with the development experience. Packages such as packr require us to add/update assets every time they change and the tool to package files up to be available on every machine.
We usually deploy Go apps with ease as we only need to deliver a single binary. There could however be cases for run time files and assets. Instead of shipping such files through build pipelines and into containers, native file embedding support allows for a more idiomatic way to access such dependencies and makes it easier to deploy our applications. Some examples I toyed with when switching from packr2.
Consider the following directory structure:
datadir
├── file1.txt
└── subdir
└── file2.txt
We can embed a read only collection of these files and walk the directory tree as follows:
//go:embed datadir
var data embed.FS
func main() {
err := fs.WalkDir(data, ".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
fmt.Printf("path=%q, isDir=%v\n", path, d.IsDir())
return nil
})
if err != nil {
log.Fatal(err)
}
}
`embed.FS` implements `fs.FS`, which allows usage with any package that understands file system interfaces. Running this with `go run *.go` returns
path=".", isDir=true path="datadir", isDir=true path="datadir/file1.txt", isDir=false path="datadir/file2.txt", isDir=false
We already have access to the file name:
fmt.Printf("path=%q, name=%s, isDir=%v\n", path, d.Name(), d.IsDir())
So a list function to return a list of all files could look like this:
func list(dir embed.FS) ([]string, error) {
var out []string
err := fs.WalkDir(dir, ".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
out = append(out, d.Name())
return nil
})
return out, err
}
it returns
list=["file1.txt" "file2.txt"]
A find function to return the byte contents of an embedded file if found we can construct analogously:
func find(dir embed.FS, filename string) ([]byte, error) {
var out []byte
err := fs.WalkDir(dir, ".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if d.Name() == filename {
out, err = fs.ReadFile(dir, path)
if err != nil {
return err
}
}
return nil
})
return out, err
}
Since we traverse the entire tree, we will find a file here recursively, from sub directories too.
Where this falls short are symlinks
datadir
├── file1.txt
├── file3.txt -> subdir/file2.txt
└── subdir
└── file2.txt
1 directory, 3 files
The `find()` function wouldn't return any results. Looking for `file3.txt` here would result in the file not found error which we should add to the above example as an exercise for the reader.
We can fix this by resolving the symlink however in a build directory and embed that from the sources.
cp -rL source build/destination # cp -RL source build/destination # on MacOS
If we're dealing with symlinks it would be nice if we can add them with our native tooling in a build step. There is the `go:generate` directive that can assist us and executes these commands with the `go generate` command:
//go:generate mkdir -p build //go:generate cp -RL datadir ./build/datadir //go:embed build/datadir var data embed.FS
Go packages can embed files which will be vendored with `go mod` and not purged because mod is sensitive to the embed statements. This works out of the box.
Armed with all this, we hopefully now have even better tooling at our disposal.
-- Response ended
-- Page fetched on Sat Apr 27 20:14:36 2024