-- Leo's gemini proxy

-- Connecting to qwertqwefsday.eu:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini;lang=en

Agate


Simple Gemini server for static files


Agate is a server for the Gemini network protocol, built with the Rust programming language. Agate has very few features, and can only serve static files. It uses async I/O, and should be quite efficient even when running on low-end hardware and serving many concurrent requests.


Since Agate by default uses port 1965, you should be able to run other servers (like e.g. Apache or nginx) on the same device.


Learn more


Home page

Agate on crates.io

Source code


Installation and Setup


1. Get a binary for agate. You can use any of the below ways:


Pre-compiled


Download and unpack the [pre-compiled binary](https://github.com/mbrubeck/agate/releases).


NixOS/Nix


Using the nix package manager run `nix-env -i agate`


_Note:_ agate is currently only in the unstable channel and will reach a release channel once the next release is tagged


Arch Linux


Install the package "agate-bin" from AUR for pre-compiled binary. Otherwise install the "agate" package from AUR to get agate compiled from source.


Cargo


If you have the Rust toolchain installed, run `cargo install agate` to install agate from crates.io.


Source


Download the source code and run `cargo build --release` inside the source repository, then find the binary at `target/release/agate`.


***

You can use the install script in the `tools` directory for the remaining steps if there is one for your system. If there is none, please consider contributing one to make it easier for less tech-savvy users!

***


2. Run the server. You can use the following arguments to specify the locations of the content directory, IP address and port to listen on, host name to expect in request URLs, and default language code to include in the MIME type for for text/gemini files: (Replace the hostname `example.com` with the address of your Gemini server.)

If you have not done it yourself, Agate will generate a private key and certificate for you on the first run, using the specified hostname(s). See the section Certificates below for more.


agate --content path/to/content/ \
      --addr [::]:1965 \
      --addr 0.0.0.0:1965 \
      --hostname example.com \
      --lang en-US

All of the command-line arguments are optional. Run `agate --help` to see the default values used when arguments are omitted.


When a client requests the URL `gemini://example.com/foo/bar`, Agate will respond with the file at `path/to/content/foo/bar`. If any segment of the requested path starts with a dot, agate will respond with a status code 52, whether the file exists or not. This behaviour can be disabled with `--serve-secret` or by an entry for the specific file in the `.meta` configuration file (see Meta-Presets). If there is a directory at that path, Agate will look for a file named `index.gmi` inside that directory.


Configuration


TLS versions


Agate by default supports TLSv1.2 and TLSv1.3. You can disable support for TLSv1.2 by using the flag `--only-tls13` (or its short version `-3`). This is *NOT RECOMMENDED* as it may break compatibility with some clients. The Gemini specification requires compatibility with TLSv1.2 "for now" because not all platforms have good support for TLSv1.3 (cf. ยง4.1 of the specification).


Directory listing


You can enable a basic directory listing for a directory by putting a file called `.directory-listing-ok` in that directory. This does not have an effect on sub-directories.

The directory listing will hide files and directories whose name starts with a dot (e.g. the `.directory-listing-ok` file itself or also the `.meta` configuration file).


A file called `index.gmi` will always take precedence over a directory listing.


Meta-Presets


You can put a file called `.meta` in any content directory. This file stores some metadata about the adjacent files which Agate will use when serving these files. The `.meta` file must be UTF-8 encoded.

You can also enable a central configuration file with the `-C` flag (or the long version `--central-conf`). In this case Agate will always look for the `.meta` configuration file in the content root directory and will ignore `.meta` files in other directories.


The `.meta` file has the following format [1]:

Empty lines are ignored.

Everything behind a `#` on the same line is a comment and will be ignored.

All other lines must have the form `<path>:<metadata>`, i.e. start with a file path, followed by a colon and then the metadata.


`<path>` is a case sensitive file path, which may or may not exist on disk. If <path> leads to a directory, it is ignored.

If central configuration file mode is not used, using a path that is not a file in the current directory is undefined behaviour (for example `../index.gmi` would be undefined behaviour).

You can use Unix style patterns in existing paths. For example `content/*` will match any file within `content`, and `content/**` will additionally match any files in subdirectories of `content`.

However, the `*` and `**` globs on their own will by default not match files or directories that start with a dot because of their special meaning.

This behaviour can be disabled with `--serve-secret` or by explicitly matching files starting with a dot with e.g. `content/.*` or `content/**/.*` respectively.

For more information on the patterns you can use, please see the documentation of `glob::Pattern`.

Rules can overwrite other rules, so if a file is matched by multiple rules, the last one applies.


Documentation of `glob::Pattern`.


`<metadata>` can take one of four possible forms:

1. empty:

Agate will not send a default language parameter, even if it was specified on the command line.

2. starting with a semicolon followed by MIME parameters:

Agate will append the specified string onto the MIME type, if the file is found.

3. starting with a gemini status code (i.e. a digit 1-6 inclusive followed by another digit) and a space:

Agate will send the metadata whether the file exists or not. The file will not be sent or accessed.

4. a MIME type, may include parameters:

Agate will use this MIME type instead of what it would guess, if the file is found. The default language parameter will not be used, even if it was specified on the command line.


If a line violates the format or looks like case 3, but is incorrect, it might be ignored. You should check your logs. Please know that this configuration file is first read when a file from the respective directory is accessed. So no log messages after startup does not mean the `.meta` file is okay.


Such a configuration file might look like this:

# This line will be ignored.
**/*.de.gmi: ;lang=de
nl/**/*.gmi: ;lang=nl
index.gmi: ;lang=en-UK
LICENSE: text/plain;charset=UTF-8
gone.gmi: 52 This file is no longer here, sorry.

If this is the `.meta` file in the content root directory and the `-C` flag is used, this will result in the following response headers:

`/` or `/index.gmi`
    -> `20 text/gemini;lang=en-UK`
`/LICENSE`
    -> `20 text/plain;charset=UTF-8`
`/gone.gmi`
    -> `52 This file is no longer here, sorry.`
any non-hidden file ending in `.de.gmi` (including in non-hidden subdirectories)
    -> `20 text/gemini;lang=de`
any non-hidden file in the `nl` directory ending in `.gmi` (including in non-hidden subdirectories)
    -> `20 text/gemini;lang=nl`

[1] In theory the syntax is that of a typical INI-like file and also allows for sections with `[section]` (the default section is set to `mime` in the parser), since all other sections are disregarded, this does not make a difference. This also means that you can in theory also use `=` instead of `:`. For even more information, you can visit the documentation of `configparser`.

documentation of `configparser`


Logging Verbosity


Agate uses the `env_logger` crate and allows you to set the logging verbosity by setting the default `RUST_LOG` environment variable. To turn off all logging use `RUST_LOG=off`. For more information, please see the documentation of `env_logger`.

documentation of `env_logger` crate


Virtual Hosts


Agate has basic support for virtual hosts. If you specify multiple `--hostname`s, Agate will look in a directory with the respective hostname within the content root directory.

For example if one of the hostnames is `example.com`, and the content root directory is set to the default `./content`, and `gemini://example.com/file.gmi` is requested, then Agate will look for `./content/example.com/file.gmi`. This behaviour is only enabled if multiple `--hostname`s are specified.

Agate also supports different certificates for different hostnames, see the section on certificates below.


If you want to serve the same content for multiple domains, you can instead disable the hostname check by not specifying `--hostname`. In this case Agate will disregard a request's hostname apart from checking that there is one.


Certificates


Agate has support for using multiple certificates with the `--certs` option. Agate will thus always require that a client uses SNI, which should not be a problem since the Gemini specification also requires SNI to be used.


Certificates are by default stored in the `.certificates` directory. This is a hidden directory for the purpose that uncautious people may set the content root directory to the current directory which may also contain the certificates directory. In this case, the certificates and private keys would still be hidden. The certificates are only loaded when Agate is started and are not reloaded while running. The certificates directory may directly contain a key and certificate pair, this is the default pair used if no other matching keys are present. The certificates directory may also contain subdirectories for specific domains, for example a folder for `example.org` and `portal.example.org`. Note that the subfolders for subdomains (like `portal.example.org`) should not be inside other subfolders but directly in the certificates directory. Agate will select the certificate/key pair whose name matches most closely. For example take the following directory structure:


.certificates
|-- cert.pem     (1)
|-- key.rsa      (1)
|-- example.org
|   |-- cert.pem (2)
|   `-- key.rsa  (2)
`-- portal.example.org
    |-- cert.pem (3)
    `-- key.rsa  (3)

This would be understood like this:

The certificate/key pair (1) would be used for the entire domain tree (exceptions below).

The certificate/key pair (2) would be used for the entire domain tree of `example.org`, so also including subdomains like `secret.example.org`. It overrides the pair (1) for this subtree (exceptions below).

The certificate/key pair (3) would be used for the entire domain tree of `portal.example.org`, so also inclduding subdomains like `test.portal.example.org`. It overrides the pairs (1) and (2) for this subtree.


Using a directory named just `.` causes undefined behaviour as this would have the same meaning as the top level certificate/key pair (pair (1) in the example above).


The files for a certificate/key pair have to be named `cert.der` and `key.der` respectively. The certificate has to be a X.509 certificate in a DER format file and has to include a subject alt name of the domain name. The private key has to be in DER format and must be either an RSA, ECDSA or Ed25519 key.

If the `--hostname` argument is used, Agate will generate certificates and Ed25519 certificates for each hostname specified.


Logging


All requests will be logged using this format:

<local ip>:<local port> <remote ip or dash> "<request>" <response status> "<response meta>"[ error:<error>]

The "error:" part will only be logged if an error occurred. This should only be used for informative purposes as the status code should provide the information that an error occurred. If the error consisted in the connection not being established (e.g. because of TLS errors), the status code `00` will be used.


By default, Agate will not log the remote IP addresses because that might be an issue because IPs are considered private data under the EU's GDPR. To enable logging of IP addresses, you can use the `--log-ip` option. Note that in this case some error conditions might still force Agate to log a dash instead of an IP address.


There are some lines apart from these that might occur in logs depending on the selected log level. For example the initial "Listening on..." line or information about listing a particular directory.


Agate uses some status codes that are not valid Gemini status codes when logging errors:

00 - there was an error establishing the TLS connection

01 - there was an error in fetching the peer's IP address


Security considerations


If you want to run agate on a multi-user system, you should be aware that all certificate and key data is loaded into memory and stored there until the server stops. Since the memory is also not explicitly overwritten or zeroed after use, the sensitive data might stay in memory after the server has terminated.


Changelog


All notable changes to this project will be documented in this file.


The format is based on Keep a Changelog and this project adheres to Semantic Versioning.

Keep a Changelog home page

Semantic versioning standard v2.0.0


[3.1.0] - 2021-06-08

Thank you to Matthew Ingwersen and Oliver Simmons (@GoodClover) for contributing to this release.


Added

tests for symlink files (#60)

Symlinks were already working before.


Fixed

A path traversal security issue was closed: Percent encoded slashes were misunderstood.


Changed

Visiting a directory without `index.gmi` and `.directory-listing-ok` now returns a different error message to better show the cause of the error.

To retain the current behaviour of showing a `51 Not found, sorry.` error, add the following line to the respective directories' `.meta` file:

index.gmi: 51 Not found, sorry.

[3.0.3] - 2021-05-24

Thank you to @06kellyjac, @cpnfeeny, @lifelike, @skittlesvampir and @steko for contributing to this release.


Added

Dockerfile for compiling Agate from source (#52, #53, #56, #57)


Fixed

If the remote IP address can not be fetched, log an error instead of panicking.

The previous handling could be exploited as a DoS attack vector. (#59)

Two tests were running on the same port, causing them to fail nondeterministically. (#51)

Rephrased the changelog for 3.0.0 on continuing to use older certificates. (#55)

Updated dependencies.


[3.0.2] - 2021-04-08

Thank you to @kvibber, @lifelike and @pasdechance for contributing to this release.


Changed

The new specfication changes are obeyed regarding rejecting request URLs that contain fragments or userinfo parts.

The default signature algorithm used for generating certificates has been changed to ECDSA since there were multiple complaints about Ed25519.


[3.0.1] - 2021-03-28

Thank you to @MidAutumnMoon and @steko for contributing to this release.


Added

Installation instructions for Arch Linux from Arch User Repositories. (#47)


Fixed

The certificate file extensions in the README example. (#45)

The certificate directory is automatically created if it does not exist. (#44)


[3.0.0] - 2021-03-27

Thank you to @ddevault for contributing to this release.


Added

Support for ECDSA and Ed25519 keys.

Agate now generates certificates and keys for each `--hostname` that is specified but no matching files exist. (#41)


Changed

The ability to specify a certificate and key with `--cert` and `--key` respectively has been replaced with the `--certs` option. (#40)

Certificates are now stored in a special directory. To migrate to this version, the keys should be stored in the `.certificates` directory (or any other directory you specify).

This enables us to use multiple certificates for multiple domains.

Note that if you want to continue to use your old certificates (recommended because of TOFU), they probably lack the `subjectAltName` directive so your old certificates should be placed at the top level of the certificates directory. Otherwise you will get an error similar to this: "The certificate file for example.com is malformed: unexpected error: The server certificate is not valid for the given name"

The certificate and key file format has been changed from PEM to DER. This simplifies loading certificate and key files without relying on unstable portions of other crates.

If you want to continue using your existing certificates and keys, please convert them to DER format. You should be able to use these commands if you have openssl installed:

openssl x509 -in cert.pem -out cert.der -outform DER
openssl rsa -in key.rsa -out key.der -outform DER

Since agate will automatically generate certificates from now on, the different format should not be a problem because users are not expected to handle certificates unless experienced enough to be able to handle DER formatting as well.


Fixed

Agate now requires the use of SNI by any connecting client.

All log lines are in the same format now:

`<local ip>:<local port> <remote ip or dash> "<request>" <response status> "<response meta>" [error:<error>]`

If the connection could not be established correctly (e.g. because of TLS errors), the status code `00` is used.

Messages from modules other than Agate itself are not logged by default.


[2.5.3] - 2021-02-27

Thank you to @littleli and @06kellyjac for contributing to this release.


Added

Automated tests have been added so things like 2.5.2 should not happen again (#34).

Version information flag (`-V` or `--version` as conventional with e.g. cargo)


Changed

Forbid unsafe code. (There was none before, just make it harder to add some.)

When logging remote IP addresses, the port is now never logged, which also changes the address format.


Fixed

Updated `url` to newest version, which resolves a TODO.

The help exits successfully with `0` rather than `1` (#37).

The GitHub workflow has been fixed so Windows binaries are compressed correctly (#36).

Split out install steps to allow for more options in the future.

Add install notes for nix/NixOS to the README (#38).

Updated dependencies.


[2.5.2] - 2021-02-12


Fixed

Semicolons are no longer considered to be starting a comment in `.mime` files.


[2.5.1] - 2021-02-12

Functionally equivalent to version 2.5.1, only releasing a new version to update README on crates.io.


Fixed

Fixed mistakes in the README.


[2.5.0] - 2021-02-12

Agate now has an explicit code of conduct and contributing guidelines.

Thank you to @ERnsTL, @gegeweb, @SuddenPineapple, and @Ylhp for contributing to this release.


Added

You can now supply multiple `--hostname`s to enable basic vhosts (#28, #31).

Disabling support for TLSv1.2 can now be done using the `--only-tls13` flag, but this is *NOT RECOMMENDED* (#12).

The tools now also contain a startup script for FreeBSD (#13).

Using central config mode (flag `-C`), all configuration can be done in one `.meta` file (see README.md for details).

The `.meta` configuration file now allows for globs to be used.


Changed

The `.meta` file parser now uses the `configparser` crate. The syntax does not change.

The changelog is now also kept in this file in addition to the GitHub releases.

Certificate chain and key file are now only loaded once at startup, certificate changes need a restart to take effect.

Hidden files are now served if there is an explicit setting in a `.meta` file for them, regardless of the `--serve-secret` flag.


Fixed

The Syntax for the IPv6 address in the README has been corrected.

Give a better error message when no keys are found in the key file instead of panicking with a range check (#33).


[2.4.1] - 2020-02-08

Fixed

Re-enabled multiple occurrences of `--addr`. This was accidentally disabled by a merge.


[2.4.0]+podman.build - 2020-02-06

This is the same as [2.4.0], only the build process has been changed so it should accommodate a wider range of architectures and devices.


[2.4.0] - 2020-02-06

Since there is a new maintainer (@Johann150), the range in pre-compiled binaries has changed a bit.


Added

Added some installation tools for Debian.

Added a sidecar file for specifying languages, MIME media types or complete headers on a per file basis (#16).


Changed

Improved logging output. Agate now also respects the `RUST_LOG` environment variable, so you can configure the log level (#22, #23).


[2.3.0] - 2020-01-17

Thanks to @Johann150.


Changed

Combine address and port back into a single command-line argument (#21).


[2.2.0] - 2020-01-16

Thank you to @gegeweb, @Johann150 and @purexo for contributing to this release.


Changed

Split address and port into separate command-line parameters.


Fixed

Listen on both IPv6 and IPv4 interfaces by default (#14, #15).

Do not serve files whose path contains a segment starting with a dot (#17, #20).

Fix redirects of URLs with query strings (#19).


[2.1.3] - 2020-01-02

Changed

Switch to the Tokio async run time.


Fixed

Send TLS close-notify message when closing a connection.

Require absolute URLs in requests.


[2.1.2] - 2020-01-01

Fixed

More complete percent-encoding of special characters in filenames.

Minor improvements to error logging.

Internal code cleanup.


[2.1.1] - 2020-12-31

Changed

List directory content in alphabetical order.


Fixed

Handle percent-escaped paths in URLs.

Percent-escape white space characters in directory listings.


[2.1.0] - 2020-12-29

Enabled GitHub Discussions. If you are using Agate, please feel free to leave a comment to let us know about it!

Thank you to @Johann150 and @KilianKemps for contributing to this release.


Added

Optional directory listings (#8, #9).


Fixed

Updated dependencies.


[2.0.0] - 2020-12-23

Thank you to @bortzmeyer, @KillianKemps, and @Ylhp for contributing to this release.


Added

New `--language` option to add a language tag to the MIME type for text/gemini responses (#6).


Changed

New format for command-line options. See the documentation or run `agate --help` for details.

Logging is enabled by default. Use the `--silent` flag to disable it.

Pre-compiled binaries are built with the [`cross`](https://github.com/rust-embedded/cross) tool, for better compatibility with older Linux systems.


[1.3.2] - 2020-12-09

This release is functionally identical to Agate 1.3.1, and users of that version do not need to update.


Fixed

Update to async-tls 0.11 because the previous version was yanked.


[1.3.1] - 2020-12-08

Thanks @dcreager for contributing this fix.


Fixed

Updated dependencies to fix `cargo install` (#7).


[1.3.0] - 2020-11-20

Thank you @Johann150, @jonhiggs and @tronje for contributing to this release!


Fixed

verify hostname and port in request URL (#4).

improved logging (#2, #3).

Don't redirect to "/" when the path is empty (#5).

Update dependencies.


[1.2.2] - 2020-09-21

Thank you to @m040601 for contributing to this release.


Changed

Switch from `tree_magic` to `mime_guess` for simpler MIME type guessing.

Built both x86_64 and ARM binaries. These binaries are built for Linux operating systems with glibc 2.28 or later, such as Debian 10 ("buster") or newer, Ubuntu 18.10 or newer, and Raspberry Pi OS 2019-06-20 or newer (#1).


Fixed

Update dependencies.

Minor internal code cleanup.


[1.2.1] - 2020-06-20

Fixed

Reduce memory usage when serving large files.

Update dependencies.


[1.2.0] - 2020-06-10

Changed

text/gemini filename extension from `.gemini` to `.gmi`.


Fixed

Handling for requests that exceed 1KB.

Reduce memory allocations and speed up request parsing.

Update dependencies.


[1.1.0] - 2020-05-22

Added

Auto-detect MIME types.


[1.0.1] - 2020-05-21

Added

Send more accurate error codes for unsupported requests.

Do more validation of request URLs.


[1.0.0] - 2020-05-21

-- Response ended

-- Page fetched on Fri Jul 23 15:03:33 2021