-- Leo's gemini proxy

-- Connecting to gmi.runtimeterror.dev:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini;lang=en-US

đź’» [runtimeterror $]

2023-12-20

Tailscale Feature Highlight: SSH, Serve, and Funnel



I've spent the past two years in love with Tailscale [1], which builds on the secure and high-performance Wireguard VPN protocol [2] and makes it really easy to configure and manage [3]. Being able to easily (and securely) access remote devices as if they were on the same LAN is pretty awesome to begin with, but Tailscale is packed with an ever-expanding set of features that can really help to streamline your operations too. Here are three of my favorites.

[1] Tailscale

[2] secure and high-performance Wireguard VPN protocol

[3] really easy to configure and manage

Tailscale SSH

Tailscale already takes care of issuing, rotating, and otherwise managing the Wireguard keys used for securing communications between the systems in your tailnet. Tailscale SSH [4] lets it do the same for your SSH keys as well. No more manually dropping public keys on systems you're setting up for remote access. No more scrambling to figure out how to get your private key onto your mobile device so you can SSH to a server. No more worrying about who has access to what. Tailscale can solve all those concerns for you - and it does it without impacting traditional SSH operations:

[4] Tailscale SSH

> *With Tailscale, you can already connect machines in your network, and encrypt communications end-to-end from one point to another—and this includes, for example, SSHing from your work laptop to your work desktop. Tailscale also knows your identity, since that’s how you connected to your tailnet. When you enable Tailscale SSH, Tailscale claims port 22 for the Tailscale IP address (that is, only for traffic coming from your tailnet) on the devices for which you have enabled Tailscale SSH. This routes SSH traffic for the device from the Tailscale network to an SSH server run by Tailscale, instead of your standard SSH server. With Tailscale SSH, based on the ACLs in your tailnet, you can allow devices to connect over SSH and rely on Tailscale for authentication instead of public key authentication.*

All you need to advertise Tailscale SSH on a system is to pass the appropriate flag in the `tailscale up` command:

sudo tailscale up --ssh

To actually use the feature, though, you'll need to make sure that your Tailscale ACL permits access. The default "allow all" ACL does this:

{
  "acls": [
    // Allow all connections.
    { "action": "accept", "src": ["*"], "dst": ["*:*"] },
  ],
  "ssh": [
    // Allow all users to SSH into their own devices in check mode.
    {
      "action": "check",
      "src": ["autogroup:member"],
      "dst": ["autogroup:self"],
      "users": ["autogroup:nonroot", "root"]
    }
  ]
}

The `acls` block allows all nodes to talk to each other over all ports, and the `ssh` block will allow all users in the tailnet to SSH into systems they own (both as nonroot users as well as the `root` account) but only if they have reauthenticated to Tailscale within the last 12 hours (due to that `check` action).

Most of my tailnet nodes are tagged with a location (`internal`/`external`) instead of belonging to a single user, and the `check` action doesn't work when the source is a tagged system. So my SSH ACL looks a bit more like this:

{
  "acls": [
    {
      // internal systems can SSH to internal and external systems
      "action": "accept",
      "users":  ["tag:internal"],
      "ports": [
        "tag:internal:22",
        "tag:external:22"
      ],
    },
  ],
  "tagOwners": {
    "tag:external":    ["group:admins"],
    "tag:internal":    ["group:admins"],
  },
  "ssh": [
    {
      // users can SSH to their own systems and those tagged as internal and external
      "action": "check",
      "src":    ["autogroup:members"],
      "dst":    ["autogroup:self", "tag:internal", "tag:external"],
      "users":  ["autogroup:nonroot", "root"],
    },
    {
      // internal systems can SSH to internal and external systems,
      // but external systems can't SSH at all
      "action": "accept",
      "src":    ["tag:internal"],
      "dst":    ["tag:internal", "tag:external"],
      "users":  ["autogroup:nonroot"],
    },
  ],
}

This way, SSH connections originating from `internal` systems will be accepted, while those originating from untagged systems will have the extra check for tailnet authentication. You might also note that this policy prevents connections from tagged systems as the `root` user, requiring instead that the user log in with their own account and then escalate as needed.


These ACLs can get pretty granular [5], and I think it's pretty cool to be able to codify your SSH access rules in a centrally-managed policy instead of having to manually keep track of which keys are on which systems.

[5] pretty granular


Once SSH is enabled on a tailnet node and the ACL rules are in place, you can SSH from a Tailscale-protected system to another as easily as `ssh [hostname]` and you'll be connected right away - no worrying about keys or fumbling to enter credentials. I think this is doubly cool when implemented on systems running in The Cloud; Tailscale provides the connectivity so I don't need to open up port 22 to the world.

ssh tsdemo
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.5.11-6-pve x86_64)
 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage
Last login: Tue Dec 19 04:17:15 UTC 2023 from 100.73.92.61 on pts/3
john@tsdemo:~$

As a bonus, I can also open an SSH session from the Tailscale admin console [6]

[6] admin console

Image: web_ssh_2

Image: web_ssh_3


That even works from mobile devices, too!

Tailscale Serve

I've mentioned in the past [7] how impressed I was (and still am!) by the Caddy webserver [8] and how effortless it makes configuring a reverse proxy with automatic TLS. I've used it for a *lot* of my externally-facing projects.

[7] mentioned in the past

[8] Caddy webserver

Caddy is great, but it's not quite as easy to use for internal stuff - I'd need a public DNS record and inbound HTTP access in order for the ACME challenge to complete and a cert to be issued and installed, or I would have to manually create a certificate and load it in the Caddy config. That's probably not a great fit for wanting to proxy my Proxmox host [9]. And that is where the capabilities of Tailscale Serve [10] really come in handy.

[9] Proxmox host

[10] Tailscale Serve

> *Tailscale Serve is a feature that allows you to route traffic from other devices on your Tailscale network (known as a tailnet) to a local service running on your device. You can think of this as sharing the service, such as a website, with the rest of your tailnet.*

<-- note -->

Tailscale Serve requires that the MagicDNS and HTTPS features be enabled on your Tailnet. You can learn how to turn those on here [11].

<-- /note -->

[11] here

The general syntax is:

tailscale serve <target>

> *`<target>` can be a file, directory, text, or most commonly the location to a service running on the local machine. The location to the location service can be expressed as a port number (e.g., `3000`), a partial URL (e.g., `localhost:3000), or a full URL including a path (e.g., `http://localhost:3000/foo`).*

The command also supports some useful flags:

  --bg, --bg=false
        Run the command as a background process (default false)
  --http uint
        Expose an HTTP server at the specified port
  --https uint
        Expose an HTTPS server at the specified port (default mode)
  --set-path string
        Appends the specified path to the base URL for accessing the underlying service
  --tcp uint
        Expose a TCP forwarder to forward raw TCP packets at the specified port
  --tls-terminated-tcp uint
        Expose a TCP forwarder to forward TLS-terminated TCP packets at the specified port
  --yes, --yes=false
        Update without interactive prompts (default false)

Tailscale Serve can be used for spawning a simple file server (like this one which shares the contents of the `/demo` directory):

sudo tailscale serve /demo
Available within your tailnet:
https://tsdemo.tailnet-name.ts.net/
|-- path  /demo
Press Ctrl+C to exit.

Image: file server

Note that this server is running in the foreground, and that it's serving the site with an automatically-generated automatically-trusted Let's Encrypt [12] certificate.

[12] Let's Encrypt

I can also use Tailscale Serve for proxying another web server, like Cockpit [13], which runs on `http://localhost:9090`:

sudo tailscale serve --bg 9090
Available within your tailnet:
=> https://cockpit-project.org/ [13] Cockpit
https://tsdemo.tailnet-name.ts.net/
|-- proxy http://127.0.0.1:9090
Serve started and running in the background.
To disable the proxy, run: tailscale serve --https=443 off

Image: cockpit

This time, I included the `--bg` flag so that the server would run in the background, and I told it to proxy port `9090` instead of a file path.

But what if I want to proxy *another* service (like netdata [14], which runs on `http://localhost:19999`) at the same time? I can either proxy it on another port, like `8443`:

sudo tailscale serve --bg --https 8443 19999
Available within your tailnet:
=> https://github.com/netdata/netdata [14] netdata
https://tsdemo.tailnet-name.ts.net/
|-- proxy http://127.0.0.1:9090
https://tsdemo.tailnet-name.ts.net:8443/
|-- proxy http://127.0.0.1:19999
Serve started and running in the background.
To disable the proxy, run: tailscale serve --https=8443 off

Or serve it at a different path:

sudo tailscale serve --bg --set-path /netdata 19999
Available within your tailnet:
https://tsdemo.tailnet-name.ts.net/
|-- proxy http://127.0.0.1:9090
https://tsdemo.tailnet-name.ts.net/netdata
|-- proxy http://127.0.0.1:19999
Serve started and running in the background.
To disable the proxy, run: tailscale serve --https=443 off

Image: netdata

<-- note -->

Not all web apps adapt well to being served at a different path than they expect. It works fine for netadata, but did not work with Cockpit (at least not without digging deeper into the configuration to change the base URL). But hey, that's why we've got options!

<-- /note -->

Tailscale Funnel

So Tailscale Serve works well for making web resources available inside of my tailnet... but what if I want to share them externally? Tailscale Funnel [15] has that use case covered, and it makes it easy to share things without having to manage DNS records, punch holes in firewalls, or configure certificates.

[15] Tailscale Funnel

> *Tailscale Funnel is a feature that allows you to route traffic from the wider internet to a local service running on a machine in your Tailscale network (known as a tailnet). You can think of this as publicly sharing a local service, like a web app, for anyone to access—even if they don’t have Tailscale themselves.*

In addition to requiring that HTTPS is enabled, Funnel also requires that nodes have the `funnel` attribute applied through the ACL policy.

There's a default policy snippet to apply this:

"nodeAttrs": [
  {
    "target": ["autogroup:member"],
    "attr":   ["funnel"],
  },
],

But I use a tag to manage funnel privileges instead so my configuration will look something like this:

{
  "acls": [
    {
      // internal systems can SSH to internal and external systems
      "action": "accept",
      "users":  ["tag:internal"],
      "ports": [
        "tag:internal:22",
        "tag:external:22"
      ],
    },
  ],
  "tagOwners": {
    "tag:external":    ["group:admins"],
    "tag:funnel":      ["group:admins"],
    "tag:internal":    ["group:admins"],
  },
  "ssh": [
    {
      // users can SSH to their own systems and those tagged as internal and external
      "action": "check",
      "src":    ["autogroup:members"],
      "dst":    ["autogroup:self", "tag:internal", "tag:external"],
      "users":  ["autogroup:nonroot", "root"],
    },
    {
      // internal systems can SSH to internal and external systems,
      // but external systems can't SSH at all
      "action": "accept",
      "src":    ["tag:internal"],
      "dst":    ["tag:internal", "tag:external"],
      "users":  ["autogroup:nonroot"],
    },
  ],
  "nodeAttrs": [
    {
      // devices with the funnel tag can enable Tailscale Funnel
      "target": ["tag:funnel"],
      "attr": ["funnel"],
    },
  ]
}

Now only nodes with the `funnel` tag will be able to enable Funnel.

From there, the process to activate Tailscale Funnel is basically identical to that of Tailscale Serve - you just use `tailscale funnel` instead of `tailscale serve`.

<-- note -->

A Funnel configuration is applied to the **port** that Tailscale Serve uses to make a resource available, not the resource itself. In the example above, I have both Cockpit and netdata being served over port `443`. If I try to use `sudo tailscale funnel --set-path /netdata 19999` to Funnel just the netdata instance, that will actually Funnel *both* resources instead of just the one.

<-- /note -->

If I want to make the netdata instance available publicly while keeping Cockpit internal-only, I'll need to serve netdata on a different port. Funnel only supports [16] ports `443`, `8443`, and `10000`, so I'll use `8443`:

sudo tailscale funnel --bg --https 8443 --set-path /netdata 19999
Available on the internet:
=> https://tailscale.com/kb/1223/funnel#limitations [16] only supports
https://tsdemo.tailnet-name.ts.net:8443/netdata
|-- proxy http://127.0.0.1:19999
Funnel started and running in the background.
To disable the proxy, run: tailscale funnel --https=8443 off

It will take 10 or so minutes for the public DNS record to get created, but after that anyone on the internet (not just within my tailnet!) would be able to access the resource I've shared.

I can use `tailscale serve status` to confirm that both Cockpit and netdata are served internally on port `443`, but only netdata is published externally on port `8443`:

sudo tailscale serve status

# Funnel on:
#     - https://tsdemo.tailnet-name.ts.net:8443
https://tsdemo.tailnet-name.ts.net (tailnet only)
|-- /        proxy http://127.0.0.1:9090
|-- /netdata proxy http://127.0.0.1:19999
https://tsdemo.tailnet-name.ts.net:8443 (Funnel on)
|-- /netdata proxy http://127.0.0.1:19999

Pretty cool, right?

Other Features

This post has covered some of my favorite (and most-used) Tailscale features, but there are *plenty* of other cool tricks up Tailscale's sleeves. These are some others that are definitely worth checking out:

Mullvad Exit Nodes

Taildrop

golink](https://tailscale.com/blog/golink) (also covered [here

And if you're particularly nerdy like me, these will probably also grab your interest:

GitOps for Tailscale ACLs

Manage Tailscale resources using Terraform

Need more inspiration? Tailscale has a pretty thorough collection of example uses cases [17] to help get the sparks flying.

[17] collection of example uses cases

I'm sure I'll have more Tailscale-centric posts [18] to share in the future, too. Stay tuned!

[18] more Tailscale-centric posts



---


📧 Reply by email



Related articles


Using a Custom Font with Hugo

Self-Hosted Gemini Capsule with gempost and GitHub Actions

Publish Services with Cloudflare Tunnel

---


Home

This page on the big web

-- Response ended

-- Page fetched on Fri May 10 10:28:48 2024