💻 [runtimeterror $]

2022-01-01 ~ 2022-07-10

Secure Networking Made Simple with Tailscale

Not all that long ago, I shared about a somewhat-complicated WireGuard VPN setup [1] that I had started using to replace my previous OpenVPN solution. I raved about WireGuard's speed, security, and flexible (if complex) Cryptokey Routing, but adding and managing peers with WireGuard is a fairly manual (and tedious) process. And while I thought I was pretty clever for using a WireGuard peer in GCP to maintain a secure tunnel into my home network without having to punch holes through my firewall, routing all my traffic through The Cloud wasn't really optimal.

[1] somewhat-complicated WireGuard VPN setup

And then I discovered Tailscale [2], which is built on the WireGuard protocol with an additional control plane on top. It delivers the same high security and high speed but dramatically simplifies the configuration and management of peer devices, and also adds in some other handy features like easy-to-configure Access Control Lists (ACLs) [3] to restrict traffic between peers and a MagicDNS [4] feature to automatically register DNS records for connected devices so you don't have to keep up with their IPs.

[2] Tailscale

[3] Access Control Lists (ACLs)

[4] MagicDNS

There's already a great write-up (from the source!) on How Tailscale Works [5], and it's really worth a read so I won't rehash it fully here. The tl;dr though is that Tailscale makes securely connecting remote systems incredibly easy, and it lets those systems connect with each other directly ("mesh") rather than needing traffic to go through a single VPN endpoint ("hub-and-spoke"). It uses a centralized coordination server to *coordinate* the complicated key exchanges needed for all members of a Tailscale network (a "tailnet [6]") to trust each other, and this removes the need for a human to manually edit configuration files on every existing device just to add a new one to the mix. Tailscale also leverages magic :tada: [7] to allow Tailscale nodes to communicate with each other without having to punch holes in firewall configurations or forward ports or anything else tedious and messy. (And in case that the typical NAT traversal techniques don't work out, Tailscale created the Detoured Encrypted Routing Protocol (DERP) to make sure Tailscale can still function seamlessly even on extremely restrictive networks that block UDP entirely or otherwise interfere with NAT traversal.)

[5] How Tailscale Works

[6] tailnet

[7] magic :tada:

<-- note -->

It's a no-brainer solution for remote access, but it's important to note that Tailscale is not a VPN *service*; it won't allow you to internet anonymously or make it appear like you're connecting from a different country (unless you configure a Tailscale Exit Node hosted somewhere in The Cloud to do just that).

<-- /note -->

Tailscale's software is open-sourced [8] so you *could* host your own Tailscale control plane and web front end, but much of the appeal of Tailscale is how easy it is to set up and use. To that end, I'm using the Tailscale-hosted option. Tailscale offers a very generous free Personal tier which supports a single admin user, 20 connected devices, 1 subnet router, plus all of the bells and whistles, and the company also sells Team, Business, and Enterprise plans [9] if you need more users, devices, subnet routers, or additional capabilities.

[8] open-sourced

[9] Team, Business, and Enterprise plans

Tailscale provides surprisingly-awesome documentation, and the Getting Started with Tailscale [10] article does a great job of showing how easy it is to get up and running with just three steps:

Sign up for an account

Add a machine to your network

Add another machine to your network (repeat until satisfied)

[10] Getting Started with Tailscale

This post will start there but then also expand some of the additional features and capabilities that have me so excited about Tailscale.

Getting started

The first step in getting up and running with Tailscale is to sign up at https://login.tailscale.com/start [11]. You'll need to use an existing Google, Microsoft, or GitHub account to sign up, which also lets you leverage the 2FA and other security protections already enabled on those accounts.

[11] https://login.tailscale.com/start

Once you have a Tailscale account, you're ready to install the Tailscale client. The download page [12] outlines how to install it on various platforms, and also provides a handy-dandy one-liner to install it on Linux:

[12] download page

After the install completes, it will tell you exactly what you need to do next:

There are also Tailscale apps available for iOS [13] and Android [14] - and the Android app works brilliantly on Chromebooks too!

[13] iOS

[14] Android

Basic `tailscale up`

Running `sudo tailscale up` then reveals the next step:

I can copy that address into a browser and I'll get prompted to log in to my Tailscale account. And that's it. Tailscale is installed, configured to run as a service, and connected to my Tailscale account. This also creates my tailnet.

That was pretty easy, right? But what about if I can't easily get to a web browser from the terminal session on a certain device? No worries, `tailscale up` has a flag for that:

That will convert the URL to a QR code that I can scan from my phone.

Advanced `tailscale up`

There are a few additional flags that can be useful under certain situations:

`--advertise-exit-node` to tell the tailnet that this could be used as an exit node for internet traffic

`--advertise-routes` to let the node perform subnet routing functions to provide connectivity to specified local subnets

`--advertise-tags` to associate the node with certain tags for ACL purposes (like `tag:home` to identify stuff in my home network and `tag:cloud` to label external cloud-hosted resources)

`--hostname` to manually specific a hostname to use within the tailnet

`--shields-up` to block incoming traffic

These flags can also be combined with each other:

Sidebar: Tailscale on VyOS

Getting Tailscale on my VyOS virtual router [15] was unfortunately a little more involved than leveraging the built-in WireGuard capability [16]. I found the vyos-tailscale [17] project to help with building a customized VyOS installation ISO with the `tailscaled` daemon added in. I was then able to copy the ISO over to my VyOS instance and install it as if it were a standard upgrade [18]. I could then bring up the interface, advertise my home networks, and make it available as an exit node with:

[15] my VyOS virtual router

[16] leveraging the built-in WireGuard capability

[17] vyos-tailscale

[18] standard upgrade

Other `tailscale` commands

Once there are a few members, I can use the `tailscale status` command to see a quick overview of the tailnet:

Without doing any other configuration beyond just installing Tailscale and connecting it to my account, I can now easily connect from any of these devices to any of the other devices using the listed Tailscale IP. Entering `ssh 100.116.31.85` will connect me to my Matrix server.

`tailscale ping` lets me check the latency between two Tailscale nodes at the Tailscale layer; the first couple of pings will likely be delivered through a nearby DERP server until the NAT traversal magic is able to kick in:

The `tailscale netcheck` command will give me some details about my local Tailscale node, like whether it's able to pass UDP traffic, which DERP server is the closest, and the latency to all Tailscale DERP servers:

Tailscale management

Now that the Tailscale client is installed on my devices and I've verified that they can talk to each other, it might be a good time to *log in* at `login.tailscale.com` [19] to take a look at the Tailscale admin console.

Image: Tailscale admin console

[19] `login.tailscale.com`

Subnets and Exit Nodes

See how the `vyos` node has little labels on it about "Subnets (!)" and "Exit Node (!)"? The exclamation marks are there because the node is *advertising* subnets and its exit node eligibility, but those haven't actually been turned on it. To enable the `vyos` node to function as a subnet router (for the `172.16.0.0/16` and `192.168.1.0/24` networks listed beneath its Tailscale IP) and as an exit node (for internet-bound traffic from other Tailscale nodes), I need to click on the little three-dot menu icon at the right edge of the row and select the "Edit route settings..." option.

Image: The menu contains some other useful options too - we'll get to those!

Image: Edit route settings

Now I can approve the subnet routes (individually or simultaneously and at the same time) and allow the node to route traffic to the internet as well.

Image: Enabled the routes

Cool! But now that's giving me another warning...

Key expiry

By default, Tailscale expires each node's encryption keys every 180 days [20]. This improves security (particularly over vanilla WireGuard, which doesn't require any key rotation) but each node will need to reauthenticate (via `tailscale up`) in order to get a new key. It may not make sense to do that for systems acting as subnet routers or exit nodes since they would stop passing all Tailscale traffic once the key expires. That would also hurt for my cloud servers which are *only* accessible via Tailscale; if I can't log in through SSH (since it's blocked at the firewall) then I can't reauthenticate Tailscale to regain access. For those systems, I can click that three-dot menu again and select the "Disable key expiry" option. I tend to do this for my "always on" tailnet members and just enforce the key expiry for my "client" type devices which could potentially be physically lost or stolen.

[20] expires each node's encryption keys every 180 days

Image: Machine list showing enabled Subnet Router and Exit Node and disabled Key Expiry

Configuring DNS

It's great that all my Tailscale machines can talk to each other directly by their respective Tailscale IP addresses, but who wants to keep up with IPs? I sure don't. Let's do some DNS. I'll start out by clicking on the DNS [21] tab in the admin console.

Image: The DNS options

[21] DNS

I need to add a Global Nameserver before I can enable MagicDNS so I'll click on the appropriate button to enter in the *Tailscale IP* of my home DNS server (which is using NextDNS [22] as the upstream resolver).

Image: Adding a global name server

[22] NextDNS

I'll also enable the toggle to "Override local DNS" to make sure all queries from connected clients are going through this server (and thus extend the NextDNS protection to all clients without having to configure them individually).

Image: Overriding local DNS configuration

I can also define search domains to be used for unqualified DNS queries by adding another name server with the same IP address, enabling the "Restrict to search domain" option, and entering the desired domain:

Image: Entering a search domain

This will let me resolve hostnames when connected remotely to my lab without having to type the domain suffix (ex, `vcsa` versus `vcsa.lab.bowdre.net`).

And, finally, I can click the "Enable MagicDNS" button to turn on the magic. This adds a new nameserver with a private Tailscale IP which will resolve Tailscale hostnames to their internal IP addresses.

Image: MagicDNS Enabled!

Now I can log in to my Matrix server by simply typing `ssh matrix`. Woohoo!

Access control

Right now, all of my Tailscale nodes can access all of my other Tailscale nodes. That's certainly very convenient, but I'd like to break things up a bit. I can use access control policies to define which devices should be able to talk to which other devices, and I can use ACL tags [23] to logically group resources together to make this easier.

[23] ACL tags

Tags

I'm going to use three tags in my tailnet:

`tag:home` to identify servers in my home network which will have access to all other servers.

`tag:cloud` to identify my cloud servers which will only have access to other cloud servers.

`tag:client` to identify client-type devices which will be able to access all nodes in the tailnet.

Before I can actually apply these tags to any of my machines, I first need to define `tagOwners` for each tag which will determine which users (in my organization of one) will be able to use the tags. This is done by editing the policy file available on the Access Controls [24] tab of the admin console.

[24] Access Controls

This ACL file uses a format called HuJSON [25], which is basically JSON but with support for inline comments and with a bit of leniency when it comes to trailing commas. That makes a config file that is easy for both humans and computers to parse.

[25] HuJSON

I'm going to start by creating a group called `admins` and add myself to that group. This isn't strictly necessary since I am the only user in the organization, but I feel like it's a nice practice anyway. Then I'll add the `tagOwners` section to map each tag to its owner, the new group I just created:

Now I have two options for applying tags to devices. I can either do it from the admin console, or by passing the `--advertise-tags` flag to the `tailscale up` CLI command. I touched on the CLI approach earlier so I'll go with the GUI approach this time. It's simple - I just go back to the Machines [26] tab, click on the three-dot menu button for a machine, and select the "Edit ACL tags..." option.

Image: Edit ACL tags

[26] Machines

I can then pick the tag (or tags!) I want to apply:

Image: Selecting the tags

The applied tags have now replaced the owner information which was previously associated with each machine:

Image: Tagged machines

ACLs

By default, Tailscale implements an implicit "Allow All" ACL. As soon as you start modifying the ACL, though, that switches to an implicit "Deny All". So I'll add new rules to explicitly state what communication should be permitted and everything else will be blocked.

Each ACL rule consists of four named parts:

`action` - what to do with the traffic, the only valid option is `allow`.

`users` - a list of traffic sources, can be specific users, Tailscale IPs, hostnames, subnets, groups, or tags.

`proto` - (optional) protocol for the traffic which should be permitted.

`ports` - a list of destinations (and optional ports).

So I'll add this to the top of my policy file:

This policy becomes active as soon as I click the Save button at the bottom of the page, and I notice a problem very shortly. Do you see it?

Earlier I configured Tailscale to force all nodes to use my home DNS server for resolving all queries, and I just set an ACL which prevents my cloud servers from talking to my home servers... which includes the DNS server. I can think of two ways to address this:

Re-register the servers by passing the `--accept-dns=false` flag to `tailscale up` so they'll ignore the DNS configured in the admin console.

Add a new ACL rule to allow DNS traffic to reach the DNS server from the cloud.

Option 2 sounds better to me so that's what I'm going to do. Instead of putting an IP address directly into the ACL rule I'd rather use a hostname, and unfortunately the Tailscale host names aren't available within ACL rule declarations. But I can define a host alias in the policy to map a friendly name to the IP:

And I can then create a new rule for `"users": ["tag:cloud"]` to add an exception for `win01:53`:

And that gets DNS working again for my cloud servers while still serving the results from my NextDNS configuration. Here's the complete policy configuration:

Wrapping up

This post has really only scratched the surface on the cool capabilities provided by Tailscale. I didn't even get into its options for enabling HTTPS with valid certificates [27], custom DERP servers [28], sharing tailnet nodes with other users [29], or file transfer using Taildrop [30]. The Next Steps [31] section of the official Getting Started doc has some other cool ideas for Tailscale-powered use cases, and there are a ton more listed in the Solutions [32] and Guides [33] categories as well.

[27] enabling HTTPS with valid certificates

[28] custom DERP servers

[29] sharing tailnet nodes with other users

[30] file transfer using Taildrop

[31] Next Steps

[32] Solutions

[33] Guides

It's amazing to me that a VPN solution can be this simple to set up and manage while still offering such incredible flexibility. **Tailscale is proof that secure networking doesn't have to be hard.**

---

📧 Reply by email

Related articles

Using a Custom Font with Hugo

Blocking AI Crawlers

Self-Hosted Gemini Capsule with gempost and GitHub Actions

---

Home

This page on the big web