-- Leo's gemini proxy
-- Connecting to gemini.smallweb.space:1965...
-- Connected
-- Sending request
-- Meta line: 20 text/gemini
Last updated:2023-08-29
For newcomers, the most frequently asked questions are, "how do I make a gemlog?" and "how do I make my own capsule?" There are a bunch of guides around, but this one's mine so that I can point folks to it when they ask.
I'm not a professional but I've researched all the items below, so feel free to provide me feedback via email.
1.1 Pick a server (home vs. VPS vs. hosted))
1.2 Install an OS
1.3 Secure the OS
1.4 Plan for content directories
1.5 Pick and install a gemini server
1.6 Buy a domain name
1.7 Setup DNS
2.1 Creating your first gemlog
2.2 Generating an ATOM feed
2.3 About tinylogs
2.4 Creating a tinylog
2.5 Aggregating your tinylog timeline
2.6 Installing a gemlog aggregator
3.1 Setting up the server
3.2 Helloworld
3.3 About environment variables
3.4 "Thanks" response script
3.5 Using Titan
----------------------------------
There are three general options:
Use your own hardware (e.g., Raspberry Pi)
Use a Virtual Private Server (VPS) -- you rent space on a server
Use a hosted server such as a Tilde / public UNIX (pubnix) that supports Gemini
Personally I've done all three but mostly the first two. I bought a Raspberry Pi 4 and hooked it up to my home network. It's a bit trickier to get your DNS to point to it but it's absolutely doable. I then migrated over to a Digital Ocean $6 USD VPS just so I wasn't opening my home network to some kind of attack vector I didn't know about. I've had no complaints with them thus far and $6 USD /mo is worth it.
I also have a capsule on Tilde.Team, but it only points to my primary capsule on my VPS. Points to consider about using a hosting service:
Setup is pretty much done for you
You don't pick the server software
Shared resources (which is important depending on how much you're planning to do in Gemini)
Possibility for a subdomain of your own (e.g., <your_sub>.tilde.cafe)
Hosting may go down since these services are provided by the community for free (make sure you backup)
If using a VPS, I believe most come with static IPs.
If using your own hardware where your home IP can change, you can use a dynamic dns service (e.g., dyndns or duckdns), or run a script that updates your domain name (see section 1.7.1).
For my VPS I picked Ubuntu, but feel free to install whatever you're comfortable with; people serve from all sorts of OSs (BSD, etc.).
I'm no security professional but I did do my research.
Generally, I:
Created an unprivileged user (i.e., no sudo access, and removed from unnecessary groups)
Setup an admin user with SUDO access
Disabled direct root login (and also via SSH)
Setup a firewall (UFW) [1]
Setup fail2ban to ban users trying to brute force their way in [1, 2]
Enabled ssh keys login (so I don't have to remember or type the passwords) [2]
(optional) Disabled SSH password login to avoid brute force attacks on passwords
NOTE: Make sure you don't lock yourself out of your server with fail2ban! Luckily I did this on my RPI and was able to log in locally to fix it.
Personally, I hand-craft my gemini capsule because I don't update all that often so it's not that much maintenance; however, there a quite a few Static Site Generators (SSGs) out there for Gemini. I've known geminauts to retrofit HTML SSGs to output both gemtext and HTML so they can mirror their blogs and gemlogs. I've not tried this.
If you choose to hand-code, simply make a directory in your unprivileged user's $HOME directory and start planning out the structure. Oftentimes the server will dictate some of this (i.e., CGI vs static content), so it's best to consult the instructions there before diving into this part.
See Section 2.1 "Creating Your First Gemlog" below.
I haven't tried any of these, but here are some SSGs for Gemini:
Picking a good server can be daunting, especially if it's your first one. I suggest using an easy static server (i.e., doesn't support dynamic content via CGI) like Agate to get started.
➢ Note: The guide sets the time limit of the self-signed certificate to expire in 1 year, I suggest 5+, or even 100. This keeps you from having to generate a new one all the time and folks wondering if something happened with your capsule. (See Section 1.5.2 for more on generating server certificates)
I've also tried Molly Brown (by @solderpunk) and am currently using GmCapsule (by @skyjake). I like GmCapsule the best thus far as it also includes Titan support
Gemini uses TLS to make secure connections between the client and server. This is mandatory, and actually a point of contention for some geminauts and also critics of the protocol. As it's a text-oriented protocol, some argue that TLS makes everything too complex (especially so on older / limited hardware), but personally I think this is a good trade-off for security, and it really doesn't add an unbearable amount of complexity from what I've seen. However, this did give way for a slimmer, non-TLS version of Gemini called Spartan.
Despite the above, if you want to run a server you're going to need to generate a certificate. Most certificates are "self-signed" certificates because they aren't backed by what's known as a trusted Certificate Authority (CA)...it's only you signing your certificate. "What's the use in that?" you may ask, well, Gemini operates on what's known as Trust On First Use (TOFU), wherein the first time you visit a server, you immediately trust the certificate and store its fingerprint for future comparision. Each time you request something from that server, it checks to see if the certificate changes, and if it has, almost all clients will throw a warning. This is to prevent a man-in-the-middle attack where a new server masquerades as the one you're actually trying to contact.
Why doesn't Gemini mandate CAs? This is actually addressed in the official Gemini FAQ:
As mentioned above there are a couple of ways to create a certificate:
Self-signed
Use a CA
If you want a free CA, most people use letsencrypt (link below), and simply copy their certs over to their gemini server (which should be run as an unprivileged user). Note that these certificates expire every 3 months and will cause a certificate change alert for your users - this is enough for most operators to skip using letsencrypt and just use a really long expiry on a self-signed certificate. I won't go over using letsencrypt as their website does a good job, especially if you're using their certbot, but once it's done you just have to navigate to "/etc/letsencrypt/live/<yoursite>/" to find the keys.
As for self-signed certificates, there are a few guides out there but the easiest way is to use Solderpunk's gemcert program to help you create one:
Note: As per the Gemcert site, it creates certificates for both your primary domain (e.g., example.com), and also a wildcard for any subdomains that you might make in the future (i.e., *.example.com). This will be important if your gemini site runs from, say, gemini.example.com
Note 2: As mentioned above, it's recommended to have your self-signed cert expire at a date way in the future, like 100+ years.
Gemcert Usage (for example, using GmCapsule):
$ ./gemcert --server --domain localhost $ mv localhost.crt cert.pem $ mv localhost.key key.pem $ mv *.pem ~/.cert/
Explanation:
Generate a server certificate, in this case, for localhost (i.e, for local testing)
The keys are already in pem format, but GmCapsule certificates require a specific naming convention
Move the keys to the location specified in your ~/.gmcapsulerc
Note: make sure you put your FULL PATH into .gmcapsulerc, and not the shortcut for home directories (e.g., ~/gritty/.certs/)
Note: You can use a dynamic dns service (see section 1.1.1), but I recommend you buy a domain.
Note 2: Tilde servers may offer subdomains for free
There are a lot of ways to do this and most are straightforward. I went with Njalla because they:
Were recommended by privacy circles
Seem like they actually respect privacy
Take crypto as payment (if I so wanted), and some anon coins I think are also included
Offer VPSs / servers and VPN if I wanted to stay with the same company
This is actually not that bad if you're using a VPS, and using a home server isn't too bad either. You have to add what's called an "A Record" that links your domain name to the static IP of your VPS.
The basic approach is to run a script that sends your current public IP address to your nameserver on a recurring basis.
Considerations / Steps
Ensure your Nameserver supports dynamic IP updates
Add a "dynamic" record to your DNS (see njalla link example below)
Run a cron job that executes the update script on a recurring bases (say, 12 hours)
Set your home server to have a static IP (this is usually set in your home router)
Forward port 1965 on your router to port 1965 on your home server (or some other port so long as your gemini server is configured to listen to that port)
Manually trigger the update script at least once and try to connect to your server
Crontab setting for every 12 hours:
> 0 */12 * * * <location/to/script>
Not only is content king on Gemini...it's the only thing. You don't have to be a voluminous writer on Gemini but it certainly helps to have an idea of what you want to say. You could, theorectically, just hang out on #Station and #BBS, but the heart and soul of Gemini is the long-form sharing of thoughts and ideas with other geminauts.
Personally, I'm not much of a writer, nor do I have that much time to do so, but something about Gemini made me want to write. I would actually catch myself thinking "this would be a good gemlog topic" on occasion, and then would write it out later. Some people post multiple times a day, and others - like myself - write a couple times a month... and THAT'S OKAY. The point here is to have good, well thought-out (or maybe stream-of-consciousness, if that's your thing) gemlog entries that others can consume.
I'll go over the manual way of writing your gemlog, as most static site generators do this for you. The exact format is pretty simple and is covered on circumlunar, the offical capsule of Gemini. Now, you can format your capsule and gemlog however you want, but if you want someone to be able to subscribe to it via their favorite reader, you'll have to comply with gmisub, or gemini subscriptions specification. See link below.
You'll also want to be familar with gemtext, which I won't cover here because the official docs cover it well enough:
Your starter folder structure may look something like this:
gemini/ ├─ cgi-bin/ ├─ gemlog/ │ ├─ drafts/ │ ├─ 20230811-hello-world.gmi │ ├─ atom-feed.xml │ ├─ index.gmi │ ├─ template.gmi ├─ index.gmi
A few things to note about this layout:
drafts/ is just as it says, where your works-in-progress go
gemlog/index.gmi - this is the gmisub page where you put links to your gemlog files
atom-feed.xml - atom feed of your gemlogs, if you have a generator (don't do this by hand)
template.gmi - I made a template for consistency that I duplicate for each new entry
/gemini/index.gmi is your capsule's main landing page, usually with a link to your gemlog
Per the gmisub specification, you need a few things:
A title
Optional sub-title
A listing of gemlogs in the format:
> => gemlog/20230811-hello-world.gmi 2023-08-11 - Hello World!
That is:
the relative link to your file
a space
date in yyyy-mm-dd format (dashes are optional)
space, dash, space
title of the article
If you do the above, all gemlog readers will be able to subscribe to your page, and you will also be able to submit your entries to Antenna and DSN Antenna
Why use Atom feeds instead of RSS? @Solderpunk describes it here:
To be honest, if you wanted to skip generating an Atom feed in favor of gmisub, you can absolutely do that. I believe most people generate Atom feeds in order to have a standardized format, and one that can be used in clients that don't support gmisub. It also supports more precise created/updated timestamps for each entry, if that's something you need. It's also an official standard, which helps.
I personally use @solderpunk's Gemfeed tool because it's just a python script that scrapes your "gemlog/" directory and generates an atom feed for you. You can do this automatically with a cron job, but I don't update a lot so I trigger it manually whenever I make an udpate. I also put my helper scripts in "gemini/scripts/" directory.
I made a "generate_atom.sh" script in my scripts/ directory:
#!/bin/bash
# taken from https://tildegit.org/solderpunk/gemfeed
BASE_URL="gemini://gemini.smallweb.space/gemlog/"
LOG_DIR="/home/gemini/gemini/gemini.smallweb.space/gemlog/"
python3 /home/gemini/gemini/scripts/gemfeed.py -b ${BASE_URL} -d ${LOG_DIR} -a "Gritty" -e "gritty@smallweb.space" -n 20
In here:
BASE_URL points to my gemlog directory, as seen from a browser
LOG_DIR is the local, server-side path to my gemlog folder (I use the full path but you can probably use relative)
-a is for Author
-e is my email
-n is the number of gemlogs you want to list (default is 10)
And that's pretty much it for Atom feeds.
Think of Twitter (now, "X") posts when you think of a tinylog, except all your posts are in one gmi file with timestamps.
For this I'm going to steal from an early gemlog of mine, where I was discovering microblogging on Gemini. There are a few microblog formats on Gemini, but tinylog is the most widely adopted, and is the one you want to use.
Tinylog has an (unofficial but widely used) RFC set out for it to define a common structure. Although "[t]he original idea and most "rules" comes from Drew/uoou/Friendo['s]" Lace script, Bacardi55 has formally defined the structure which allows for easier aggregation.
Tinylog:
Is Gemtext
Is defined by an RFC
Has tools for consuming/reading, subscribing, and aggregating/publishing
Is supported by Station microblogging/socialization platform for every user
Is supported by BBS (by @skyjake) - publishing only, not subscribing
Is supported by Pollux.Casa capsule hosting service
Has a list of known tinylogs, but may not be comprehensive (Station isn't included)
I want to emphasize the first point - while twtxt is plaintext, Tinylogs are Gemtext, and thus can be read by *any* Gemini client natively, in whatever native format that client renders (i.e., Lagrange rendering vs Kristall, for example).
If you follow the RFC, creating a tinylog is straightforward - it's just a .gmi file with a particular format. Each entry is separated by a timestamp.
Here's a snapshot of the top of mine at the time of this writing:
# Gritty's tinylog Thoughts too small for a normal gemlog author: @gritty@gemini.smallweb.space ## 2023-08-11 00:41 UTC I wonder if 2FA / totp could theoretically be used on Gemini for logins... ## 2023-08-09 16:27 UTC I updated my gemroll list with new resources on gemini and added a HOWTO for creating your own capsule => ../HOWTO/managing-your-own-capsule.gmi Managing your own capsule
In the above:
Line 1: title of your tinylog (use a single "#")
Next, the subtitle, which I believe is optional
Then the author - I chose activitypub format, but check the RFC for options
1st entry: level 2 header (##) along with the date and time and timezone. I use UTC to give myself a bit of privacy
Everything after the date/time is the entry, up until the empty line and start of a new level 2 header and date/time. As you can see, all valid gemtext markup is allowed.
And that's it.
Tip: if SSH'ing into your capsule to update your tinylog is time consuming, you can make a CGI script to update from your phone / browser. Check the CGI section.
As mentioned in the sections above, you can use the GTL tool on the commandline to get a Twitter-like microblog interface. It's a TUI (Terminal User Inferface) that stitches together the tinylogs you subscribe to into a timeline. The reason this is necessary, is that unlike Twitter (now "X"), all tinylogs are flat files hosted on a variety of servers and not in a central database (thus, out of order). You can also edit your own tinylog and reply to others within the program.
That being said, if you're like me, you're on your phone most of the time and not logging into a console to fire up GTL. The neat thing about GTL is that you can make it output .gmi files to your capsule on a recurring basis through a cron job. This way, you can just use your phone to navigate to a normal gemini file that has your subscriptions already in a timeline format.
Once you have GTL installed somewhere and have subscribed to a few tinylogs, just edit your crontab:
> crontab -e
here's my entry:
0 */6 * * * /home/gemini/bin/gtl --mode gemini --limit 55 > /home/gemini/gemini/gemini.smallweb.space/tinylog-agg.gmi
Let's break this down:
I'm having cron run this every 6 hours
gtl is in my ~/bin/ directory
"--mode gemini" sets the output to gemtext
"--limit 55" limits to 55 entries of output
the next part redirects the output from the screen to a .gmi file in my capsule
Now all you have to do is link to this file from somewhere, like your landing page on your capsule.
And that's it, you can now read your tinylog timeline from any gemini browser.
Here's a few reasons you may want to install an aggregator on your capsule:
You don't have, or don't want to use, a browser's built-in subscribe function
You use multiple browsers with no easy way to sync subscriptions
You want to share your subscriptions
Whatever your reason, there are several to pick from. Here are a few:
Personally I use comitium since I read a gemlog a while back comparing a few (I didn't save the link), and I liked the features. Here's an excerpt from the author, @nytpu:
> There are many Gemini aggregators out there, and yet not one does everything I want it to. I want an aggregator that:
> 1. Works with Gemini and Gopher (and possibly http)
> 2. Supports Atom, RSS, and Gemini feeds (and possibly JSON feeds)
> 3. Can watch a page for changes
> 4. Simple and easy setup & usage
> All of the aggregators always have really cool and unique features, and yet all that I've found don't meet one or more of these basic (IMO) criterion. Hence, going and writing my own.
> Also, it's nice if it doesn't tie me to a specific browser/service; i.e. trivially self-hostable, ideally as CGI or static pages that can use an existing server setup rather than needing vhosting and routes.
The quickstart guide is really good and will do exactly what it says:
Food for thought on aggregators and centralized services
I've been around the web since the mid-1990's and even then I didn't understand or use CGI all that much; it was foreign to me until, ironically, 2022 - a date in which CGI should be considered the way of the dodo. But this is Gemini, and we are harkening back to the simpler times. CGI is "well documented" on the web, but it wasn't really that understandable to me and how exactly to use it on a Gemini server.
A few resources helped me get on the right track:
I highly suggest reading the SCGI article above, and then watch Tomasino's videos, they're quite helpful.
TLDR: CGI apps are scripts/program (Python, Bash, etc.) on your server that output gemtext. CGI starts and ends a program for every call, where SCGI keeps a server up and running.
I am using GmCapsule for the following examples. Please consult your server's documentation on how to setup CGI. I don't want to repeat too much of the User Manual but I'll cover some general guidelines.
GmCapsule supports virtual hosts so it expects a certain directory structure with the name of your domain. For both the static content and CGI directories, you will need to have a subdirectory with the domain you are referencing so the server knows which content goes to which site when it receives a request for a certain domain. Please see the example from my capsule below:
~gemini/ ├─ gemini/ │ ├─ gemini.smallweb.space/ │ │ ├─ <static content> │ ├─ cgi-bin/ │ │ ├─ gemini.smallweb.space/ │ │ │ ├─ <dynamic content / scripts>
As you can see above, both static and dynamic (CGI) content goes under a domain-named subdirectory (gemini.smallweb.space, in my case)
The above must be configured in GmCapsule's config file. It's a simple ini-style file that sits in the home directory of the user running the server in a file named ~/.gmcapsulerc.
Here's mine:
; By default, configuration is read from ~/.gmcapsulerc. Use the -c option ; to specify some other configuration file. [server] host = gemini.smallweb.space port = 1965 certs = /home/gemini/.certs [static] root = /home/gemini/gemini [cgi] bin_root = /home/gemini/gemini/cgi-bin
From the above, GmCapsule will look for subdirectories defined in "host" at the defined [static] and [cgi] root directories. For CGI, any executable file in the cgi-bin directory will be executed if called by the browser.
--Location of Files--
From the example, my static content is available at:
gemini://gemini.smallweb.space/
and dynamic content is located at
gemini://gemini.smallweb.space/
Notice that you don't need a subdirectory called "cgi-bin" or something similar. This has the advantage of being able to reference static content from your cgi scripts/directory, if similarly named (e.g., an /Antenna directory in both static and cgi-bin)
Next up is to see if it works (see next section)
After you have your CGI configured it's time to test it out. Note that if you're on a shared server you just need to consult the documentation or server owner for the location where user CGI scripts are served (if at all).
Let's start with a hello world bash script.
> cd cgi-bin/
> touch hello
> vim hello
And save the following:
#!/bin/sh printf "20 text/gemini; charset=utf-8\r\n" printf "# Hello World! \n" printf "## I got CGI working \n"
Finally, make it executable:
> chmod +x hello
If all goes well you can see this script (per our example) at:
gemini://gemini.smallweb.space/hello
(I don't have the example script at that location)
-- Breakdown of Script --
In the above we start with a "she-bang" and then the location to the shell that's going to execute the code below the she-bang. You can change this to /bin/bash if you wanted or even /usr/bin/python3 if you wanted to use python instead (which we use in later examples).
Per the gemini spec, a browser expects a "20" response if everything is OK and you want to display text to the user (see the link below to reference the spec). The next 2 lines simply print out gemtext, which in this case a level 1 and level 2 header.
-- Errors --
If all does not go well, check your configurations, paths, and directory and file naming. Also make sure to make your script executable.
Nearly all servers that implement CGI pass along a defined set of CGI variables that your programs can access in order to make decisions and process dynamic content. This can be a client's certificate, data from the user for a type "10" input (see Gemini spec for 10/input), or the user's current path.
The CGI spec defines particular variables but your server may have a subset of these or even new ones (such as Titan). GmCapsule defines these:
REMOTE_ADDR
QUERY_STRING
PATH_INFO
SCRIPT_NAME
SERVER_PROTOCOL
SERVER_NAME
AUTH_TYPE
TLS_CLIENT_HASH (when client certificate provided)
REMOTE_IDENT (fingerprints of client certificate and public key)
REMOTE_USER (when client certificate provided)
CONTENT_LENGTH (Titan protocol only)
CONTENT_TYPE (Titan protocol only)
TITAN_TOKEN (Tital protcol only)
-- Accessing these in your code (Python) --
To access in Python:
> os.getenv('<env_var')
for example:
> os.getenv('QUERY_STRING')
which gets the query string passed to the script.
If you want to see what these all do, just make a script to print them out:
#!/usr/bin/python3
import os
print("20 text/gemini\r")
print ("Client cert hashes:", os.getenv('REMOTE_IDENT'))
After that, it's figuring out what you want to do with all of this. For me, I made a tinylog updater (see other HOWTO).
Skyjake has examples using Titan with GmCapsule:
If you've read some of my BBS / Station posts or through my capsule, you'll know that I don't have much time to login via terminal to my capsule much, so content updates to my site should ideally be through my phone. I had no easy way to update my tinylog (even Termux is cumbersome), and so I decided to just have a cert-restricted tinylog update script for my capsule.
As this was my first real project I relied heavily on Tomasino's starter project to get me going:
The general idea is to:
Get a "10" input from the user
Paste it into tinylog.txt file as a new entry
Make sure the dates are correct
Limit access to just certain certificate(s)
Taking the lead from Tomasino, I created a "helpers.py" file that housed all my common routines that may be reused in other parts of my capsule. You can see in the file (below), but here are the stubs:
get_client_cert(cert_reqd = False, restricted_auth = False)
This checks for, and gets the client certificate
show_wrong_cert()
Error if wrong cert presented
get_query_string(unparsed = False)
simply grab the query string, if it exists
show_header_ok()
print the "20" response
temp_redirect(url)
Temporary redirect to url (code 30)
show_header_cert_required()
cert required (send code 60)
show_query_string_required(msg)
prompt for input (code 10)
get_path_info()
simply return the path environment variable
read_file(fullPath)
read in a text file (such as a tinylog)
-- The Update Script --
I won't go line-by-line on how this works here (maybe in a gemlog post), but you can see the script I use here:
-- Basic breakdown of the code --
I first import my helper library
Get all the information I need (path, query string, etc.)
see if user is on the allowed certs to use this service
Read in and display a gemini file as the "landing page"
If we have a querystring, process it depending on what it is
for the update, I scan the file until I find the "mark" and then insert the formatted date/time that tinylog requires and then paste in the status, and then write the rest of the file.
at the end it displays if the udpate was successful
And that's pretty much it
I took the "thanks" idea from @Morgan. The basic idea is that visual responses like thumbs up or down on posts or comments can skew someone's perception of an article, post, or comment. What's important is that the author gets direct feedback about their post without the readers seeing it.
Again, I'll post the code and briefly go over how it works here:
-- Overview --
I import my helpers.py file (see tinylog HOWTO for more on this file)
I grab the data from the query string, which is coded into the link on the gemlog, like this:
> gemini://gemini.smallweb.space/thanks?page=/gemlog/20230804-DSN-note.gmi&feedback=thanks
I screen for valid input
I then open a JSON file that has my data, read it in, and if the entries for that gemlog already exists, I just update the count, if not, I create a new entry with the counter at 1 for the user's selection ("thanks", "not helpful" or "no opinion")
Once done I write over the JSON file with the new data
Then I redirect to a generic page thanking for the input
What this script doesn't do is limit how many times you can click a link... so someone could spam a bunch of negative feedback if they wanted.
Currently I have to log into my capsule to view that JSON file, but I plan on making a certificate-restricted page so I can easily see the updates for myself.
Currently I'm not using Titan on my capsule but I do want to use it to make gemlog posts from my phone. This is a future project of mine. Currently, the best source to get a helloworld Titan example running on GmCapsule is from its homepage:
Example CGI script by @skyjake on adding, modifying, and deleting files via Titan:
If anyone has additional examples on using Titan that they want to share, please feel free to reach out to me (gritty@smallweb.space)
-- Response ended
-- Page fetched on Tue May 7 20:38:05 2024