Migrating prosody internal storage to SQLite on OpenBSD

Author: Solène

Date: 18 August 2023

Tags: prosody xmpp openbsd

Comment on Mastodon

Introduction

As some may know, I'm an XMPP user, an instant messaging protocol which used to be known as Jabber. My server is running Prosody XMPP server on OpenBSD. Recently, I got more users on my server, and I wanted to improve performance a bit by switching from the internal storage to SQLite.

Actually, prosody comes with a tool to switch from a storage to another, but I found the documentation lacking and on OpenBSD the migration tool isn't packaged (yet?).

The switch to SQLite drastically reduced prosody CPU usage on my small server, and went pain free.

Prosody documentation: Prosody migrator

Setup

For the migration to be done, you will need a few prerequisites:

know your current storage, which is "internal" by default

know the future storage you want to use

know where prosody stores its files

the migration tool

On OpenBSD, the migration tool can be retrieved by downloading the sources of prosody. If you have the ports tree available, just run `make extract` in `net/prosody` and cd into the newly extracted directory. The directory path can be retrieved using `make show=WRKSRC`.

The migration tool can be found in the subdirectory `tools/migration` of the sources, the program `gmake` is required to build the program (it's only replacing a few variables in it, so no worry about a complex setup).

In the migration directory, run `gmake`, you will obtain the migration tool `prosody-migrator.install` which is the program you will run for the migration to happen.

Prepare the configuration file

In the migration directory, you will find a file `migrator.cfg.lua.install`, this is a configuration file describing your current prosody deployment and what you want with the migration, it defaults to a conversion from "internal" to "sqlite" which is what most users will want in my opinion.

Make sure the variable `data_path` in the file refers to `/var/prosody` which is the default directory on OpenBSD, and check the hosts in the "input" part which describe the current storage. By default, the new storage will be in `/var/prosody/prosody.sqlite`.

Run the tool

Once you have the migrator and its configuration file, it's super easy to proceed:

Stop the prosody server with `rcctl stop prosody`

Modify `/etc/prosody/prosody.cfg.lua` to use the sql driver instead of internal

Backup `/var/prosody` in case something is going bad

Run the migration with `lua53 prosody-migrator.install --config ./migrator.cfg.lua.install`

Verify the file `/var/prosody/prosody.sqlite` exists and isn't empty

Chown `/var/prosody/prosody.sqlite` to `_prosody:_prosody`

Start the prosody server with `rcctl start prosody`, check everything is working fine

If you had an error at the migration step, check the logs carefully to check if you missed something, a bad path, maybe.

Conclusion

Prosody comes with a migration tool to switch from a storage backend to another, that's very handy when you didn't think about scaling the system correctly at first.

The migrator can also be used to migrate from the server ejabberd to prosody.

Thanks prx for your report about some missing steps!