-- Leo's gemini proxy

-- Connecting to perso.pw:1965...

-- Connected

-- Sending request

-- Meta line: 20 text/gemini;

Faster packages updates with OpenBSD


Author: Solène

Date: 06 August 2021

Tags: openbsd


Comment on Mastodon


Introduction


On OpenBSD, pkg_add is not the fastest package manager around but it is possible to make a simple change to make yours regular updates check faster.


Disclaimer: THIS DOES NOT WORK ON -current/development version!


Explanation


When you configure the mirror url in /etc/installurl, on release/stable installations when you use "pkg_add", some magic happens to expand the base url into full paths usable by PKG_PATH.


http://ftp.fr.openbsd.org/pub/OpenBSD

becomes

http://ftp.fr.openbsd.org/pub/OpenBSD/%v/packages-stable/%a/:http://ftp.fr.openbsd.org/pub/OpenBSD/%v/packages/%a/

The built string passed to PKG_PATH is the concatenation (joined by a ":" character) of the URL toward /packages/ and /packages-stable/ directories for your OpenBSD version and architecture.


This is why when you use "pkg_info -Q foobar" to search for a package and that a package name matches "foobar" in /packages-stable/ pkg_info will stop, it search for a result in the first URL given by PKG_PATH, when you add -a like "pkg_info -aQ foobar", it will look in all URL available in PKG_PATH.


Why we can remove /packages/


When you run your OpenBSD system freshly installed or after an upgrade, when you have your packages sets installed from the repository of your version, the files in /packages/ in the mirrors will NEVER CHANGE. When you run "pkg_add -u", it's absolutely 100% sure nothing changed in the directory /packages/, so checking for changes against them every time make no sense.


Using "pkg_add -u" with the defaults makes sense when you upgrade from a previous OpenBSD version because you need to upgrade all your packages. But then, when you look for security updates, you only need to check against /packages-stable/.


How to proceed


There are two ways, one reusing your /etc/installurl file and the other is hard coding it. Pick the one you prefer.


# reusing the content of /etc/installurl
env PKG_PATH="$(cat /etc/installurl)/%v/packages-stable/%a/" pkg_add -u

# hard coding the url
env PKG_PATH="http://ftp.fr.openbsd.org/pub/OpenBSD/%v/packages-stable/%a/" pkg_add -u

Be careful, you will certainly have a message like this:


Couldn't find updates for ImageMagick-6.9.12.2 adwaita-icon-theme-3.38.0 aom-2.0.2 argon2-20190702 aspell-0.60.6.1p10 .....

This is perfectly normal, as pkg_add didn't find the packages in /packages-stable/ it wasn't able to find the current version installed or an update, as we only want updates it's fine.


Simple benchmark


On my server running 6.9 with 438 packages I get these results.


packages-stable only: 44 seconds

all the packages: 203 seconds


I didn't measure the bandwidth usage but it should scale with the time reduction.


Conclusion


This is a very simple and reliable way to reduce the time and bandwidth required to check for updates on OpenBSD (non -current!). I wonder if this would be a good idea to provide it as a flag for pkg_add, like "only check for stable updates".

-- Response ended

-- Page fetched on Thu Mar 28 20:33:37 2024