How to use cpan or pip packages on Nix and NixOS

Author: Solène

Date: 18 September 2021

Tags: nixos nix perl python

Comment on Mastodon

Introduction

When using Nix/NixOS and requiring some development libraries available in pip (for python) or cpan (for perl) but not available as package, it can be extremely complicated to get those on your system because the usual way won't work.

Nix-shell

The command nix-shell will be our friend here, we will define a new environment in which we will have to create the package for the libraries we need. If you really think this library is useful, it may be time to contribute to nixpkgs so everyone can enjoy it :)

The simple way to invoke nix-shell is to use packages, for example the command ` nix-shell -p python38Packages.pyyaml` will give you access to the python library pyyaml for Python 3.8 as long as you run python from this current shell.

The same way for Perl, we can start a shell with some packages available for databases access, multiples packages can be passed to "nix-shell -p" like this: `nix-shell -p perl532Packages.DBI perl532Packages.DBDSQLite`.

Defining a nix-shell

Reading the explanations found on a blog and help received on Mastodon, I've been able to understand how to use a simple nix-shell definition file to declare new cpan or pip packages.

Mattia Gheda's blog: Introduction to nix-shell

Mastodon toot from @cryptix@social.coop how to declare a python package on the fly

What we want is to create a file that will define the state of the shell, it will contain new packages needed but also the list of packages.

Skeleton

Create a file with the nix extension (or really, whatever the file name you want), special file name "shell.nix" will be automatically picked up when using "nix-shell" instead of passing the file name as parameter.

with (import <nixpkgs> {});
let
    # we will declare new packages here
in
mkShell {
  buildInputs = [ ]; # we will declare package list here
}

Now we will see how to declare a python or perl library.

Python

For python, we need to know the package name on pypi.org and its version. Reusing the previous template, the code would look like this for the package Crossplane

with (import <nixpkgs> {}).pkgs;
let
  crossplane = python37.pkgs.buildPythonPackage rec {
    pname = "crossplane";
    version = "0.5.7";
    src = python37.pkgs.fetchPypi {
      inherit pname version;
      sha256 = "a3d3ee1776bcccebf7a58cefeb365775374ab38bd544408117717ccd9f264f60";
    };

    meta = { };
  };


in
mkShell {
  buildInputs = [ crossplane python37 ];
}

If you need another library, replace crossplane variable name but also pname value by the new name, don't forget to update that name in buildInputs at the end of the file. Use the correct version value too.

There are two references to python37 here, this implies we need python 3.7, adapt to the version you want.

The only tricky part is the sha256 value, the only way I found to find it easily is the following.

1. declare the package with a random sha256 value (like echo hello | sha256)

2. run nix-shell on the file, see it complaining about the wrong checksum

3. get the url of the file, download it and run sha256 on it

4. update the file with the new value

Perl

For perl, it is required to use a script available in the official git repository when packages are made. We will only download the latest checkout because it's quite huge.

In this example I will generate a package for Data::Traverse.

$ git clone --depth 1 https://github.com/nixos/nixpkgs
$ cd nixpkgs/maintainers/scripts
$ nix-shell -p perlPackages.{CPANPLUS,perl,GetoptLongDescriptive,LogLog4perl,Readonly}
$ ./nix-generate-from-cpan.pl Data::Traverse
attribute name: DataTraverse
module: Data::Traverse
version: 0.03
package: Data-Traverse-0.03.tar.gz (Data-Traverse-0.03, DataTraverse)
path: authors/id/F/FR/FRIEDO
downloaded to: /home/solene/.cpanplus/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz
sha-256: dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f
unpacked to: /home/solene/.cpanplus/5.34.0/build/EB15LXwI8e/Data-Traverse-0.03
runtime deps:
build deps:
description: Unknown
license: unknown
License 'unknown' is ambiguous, please verify
RSS feed: https://metacpan.org/feed/distribution/Data-Traverse
===
  DataTraverse = buildPerlPackage {
    pname = "Data-Traverse";
    version = "0.03";
    src = fetchurl {
      url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz";
      sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f";
    };
    meta = {
    };
  };

We will only reuse the part after the ===, this is nix code that defines a package named DataTraverse.

The shell definition will look like this:

with (import <nixpkgs> {});
let
  DataTraverse = buildPerlPackage {
    pname = "Data-Traverse";
    version = "0.03";
    src = fetchurl {
      url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz";
      sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f";
    };
    meta = { };
  };

in
mkShell {
  buildInputs = [ DataTraverse perl ];
  # putting perl here is only required when not using NixOS, this tell you want Nix perl binary
}

Then, run "nix-shell myfile.nix" and run you perl script using Data::Traverse, it should work!

Conclusion

Using not packaged libraries is not that bad once you understand the logic of declaring it properly as a new package that you keep locally and then hook it to your current shell session.

Finding the syntax, the logic and the method when you are not a Nix guru made me despair. I've been struggling a lot with this, trying to install from cpan or pip (even if it wouldn't work after next update of my system and I didn't even got it to work.

-- Response ended

-- Page fetched on Fri Apr 26 14:08:20 2024