I recently wanted to try out (aider)[https://aider.chat], and ran into a problem that I continually run into: how do I install one-off, experimental software into my environment, without being a nix master, or spending an extreme amount of time on every new package.

In this case, it is a new python package available in pip. It isn’t available in nixpkgs, and isn’t nixified. pip2nix doesn’t work on arm64 Macs (bug here)[https://github.com/nix-community/pip2nix/issues/88], so I can’t use that to try and create a flake that works with this package. It isn’t using poetry, so poetry2nix is out.

How are you dealing with this problem? Are you all experts in nix, and writing flakes for every piece of software that you want to play around with? Do you have a “dirty” part of your environment that you install this kind of stuff into? (I looked into using pipx to install this, however that (also has a bug)[https://github.com/NixOS/nixpkgs/issues/171429] in nixpkgs.)

Thoughts?

  • thejevansEnglish
    2•1 year ago

    I’m certainly still learning and not an expert. For my own python projects, I’ve converted them to poetry and made a flake that I can easily tweak for development environments. For adding something like paul-gauthier/aider, there is fairly decent documentation on how to make a python nix package with examples here. It will probably be a bit slow your first time because you will have to follow along the documentation, but the second and third time you do this, it will be fairly quick.

    EDIT: specifically, you want to look at buildPythonPackage

    • @SpottOPEnglish
      1•1 year ago

      Maybe my issue here is that the whole “buildPythonPackage” ecosystem of nix is fundamentally at odds with the python ecosystem of late, and the rest of the nix ecosystem is better.

      Python has been moving toward a more “define your exact dependencies” model lately (see pipenv, poetry, piptools, etc.), while nixpkgs expects that python package dependencies can be replaced with whatever version is in the nixpkgs repo (essentially the opposite view). Nix also assumes that tests will catch any problem caused by a version mismatch, which means defensive dependency pinning won’t work. (A dependency that is pinned to be under some assumed api breaking change is trying to prevent code from breaking before it does. If the dependency changes behavior in a way that changes results but still works, it will break the code in ways that tests don’t necessarily catch)

      This is wrong more often than it is right. Partly because of the pain of the arm64 transition on Mac, and the Mac vs linux issue, and the fact that data science based python packages frequently have dependencies on C libraries (which just multiplies the dependency space), I have yet to have a python package build correctly the first time from nix.

      But this is partly beside the point. If I have to learn a new way to install packages for every language, then I kinda have to be knowledgable in nix and the language to install a throwaway piece of software in whatever ecosystem I want to try out. I can probably make the buildPythonPackage thing work for python, because I know python… but if I have to do that for a nodejs project? or a ruby project? When I just want to use the output of the package? That is a ton of work to use a nix system… is that really the only way?

      • thejevansEnglish
        4•1 year ago

        How about this. I just tested this out on my NixOS machine.

        nix-shell -p poetry
poetry new aider-playground
cd aider-playground
poetry add aider-chat
# I needed to better specify the python version in pyproject.toml here for scipy, but the error message told me exactly how to specify it.
poetry shell
aider demo.py

        This should give you quick access to aider to play around with it.

  • @[email protected]English
    1•1 year ago

    This is one of those times I would throw my hands up and look at virtualization instead. Containerization with docker/podman is an option. I recall some time ago there was a utility that sort of embeded different distributions within your own, and you could run commands natively on the host machine but in the environment of the VM. I might also be confusing that with VanillaOS or BlendOS features that have that built in to the OS. APX is VanillaOS’s package manager that might be another option, although its getting messy at this point.