LLM containers

In NICHD’s Bioinformatics and Scientific Programming Core, we wanted to use multiple LLM agent tools in a secure way on multiple systems.

Specifically, we wanted to run them in isolated containers (Docker/Podman, Singularity) to prevent agents from seeing arbitrary directories on the system. See Why containers for running agents? for more on this topic.

There was nothing available for our particular requirements, so this repo now supports:

Multiple agent harnesses depending on your preferences:

Multiple container runtimes for different systems:

  • Podman images, for running containers on a local Mac

  • Singularity images, for running containers on a Linux HPC system

Tools to make it as easy as possible to authenticate and launch while still remaining secure:

  • refresh.py to refresh your credentials and optionally push them to a remote system

  • launch.py to launch a container running the LLM tool

  • build.py to build container images (only required if you want to build your own; you can use our hosted images)

Additional features

  • Handle enterprise SSL/TLS interception

  • Mount existing conda environments and prepend them to the PATH so agents can use them

When everything is set up, usage looks like this:

refresh.py        # refresh credentials if needed
launch.py codex   # run Codex in a container
launch.py claude  # or Claude Code
launch.py pi      # or pi

Or, to use on a remote machine:

# Run this on a *local* machine to refresh and push credentials to the
# right place on the remote.
refresh.py --remote $REMOTE_HOST

# then log in to the remote host and run:
launch.py codex  # or claude or pi

See https://nichd-bspc.github.io/llm for documentation.

See https://github.com/nichd-bspc/llm for code.

Note

While most of this documentation applies to any type of system, there are some NIH-specific components that are indicated by NIH-specific.

Contents

Getting started

Next steps

Details