PEP-723 and Me

I didn't realize how much I wanted "Inline Script Metadata", otherwise (better?) known as PEP-723, in my life. When it first came out early last year, I was indifferent. Seemed neat, but whatever.

Since using it, however, I wish more languages supported something similar.

Taken directly from the PEP-723 abstract:

This PEP specifies a metadata format that can be embedded in single-file Python scripts to assist launchers, IDEs and other external tools which may need to interact with such scripts.

Blah, that's a bit boring. It just talks about how you can help tooling, not developers themselves (but later on in the "Rationale" section there are more examples).

Let's humanize this

Briefly, I write a python script and run it with python file.py. Great.

If there are dependencies, though, then I need to add a pyproject.toml or requirements.txt, then go through a small song and dance.

% python3 -m venv .venv
% source .venv/bin/activate 
(.venv) % pip install -r requirements.txt
(.venv) % python file.py

That annoys me so much that I have a few aliases to make my life marginally easier (naturally using uv for performance):

ve='source .venv/bin/activate'
venv='uv venv .venv; ve'
vreq='uv pip install -r requirements.txt'

This is all well and good if I have a project with lots of dependencies, lots of files, and I'll spend a few hours working on it. Alternatively, I'll setup Pants and use that to do whatever I need.

But like, for a 10 line file that needs one or two dependencies? That's lame.

A better way

Let's say I want to show someone how numpy arrays look when they're returned in a fastapi response on a newer version of Python. PEP-723 makes that trivial:

# /// script
# dependencies = [
#   "fastapi==0.121.3",
#   "numpy==2.3.5",
#   "uvicorn==0.38.0",
# ]
# requires-python = ">=3.13"
# ///

from fastapi import FastAPI
import numpy as np
import uvicorn

app = FastAPI()
data = np.array([[1, 2], [3, 4]])

@app.get("/")
async def root() -> dict[str, str]:
    return {"message": f"Hello Data: {data}"}

if __name__ == "__main__":
    uvicorn.run(app, host="127.0.0.1", port=8000)

And then we just run it like so:

% python3 file.py

  Traceback (most recent call last):
    File "/Users/sj/Developer/rpj/sureshjoshi.com/file.py", line 9, in <module>
      from fastapi import FastAPI
  ModuleNotFoundError: No module named 'fastapi'

% python3 -m pip file.py
  ERROR: unknown command "file.py"

Ah... Interesting. Neither Python nor Pip know what's going on. Python is just the runner/interpreter, so it really shouldn't know too much about installing dependencies. But, after a quick search, it still looks like pip doesn't have PEP-723 support.

Enter better tooling

There are several tools that support PEP-723, but I'll just focus on the two that I use often.

uv

With uv installed, running that script is as simple as:

% uv run file.py
  INFO:     Started server process [33415]
  INFO:     Waiting for application startup.
  INFO:     Application startup complete.
  INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

  # Checking http://127.0.0.1:8000
  {message	"Hello Data: [[1 2]\n [3 4]]"}

pex

pex is an integral part of python support for Pants, and I use it for creating single-file Python projects that are unpacked and run on-the-fly (either in Docker containers, or on my machine, or in sandboxes).

More recently, I've gone all-in on packaging pex files with Pants and the science CLI tool to merge projects with a python interpreter (lazily installed, or pre-packaged). It also comes with the ability to add busybox-esque commands - which I make heavy use of in Pantsible, to essentially re-direct calls to ansibles commands into this downloadable single-file ansible wrapper.

% pantsible
Error: Could not determine which command to run.

Ansible with an embedded Python interpreter.

Please select from the following boot commands:

ansible
ansible-config
ansible-console
ansible-doc
ansible-galaxy
ansible-inventory
ansible-playbook
ansible-pull
ansible-vault

Using pex to run a PEP-723 script , you'd use the following:

# If pex is installed globally
pex --exe file.py
# Otherwise, use `uvx` to run it
uvx pex@2.70.0 --exe file.py
  INFO:     Started server process [34684]
  INFO:     Waiting for application startup.
  INFO:     Application startup complete.
  INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

  # Checking http://127.0.0.1:8000
  {message	"Hello Data: [[1 2]\n [3 4]]"}

A few usecases

Without re-hashing the example above, where you can quickly share runnable scripts including dependencies, here are some use cases where I use PEP-723.

Infrequently run single-file scripts

I have files like these littered throughout utils repos, or on my own machine. They're used by people who work on projects, but maybe not on Python specifically. So, I might have an iOS project, where I include a python script to seed a test database, download assets locally, or setup some other preconditions that matter.

On my own machine, I quickly found an example where I was hitting a BitBucket API to get a list of repos, and then I cloned them and their issues all locally. I haven't actively used BitBucket in years, so this was part of a backup system I wrote semi-recently.

# /// script
# dependencies = [
#     "requests==2.32.3",
# ]
# requires-python = ">=3.13"
# ///

from dataclasses import dataclass
import json
from pathlib import Path

import requests

@dataclass
class Credentials:
    login: str
    password: str

# ... several support functions and classes, e.g. ...

def get_issues(self, repo_name: str) -> dict | None:
    url = f"{BitBucket.BASE_URL}/{repo_name}/issues"
    response = requests.get(url, auth=self.auth)
    return response.json() if response.ok else None

def main():
    bb = BitBucket(Credentials("MY_USERNAME", "MY_TOKEN"))
    # ... setup ...
    for workspace in workspaces:
        issues = bb.get_issues(workspace.repo)
        # ... do something ... 

if __name__ == "__main__":
    main()

Reproducible bug report tests

I tried this for the first time this past week because I never even thought to check that I could run pytest without the pytest CLI. Once I discovered that was possible, including PEP-723 scripts with bug reports seems downright obvious.

I have a full gist here of that report, but here's an example of what I mean:

# /// script
# dependencies = [
#   "pytest==9.0.1",
#   "MY-FOOLIB-EXAMPLE==99.99.99",
# ]
# requires-python = ">=3.11"
# ///

import pytest
import sys

from MY_FOOLIB_EXAMPLE import add

@pytest.mark.parametrize(
    "a, b, expected",
    [(1, 1, 2), (2, 2, 4), (-1, -1, -2)],
)
def test_addition(a: int, b: int, expected: int):
  assert add(a, b) == expected

if __name__ == "__main__":
    sys.exit(pytest.main([__file__, "-vv", f"--config-file={__file__}"]))

This snippet is runnable by anyone who wants to confirm or work on this bug, and by updating the release version, it's also a fully standalone confirmation of functionality. I would love to have reproductions like this when people leave issues on my projects.

A zany maybe bug

This bug(?) is bizarre, and feels more like a side-effect of PEP-723 - as I don't think the PEP itself has this preceise case covered.

What happens if there is a published library, whose import is shadowed by a folder in the same directory where you're running the script? For example, if I have a bunch of standalone utility scripts in the root of a project, with similar naming to an import shared in the utility scripts?

Contrived example incoming:

requests/
requests/__init__.py # <-- Contains `__version__ = "dev"`
requests/...
main.py
script.py

And in script.py, we use the normal requests to download some file, unrelated to this folder.

# /// script
# dependencies = [
#   "requests==2.32.5",
# ]
# requires-python = ">=3.13"
# ///

from requests import __version__

if __name__ == "__main__":
    print(f"Version: {__version__}")

So, running with python will work, because the module is the local folder, and everything else are just comments:

% python3 script.py
  Version: dev

Let's run with uv:

% uv run script.py
  Installed 5 packages in 5ms
  Version: dev

Interesting, so it grabbed the PEP-723 dependencies, but ran the local module.

Lastly, let's try pex:

% uvx pex@2.70.0 --exe script.py
  Version: 2.32.5

Check that out, pex specifically runs from the PEP-723 dependency and ignores the local module.

Who is right?

I neither know, nor do I care.

This was a contrived example, and while I could see some variant of it happening in the field somewhere, it feels like a very edge case. In reading through the PEP, it didn't clearly seem to be stated one way or another what happens in this case. The PEP mostly focuses on the specification of the script comments. Perhaps the earlier PEP-722 covered this, or maybe it's in a Github issue somewhere.

I also ran into this page where uv shows off some additions they've made (again, maybe captured in a later PEP or might just be uv-specific).

# /// script
# dependencies = [
#   "requests",
# ]
# [tool.uv]
# exclude-newer = "2023-10-16T00:00:00Z"
# ///

They also showed a mechanism to make Python scripts runnable by uv without calling uv run.

#!/usr/bin/env -S uv run --script

print("Hello, world!")

Neat... I guess...