Macabre

macabre : (adj) tending to produce horror in a beholder

This is an experimental compiler written in Gleam to compile Gleam source code (using the glance package) to Python.

It covers all Gleam syntax and is now self-hosted: macabre can compile its own source code to Python, and the Python-compiled compiler produces byte-identical output to the Erlang-compiled one. It also passes its own test suite (217 tests) and is idempotent — running the generated compiler on itself produces the same output every time.

My original vision was to have fun while making this a self-hosted compiler. That's done now, so I've come up with a new vision: maybe something to do with supporting other compilation targets, such as Gleam's native Erlang and Javascript environments, or even WASM or Rust.

Usage

Build the compiler (this uses the official Gleam build tool; the gleam.toml in this repo uses hex dependencies so everything works like a normal Gleam project):

gleam build

Export it as a single-file escript binary:

gleam export escript

This produces a macabre executable in the repo root. Pass it a properly structured macabre folder as the only argument:

./macabre some_package_folder

(You can also run it from source with gleam run -- some_package_folder.)

The package folder should have a structure similar to a normal Gleam project:

folder
├── macabre.toml
├── build
│   └── <generated stuff>
└─ src
    ├── <repo_name>.gleam
    ├── some_folder
    │   ├── something.gleam
    │   └── bindings.py
    ├── some_file.gleam
    └── some_bindings.py

The macabre.toml (it falls back to gleam.toml if macabre.toml doesn't exist) only supports two keys, name and dependencies. Dependencies can be git repositories, hex packages, or local paths:

name = "example"

[dependencies]
glance = "7.0.0"
macabre_stdlib = {
  git = "git@github.com:dusty-phillips/macabre_stdlib.git",
  ref = "main"
}

Git dependencies use the same syntax as the official Gleam build tool: a table with a git field for the repository URL and a ref field for the commit, branch, or tag to check out. Like the official tool, a commit sha is preferred for reproducibility. An optional path field can point at a subdirectory for monorepos that contain several packages.

Dependencies are resolved transitively: after a package is cloned, its own config file is read and any git or local packages it depends on are cloned too. Only direct dependencies need to be listed in macabre.toml.

Hex deps of a dependency are not currently followed, and macabre has no version solver. At the moment, the nascent macabre ecosystem mirrors each standard library as a macabre_* fork, so a dependency's gleam.toml lists the original hex packages (e.g. gleam_stdlib) rather than the ports macabre can compile; resolving those ranges would drag in the un-compilable originals, and guessing a matching macabre_* package for each one is a dependency-confusion and security hazard.

For now, list any package a transitive dependency needs directly in macabre.toml. Hex dependencies are version strings like "7.0.0", or version ranges like ">= 1.0.0 and < 2.0.0". If the project has a manifest.toml lockfile (generated by the official Gleam tool), macabre reads the exact versions from it. Hex packages are downloaded as tarballs from repo.hex.pm.

The compiler expects git to be installed (for git deps) and curl and tar (for hex tarballs).

Cloned dependencies are kept in build/packages and reused across runs: git clones are updated to their requested ref, and hex packages (which are versioned and immutable) are reused as-is. Only the copied sources and generated output are rebuilt each time.

Macabre copies the src/ folder of each package into the build directory and then builds all dependencies. Your source files are also copied into this folder.

Your main module will always be <repo_name>.gleam where <repo_name> is whatever you put in the name in gleam.toml.

Your files are compiled to build/dev/python. If your <repo_name>.gleam has a main function in it, then the compiler will generate a build/dev/python/__main__.py to call that function.

The project's own test/ and dev/ directories are compiled too (a dependency's are not — only its src/ is used). A test or dev module whose dependencies aren't available in the build is skipped rather than failing the whole build, so e.g. macabre's own test modules that need temporary or pprint are not compiled when macabre builds itself.

Use this command to invoke it:

python3 build/dev/python

Running tests

With the macabre_gleeunit port of gleeunit in your macabre.toml dependencies, a test suite is compiled alongside your project. Add a test entry that calls gleeunit.main():

// test/yourapp_test.gleam
import gleeunit

pub fn main() {
  gleeunit.main()
}

Compile the project and run its tests in one step:

macabre yourapp test

This builds the project (including its test/ directory) and runs the compiled <yourapp>_test module with Python. The exit status is 0 when all tests pass and 1 otherwise. Any module with a main function is written with its own if __name__ == "__main__" block, so the test entry can also be run directly:

python3 build/dev/python/yourapp_test.py

Pythonic libraries that work with macabre

The standard Gleam libraries use Erlang and JavaScript externals, which macabre can't use. A set of ported libraries with Python bindings (@external(python, ...)) live in my macabre_* repositories:

• macabre_stdlib — the Gleam stdlib, ported to Python bindings (list, dict, string, int, io, and friends)
• macabre_argv
• macabre_simplifile
• macabre_shellout
• macabre_tom
• macabre_glexer
• macabre_filepath
• macabre_splitter
• macabre_gleeunit — a port of gleeunit with a Python test runner

Macabre itself uses these forks — see macabre.toml in this repo.

Development

Run tests with gleeunit:

gleam test

Run tests in watch mode:

fd .gleam | entr gleam test

gleam.toml is the normal hex configuration, so the official Gleam toolchain (including the LSP) works on this repo. macabre.toml is the configuration macabre uses when compiling itself: it points at the macabre_* port libraries listed above.

Contributing

PRs are welcome.

The main entry point is macabre.gleam, which handles all file loading and other side effects.

The package depends on the glance AST parser, and glimpse, a package I wrote to wrap glance to support multiple inter-dependent modules. Glimpse also does typechecking, via glimpse.load_package.

The compiler is pure gleam. Most of the work happens in transformer.gleam and generator.gleam. The former converts the Gleam AST to a Python AST, the latter generates python code. There are tons of helper functions in various other files.

The Python AST is in python.gleam. This doesn't model all of python; just the subset that is needed to map Gleam expressions to.

Some tasks below are marked easy if you want to get started.

Outstanding tasks

High Pri

• non-byte-aligned bitstrings are now supported for both construction and pattern matching (verified: <<1:3, 2:5, 3:4>> packs to 12 bits and matches back correctly). Remaining gap: the bit_size/byte_size runtime helpers still assume byte alignment for non-byte-aligned values.
• (EASY) Should be putting public types, functions, and constants in __all__

Low Pri

• (EASY) maybe call ruff on the files after they are output, if they are installed. (shellout is already available)
• See if there are ways to leverage the gleam_package_interface

AI Implementation notes

Most of the initial prototype was handcoded. I ran out of time and despaired of completing the project so on a whim I handed it to Deepseek V4 flash and... it seems to have worked. It's more or less vibe coded with careful oversight so it should be treated more like a prototype than good code