python

python/ — Conda configuration + ML environment starter

TL;DR: Conda solves the hardest problem in Python tooling — getting C-extension-heavy packages (NumPy, SciPy, PyTorch, TensorFlow) to install correctly across macOS / Linux / Apple Silicon / Intel without compiler errors. It is NOT a replacement for pip; the two coexist, with a clear division of labor. This section ships an opinionated .condarc (fast solver, broad channel coverage, no auto-activation) and a minimal ML starter env (ml-env.yml.example) — both written to be portable across machines.

This section assumes you've already installed Miniconda (recommended over the heavier Anaconda Distribution). If not: Miniconda install docs.

---

Why conda, and when NOT to use it

Situation	Use…
Pure-Python library work (FastAPI, Django, Flask, web scrapers)	python -m venv or uv — lighter than conda, no need for conda's complexity
ML/data-science with NumPy/SciPy/sklearn/matplotlib	conda — handles BLAS/LAPACK linking correctly out of the box
Deep learning (PyTorch with MPS, TF with Metal) on Apple Silicon	conda for the env + pip for the framework — see DL section below
Multiple Python versions side-by-side	conda or pyenv — both work; conda is simpler if you also need C-extension packages
One Python project, no C extensions needed	python -m venv — don't add conda just to manage a venv

The rule of thumb: if the project's requirements.txt includes numpy, scipy, scikit-learn, pandas, pytorch, or tensorflow — reach for conda. If it doesn't — venv is enough.

---

Files in this section

condarc.example

A .condarc configuration that fixes conda's three biggest default-behavior issues:

1. Slow solver → solver: libmamba (5–10x faster than the classic solver)
2. Narrow package coverage → channels: [conda-forge, defaults] with strict priority
3. Auto-activated base on every shell startup → auto_activate_base: false

Copy to ~/.condarc to apply:

cp condarc.example ~/.condarc
conda config --show     # verify settings were picked up

ml-env.yml.example

A nine-package starter env covering classical ML and data science: Python 3.11, NumPy, pandas, scikit-learn, matplotlib, seaborn, JupyterLab, Jupyter, imbalanced-learn. Designed to be portable — created with --from-history so dependencies resolve fresh on whatever machine runs it.

Create the env:

conda env create -f ml-env.yml.example
conda activate ml-env

Rename on creation (recommended — multiple projects shouldn't share one ml-env):

conda env create -f ml-env.yml.example -n project-x-ml

---

The conda + pip discipline

The most common failure mode in Python tooling is treating conda and pip as competing tools that should be picked between. They're complementary — but only if you respect the division.

Tool	Manages	Use it for
conda	Python interpreters, C-extension binaries, the env itself	python=3.11, numpy, scipy, pytorch, the env file
pip	Pure-Python packages, anything not in conda-forge	FastAPI, langchain, transformers, project-specific libs

The discipline: install via conda first, fall back to pip only for things conda doesn't have. When pip and conda both manage the same package, conda's solver gets confused and can break the env on the next conda install.

Capturing a conda + pip hybrid env

conda env export --from-history only sees conda install-ed packages. If your env has heavy pip-installed deps (common for AI/LLM work), one YAML file isn't enough — you need two artifacts:

# 1. Conda foundation (Python version + C-extension packages)
conda env export -n myenv --from-history > environment.yml

# 2. Pip-installed packages
pip freeze > requirements.txt

Recreating on a new machine:

conda env create -f environment.yml
conda activate myenv
pip install -r requirements.txt

Two files, two tools, clean separation. Don't try to fold pip packages into environment.yml via the pip: subsection unless every reader of your env also has the same conda+pip versions — it works, but it's brittle.

---

The --from-history rule (the most important conda habit)

Most conda tutorials teach this command:

conda env export > environment.yml   # WRONG (usually)

That dumps the full transitive dependency graph with exact build hashes. The resulting YAML works ONLY on the exact OS + architecture it was exported from. Try to conda env create -f it on a different machine and you'll get failures like:

ResolvePackageNotFound:
  - pytorch=2.0.1=py3.11_h7e6e0d2_0_cpu
  - numpy=1.24.3=py311h6f50973_0

The right command is:

conda env export --from-history > environment.yml

This exports ONLY the packages you explicitly installed. Conda re-resolves their dependencies on the target machine. Result: portable, short, future-proof.

Caveat: --from-history can fail with CondaValueError: Requested package 'X' is not found in 'explicit_packages'. This means the env's history log has desynced from its actually-installed packages — usually because of mixed conda+pip installs, partial removals, or solver switches mid-env-life. Fix options:

• Soft fix: conda env export --no-builds > environment.yml — full export but with build hashes stripped. Less portable than --from-history but more portable than the default.
• Hard fix: Recreate the env from scratch — conda list -n broken-env, note what's there, create a clean new env, reinstall.

---

Apple Silicon ML/DL setup

The starter env (ml-env.yml.example) deliberately excludes PyTorch and TensorFlow — both have Apple-Silicon-specific install paths that don't fit cleanly in a generic environment.yml.

PyTorch with MPS (Metal Performance Shaders)

conda activate ml-env
pip install torch torchvision torchaudio

PyTorch's official wheels on Apple Silicon include MPS support out of the box. Verify with:

import torch
print(torch.backends.mps.is_available())   # should print True

Authoritative reference: pytorch.org/get-started/locally — picks the right install command for your platform automatically.

TensorFlow with Metal

conda activate ml-env
pip install tensorflow tensorflow-metal

The tensorflow-metal plugin is what enables GPU acceleration on Apple Silicon. Verify with:

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))   # should list 1+ device

Authoritative reference: developer.apple.com/metal/tensorflow-plugin — Apple maintains the install instructions here.

---

Never pip install outside an env

This is the single most important Python discipline, and worth stating explicitly:

# DON'T do this — installs into your base Python or worse, system Python
pip install pandas

# DO this — install into an explicit env
conda activate myenv
pip install pandas

pip install with no active env can land packages in unpredictable places: system Python (breaks OS tooling on Linux), Homebrew Python (clobbers Homebrew's package management), or conda's base env (pollutes the env you should never use for project work). The fix is muscle memory: always run conda activate <env> before pip install, and accept the prompt-modified shell as your signal that an env is active.

If you ever DO need a system-wide CLI tool that ships as a Python package (e.g., uv, pre-commit, pipx itself), use pipx:

brew install pipx     # or apt install pipx on Linux
pipx install <tool>   # installs into an isolated env, exposes the CLI globally

pipx is the right answer for "I want this Python CLI tool available everywhere." Plain pip install is never the right answer outside an active env.