- Install pipx
- Install poetry using
pipx install poetry git clone https://github.com/Knowledge-Graph-Hub/kg-microbe.gitcd kg-microbepoetry install
poetry run kg download: This will download the resources needed for this project.
poetry run kg transform: This transforms the resources into knowledge graphs (KGs).
poetry run kg merge: This merges all transformed graphs above.
The standard merge writes data/merged/merged-kg.tar.gz, containing
merged-kg_nodes.tsv and merged-kg_edges.tsv, and writes statistics to
merged_graph_stats.yaml. To work with the TSV files directly:
tar -xzf data/merged/merged-kg.tar.gz -C data/merged
make run-summarymake run-summary also reads the archive directly, so extraction is optional.
The canonical merge.yaml build is the release graph. Other merge configs use
distinct names such as merged-kg-minimal.tar.gz,
merged-kg-no-metatraits.tar.gz, and merged-kg-prego-full.tar.gz, so running a
variant cannot masquerade as the canonical graph. The source differences are
defined in config/merge_variants.yaml.
The release workflow publishes the checksum-verified Jenkins artifact as
kg-microbe-YYYYMMDD.tar.gz. It contains the canonical
merged-kg_nodes.tsv, merged-kg_edges.tsv, and dated graph statistics. The
release also includes artifact-provenance.txt with the build URL, checksum,
and workflow revision.
- Immutable dated builds:
https://kg-hub.berkeleybop.io/kg-microbe/YYYYMMDD/kg-microbe.tar.gz - Mutable most-recent build:
https://kg-hub.berkeleybop.io/kg-microbe/current/kg-microbe.tar.gz - Latest curated GitHub release: releases/latest
Use a dated URL or a GitHub release checksum for reproducible downstream work;
the current URL changes after a successful pipeline publication.
In order to be able to make KG releases on this repository, you'll need:
- Appropriate permissions to this repository.
- A GitHub token that has permissions on this repository. This is how you set it in GitHub. Make sure your token has access to this project.
- Save this token locally in the environment variable
GH_TOKEN:GitHub CLI reads the variable directly. Do not place the token in a Git URL, repository file, or shell history.export GH_TOKEN=XXXX
It should be noted that the KG construction process, particularly the transform step involving trimming of NCBI Taxonomy for any KG and the steps involving the microbial UniProt dataset for KG-Microbe-Function and KG-Microbe-Biomedical-Function, is computationally intensive. Successful execution on a local machine may require significant memory resources (e.g., >500 GB of RAM), further details can be found in the project's code repository.
Please remember to run poetry run tox before every commit to make sure the code you commit is error-free.
If you need to use environment variables for this project, copy .env.example to .env and set the environment variables accordingly:
cp .env.example .envThen edit the .env file to configure the required environment variables for your setup.
This cookiecutter project was developed from the kg-cookiecutter template and will be kept up-to-date using cruft.