Skip to content

ML-Bioinfo-CEITEC/llm_and_compression

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

249 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LLM & Compression: genomic sequences

Progress report

This work was defended as an M.Sc. thesis at the Faculty of Informatics, Masaryk University, and published as a workshop paper at the GenBio workshop at ICML 2026: [OpenReview], [poster].

Interactive information-content map: https://genomeinfo.dyn.cloud.e-infra.cz/.

Tasks

  1. Test existing genomic language models.
  2. Determine their compression ratio on the human genome (or other organisms) and compare it with standard compression models (e.g., gzip, 7zip).
  3. It could also be interesting to analyze how much of the compression is attributed to the tokenizer versus the language model itself.
  4. Since, unlike natural language, the compressibility of different genomic regions will vary significantly, it might be interesting to create an "information content map" across the human genome.
  • src/llm_and_compression/arithmetic_encoder/GPTProbabilityModel.py - LLM interface responsible for token-by-token language modelling designed to be plugged into Arithmetic encoder.
  • src/llm_and_compression/arithmetic_encoder/coder.py - implementation of the encoding algorithm.

How to get started

  1. Clone the repository.
  2. Install uv package:
pip install uv
  1. Install the package in editable mode:
uv pip install -e .

Example compression notebook

An example notebook demonstrating how to use the implemented framework for compressing genomic sequences using LLMs can be found at: src/notebooks/compression_demo.ipynb.

About

Resources

License

Stars

1 star

Watchers

3 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors