This project proposes helm charts deployment for llm use and serving. It is based on the library and docker package vLLM.
This project is currently a work in progress (WIP) and might be subject to changes and bugs.
This project aims to be deployed on Onyxia but should be general enough that it can be adapted to to other kubernetes-based platforms.