dare-ties

Tools for merging pretrained Large Language Models and create Mixture of Experts (MoE) from open-source models.

End-to-end workflow to build a Case-law Citation Helper by fine-tuning two specialized LoRA “experts” and then merging them into a single adapter using TIES and DARE-style strategies.

Mergekit Assistant is a cutting-edge toolkit designed for the seamless merging of pre-trained language models. It supports an array of models, offers various merging methods, and optimizes for low-resource environments with both CPU and GPU compatibility.