Skip to content

Finetuning pipeline code. #16

Description

@ashmalvayani

Hello, thank you for the great work. I have the following question.

For the fine-tuning stage where you used the following datasets (please correct me if I am wrong):
Quora-Arabic, Code-Alpaca-Arabic, Alpaca-Arabic, Evol-Instruct-Arabic, and ShareGPT (English original version 80k samples). These datasets are also open-sourced by you on your official HuggingFace repository.

Which training pipeline/ codebase you used for the Localized Supervised Fine-Tuning on the above datasets? I am using FastChat repository to reproduce the results.

Also, can you share the results of Localized Supervised Finetuning and after RLAI for ArabicMMLU and Exams? Table 9 only shows the results for base models and not sure if it's before RLAI or after RLAI. can you please provide some clarification on it?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions