Skip to content

[Bug]: DeepSeek Models are producing Inaccurate generated Answers #225

Description

@shailajafissionqa

FloTorch Version

V2.0.3

Bug Description

DeepSeek Models are producing Inaccurate Answers and resulting the Poor Evaluation Metrics

Steps to Reproduce

  1. Login to FloTorch.ai
  2. Click on Create Project
  3. Enter the details of the Project, select the Region
  4. Select the Knowledge base type from the drop down
  5. Upload the Knowledge Base
  6. Upload the Ground Truth json file and click on Next button
  7. Select the Chucking, Embedding Model, Vector Dimension and Indexing Algorithm from the drop down and click on Next button
  8. Select the N shot Prompts, and upload the N shot prompt file
  9. Select the KNN value from the KNN dropdown
  10. Select any of the Deepseek model's from Inferencing Model dropdown
  11. Select the Inferencing Model Temperature and click on Next
  12. Select the Evaluation Service as Ragas, Embedding model and Inferencing Model from the dropdown and click on Submit button
  13. Select the Valid experiments and click on Run button
  14. Click on Confirm button

Expected Result : Generated Answers should be accurate and should match with the Ground Truth Answers.
Actual Result : Generated Answers are inaccurate and not matching with the Ground Truth Answers.

Browser and Version

Chrome version : 133.0.6943.54

Runtime Version

Version : 3.12

Error Messages

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug 🐞Something isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions