video queries meet wrong result

[Meterwalon]

Hi, I run this script and try to lei it describe a video. But the result is ['!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!']
Could someone give me advices? Thanks ~

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

checkpoint = "/data/HuggingFace/MOSS-VL-Instruct-0708/"

processor = AutoProcessor.from_pretrained(checkpoint, trust_remote_code=True, force_reload=True)
model = AutoModelForCausalLM.from_pretrained(
    checkpoint, trust_remote_code=True, device_map="cuda", torch_dtype=torch.bfloat16
)

queries = [
  {
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "video",
            "video": "test.mp4"
          },
          {
            "type": "text",
            "text": "发生甚么事了"
          }
        ]
      }
    ],
    "media_kwargs": {
      "video_fps": 1.0,
      "min_frames": 1,
      "max_frames": 256
    },
    "generate_kwargs": {
      "max_new_tokens": 512,
      "temperature": 1.0,
      "top_k": 50,
      "top_p": 1.0,
      "repetition_penalty": 1.0,
      "do_sample": False,
      "vision_chunked_length": 64
    }
  }
]


with torch.no_grad():
    result = model.offline_batch_generate(processor, queries)

print([item["text"] for item in result["results"]])

[SSSSuperC]

Hi Meterwalon, thanks for reporting this issue.

We tried to reproduce it with the same inference settings, but the model produced a normal response.
Could you please provide:

1. The exact Hugging Face checkpoint link you used, including the revision if applicable.
2. The video file or a downloadable link to it. If the video cannot be shared, please provide its basic information,
   such as duration, resolution, FPS, and codec.
3. Please run the example provided in the repository using:
   • inference/run_inference.py
   • inference/video_queries.json
   • the sample video under inference/assets/videos/
     Please let us know whether the repository example also produces repeated ! characters. This will help us determine
     whether the issue is related to the checkpoint, the input video, or the inference environment.