Why format rewads are always 0 using Qwen3-VL-8B-Instrcut

[ZhaoyangLi-1]

I implement the Qwen3-VL model and run grpo_classification.py. However, my forward rewards are always 0, the format_reward function didn't revise.

[zhouxinwen-cpu]

Because the content also contains the question prompt ， which leads to more than one think and answer pair in the sentence to be matched .

content: Detect all objects belonging to the category 'car' in the image, and provide the bounding boxes (between 0 and 1000, integer) and confidence (between 0 and 1, with two decimal places). If no object belonging to the category 'car' in the image, return 'No Objects'. Output the thinking process in <think> </think> and final answer in <answer> </answer> tags.The output answer format should be as follows: <think> ... </think> <answer>[{'Position': [x1, y1, x2, y2], 'Confidence': number}, ...]</answer> Please strictly follow the format.
assistant
<think>

</think>

<answer>[{'Position': [427, 291, 498, 363], 'Confidence': 1}, {'Position': [429, 266, 507, 319], 'Confidence': 1}]</answer>

[rahulismyname]

Hi @ZhaoyangLi-1 , Did you made any other changes in the trainer apart from updating the `model = Qwen3VLForConditionalGeneration.from_pretrained(model, **model_init_kwargs)´ part ?

[JuzhengMiao]

I implement the Qwen3-VL model and run grpo_classification.py. However, my forward rewards are always 0, the format_reward function didn't revise.

May I ask you what kind of GPU you need to train this 8B model? I can train the Qwen2-VL-7B model using one 40 GB GPU. However, when I try to adapt the code to Qwen3-VL-8B-Instrcut, the code will stop automatically. Did you encounter this problem?