Regarding training data format for gliner-bi-large-v2

[I-Am-Alam]

I have been trying to fine tune the knowledgator/gliner-bi-large-v2.0. I was following the urchade/GLiNER/examples/finetune.ipynb.

Below is the cleaned data for training:

This my count of data:

After finetuning under following configuration:

num_steps = 600
batch_size = 8
data_size = len(train_dataset)
num_batches = data_size // batch_size
num_epochs = max(1, num_steps // num_batches)

training_args = TrainingArguments(
    output_dir="/content/drive/MyDrive/model_gliners",
    learning_rate=2e-5,
    weight_decay=0.01,
    others_lr=1e-5,
    others_weight_decay=0.01,
    lr_scheduler_type="linear", #cosine
    warmup_ratio=0.1,
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=batch_size,
    focal_loss_alpha=0.75,
    focal_loss_gamma=2,
    num_train_epochs=num_epoch,
    save_steps = 50,
    save_total_limit=10,
    dataloader_num_workers = 0,
    use_cpu = False,
    report_to="none",
    )

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
    tokenizer=model.data_processor.transformer_tokenizer,
    data_collator=data_collator,
)

trainer.train()

Result

All the predictions are same label, with nearby score of 0.5

[{'start': 1, 'end': 18, 'text': 'Cristiano Ronaldo', 'label': 'PERSON', 'score': 0.5005358457565308}, {'start': 19, 'end': 22, 'text': 'dos', 'label': 'PERSON', 'score': 0.5004287362098694}]

If anyone can resolve, It will be great help to new enthusiasts.

[dlb-pub2]

Same issue here since 2 weeks ago with urchade/gliner_multiv2.1 model (actually a copy from 2024), I am trying to figure out if trainer had any changes reviewing @urchade's commits

[filipsam789]

Having the same issue. Were you able to find anything?
After more than 30 fine‑tuning runs with different hyperparameters and different mixes of my custom data and Pile‑NER, results are consistently terrible. The model assigns a single random label to almost every token. I also reran finetune.ipynb from the repository examples, with the original code and dataset, and saw the same behavior.
@urchade @Ingvarstep maybe this needs to be investigated.

[dlb-pub2]

so far overriding original Trainer.training_step is making an impact in my trainings, just commenting this that bcs i wont be able to answer in a while, you need to further investigate:
class CustomTrainer(Trainer):
def training_step(self, model, inputs, num_items_in_batch):
return super().training_step(model, inputs)

[jzamora-281992]

Same issue here but with this model: urchade/gliner_small-v2.1. Looking forward to an answer.

[filipsam789]

Looks like they resolved it here: #294. We will need to test it out

[I-Am-Alam]

@filipsam789 Sure will try it out

[I-Am-Alam]

@filipsam789

Looks like they resolved it here: #294. We will need to test it out

I have tried to fine tune the "urchade/gliner_small-v2.1" model using the finetune.ipynb notebook based on discussion in #294.

"urchade/gliner_small-v2.1" fine tune output:

But when trying to fine tune "knowledgator/gliner-bi-large-v2.0", I am getting the keyerror:
The error comes during loading model from huggingface.
Errro:

    model = GLiNER.from_pretrained("knowledgator/gliner-bi-large-v2.0")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/imran/GLiNER/venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/imran/GLiNER/venv/lib/python3.12/site-packages/huggingface_hub/hub_mixin.py", line 568, in from_pretrained
    instance = cls._from_pretrained(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/root/imran/GLiNER/venv/lib/python3.12/site-packages/gliner/model.py", line 846, in _from_pretrained
    config = GLiNERConfig(**config_)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/imran/GLiNER/venv/lib/python3.12/site-packages/gliner/config.py", line 44, in __init__
    encoder_config = CONFIG_MAPPING[encoder_config["model_type"]](**encoder_config)
                     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/imran/GLiNER/venv/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 725, in __getitem__
    raise KeyError(key)
KeyError: 'modernbert'

Looking forward for engagement on this issue.

[MohamedAmineDHIAB]

@I-Am-Alam to fix that just do:

pip install -U transformers

This did the job for me, as modernbert was recently added to Transformers and it could be that the auto configuration is not finding the key modernbert automatically so it needs an update.