One encoder for the structured half of AI workloads
We have been developing efficient, general-purpose models based on the idea that many AI tasks have structured outputs that can be predicted directly. When an autoregressive language model extracts JSON from text, it generates field names, punctuation, and values as a sequence of tokens. Constrained decoding can enforce valid syntax and supported schema constraints, but it does not guarantee that the extracted values or their relationships are correct. Autoregressive decoding also introduces sequential dependencies that limit parallelism within each response. GLiFormer instead predicts task-specific structures from a shared source representation and assembles the output deterministically.
What if we predict the components of a structured output in parallel and assemble the JSON deterministically? This is what we achieved with the first release of GLiFormer. In this release, we expand it to many more useful tasks: NER, classification, extracting relations and events from text, arithmetic and regression tasks, object detection in images, and sequence editing for tasks like grammar correction. If we look at task usage on OpenRouter, these tasks account for almost 30% of all requests. Imagine doing them more reliably, and 100x cheaper and faster.
How it works

Our GLiFormer models use an encoder-encoder architecture with task-specific heads. Each head represents its own combinatorial space. In relation extraction, for example, it covers all possible entity interactions rather than all possible tokens: the model learns which entities interact and aligns each pair’s representation with the correct relation label. Labels, relation types, and JSON schemas are passed at inference time, so the same model works for new labels and schemas without retraining, and several tasks can run in one pass.
Compared with the first generation of our models and GLiNER, the new omni line works directly with tokens rather than words. It no longer depends on how well the text is split into words, which is hard for languages such as Japanese.
Three sizes
We release three model sizes. GLiFormer 0.4B is text-only and based on DeBERTa: it is very parameter-efficient and works on par with multi-billion-parameter LLMs on tasks such as NER and structuring. GLiFormer 0.8B and GLiFormer 2B are based on Qwen3.5 0.8B and 2B and support image classification and object detection in addition to all text tasks. The size in the name is the size of the backbone; the task heads add roughly 0.4B parameters on top.
GLiFormer - a knowledgator Collection
What the models can do

- Named entity recognition. Give any entity types, get every matching span with its type and score.
- Text classification. Single- and multi-label classification over any label set, including several named label groups at once (e.g. sentiment and topic).
- Relation extraction. Entities and the relations between them in one pass; the head scores entity pairs, not tokens.
- Structured extraction. Fill a JSON schema with records, from flat lists to deeply nested objects (Pydantic models work too). Fields can be typed: text, numbers, dates, and categorical fields that pick from a fixed set of options.
- Calculation. A field marked as calculated is answered by a small program (copy, add, subtract, multiply, divide, …) over numbers found in the text. The model predicts the program; the arithmetic is executed exactly, so there are no hallucinated digits.
- Normalization. Numbers, dates, and times written in any form (“4.3 billion”, “March 3rd”) are read into machine values, which also power the regression-style numeric fields.
- Sequence editing. Per-token keep, delete, replace, and insert operations for grammar correction and other light rewriting, applied in a few parallel passes instead of regenerating the text.
- Text embeddings. Sentence vectors for search and similarity from the same encoder.
- Vision (0.8B and 2B). Image classification and object detection with labels given at inference time.
Training
We trained the models on millions of high-quality data points in several stages. First, we did broad pre-training where each batch contains a single task. Then we post-trained on mixed batches, so every optimization step sees several tasks at once. Surprisingly, the 2B model, after this pre-training, performs better on most tasks even when you fine-tune it on a single one. This suggests the model builds a joint space for all tasks, so gradients that change the weights have a similar effect across tasks. We see something similar in multilingual models, where English-only fine-tuning enables the same capabilities in other languages.
Evaluation
We evaluate the models across dozens of datasets and all of our tasks. Despite their size, the models are highly competitive and about 100–200x faster than LLMs. The difference is especially noticeable for JSON extraction: when the extracted JSON is larger than the input text, as it is in many examples, generation means longer inference times and higher costs.
Compared with other models
GLiFormer v1.5 against LLMs, GLiNER, and our first generation. LLM, GLiNER, and v1 numbers are from the GLiFormer paper; LLMs are prompted to extract or classify.

CrossNER is the mean over its five domains, classification over 13 datasets, relations over DocRED, CrossRE, and FewRel; the average is over these three columns. Baseline sample sizes and protocols follow their sources, so the relation column is indicative.
The 2B model is within 1.5 points of 31B-120B LLMs on CrossNER and classification, and the 0.4B model beats every LLM on relation extraction. The same Qwen3.5 backbones prompted as LLMs score 21–26 F1 on CrossNER; trained as GLiFormer encoders, they reach 66–70.

The 2B model leads on entities, classification, and flat records. The 0.4B DeBERTa model is the strongest on relations and nested JSON, showing how far a small, parameter-efficient encoder can go. Calculation is where the 2B model still lags: it picks the right program as often as the smaller models, but misreads some numbers. JSON scores are order-free and tolerant of small boundary differences; NER uses up to 500 test examples per dataset. Most evaluation datasets contributed a few dozen training examples to the final fine-tuning set; full per-dataset results are in the model cards.
Try it
The models are available on Hugging Face as knowledgator/gliformer-0.4b-v1.5, knowledgator/gliformer-0.8b-v1.5, and knowledgator/gliformer-2b-v1.5, and work with pip install gliformer.
Or alternatively, you can use our inference and fine-tuning platform:
https://platform.knowledgator.com
Read more about the architecture in our paper: https://knowledgator.com/research/gliformer