GLiFormer: A Generalist Multitask Transformer Encoder

We introduce GLiFormer, a compact schema-conditioned encoder framework that unifies multiple information-processing tasks through a shared representation and generalized anchor formulation. Runtime labels are matched against anchors representing entities, relations, classes, or records, allowing named-entity recognition, relation extraction, classification, and hierarchical text structuring to share a single source encoding.

For hierarchical structuring, GLiFormer grounds field values directly in the source, assigns them to unordered record anchors, and predicts parent–child relations before deterministically assembling nested JSON, eliminating autoregressive output generation. The framework additionally supports document inputs with spatial and optional visual features.

We initialize GLiFormer-base from a DeBERTa backbone further pretrained on 100 billion diverse tokens and train the resulting models using broad multitask training followed by task-focused post-training.

Across 26 NER and 13 classification datasets, GLiFormer demonstrates strong performance relative to its parameter count. On a 500-example multilevel structuring benchmark, GLiFormer-large achieves 91.10% F1, compared with 91.96% for GPT-5.6-luna. On a separate depth-balanced efficiency benchmark, GLiFormer-base achieves median end-to-end latencies of 69 ms on GPU and 547 ms on CPU; under explicit autoregressive throughput assumptions, the corresponding GPU workload is estimated to be up to 95.8× faster.

These results show that a single compact encoder can support diverse schema-conditioned tasks while providing accurate, source-grounded structured extraction without autoregressive generation.