---
# System prepended metadata

title: Comparison of NLP models

---

# Comparison of NLP models

| | CNN-DailyMail | Extra Remarks |
|------|-----|---|
| Brio | 47.78/23.55/44.57 ||
| Pegasus | 44.16/21.56/41.30 ||
| BigBird | 43.48/21.11/40.47 | benchmark not really suited |
| t5 | 43.52/21.55/40.69||
| Longformer Encoder-Decoder | ||

### Pegasus

Pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective. In PEGASUS, important sentences are removed/masked from an input document and are generated together as one output sequence from the remaining sentences, similar to an extractive summary.

### BigBird

Sparse-attention based transformer, which gives it the capability to handle much longer sequences. This drastically improves performance on NLP tasks like question answering and text summarization of longer documents.
