Embedded Files
Overview
Overview
Bag of words.
Words of Embeddings.
RNN based models.
LSTM based models.
Bi-directional LSTM.
Attention based.
Transformers.
Timeline of natural language processing models
Figure. The Transformer (Muppet) family | Source: PLM Papers
Papers
Papers
Self-Supervised Learning
Self-Supervised Learning
Ryan Kiros, et al. “Skip-Thought Vectors”
Tomas Mikolov, et al. “Efficient Estimation of Word Representations in Vector Space”
Bjarke Felbo, et al. “Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm”
Encoder (Autoencoder) Models
Encoder (Autoencoder) Models
[ALBERT] Zhenzhong Lan, et al. “ALBERT: A Lite BERT for Self-supervised Learning of Language Representations”
[BERT] Jacob Devlin, et al. “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”
[DistilBERT]
[DeBERTaV3]
[Electra]
[RoBERTa] Yinhan Liu, et al. “RoBERTa: A Robustly Optimized BERT Pretraining Approach”
Decoder (Autoregressive) Models
Decoder (Autoregressive) Models
[BioGPT]
[CodeGen]
[LLaMa]
[GPT] Alec Radford, et al. “Improving Language Understanding by Generative Pre-Training”
[GPT-2]
[GPT-J]
[GPT-NEO]
[GPT-NEOX]
[NeMo Megatron-GPT]
[OPT]
[BLOOM]
[GLM]
[YaLM]
Encoder + Decoder (seq2seq) Models
Encoder + Decoder (seq2seq) Models
[T5]
[FLAN-T5]
[Code-T5]
[BART] Mike Lewis, et al. “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”
[Pegasus]
[MT5]
[UL2]
[FLAN-UL2]
[EdgeFormer]
Unfilter
Unfilter
[RoBERTa]
[XLNet]
[LongFormer]
[Reformer]
[T5 Transformer]
Bjarke Felbo, et al. “Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm”
Building Apps with LLMs
Building Apps with LLMs
Books
Books
Page updated
Google Sites
Report abuse