On the other hand, BEiT is a vision model pretrained on a masked image modeling task which masks some of the image patches and requires the model to predict the masked patches (similar to the masked language modeling objective).
R
recurrent neural network (RNN)
A type of model that uses a loop over a layer to process texts.
representation learning
A subfield of machine learning which focuses on learning meaningful representations of raw data.