imageomics/bioclip-2

Zero-Shot Image Classification • Updated 5 days ago • 5.96k • 14

Note BioCLIP 2 is a foundation model for biology organismal images. It is trained on TreeOfLife-200M on the basis of a CLIP model (ViT-14/L) pre-trained on LAION-2B. BioCLIP 2 yields state-of-the-art performance in recognizing various species. More importantly, it demonstrates emergent properties beyond species classification after extensive hierarchical contrastive training.

imageomics/bioclip

Zero-Shot Image Classification • Updated May 17, 2024 • 165k • 51

Note BioCLIP is a foundation model for the tree of life, built using CLIP architecture as a vision model for general organismal biology. It is trained on TreeOfLife-10M, our specially-created dataset covering over 450K taxa--the most biologically diverse ML-ready dataset available to date.

Note Nearly 214 million images representing 952,257 taxa across the tree of life used to train BioCLIP 2. This dataset combines images and metadata from four core biodiversity data providers: Global Biodiversity Information Facility (GBIF), Encyclopedia of Life (EOL), BIOSCAN-5M, and FathomNet to more than double the number of unique taxa covered by TreeOfLife-10M.

Note Over 10 million images covering 454 thousand taxa in the tree of life used to train BioCLIP. This dataset of images of biological organisms paired with their associated taxonomic labels expands on the foundation established by existing high-quality datasets, such as iNat21 and BIOSCAN-1M, by further incorporating newly curated images from the Encyclopedia of Life (eol.org), which supplies most of TreeOfLife-10M’s data diversity.

Note This dataset was generated alongside TreeOfLife-10M as a benchmark for BioCLIP; data (images and text) were pulled from Encyclopedia of Life (EOL) to generate a dataset consisting of rare species for zero-shot-classification and more refined image classification tasks. Here, we use "rare species" to mean species listed on The International Union for Conservation of Nature (IUCN) Red List as Near Threatened, Vulnerable, Endangered, Critically Endangered, and Extinct in the Wild.

Note IDLE-OO Camera Traps is a 5-dataset benchmark of camera trap images from the Labeled Information Library of Alexandria: Biology and Conservation (LILA BC) with a total of 2,586 images for species classification. Each of the 5 benchmarks is balanced to have the same number of images for each species within it (between 310 and 1120 images), representing between 16 and 39 species.

imageomics/bioclip-vit-b-16-inat-only

Zero-Shot Image Classification • Updated Feb 6, 2024 • 4

Note This model is trained on iNat21, different from BioCLIP which is trained on TreeOfLife-10M.