Hey there! I've been keeping an eye on the latest AI papers this week, and these are three of my favorites.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
This paper presents BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). The paper finds that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning.
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
The paper discusses how the CLIP model can be improved by using modern Hopfield networks to tackle the problem of explaining away. The new model, CLOOB, is shown to outperform CLIP in zero-shot transfer learning across all considered architectures and datasets.
Self-conditioned Embedding Diffusion for Text Generation
This paper proposes a new method for continuous diffusion models that overcomes the limitations of previous models. The new model, self-conditioned embedding diffusion, is more efficient on accelerator hardware and produces comparable results to standard autoregressive language models.
Hope you found this issue useful and informative. If so, please share it with your network!