Hey there! I've been keeping an eye on the latest AI papers this week, and these are three of my favorites.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
This paper presents BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). The paper finds that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning.
→ Go to paper
CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP
The paper discusses how the CLIP model can be improved by using modern Hopfield networks to tackle the problem of explaining away. The new model, CLOOB, is shown to outperform CLIP in zero-shot transfer learning across all considered architectures and datasets.
→ Go to paper
Self-conditioned Embedding Diffusion for Text Generation
This paper proposes a new method for continuous diffusion models that overcomes the limitations of previous models. The new model, self-conditioned embedding diffusion, is more efficient on accelerator hardware and produces comparable results to standard autoregressive language models.
→ Go to paper
Hope you found this issue useful and informative. If so, please share it with your network!