Issue 8 March 30th 2021

News

Hugging Face Raises Series B!

📣 We are so excited to announce our $40M series B led by Lee Fixel at Addition with participation from Lux Capital, A.Capital Ventures, and betaworks!

Thank you to all our open source contributors, pull requesters, issue openers, notebook creators, model architects, tweeting supporters & community members all over the world 🌎!

We couldn't do what we do & be where we are - in a field dominated by big tech - without you! 🙏🏻

Check us out on TechCrunch and VentureBeat!

🧠Train Transformers faster with Hugging Face in Amazon SageMaker 🎉

We partnered with Amazon SageMaker to enable faster training of Transformers in your AWS cloud! 🔥

Head to our blog for walkthroughs, documentation and sample notebooks showing you how to use the new Hugging Face Deep Learning Containers (DLCs) with the SageMaker Python SDK to train models with PyTorch and TensorFlow, and
🏎Data Parallelism
🚀Model Parallelism
💸Spot Instances
📈Custom Metrics

🤗 Transformers v4.4 gets 5 new models!

1️⃣ 🌐 Multilingual w/ M2M100 and 2️⃣ mBART-50
3️⃣ 🎤 Speech w/ Wav2Vec2-XLSR
4️⃣ Quantization w/ I-BERT
5️⃣ 🥇 SOTA NLU w/ DeBERTa-v2

Not to mention:
⚙️ TF models support XLA & AMP
➡️ Trainer supports SageMaker model Parallelism

New v1.5 of 🤗 Datasets is out 💥

💾 Tokenized datasets now 4x (!) smaller
🔥 Simpler from_csv/json/text
📈 New datasets: Common Voice, SST, e.g.:
🎤 Common Voice: speech data in 60 languages!
👩🏻‍🎤 fashionMNIST for CV.

🚀 Shout out to over 800+ users who are already sharing and hosting their datasets on the Hub!

🐍 Any dataset can be loaded with one line of python.

👉🏻 Check out the full list here
📝 Learn how to add yours here

Dark Mode is Here!

🌗 🌘 🌑 Get your equipment because it's getting very dark in here... The long-awaited dark mode is now available on Hugging Face 🚀

To try it out, activate in your user settings/theme (you have to be a registered user).

Facebook AI's Wav2Vec2 is now available in Transformers!

Not only for English but for 53 Languages 🤯

Check out the tutorials:
👉🏻 Train Wav2Vec2 on TIMIT
👉🏻 Train XLSR-Wav2Vec2 on Common Voice

For more interaction, take a look at these colab tutorials:
👉🏻 Train Wav2Vec2 on TIMIT
👉🏻 Train XLSR-Wav2Vec2 on Common Voice

AutoNLP is Here!

🔎 Looking for a sneak peek of AutoNLP in action?

Check out this exclusive preview video by Abhishek Thakur that shows just how easy it is to train models using AutoNLP!

FairScale Support Release

FairScale just released support for ZeRO-DP3 and ZeRO-offload (to make large model fine-tuning easier), and you can already start playing with it in 🤗 Transformers!

This is still highly experimental so expect a few (maybe a lot) of rough edges. The PR gives a few examples, refer to the master documentation for more information.

For more information on what ZeRO-DP and ZeRO-offload are, you're in luck! Sylvain Gugger gave a talk about the topic at PyDataMTL.

The new SOTA is in Transformers!

DeBERTa-v2 beats the human baseline on SuperGLUE and up to a crazy 91.7% dev accuracy on MNLI task. It even beats T5 while 10x smaller!

DeBERTa-v2 was contributed by Pengcheng He from Microsoft Research

Try it directly on the hub or in 🤗 Transformers by installing from source!

DeBERTa will be available from pypi/anaconda as early as v4.4.0 is out!

📣 Announcing a new Ray + Hugging Face integration!

RAG is a new NLP model that uses external documents to augment its knowledge. The RAG model by Aleksandra Piktus, Patrick Lewis, and more Facebook AI colleagues leverages external knowledge sources like Wikipedia to have direct and dynamic access to information at inference time.

🚀 This integration with RAG:

Speeds up retrieval calls by 2x
Improves the scalability of fine-tuning

📝 Check out our newest guest post by Amog Kamsetty and the Ray team on training a Retrieval Augmented Generation Model.

🧙‍♂️Train a Text Classifier with Unlabeled Data

We’ve added a script to 🤗 Transformers that allows you to train a text classifier with nothing but a set of specified class names and some unlabeled data!

The script generates proxy-labels on your data from our zero-shot classification pipeline and performs knowledge distillation by training a smaller student model 💪

The result is an efficient classifier that speeds up inference by 100x or more compared to zero-shot classification 🚀

📝 Script

📕 Walkthrough colab notebook

🚀 Model Hub Highlights 🚀

Translate text to or between 50 languages with mBART-50 from Facebook AI!
🇺🇳 One-to-Many model: translate from English to 49 other languages
↔️ Many-to-Many model: translation between any pair of 50 languages

Check out all the mBART-50 models

I-BERT
🔥Brought to you by UC Berkeley, I-BERT is the first quantized model in 🤗Model Hub! Everything is integer in I-BERT. It brings you 4x speed-up with TensorRT!!

Community

📚 Hugging Face Reading Group

The Hugging Face Reading Group is back!

We frequently need to manipulate extremely long sequences for application such as document summarization and also in modalities outside of NLP. But how do you efficiently process sequences of over 64K tokens with Transformers?

We took a deep dive into long-range transformers to understand the different approaches, their assumptions, as well as their strengths and weaknesses.

To learn how you can use it for your use-cases, check out the paper summaries and our discussion!

Collaboration between Hugging Face and Tensorflow's implementation of RAG

An incredible collaboration between Hugging Face and Tensorflow's implementation of RAG by (atthachat Chatpatanasiri (Jung)!

ICYMI: RAG is an AI prototype that can read articles to give answers to any questions!

With appropriate training data like ELI5, it can even give free-form answers with reasonable arguments!

🗺 Languages at Hugging Face

Join our new "Languages at Hugging Face" initiative! You can discuss language-specific tools and resources, connect with local groups, and share initiatives to make #NLP easier in your own language!

🔥 Hot Hugging Face Forum Topics 🔥

Check out our joint blog post with Google which showcases the integration and examples.