📙Chapter 1: Transformer models
This chapter will teach you about Transformer models as well as the pipeline method and how to apply it to NLP tasks such as text generation, text classification, question answering, and many others. Finally, you will learn about encoders, decoders, and encoder-decoder models.
Join either live session to cover Chapter 1 with us!
Chapter 1 with Lysandre (Twitter/LinkedIn): Wednesday, June 16th (8:00-9:00 UTC)
Chapter 1 with Sylvain (Twitter/LinkedIn): Thursday, June 17th (18:00-19:00 UTC)
Check out the course introduction!
CSO Thomas Wolf will be speaking at Ray Summit (Sept. 30 - Oct. 1). Learn more about scalable machine learning and Python from Thomas and other industry experts. You can register here.
♾️ Getting started with Hugging Face Infinity
Hugging Face Infinity is our new containerized solution to deploy fully optimized inference pipelines for state-of-the-art Transformer models into your own production environment 🔥. Watch Philipp Schmid optimize a Sentence-Transformer to achieve 1.Xms latency with Hugging Face Infinity on GPU!
If you are interested in trying out Infinity, sign-up for a trial at hf.co/infinity-trial
In this notebook by Research Engineer Sylvain Gugger, you'll learn how to fine-tune one of the 🤗 Transformers model on a language modeling tasks. We cover two types of language modeling tasks:
You can easily load and preprocess the dataset for each one of those tasks and use the Trainer API to fine-tune a model on it.
🚨New pre-print on avoiding dataset biases
Led by Research Scientist Victor Sanh, we show a method to train a model to ignore dataset biases without explicitly identifying/modeling them by learning from the errors of a “dumb” model.
Special shout to collaborators: Thomas Wolf, Yonatan Belinkov, and Sasha Rush.
Our work builds on top of previous works (1, 2, 3) which explicitly construct a biased model (e.g. a hypothesis-only model for NLI) and use it to improve the robustness of the main model via product-of-experts.
The assumption of knowledge of the underlying dataset bias is quite restrictive: finding biases in datasets can be costly and time-consuming.
We show that we don't need such an explicit formulation of the dataset biases.
More results & analysis in the paper and see Victor's full tweet here!
Our team is working hard alongside the community to add incredible NLP datasets to our 🤗 Datasets library. Here's a spotlight of a few of our recent additions.
CommonGen
One of the key metrics for success in creating world-changing text generation is a model's ability to use "common sense" 🧠
A group of researchers from University of Southern California have introduced a new benchmark called CommonGen, already included in the 🤗 Datasets library, that measures a model's ability to connect a set of related concepts using commonsense knowledge.
Amazon Reviews
Amazon Reviews is one of the largest canonical datasets for sentiment analysis in NLP. The dataset contains the text for over 130 million customer reviews of Amazon products with annotated data including the 1-5 star rating, the number of “helpful” votes, and the product ID and category ⭐️⭐️⭐️⭐️⭐️
If you’re interested in enormous datasets with continuous sentiment ratings and plentiful metadata, check this one out 🔢
XNLI Training Set
We already had the canonical test and validation sets, and now we’ve added the recently translated training sets as well. XNLI is a cross-lingual version of the MultiNLI dataset with translations in 15 languages 🌏
This data can be used to evaluate cross-lingual models or even train a multilingual zero-shot text classifier like this one. Check it out!
Quail QA: Multi-domain Reading Comprehension
This brand new dataset includes passages, questions, and answers for reading comprehension. The domains include news 📰, blogs 🖥, fiction 📖, and user stories 📝 with hundreds of examples in each category. Domain diversity mitigates the issue of possible overlap between training and test data of large pre-trained models, which the current SOTA systems are based on. Try it out!
Original Chinese Natural Language Inference (OCNLI)
OCNLI is a corpus for Chinese Natural Language Inference, collected following closely the procedures of MNLI, but with enhanced strategies aiming for more challenging inference pairs. Unlike some other non-English NLI datasets, this OCNLI does not use either human- or machine- translated examples – all are original Chinese texts. The dataset is included as part of the clue dataset. Check out the paper here.