You may want to understand HuggingFace Transformers

7 min readOct 19, 2024

A graceful practical introduction to huggingface transfromers.

Baimam Boukar | HuggingFace Transformers 101

In 2024, we find ourselves in the midst of the “AI scramble” era. In this dynamic, Hugging Face is changing the ML landscape by democratizing access to state-of-the-art machine learning models. This article aims to provide a practical introduction to Hugging Face Transformers, a technology that’s reshaping the AI landscape.

Fun Fact! The name “Hugging Face” was inspired by the hugging face emoji (🤗), chosen to convey a caring and friendly image to teenagers using their original chatbot.

Wait, HuggingFace ?

Hugging Face is the “brainchild” of three French entrepreneurs: Clément Delangue, Julien Chaumond, and Thomas Wolf. They had a Eureka time: building a platform that not only showcases state-of-the-art ML but also makes it accessible and enjoyable for everyone.

Interestingly, Hugging Face wasn’t always the AI platform we know today. Back in 2016, it started as a chatbot designed to entertain and support teenagers. The name, derived from the hugging face emoji (🤗), was chosen to appear caring and friendly to its young users. Read more here…

Hugging Face wants to become your artificial BFF | TechCrunch

Meet Hugging Face, a new chatbot app for bored teenagers. The New York-based startup is creating a fun and emotional…

techcrunch.com

Fast forward to 2024, and Hugging Face has transformed into the go-to platform for all things ML. It hosts datasets, models, spaces, and daily trending ML papers. As of October 2024, the platform boasts more than 750,000 models, 300,000 datasets, and 150,000 spaces . It has become the ideal platform for AI engineers, practitioners, and enthusiasts, hosting latest models from tech giants like Google, IBM, Intel, Microsoft, Apple, Meta, and NVIDIA.

Hugging Face secured $235 million in Series D funding in August 2023, raising its valuation to $4.5 billion.

Not bad for a former emotional support chatbot 😅. Enough, let’s talk about HuggingFace Transformers now.

Understanding Hugging Face Transformers

Hugging Face Transformers is an open-source library that provides state-of-the-art machine learning models for various tasks. At its core, it is built on the transformer architecture.

The library offers a large collection of pre-trained models for diverse tasks, from text classification and named entity recognition to question answering and text generation. These models are not just powerful; they are also incredibly easy to use.

From a technical perspective, Transformers leverage the power of attention mechanisms, allowing models to focus on different parts of the input when producing output. This approach has proven particularly effective in capturing long-range dependencies in data, a challenge that previous architectures struggled with. Read more about the original paper here…

Attention is all you need | Proceedings of the 31st International Conference on Neural Information…

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include…

dl.acm.org

Another Fact! The Transformers library supports over 100,000 pre-trained models as of October 2024 with tasks covered in 100+ languages.

Advantages of Hugging Face Transformers

Hugging Face Transformers offer several important advantages:

Reduced infrastructure costs. Leveraging pre-trained models decreases computational resource requirements.
Lower carbon footprint. Fine-tuning existing models reduces energy consumption. The amount of carbon footprint will be far more significant if training a model from scratch.
Time savings. Building on existing models accelerates your work and enables faster development and deployment.
Resource conservation.

One of the most powerful features of Transformers is their interoperability. They seamlessly integrate with popular deep learning frameworks like PyTorch, TensorFlow, and JAX. While PyTorch is the stable choice for all models, this flexibility allows developers to work within their preferred ecosystem.

Fun Fact! Hugging Face Transformers can be used for tasks beyond text processing, including image classification, speech recognition, and even protein folding prediction.

Fine-tuning with Transformers

Transformers excel at fine-tuning. This technique allows practitioners to use pre-existing pre-trained models and adapt them to specific tasks or domains. Fine-tuning involves taking a pre-trained model and further training it on a smaller, task-specific dataset. This process allows the model to adapt its general knowledge to the nuances of a particular application. For instance, a foundational model pre-trained on a large corpus of English text could be fine-tuned for sentiment analysis in product reviews.

This practice has many advantages, among which:

1. Reduced training time and computational resources
2. Improved performance on tasks with limited labeled data
3. Better generalization to new, unseen examples

Fun Fact! HuggingFace Transformers has over 134,000 stars on GitHub. Only Google Tensorflow has more in the category of AI and ML libraries.

Now, when to Use Hugging Face Transformers

Hugging Face Transformers are particularly useful in scenarios where you need to leverage state-of-the-art machine learning without the resources to train models from scratch. I am sure it may help you in these common use cases:

1. Rapid prototyping of AI-powered applications in general
2. Building chatbots or conversational AI systems
3. Implementing text classification for sentiment analysis or content moderation
4. Creating translation and summarization systems
5. Developing question-answering systems for customer support or information retrieval
6. Generating human-like text for content creation or data augmentation

Another Fact! “IBM has contributed over 200 open models and datasets on Hugging Face, including the recent release of the Geospatial Foundation Model in partnership with NASA. It is the largest geospatial foundation model on Hugging Face and the first-ever open-source AI foundation model built in collaboration with NASA.”

Getting Started with Hugging Face Transformers

Getting started with Hugging Face is really straightforward. As we mentioned earlier, it is very easy to use.

1. Start by creating a free account on the Hugging Face website.
2. Install the Transformers library pip install transformers
3. Start experimenting with pre-trained models.

That’s all you need for the setup! It is advised to use Google Collab for this task.

Now, let’s get our hands dirty by trying out transformers. Again, I will repeat it. The beautiful thing with transformers is their ease of getting started with. The pipeline() function is the easiest way to use transformers.

Let’s dive into these hands-on examples with Hugging Face Transformers.

Question Answering

Let’s start with the question-answering transformer.

from transformers import pipeline

# Question Answering example
qa_model = pipeline("question-answering")
context = "Hugging Face was founded in 2016 as a chatbot for teenagers."
question = "When was Hugging Face founded?"
answer = qa_model(question=question, context=context)
print(f"Answer: {answer['answer']}")
print(f"Confidence Score: {answer['score']:.2f}")

In this example, we’re using the `pipeline()` function. By specifying “question-answering” as the task, we are using a pre-trained model optimized for answering questions based on a given context. The model processes this information and returns an answer along with a confidence score. The output might look something like this.

Answer: 2016
Confidence Score: 0.98

2. Sentiment Analysis

Now, let’s explore the NLP capabilities with sentiment analysis

from transformers import pipeline

classifier = pipeline("sentiment-analysis")
inputs = ["The Beautiful Thing With Learning Is That No Once Can Take It Away From You!",
          "HuggingFace Is cool", "I want problems, always 🥷"]
output = classifier(inputs)

for index, result in enumerate(output):
    print(f"🟢 {inputs[index]} | label => {result['label']} | score => {round(result['score'], 4)}")Here, we’re using a specific model for English to French translation. The `pipeline()` function allows us to specify the model we want to use, giving us fine-grained control over our NLP tasks.

3. Audio Classification

Finally, let’s look at audio classification.

from transformers import pipeline

# Audio Classification Task
audio_classifier = pipeline("audio-classification")
audio_file = "reference.mp3"
results = audio_classifier(audio_file)
print(f"Audio classification results for {audio_file}:")
for result in results[:3]:
    print(f"- {result['label']} (Confidence: {result['score']:.2f})")

With just these few lines of code, we have leveraged state-of-the-art models for question answering, sentiment analysis, and audio classification. Pretty cool! If you wish to explore more and practice with more example, I recommend the official documentation.

Pipelines for inference

We're on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

Putting it All Together

Hugging Face is revolutionizing the way we interact with AI models. It is arguably the fastest way to run inference on existing state-of-the-art models. By democratizing access to state-of-the-art ML models, Hugging Face Transformers are empowering AI researchers and curious beginners to leverage or build on top of various foundation models.

Thanks for reading ❤️‍🔥✨

I hope this introduction to Hugging Face and Transformers is cool. My name is Baimam Boukar. I’m a software engineer, and I enjoy sharing my knowledge through blog posts. I write about Machine Learning, Cloud, Research, and Software Development. Let’s stay connected

I am on HuggingFace
Find me on Github
Let’s connect on LinkedIn