Mô tả

Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.

In this course, we cover everything you need to get started with building cutting-edge performance NLP applications using transformer models like Google AI's BERT, or Facebook AI's DPR.

We cover several key NLP frameworks including:

  • HuggingFace's Transformers

  • TensorFlow 2

  • PyTorch

  • spaCy

  • NLTK

  • Flair

And learn how to apply transformers to some of the most popular NLP use-cases:

  • Language classification/sentiment analysis

  • Named entity recognition (NER)

  • Question and Answering

  • Similarity/comparative learning

Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.

All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:

  • History of NLP and where transformers come from

  • Common preprocessing techniques for NLP

  • The theory behind transformers

  • How to fine-tune transformers

We cover all this and more, I look forward to seeing you in the course!

Bạn sẽ học được gì

Industry standard NLP using transformer models

Build full-stack question-answering transformer models

Perform sentiment analysis with transformers models in PyTorch and TensorFlow

Advanced search technologies like Elasticsearch and Facebook AI Similarity Search (FAISS)

Create fine-tuned transformers models for specialized use-cases

Measure performance of language models using advanced metrics like ROUGE

Vector building techniques like BM25 or dense passage retrievers (DPR)

An overview of recent developments in NLP

Understand attention and other key components of transformers

Learn about key transformers models such as BERT

Preprocess text data for NLP

Named entity recognition (NER) using spaCy and transformers

Fine-tune language classification models

Yêu cầu

  • Knowledge of Python
  • Experience in data science a plus
  • Experience in NLP a plus

Nội dung khoá học

14 sections

Introduction

8 lectures
Introduction
02:25
Course Overview
06:33
Hello! and Further Resources
02:44
Environment Setup
06:13
Alternative Local Setup
01:02
Alternative Colab Setup
01:52
CUDA Setup
03:16
Apple Silicon Setup
00:37

NLP and Transformers

10 lectures
The Three Eras of AI
06:58
Pros and Cons of Neural AI
04:44
Word Vectors
04:52
Recurrent Neural Networks
04:01
Long Short-Term Memory
01:56
Encoder-Decoder Attention
05:38
Self-Attention
04:03
Multi-head Attention
02:41
Positional Encoding
08:58
Transformer Heads
09:40

Preprocessing for NLP

9 lectures
Stopwords
05:39
Tokens Introduction
06:53
Model-Specific Special Tokens
05:44
Stemming
05:27
Lemmatization
03:40
Unicode Normalization - Canonical and Compatibility Equivalence
05:22
Unicode Normalization - Composition and Decomposition
04:32
Unicode Normalization - NFD and NFC
05:09
Unicode Normalization - NFKD and NFKC
06:51

Attention

6 lectures
Attention Introduction
02:16
Alignment With Dot-Product
11:59
Dot-Product Attention
04:25
Self Attention
05:39
Bidirectional Attention
02:23
Multi-head and Scaled Dot-Product Attention
06:04

Language Classification

5 lectures
Introduction to Sentiment Analysis
09:17
Prebuilt Flair Models
08:22
Introduction to Sentiment Models With Transformers
06:12
Tokenization And Special Tokens For BERT
06:45
Making Predictions
06:10

[Project] Sentiment Model With TensorFlow and Transformers

7 lectures
Project Overview
02:56
Getting the Data (Kaggle API)
07:17
Preprocessing
13:42
Building a Dataset
05:31
Dataset Shuffle, Batch, Split, and Save
06:42
Build and Save
13:00
Loading and Prediction
10:39

Long Text Classification With BERT

2 lectures
Classification of Long Text Using Windows
21:33
Window Method in PyTorch
16:12

Named Entity Recognition (NER)

10 lectures
Introduction to spaCy
07:46
Extracting Entities
06:22
NER Walkthrough
1 question
Authenticating With The Reddit API
08:03
Pulling Data With The Reddit API
12:44
Extracting ORGs From Reddit Data
06:24
Getting Entity Frequency
03:53
Entity Blacklist
03:41
NER With Sentiment
19:20
NER With roBERTa
08:36

Question and Answering

7 lectures
Open Domain and Reading Comprehension
03:18
Retrievers, Readers, and Generators
06:25
Intro to SQuAD 2.0
05:38
Processing SQuAD Training Data
06:53
(Optional) Processing SQuAD Training Data with Match-Case
04:20
Processing SQuAD Dev Data
1 question
Our First Q&A Model
08:36

Metrics For Language

6 lectures
Q&A Performance With Exact Match (EM)
04:51
Introducing the ROUGE Metric
04:20
ROUGE in Python
04:36
Applying ROUGE to Q&A
08:15
Recall, Precision and F1
04:54
Longest Common Subsequence (LCS)
03:10

Reader-Retriever QA With Haystack

14 lectures
Intro to Retriever-Reader and Haystack
03:37
What is Elasticsearch?
06:45
Elasticsearch Setup (Windows)
01:46
Elasticsearch Setup (Linux)
01:55
Elasticsearch in Haystack
08:23
Sparse Retrievers
03:59
Cleaning the Index
05:17
Implementing a BM25 Retriever
02:17
What is FAISS?
09:26
Further Materials for Faiss
00:57
FAISS in Haystack
13:46
What is DPR?
08:23
The DPR Architecture
02:17
Retriever-Reader Stack
12:03

[Project] Open-Domain QA

3 lectures
ODQA Stack Structure
01:42
Creating the Database
07:26
Building the Haystack Pipeline
07:59

Similarity

6 lectures
Introduction to Similarity
06:27
Extracting The Last Hidden State Tensor
05:42
Sentence Vectors With Mean Pooling
06:57
Using Cosine Similarity
05:01
Similarity With Sentence-Transformers
03:38
Further Learning
00:06

Pre-Training Transformer Models

15 lectures
Visual Guide to BERT Pretraining
09:45
Introduction to BERT For Pretraining Code
05:21
BERT Pretraining - Masked-Language Modeling (MLM)
08:40
BERT Pretraining - Next Sentence Prediction (NSP)
06:20
The Logic of MLM
12:47
Pre-training with MLM - Data Preparation
12:44
Pre-training with MLM - Training
13:40
Pre-training with MLM - Training with Trainer
03:14
The Logic of NSP
04:39
Pre-training with NSP - Data Preparation
13:32
Pre-training with NSP - DataLoader
03:37
Setup the NSP Pre-training Training Loop
1 question
The Logic of MLM and NSP
05:35
Pre-training with MLM and NSP - Data Preparation
09:00
Setup DataLoader and Model Pre-training For MLM and NSP
3 questions

Đánh giá của học viên

Chưa có đánh giá
Course Rating
5
0%
4
0%
3
0%
2
0%
1
0%

Bình luận khách hàng

Viết Bình Luận

Bạn đánh giá khoá học này thế nào?

image

Đăng ký get khoá học Udemy - Unica - Gitiho giá chỉ 50k!

Get khoá học giá rẻ ngay trước khi bị fix.