Essential NLP Tasks and Applications

Overview

In our previous lessons, we've explored the fundamental components of modern NLP systems, from text preprocessing and tokenization to transformer architectures, different generations techniques, and the evolution of language models. Now it's time to see how these foundational concepts come together in practical applications.

This lesson focuses on practical applications in NLP, covering tasks such as text classification, named entity recognition (NER), and question answering. We'll explore real-world use cases, implementation approaches, and evaluation metrics for each task, providing you with hands-on experience in building and deploying practical NLP solutions.

Learning Objectives

After completing this lesson, you will be able to:

• Identify common NLP tasks and their appropriate applications
• Implement text classification solutions for sentiment analysis and topic categorization
• Develop named entity recognition systems for information extraction
• Build question answering models for information retrieval
• Select appropriate evaluation metrics for each NLP task
• Apply best practices for model selection and deployment

Text Classification: Understanding and Categorizing Content

What is Text Classification?

Text classification is the task of assigning predefined categories to text documents. It's one of the most fundamental and widely used NLP tasks, with applications ranging from sentiment analysis and spam detection to content categorization and intent recognition.

Interactive Exploration: Text Classification in Action

Before diving into the theory, let's explore how text classification works with different preprocessing approaches:

This visualization shows how BERT's bidirectional training enables better text understanding for classification tasks.

Evaluating Text Classification Models

Selecting the right evaluation metrics is crucial for assessing text classification performance. Different metrics emphasize different aspects of performance and are appropriate for different scenarios.

Common Evaluation Metrics

Text classification models are typically evaluated using metrics like:

• Accuracy: Percentage of correctly classified instances
• Precision: Proportion of true positives among positive predictions
• Recall: Proportion of true positives identified among all actual positives
• F1 Score: Harmonic mean of precision and recall
• AUC-ROC: Area under the Receiver Operating Characteristic curve

The performance of these metrics can vary significantly between balanced and imbalanced datasets.

Which Metrics to Use When

1. Accuracy:

   • The proportion of correctly classified instances
   • Best for balanced datasets with equal importance for all classes
   • Can be misleading for imbalanced datasets

2. Precision:

   • The ratio of true positives to all predicted positives
   • Important when the cost of false positives is high
   • Example: Spam detection (misclassifying legitimate emails is costly)

3. Recall:

   • The ratio of true positives to all actual positives
   • Important when the cost of false negatives is high
   • Example: Toxic content detection (missing toxic content is costly)

4. F1 Score:

   • The harmonic mean of precision and recall
   • Balances precision and recall
   • Good for imbalanced datasets

5. AUC-ROC:

   • Area under the Receiver Operating Characteristic curve
   • Measures discrimination ability across thresholds
   • Less sensitive to class imbalance

Visualizing a confusion matrix (toy data)

Notice how the same word can represent different entity types depending on context - this is why contextual embeddings are crucial for accurate NER.

Common NER Entity Types

The standard types of named entities include:

1. Person (PER): Names of individuals
2. Organization (ORG): Companies, institutions, agencies
3. Location (LOC): Countries, cities, geographical features
4. Date/Time (DATE): Temporal expressions
5. Money (MONEY): Monetary values
6. Percentage (PERCENT): Percentage values
7. Product (PROD): Products, works of art
8. Event (EVENT): Named events like wars, sports events
9. Miscellaneous (MISC): Entities that don't fit into other categories

Domain-specific NER systems may include additional categories like genes, proteins, diseases, drugs, etc.

NER Approaches

Traditional Sequence Labeling Approaches

NER is typically framed as a sequence labeling problem, where each token is assigned a tag:

1. BIO Tagging Scheme:

   • B-X: Beginning of entity of type X
   • I-X: Inside of entity of type X
   • O: Outside of any entity

2. Traditional Models:

   • Hidden Markov Models (HMMs)
   • Conditional Random Fields (CRFs)
   • Maximum Entropy Markov Models (MEMMs)

Example: CRF-based NER

Copy
import numpy as np
from sklearn.feature_extraction import DictVectorizer
from sklearn.preprocessing import LabelEncoder
from sklearn_crfsuite import CRF
from sklearn_crfsuite.metrics import flat_classification_report

# Example data (usually this would be much larger)
train_data = [
    [("Apple", "B-ORG"), ("Inc.", "I-ORG"), ("is", "O"), ("based", "O"), ("in", "O"), ("Cupertino", "B-LOC"), ("California", "B-LOC"), (".", "O")],
    [("Tim", "B-PER"), ("Cook", "I-PER"), ("is", "O"), ("the", "O"), ("CEO", "O"), ("of", "O"), ("Apple", "B-ORG"), (".", "O")]
]

# Test data
test_data = [
    [("Microsoft", "B-ORG"), ("is", "O"), ("based", "O"), ("in", "O"), ("Redmond", "B-LOC"), (".", "O")]
]

# Feature extraction function
def word2features(sent, i):
    word = sent[i][0]
    
    # Basic features
    features = {
        'bias': 1.0,
        'word.lower()': word.lower(),
        'word[-3:]': word[-3:],
        'word[-2:]': word[-2:],
        'word.isupper()': word.isupper(),
        'word.istitle()': word.istitle(),
        'word.isdigit()': word.isdigit()
    }
    
    # Features for previous word
    if i > 0:
        word1 = sent[i-1][0]
        features.update({
            '-1:word.lower()': word1.lower(),
            '-1:word.istitle()': word1.istitle(),
            '-1:word.isupper()': word1.isupper()
        })
    else:
        features['BOS'] = True
        
    # Features for next word
    if i < len(sent)-1:
        word1 = sent[i+1][0]
        features.update({
            '+1:word.lower()': word1.lower(),
            '+1:word.istitle()': word1.istitle(),
            '+1:word.isupper()': word1.isupper()
        })
    else:
        features['EOS'] = True
        
    return features

# Extract features for a sentence
def sent2features(sent):
    return [word2features(sent, i) for i in range(len(sent))]

# Extract labels from a sentence
def sent2labels(sent):
    return [label for token, label in sent]

# Prepare data
X_train = [sent2features(s) for s in train_data]
y_train = [sent2labels(s) for s in train_data]

X_test = [sent2features(s) for s in test_data]
y_test = [sent2labels(s) for s in test_data]

# Train CRF model
crf = CRF(
    algorithm='lbfgs',
    c1=0.1,
    c2=0.1,
    max_iterations=100,
    all_possible_transitions=True
)

crf.fit(X_train, y_train)

# Make predictions
y_pred = crf.predict(X_test)

# Print predictions
for i, sentence in enumerate(test_data):
    print("Sentence:", " ".join([word for word, _ in sentence]))
    print("Predicted:", " ".join(y_pred[i]))
    print("Actual:", " ".join(y_test[i]))
    print()

Deep Learning Approaches for NER

Modern NER systems use neural network architectures:

1. Bi-LSTM-CRF:

   • Bidirectional LSTM to capture context in both directions
   • CRF layer to model label dependencies
   • Often combined with word and character embeddings

2. Transformer-based Models:

   • Fine-tuning pre-trained models like BERT, RoBERTa, XLNet
   • Token classification heads for sequence labeling
   • Contextual embeddings capture rich semantic information

Example: BERT for NER

Copy
from transformers import AutoModelForTokenClassification, AutoTokenizer
from transformers import pipeline
import torch

# Load pre-trained model and tokenizer
model_name = "dbmdz/bert-large-cased-finetuned-conll03-english"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForTokenClassification.from_pretrained(model_name)

# Create NER pipeline
ner_pipeline = pipeline('ner', model=model, tokenizer=tokenizer, aggregation_strategy="simple")

# Test on an example
text = "Apple Inc. was founded by Steve Jobs, Steve Wozniak, and Ronald Wayne in Cupertino, California."
results = ner_pipeline(text)

# Process and print results
for entity in results:
    print(f"Entity: {entity['word']}")
    print(f"Type: {entity['entity_group']}")
    print(f"Confidence: {entity['score']:.4f}")
    print(f"Start-End: {entity['start']}-{entity['end']}")
    print()

# Manual prediction with more control
def predict_entities(text):
    # Tokenize input
    inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
    
    # Get predictions
    with torch.no_grad():
        outputs = model(**inputs)
    
    # Get predicted labels
    predictions = torch.argmax(outputs.logits, dim=2)
    
    # Convert IDs to labels
    predicted_labels = [model.config.id2label[label_id.item()] for label_id in predictions[0]]
    
    # Get tokens
    tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
    
    # Align tokens with predictions
    aligned_predictions = []
    current_entity = None
    
    for token, label in zip(tokens, predicted_labels):
        # Skip special tokens
        if token in [tokenizer.cls_token, tokenizer.sep_token, tokenizer.pad_token]:
            continue
            
        # Handle subword tokens (starting with ##)
        if token.startswith("##"):
            if current_entity:
                current_entity["word"] += token[2:]
            continue
        
        # B- prefix indicates beginning of entity
        if label.startswith("B-"):
            if current_entity:
                aligned_predictions.append(current_entity)
            current_entity = {"word": token, label: label[2:]}
        
        # I- prefix indicates continuation of entity
        elif label.startswith("I-") and current_entity and current_entity["label"] == label[2:]:
            current_entity["word"] += " " + token
        
        # O indicates outside any entity
        elif label == "O":
            if current_entity:
                aligned_predictions.append(current_entity)
                current_entity = None
    
    # Add final entity if exists
    if current_entity:
        aligned_predictions.append(current_entity)
    
    return aligned_predictions

# Example usage of manual prediction
manual_predictions = predict_entities(text)
print("
Manual Prediction Results:")
for entity in manual_predictions:
    print(f"Entity: {entity['word']} - Type: {entity['label']}")

Evaluating NER Systems

Evaluation for NER requires special consideration:

Entity-Level vs. Token-Level Evaluation

1. Token-level metrics: Evaluate each token's prediction independently

   • Standard precision, recall, F1 score for each token
   • Doesn't account for entity boundaries

2. Entity-level metrics: Evaluate complete entity predictions

   • An entity is correct only if both the type and boundaries are correct
   • More reflective of real-world performance

Common NER Evaluation Metrics

Calculating Entity-level F1 Score

Copy
from seqeval.metrics import classification_report, f1_score, precision_score, recall_score

# Example: ground truth and predictions
true_tags = [
    ['O', 'B-PER', 'I-PER', 'O', 'O', 'B-ORG', 'I-ORG', 'O'],
    ['O', 'B-LOC', 'I-LOC', 'O', 'B-PER', 'I-PER', 'O', 'O']
]

pred_tags = [
    ['O', 'B-PER', 'I-PER', 'O', 'O', 'B-ORG', 'O', 'O'],     # Incomplete ORG entity
    ['O', 'B-LOC', 'I-LOC', 'O', 'B-ORG', 'I-ORG', 'O', 'O']  # Misclassified PER as ORG
]

# Calculate metrics
print(classification_report(true_tags, pred_tags))
print(f"Overall F1: {f1_score(true_tags, pred_tags):.4f}")
print(f"Precision: {precision_score(true_tags, pred_tags):.4f}")
print(f"Recall: {recall_score(true_tags, pred_tags):.4f}")

# Calculate metrics for specific entity types
print("
Metrics by entity type:")
entity_types = ['PER', 'ORG', 'LOC']
for entity_type in entity_types:
    # Filter for specific entity type using list comprehension
    true_filtered = [[t if t == f'B-{entity_type}' or t == f'I-{entity_type}' else 'O' for t in seq] for seq in true_tags]
    pred_filtered = [[t if t == f'B-{entity_type}' or t == f'I-{entity_type}' else 'O' for t in seq] for seq in pred_tags]
    
    # Calculate metrics
    entity_f1 = f1_score(true_filtered, pred_filtered)
    print(f"{entity_type} F1: {entity_f1:.4f}")

Tokenization Impact on NLP Tasks

Before moving to Question Answering, let's explore how different tokenization strategies affect NLP task performance:

Attention patterns show how the model focuses on different parts of the input when generating answers. Cross-attention is particularly important for QA tasks.

Evaluating Question Answering Systems

Evaluation Metrics for QA Systems

Calculating F1 Score for QA

Copy
def normalize_answer(s):
    """Normalize answer by removing articles, punctuation, and lowercase."""
    import re
    import string
    
    def remove_articles(text):
        regex = re.compile(r'\b(a|an|the)\b', re.UNICODE)
        return re.sub(regex, ' ', text)
    
    def white_space_fix(text):
        return ' '.join(text.split())
    
    def remove_punc(text):
        exclude = set(string.punctuation)
        return ''.join(ch for ch in text if ch not in exclude)
    
    def lower(text):
        return text.lower()
    
    return white_space_fix(remove_articles(remove_punc(lower(s))))

def compute_exact_match(prediction, ground_truth):
    """Compute exact match between prediction and ground truth."""
    return normalize_answer(prediction) == normalize_answer(ground_truth)

def compute_f1(prediction, ground_truth):
    """Compute token F1 score."""
    prediction_tokens = normalize_answer(prediction).split()
    ground_truth_tokens = normalize_answer(ground_truth).split()
    
    # If either is empty, return 0
    if len(prediction_tokens) == 0 or len(ground_truth_tokens) == 0:
        return 0
    
    # Count common tokens
    common = sum(1 for token in prediction_tokens if token in ground_truth_tokens)
    
    # If no common tokens, return 0
    if common == 0:
        return 0
    
    # Compute precision, recall, and F1
    precision = common / len(prediction_tokens)
    recall = common / len(ground_truth_tokens)
    f1 = 2 * precision * recall / (precision + recall)
    
    return f1

# Example usage
prediction = "Maida Vale, London"
ground_truth = "Maida Vale in London"

em = compute_exact_match(prediction, ground_truth)
f1 = compute_f1(prediction, ground_truth)

print(f"Prediction: {prediction}")
print(f"Ground Truth: {ground_truth}")
print(f"Exact Match: {em}")
print(f"F1 Score: {f1:.4f}")

# Multiple references
references = ["Maida Vale in London", "London", "Maida Vale, London, England"]
max_f1 = max(compute_f1(prediction, reference) for reference in references)
any_em = any(compute_exact_match(prediction, reference) for reference in references)

print(f"
Best F1 Score across references: {max_f1:.4f}")
print(f"Any Exact Match: {any_em}")

Question Answering: Finding Answers in Context

What is Question Answering?

Question Answering (QA) is the task of providing accurate answers to questions based on relevant context. Modern QA systems can extract answers from provided passages, retrieve relevant documents from a large corpus, or generate answers based on their learned knowledge.

Analogy: The Helpful Librarian

Think of QA systems as skilled librarians:

• A librarian listens to your question and understands what you're looking for
• They search through their collection to find relevant information
• They can point to a specific passage in a book or synthesize information from multiple sources
• They return a precise answer rather than simply a stack of books

Just as good librarians save time by providing direct answers rather than making users read entire books, QA systems extract or generate the specific information users need.

Conclusion: From Fundamentals to Production

The practical NLP tasks we've explored—text classification, named entity recognition, and question answering—form the foundation of many real-world NLP applications. By understanding these fundamental tasks and how to implement them effectively, you're well-positioned to build sophisticated NLP systems that solve real problems.

What You've Learned in This Course

Throughout this NLP Fundamentals course, you've built a comprehensive understanding of:

1. Text Processing Foundations: From basic preprocessing to advanced tokenization techniques
2. Representation Learning: Traditional word embeddings to modern contextual representations
3. Architecture Evolution: The journey from RNNs to the transformer revolution
4. Generation Methods: Both deterministic and probabilistic approaches to text generation
5. Model Landscape: Understanding of modern language models and their capabilities
6. Practical Applications: Core NLP tasks and how to approach them effectively

Ready for Advanced Topics?

With this foundation, you're now prepared to tackle the engineering and production aspects of NLP. If you're interested in learning how to:

• Train and fine-tune large language models from scratch
• Optimize models for production deployment through quantization and acceleration
• Build production systems like RAG applications and monitoring infrastructure
• Implement alignment techniques to ensure model safety and helpfulness

Consider continuing with our "Advanced NLP: Training & Production Systems" course, which builds directly on the concepts you've learned here.

Key Takeaways

1. Text Classification is essential for categorizing content, enabling applications from sentiment analysis to content moderation.

2. Named Entity Recognition extracts structured information from unstructured text, providing the foundation for information extraction systems.

3. Question Answering combines retrieval, extraction, and generation to provide direct answers to user queries.

4. Evaluation Matters: Selecting appropriate evaluation metrics is critical for understanding model performance in real-world scenarios.

5. Modern Approaches: Deep learning and transformer-based models have dramatically improved performance across all these tasks.

Modern Model Performance Comparison

Explore how different modern language models perform across these practical NLP tasks: