🔧 Classification Metrics: Understanding Their Role, Usage, and Examples

🕛 Zeit seit Veröffentlichung: 53 Tage, 2 Stunden 59 Minuten
📆 Veröffentlicht am: 02.03.2025 um 20:11 Uhr
Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to

Classification Metrics: Understanding, Usage, and Examples

In machine learning, classification metrics play a crucial role in evaluating the performance of classification models. Since different classification problems have varying requirements, selecting the right metric ensures that models align with real-world needs. In this article, we’ll explore the different classification metrics, their importance, and when to use them, with examples.

1. Accuracy

Definition

Accuracy is the proportion of correctly classified instances out of the total instances:

where:

• TP (True Positives): Correctly predicted positive cases
• TN (True Negatives): Correctly predicted negative cases
• FP (False Positives): Incorrectly predicted positive cases
• FN (False Negatives): Incorrectly predicted negative cases

When to Use Accuracy

• Works well when the classes are balanced (i.e., equal number of positive and negative examples).
• Not suitable for imbalanced datasets, as it can give misleading results.

Example

If we have a spam email classifier with 1000 emails (900 non-spam, 100 spam), and the model predicts all emails as non-spam, the accuracy would be:

Even though 90% seems high, the model fails to detect any spam emails, showing that accuracy is not always reliable.

2. Precision

Definition

Precision measures how many of the predicted positive instances are actually positive:

When to Use Precision

• Useful in cases where false positives are costly (e.g., detecting fraud, medical diagnosis).
• Helps when false alarms must be minimized.

Example

In a fraud detection system, a model classifies 100 transactions as fraudulent, out of which only 70 are actually fraudulent. The precision is:

A high precision means that when the model says "fraud," it is likely correct.

3. Recall (Sensitivity or True Positive Rate)

Definition

Recall measures how many actual positive instances were correctly predicted:

When to Use Recall

• Important when false negatives are costly (e.g., detecting diseases, security threats).
• Used when missing a positive case is more dangerous than predicting extra positives.

Example

If a cancer detection model correctly identifies 80 cancerous patients out of 100 actual cases, its recall is:

A low recall would mean many cancer patients go undetected, which is dangerous.

4. F1-Score

Definition

F1-Score is the harmonic mean of precision and recall:

When to Use F1-Score

• Best when there is a trade-off between precision and recall (e.g., fraud detection, medical diagnoses).
• Helps in imbalanced datasets where accuracy is misleading.

Example

If a model has 70% precision and 80% recall, the F1-score is:

A high F1-score balances precision and recall well.

5. Specificity (True Negative Rate)

Definition

Specificity measures how well the model identifies negative instances:

When to Use Specificity

• When true negatives matter, such as in medical screening tests.
• Used in combination with recall for a full assessment of model performance.

Example

If a COVID-19 test correctly identifies 950 healthy people out of 1000 non-infected individuals, its specificity is:

6. ROC-AUC (Receiver Operating Characteristic – Area Under Curve)

Definition

ROC-AUC measures the model’s ability to distinguish between classes. It plots True Positive Rate (Recall) vs. False Positive Rate (1 - Specificity).

• AUC = 1 → Perfect classifier
• AUC = 0.5 → Random guessing
• AUC < 0.5 → Worse than random guessing

When to Use ROC-AUC

• Best for imbalanced datasets and comparing different models.
• Used in binary classification tasks like fraud detection and medical diagnoses.

Example

A fraud detection model with an AUC of 0.95 is much better than one with AUC of 0.6, as it better differentiates fraud from normal transactions.

7. Logarithmic Loss (Log Loss)

Definition

Log Loss evaluates the uncertainty of a classification by penalizing wrong predictions with confidence:

where yi is the actual class (0 or 1) and yhati is the predicted probability.

When to Use Log Loss

• Used for probabilistic models, where output is a probability instead of a binary decision.
• Suitable for multi-class classification tasks.

Example

In a weather prediction model, if the probability of rain is predicted as 0.9 but it doesn’t rain, the log loss will be high, penalizing overconfidence in a wrong prediction.

Conclusion

Choosing the right classification metric depends on the problem at hand.

...