Tutorial 1.1: Tensor Basics
This is the first part of my comprehensive PyTorch tutorial series. Full series available on GitHub.
Learning Objectives
- Understand what a Tensor is
- Master tensor creation methods
- Learn basic tensor operations
1. What is a Tensor?
A Tensor is the fundamental data structure in PyTorch, similar to NumPy's multi-dimensional arrays.
| Dimensions | Name | Example |
|---|---|---|
| 0D | Scalar | A single number 5 |
| 1D | Vector | [1, 2, 3] |
| 2D | Matrix | [[1,2], [3,4]] |
| 3D+ | Tensor | Image data [batch, channels, height, width] |
2. Tensor Types
PyTorch supports various data types:
| Data Type | CPU Tensor | GPU Tensor |
|---|---|---|
| 32-bit float (default) | torch.FloatTensor | torch.cuda.FloatTensor |
| 64-bit float | torch.DoubleTensor | torch.cuda.DoubleTensor |
| 32-bit integer | torch.IntTensor | torch.cuda.IntTensor |
| 64-bit integer | torch.LongTensor | torch.cuda.LongTensor |
3. Creating Tensors
3.1 From Python Lists
import torch
# Create from list
x = torch.tensor([1, 2, 3])
print(x) # tensor([1, 2, 3])
# Create 2x3 matrix
matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(matrix)
# tensor([[1, 2, 3],
# [4, 5, 6]])
3.2 Special Value Tensors
# All zeros
zeros = torch.zeros(2, 3) # 2 rows, 3 columns
# All ones
ones = torch.ones(2, 3)
# Identity matrix
eye = torch.eye(3) # 3x3 identity matrix
# Fill with specific value
full = torch.full((2, 3), 7) # 2x3, filled with 7
3.3 Random Tensors
# Uniform distribution [0, 1)
rand = torch.rand(2, 3)
# Standard normal distribution
randn = torch.randn(2, 3)
# Random integers in range
randint = torch.randint(0, 10, (2, 3)) # Range [0, 10)
3.4 Sequences
# arange: similar to Python range
x = torch.arange(0, 10, 2) # start, end, step
print(x) # tensor([0, 2, 4, 6, 8])
# linspace: n evenly spaced points
x = torch.linspace(0, 10, 5) # start, end, num_points
print(x) # tensor([ 0.0, 2.5, 5.0, 7.5, 10.0])
4. Tensor Attributes
x = torch.randn(3, 4)
print(x.shape) # torch.Size([3, 4])
print(x.dim()) # 2 (number of dimensions)
print(x.numel()) # 12 (total elements)
print(x.dtype) # torch.float32
print(x.device) # cpu
5. Indexing and Slicing
x = torch.tensor([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
# Single element
print(x[0, 0]) # tensor(1)
# Get a row
print(x[0]) # tensor([1, 2, 3])
# Get a column
print(x[:, 0]) # tensor([1, 4, 7])
# Slicing
print(x[0:2, 1:3]) # tensor([[2, 3], [5, 6]])
6. Shape Operations
6.1 Reshaping
x = torch.arange(12)
# view: reshape (shares memory)
y = x.view(3, 4)
# Use -1 to auto-infer dimension
z = x.view(2, -1) # becomes (2, 6)
# reshape: more flexible
w = x.reshape(4, 3)
6.2 Adding/Removing Dimensions
x = torch.tensor([1, 2, 3]) # shape: [3]
# Add dimension
y = x.unsqueeze(0) # shape: [1, 3]
y = x.unsqueeze(1) # shape: [3, 1]
# Remove dimension (size 1 only)
z = y.squeeze() # back to [3]
6.3 Transpose
x = torch.tensor([[1, 2, 3],
[4, 5, 6]]) # shape: [2, 3]
y = x.t() # shape: [3, 2]
z = x.transpose(0, 1) # same as t()
7. Practice Exercises
Exercise 1: Create a 3x3 zero matrix, a 2x4 random normal matrix, and a vector from 1 to 9.
Exercise 2: Given x = torch.arange(1, 13).view(3, 4), get the second row, last column, and bottom-right 2x2 submatrix.
Exercise 3: Transform a tensor of shape (2, 3, 4) to shape (6, 4).
Next in Series
1.2 Tensor Operations
1.3 Data Loading
2.1 Neural Network Modules- ... and 9 more tutorials!
👉 Full tutorial series: GitHub Repo
⭐ If you find this helpful, please star the repo and follow for more tutorials!