1. Loading data
First reference the necessary libraries:
import torch from torch import nn from torch.utils.data import DataLoader from torchvision import datasets from torchvision.transforms import ToTensor, Lambda, Compose import matplotlib.pyplot as plt
Like in-depth learning on other machines, Pytorch runtime requires datasets and tags.
Pytorch provides databases in many fields, such as text, voice, video, etc. This paper uses the FashionMNIST video database dataset.
Each Torch video library contains two important parameters: samples and tags.
# Download training set from open library
training_data = datasets.FashionMNIST(
root="data",
train=True,
download=True,
transform=ToTensor(),
)
# Download test datasets from open Libraries
test_data = datasets.FashionMNIST(
root="data",
train=False,
download=True,
transform=ToTensor(),
)
The following iterates over the dataset to print some data information:
batch_size = 64
# Create data loaders.
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)
for X, y in test_dataloader:
print("Shape of X [N, C, H, W]: ", X.shape)
print("Shape of y: ", y.shape, y.dtype)
break
The output is as follows: the batch size we defined is 64, and the iterator returns 64 features and labels per batch.
Shape of X [N, C, H, W]: torch.Size([64, 1, 28, 28]) Shape of y: torch.Size([64]) torch.int64
2. Creating models
Use nn.Module to create a neural network in PyTorch. Sequential networks are defined here.
A layer of neural networks is defined below, and GPU s can be used to speed up operations.
# Get cpu or gpu device for training.
device = "cuda" if torch.cuda.is_available() else "cpu"
print("Using {} device".format(device))
# Define model
class NeuralNetwork(nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__()
self.flatten = nn.Flatten()
self.linear_relu_stack = nn.Sequential(
nn.Linear(28*28, 512),
nn.ReLU(),
nn.Linear(512, 512),
nn.ReLU(),
nn.Linear(512, 10)
)
def forward(self, x):
x = self.flatten(x)
logits = self.linear_relu_stack(x)
return logits
model = NeuralNetwork().to(device)
print(model)
- torch.nn.Linear(input_data,hidden_layer) completes the linear transformation from the input layer to the hidden layer.
- torch.nn.ReLU() is the activation function;
- torch.nn.Linear(hidden_layer, output_data) completes the linear transformation from the hidden layer to the output layer.
Introduction to torch.nn.Sequential:
The torch.nn.Sequential class is a sequence container in torch.nn, in which various implementation neural networks can be nested to complete the building of the neural network model, and the parameters will be passed down automatically according to the sequence we defined.
Another way to pass parameters is orderdict,
torch.nn.Linear
The torch.nn.Linear class is used to define the linear layer of a model, that is, to complete the linear transformation between the different layers mentioned earlier.
3. torch.nn.ReLU
The torch.nn.ReLU class is a non-linear activation class and does not require an incoming parameter by default when it is defined.
This step prints the network structure as follows:
3. Adjusting Model Parameters
In order to train the model, a loss function and an optimizer are defined here.
loss_fn = nn.CrossEntropyLoss() optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
In each round of training, the model performs predictions on the training set and adjusts model parameters based on reflection propagation.
def train(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
model.train()
for batch, (X, y) in enumerate(dataloader):
X, y = X.to(device), y.to(device)
# Compute prediction error
pred = model(X)
loss = loss_fn(pred, y)
# Backpropagation
optimizer.zero_grad()
loss.backward()
optimizer.step()
if batch % 100 == 0:
loss, current = loss.item(), batch * len(X)
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
Test sets are also needed to optimize and examine the model.
def test(dataloader, model, loss_fn):
size = len(dataloader.dataset)
num_batches = len(dataloader)
model.eval()
test_loss, correct = 0, 0
with torch.no_grad():
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
test_loss += loss_fn(pred, y).item()
correct += (pred.argmax(1) == y).type(torch.float).sum().item()
test_loss /= num_batches
correct /= size
print(f"Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \n")
The training process consists of several cycles, each of which is learned through parameters to obtain a better inference model. The precision and loss functions of the iteration process are printed in the following code:
epochs = 5
for t in range(epochs):
print(f"Epoch {t+1}\n-------------------------------")
train(train_dataloader, model, loss_fn, optimizer)
test(test_dataloader, model, loss_fn)
print("Done!")
4. Save Model
The internal state dictionary is usually serialized and saved as a model.
torch.save(model.state_dict(), "model.pth")
print("Saved PyTorch Model State to model.pth")
5. Loading models
model = NeuralNetwork()
model.load_state_dict(torch.load("model.pth"))
The following uses models to perform reasoning:
classes = [
"T-shirt/top",
"Trouser",
"Pullover",
"Dress",
"Coat",
"Sandal",
"Shirt",
"Sneaker",
"Bag",
"Ankle boot",
]
model.eval()
x, y = test_data[0][0], test_data[0][1]
with torch.no_grad():
pred = model(x)
predicted, actual = classes[pred[0].argmax(0)], classes[y]
print(f'Predicted: "{predicted}", Actual: "{actual}"')
Inference:
Full source code:
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets
from torchvision.transforms import ToTensor, Lambda, Compose
import matplotlib.pyplot as plt
# Download training data from open datasets.
training_data = datasets.FashionMNIST(
root="data",
train=True,
download=True,
transform=ToTensor(),
)
# Download test data from open datasets.
test_data = datasets.FashionMNIST(
root="data",
train=False,
download=True,
transform=ToTensor(),
)
batch_size = 64
# Create data loaders.
train_dataloader = DataLoader(training_data, batch_size=batch_size)
test_dataloader = DataLoader(test_data, batch_size=batch_size)
for X, y in test_dataloader:
print("Shape of X [N, C, H, W]: ", X.shape)
print("Shape of y: ", y.shape, y.dtype)
break
# Get cpu or gpu device for training.
device = "cuda" if torch.cuda.is_available() else "cpu"
print("Using {} device".format(device))
# Define model
class NeuralNetwork(nn.Module):
def __init__(self):
super(NeuralNetwork, self).__init__()
self.flatten = nn.Flatten()
self.linear_relu_stack = nn.Sequential(
nn.Linear(28*28, 512),
nn.ReLU(),
nn.Linear(512, 512),
nn.ReLU(),
nn.Linear(512, 10)
)
def forward(self, x):
x = self.flatten(x)
logits = self.linear_relu_stack(x)
return logits
model = NeuralNetwork().to(device)
print(model)
loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)
def train(dataloader, model, loss_fn, optimizer):
size = len(dataloader.dataset)
model.train()
for batch, (X, y) in enumerate(dataloader):
X, y = X.to(device), y.to(device)
# Compute prediction error
pred = model(X)
loss = loss_fn(pred, y)
# Backpropagation
optimizer.zero_grad()
loss.backward()
optimizer.step()
if batch % 100 == 0:
loss, current = loss.item(), batch * len(X)
print(f"loss: {loss:>7f} [{current:>5d}/{size:>5d}]")
def test(dataloader, model, loss_fn):
size = len(dataloader.dataset)
num_batches = len(dataloader)
model.eval()
test_loss, correct = 0, 0
with torch.no_grad():
for X, y in dataloader:
X, y = X.to(device), y.to(device)
pred = model(X)
test_loss += loss_fn(pred, y).item()
correct += (pred.argmax(1) == y).type(torch.float).sum().item()
test_loss /= num_batches
correct /= size
print(f"Test Error: \n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \n")
epochs = 5
for t in range(epochs):
print(f"Epoch {t+1}\n-------------------------------")
train(train_dataloader, model, loss_fn, optimizer)
test(test_dataloader, model, loss_fn)
print("Done!")
torch.save(model.state_dict(), "model.pth")
print("Saved PyTorch Model State to model.pth")
model = NeuralNetwork()
model.load_state_dict(torch.load("model.pth"))
classes = [
"T-shirt/top",
"Trouser",
"Pullover",
"Dress",
"Coat",
"Sandal",
"Shirt",
"Sneaker",
"Bag",
"Ankle boot",
]
model.eval()
x, y = test_data[0][0], test_data[0][1]
with torch.no_grad():
pred = model(x)
predicted, actual = classes[pred[0].argmax(0)], classes[y]
print(f'Predicted: "{predicted}", Actual: "{actual}"')
Run result:
(pytorch) appledeMacBook-Pro:pytorch apple$ python3 learn1.py
/opt/miniconda3/envs/pytorch/lib/python3.7/site-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:180.)
return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)
Shape of X [N, C, H, W]: torch.Size([64, 1, 28, 28])
Shape of y: torch.Size([64]) torch.int64
Using cpu device
NeuralNetwork(
(flatten): Flatten(start_dim=1, end_dim=-1)
(linear_relu_stack): Sequential(
(0): Linear(in_features=784, out_features=512, bias=True)
(1): ReLU()
(2): Linear(in_features=512, out_features=512, bias=True)
(3): ReLU()
(4): Linear(in_features=512, out_features=10, bias=True)
)
)
Epoch 1
-------------------------------
loss: 2.314974 [ 0/60000]
loss: 2.301318 [ 6400/60000]
loss: 2.279395 [12800/60000]
loss: 2.260673 [19200/60000]
loss: 2.256194 [25600/60000]
loss: 2.230177 [32000/60000]
loss: 2.232735 [38400/60000]
loss: 2.197281 [44800/60000]
loss: 2.199913 [51200/60000]
loss: 2.165345 [57600/60000]
Test Error:
Accuracy: 46.0%, Avg loss: 2.161378
Epoch 2
-------------------------------
loss: 2.181149 [ 0/60000]
loss: 2.164842 [ 6400/60000]
loss: 2.110326 [12800/60000]
loss: 2.114110 [19200/60000]
loss: 2.074363 [25600/60000]
loss: 2.027263 [32000/60000]
loss: 2.043373 [38400/60000]
loss: 1.965748 [44800/60000]
loss: 1.974563 [51200/60000]
loss: 1.901750 [57600/60000]
Test Error:
Accuracy: 58.3%, Avg loss: 1.897587
Epoch 3
-------------------------------
loss: 1.938917 [ 0/60000]
loss: 1.899398 [ 6400/60000]
loss: 1.785908 [12800/60000]
loss: 1.814409 [19200/60000]
loss: 1.720331 [25600/60000]
loss: 1.681701 [32000/60000]
loss: 1.687258 [38400/60000]
loss: 1.586764 [44800/60000]
loss: 1.616351 [51200/60000]
loss: 1.508570 [57600/60000]
Test Error:
Accuracy: 60.8%, Avg loss: 1.523732
Epoch 4
-------------------------------
loss: 1.598114 [ 0/60000]
loss: 1.553433 [ 6400/60000]
loss: 1.402492 [12800/60000]
loss: 1.467502 [19200/60000]
loss: 1.366623 [25600/60000]
loss: 1.368079 [32000/60000]
loss: 1.371022 [38400/60000]
loss: 1.290068 [44800/60000]
loss: 1.333592 [51200/60000]
loss: 1.231188 [57600/60000]
Test Error:
Accuracy: 62.9%, Avg loss: 1.253851
Epoch 5
-------------------------------
loss: 1.339722 [ 0/60000]
loss: 1.310711 [ 6400/60000]
loss: 1.143826 [12800/60000]
loss: 1.242881 [19200/60000]
loss: 1.134900 [25600/60000]
loss: 1.168011 [32000/60000]
loss: 1.180709 [38400/60000]
loss: 1.111995 [44800/60000]
loss: 1.158805 [51200/60000]
loss: 1.070515 [57600/60000]
Test Error:
Accuracy: 64.0%, Avg loss: 1.088987
Done!
Saved PyTorch Model State to model.pth
Predicted: "Ankle boot", Actual: "Ankle boot"