PyTorch

Summary

python base로 딥러닝 연구 시 가장 많이 사용되는 라이브러리.

Model Define

---

Multi column

Hint

import torch
import torch.nn as nn
# create a linear regression model in pyTorch
class LinearRegressionModel(nn.Module):             # (1)
	def __init__(self):   # (2)
		super().__init__()            # (3)
		
		# initialize model parameters
		self.weights = nn.Parameter(torch.randn(1, requires_grad=True, dtype=torch.float))          # (4)
		self.bias = nn.Parameter(torch.randn(1, requires_grad=True, dtype=torch.float))          # (5)
		
	# foward() defines the computation in the model
	def forward(self, x: torch.Tensor) -> torch.Tensor:             # (6)
	   return self.weights * x + self.bias                     # (7)
	   

Check

class LinearRegressionModel(nn.Module):             # (1)

여기에서는 torch.nn.Module(subclass in torch) 를 상속.
→ 내부에 구현되어 있는, 아래와 같은 함수들을 별도 구현 없이 그대로 사용.

• 파라미터 추적
• GPU 이동: (.to method)
• train / eval mode 전환 : .train(), .eval()
• 모델 저장: .save_dict()

def __init__(self):   # (2)
	super().__init__()            # (3)
	
	# initialize model parameters
	self.weights = nn.Parameter(torch.randn(1, requires_grad=True, dtype=torch.float))          # (4)
	self.bias = nn.Parameter(torch.randn(1, requires_grad=True, dtype=torch.float))          # (5)

python의 constructor.
여기서 model의 stucture 정의하고, 초기화도 여기서 함.

• “requires_grad = True”는 torch에서 gradient 추적을 허용.
• torch.autograd()를 사용하기 위한 옵션.

# foward() defines the computation in the model
def forward(self, x) -> torch.Tensor:             # (6)
   return self.weights * x + self.bias                     # (7)

실제 model의 foward pass를 계산하는 함수라, 모든 subclass of nn.Module들은 이 함수를 override 해야 함.

Tip

위에서 construct 된 linear model은 torch.nn 안에 구현되어 있다.

self.linear_layer = nn.Linear(in_features=input_dim, out_features=output_dim)

이렇게 대체 가능.

Training-Testing Loop Code

---

Check

### Training Loop
# put the model in training mode(train state is default)
model.train()
 
# pass the data through the model for a number of epochs
for epoch in range(epochs):
	# 1. forward pass on train data
	y_pred = model.forward(X_train)
	
	# 2. calculate the loss
	loss = loss_fn(y_pred, y_true)
	
	# 3. zero the gradients of the optimizer(default: accumulated)
	optimizer.zero_grad()
	
	# 4. backward pass (gradient calculation)
	loss.backward()
	
	#5. progress the optimier (GD)
	optimizer.step()
	
	### Testing Looop
	# put the model to evaluation mode(train state is default)
	model.eval()
	
	# turn on inference mode context message 
	with torch.inference_mode():
		# 1. do the forward pass
		test_pred = model(X_test)
		
		# 2. calculate the loss
		test_loss = loss_fn(test_pred, y_test)
		
	# print out
	print(f"Epoch:{epoch} | Train loss: {loss: .4f} | Test loss: {test_loss: .4f})