Newbe Dev Stack

Answer : Following should help for L2 regularization: optimizer = torch.optim.Adam(model.parameters(), lr=1e-4, weight_decay=1e-5) This is presented in the documentation for PyTorch. Have a look at http://pytorch.org/docs/optim.html#torch.optim.Adagrad. You can add L2 loss using the weight decay parameter to the Optimization function. For L2 regularization, l2_lambda = 0.01 l2_reg = torch.tensor(0.) for param in model.parameters(): l2_reg += torch.norm(param) loss += l2_lambda * l2_reg References: https://discuss.pytorch.org/t/how-does-one-implement-weight-regularization-l1-or-l2-manually-without-optimum/7951. http://pytorch.org/docs/master/torch.html?highlight=norm#torch.norm.

Answer : PyTorch doesn't have a function to calculate the total number of parameters as Keras does, but it's possible to sum the number of elements for every parameter group: pytorch_total_params = sum(p.numel() for p in model.parameters()) If you want to calculate only the trainable parameters: pytorch_total_params = sum(p.numel() for p in model.parameters() if p.requires_grad) Answer inspired by this answer on PyTorch Forums . Note: I'm answering my own question. If anyone has a better solution, please share with us. To get the parameter count of each layer like Keras, PyTorch has model.named_paramters() that returns an iterator of both the parameter name and the parameter itself. Here is an example: from prettytable import PrettyTable def count_parameters(model): table = PrettyTable(["Modules", "Parameters"]) total_params = 0 for name, parameter in model.named_parameters(): if not parameter.requires_grad: continue...

Answer : I've found this page on their github repo, I'll just paste the content here. Recommended approach for saving a model There are two main approaches for serializing and restoring a model. The first (recommended) saves and loads only the model parameters: torch.save(the_model.state_dict(), PATH) Then later: the_model = TheModelClass(*args, **kwargs) the_model.load_state_dict(torch.load(PATH)) The second saves and loads the entire model: torch.save(the_model, PATH) Then later: the_model = torch.load(PATH) However in this case, the serialized data is bound to the specific classes and the exact directory structure used, so it can break in various ways when used in other projects, or after some serious refactors. It depends on what you want to do. Case # 1: Save the model to use it yourself for inference : You save the model, you restore it, and then you change the model to evaluation mode. This is done because you usually have BatchNorm and Dropout layers...