#FLOPs Tool

#Definition of FLOPs

FLOPs is a common tool used to evaluate the size of neural networks. Common FLOPs tools generally calculate two types of operations, one is convolutional layer related operations, and the other is fully-connected layer related operations. Both are related to multiplication and addition.

Take the common torch.nn.Conv2d as an example, the shape of the input data is bs * c_in * w_in * h_in, the shape of the output data is bs * c_out * w_out * h_out, and the size of the convolution kernel is f. Then the FLOPs of this Conv2d are 2 * bs * f * f * c_in * c_out * w_out * h_out. Here 2 denotes half of the addition operations and half of the multiplication operations.

In the case of torch.nn.Linear, the number of neurons in the input data is c_in, and the number of neurons in the output data is c_out. In fact, this fully-connected layer can be used as a special convolutional layer, where the sizes of both input and output are 1x1 and the size of the convolutional kernel is also 1x1. Then the FLOPs of the fully connected layer is bs * (c_in * c_out + c_out). Note that the ops of multiplication and addition are not exactly the same here.

#The Use of FLOPs Tool

The current FLOPs tool supports the calculation of three types of operations, i.e., torch.nn.Conv2d, torch.nn.Linear and torch.nn.ConvTranspose2d.

In config , the main keys affecting the op counting are test_model (or model) and test_inputs. The model determines the model to be checked by the tool, while test_inputs determines the input size. The input shape can also be determined by the input parameters of -input-shape B,C,H,W .

Note that the calops tool can also support matmul by selecting fx for method. However, this method requires that the model itself can support fx.

#FLOPs of Common Classification Models (Input Size:1x3x224x224)

#Descriptions

In general, there are two cases when using calops, which may cause incorrect results or calculation errors.

1. When hook is selected for method, but the model contains matmul or other operators that are explicitly not covered.

2. When fx is selected for method, but the model itself does not support fx, or contains other operators that are explicitly not covered.