Based on the given input shape, a random tensor according to the shape will be generated and used. Therefore, torch model requires a shape of input tensor for inference as a input of predictor.predict(). For Torch models, the shape of feature maps is unknown merely based on the given network structure, which is, however, significant parameters in latency prediction. In predictor.predict(), the allowed items of the parameter model_type include, representing model types of tensorflow, torch, onnx, nn-meter IR graph and NNI IR graph, respectively. If the predictor file doesn't exist, it will download from the Github release. nn-Meter will try to find the right predictor file in ~/.nn_meter/data. predict( model, model_type) # the resulting latency is in unit of msīy calling load_latency_predictor, user selects the target hardware and loads the corresponding predictor. To apply latency prediction for torchvision model in command line, onnx and onnx-simplifier packages are required.įrom nn_meter import load_latency_predictor predictor = load_latency_predictor( hardware_name, hardware_predictor_version) # case insensitive in backend # build your model (e.g., model instance of torch.nn.Module) model =. The string followed by -torchvision should be exactly one or more string indicating name(s) of some existing torchvision models. It should also be noted that for PyTorch model, nn-meter can only support existing models in torchvision model zoo. To predict latency for multiple models in the same model type once, user should collect all models in one folder and state the folder after - liked argument. Nn-Meter can support batch mode prediction. When the predictor version is not specified by users, nn-meter will use the latest version of the predictor. predictor-version arguments is optional. Nn-meter predict -predictor cortexA76cpu_tflite21 -predictor-version 1.0 -nn-meter-ir mobilenetv3small_0.json Nn-meter predict -predictor -nn-meter-ir #Example Usage Nn-meter predict -predictor cortexA76cpu_tflite21 -predictor-version 1.0 -torchvision mobilenet_v2 Nn-meter predict -predictor -torchvision. # for torch model from torchvision model zoo (str) Nn-meter predict -predictor cortexA76cpu_tflite21 -predictor-version 1.0 -onnx mobilenetv3small_0.onnx Nn-meter predict -predictor -onnx #Example Usage Nn-meter predict -predictor cortexA76cpu_tflite21 -predictor-version 1.0 -tensorflow mobilenetv3small_0.pb Nn-meter predict -predictor -tensorflow # Example Usage Users can get all predefined predictors and versions by running Currently, nn-Meter supports prediction on the following four configs: Predictor (device_inferenceframework) In both methods, users could appoint predictor name and version to target a specific hardware platform (device). Json file in the format of nn-Meter IR Graphĭict object following the format of nn-Meter IR Graph onnxĬheckpoint file dumped by onnx.save() or model loaded by onnx.load() pbĬheckpoint file dumped by () or onnx.save() and end with. pbĬheckpoint file dumped by tf.saved_model and end with. Testing Model TypeĬheckpoint file dumped by tf.saved_model() and end with. Here is a summary of supported inputs of the two methods.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |