You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
inputs = [example_tensor, # Provide example tensor for input shape or...
61
+
torch_tensorrt.Input( # Specify input object with shape and dtype
62
+
min_shape=[1, 3, 224, 224],
63
+
opt_shape=[1, 3, 512, 512],
64
+
max_shape=[1, 3, 1024, 1024],
65
+
# For static size shape=[1, 3, 224, 224]
66
+
dtype=torch.half) # Datatype of input tensor. Allowed options torch.(float|half|int8|int32|bool)
67
+
],
68
+
enabled_precisions = {torch.half}, # Run with FP16)
69
+
70
+
result = trt_ts_module(input_data) # run inference
71
+
torch.jit.save(trt_ts_module, "trt_torchscript_module.ts") # save the TRT embedded Torchscript
74
72
```
75
73
76
74
> Notes on running in lower precisions:
77
75
>
78
76
> - Enabled lower precisions with compile_spec.enabled_precisions
79
77
> - The module should be left in FP32 before compilation (FP16 can support half tensor models)
80
-
> -In FP16 only input tensors by default should be FP16, other precisions use FP32. This can be overrided by setting Input::dtype
78
+
> -Provided input tensors dtype should be the same as module before compilation, regardless of `enabled_precisions`. This can be overrided by setting `Input::dtype`
0 commit comments