You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
inputs = [example_tensor, # Provide example tensor for input shape or...
45
+
torch_tensorrt.Input( # Specify input object with shape and dtype
46
+
min_shape=[1, 3, 224, 224],
47
+
opt_shape=[1, 3, 512, 512],
48
+
max_shape=[1, 3, 1024, 1024],
49
+
# For static size shape=[1, 3, 224, 224]
50
+
dtype=torch.half) # Datatype of input tensor. Allowed options torch.(float|half|int8|int32|bool)
51
+
],
52
+
enabled_precisions = {torch.half}, # Run with FP16)
53
+
54
+
result = trt_ts_module(input_data) # run inference
55
+
torch.jit.save(trt_ts_module, "trt_torchscript_module.ts") # save the TRT embedded Torchscript
58
56
```
59
57
60
58
> Notes on running in lower precisions:
61
59
> - Enabled lower precisions with compile_spec.enabled_precisions
62
60
> - The module should be left in FP32 before compilation (FP16 can support half tensor models)
63
-
> -In FP16 only input tensors by default should be FP16, other precisions use FP32. This can be overrided by setting Input::dtype
61
+
> -Provided input tensors dtype should be the same as module before compilation, regardless of `enabled_precisions`. This can be overrided by setting `Input::dtype`
0 commit comments