-
Notifications
You must be signed in to change notification settings - Fork 382
Open
Labels
feature requestNew feature or requestNew feature or request
Description
Is your feature request related to a problem? Please describe.
We now see these warnings:
WARNING:torch_tensorrt [TensorRT Conversion Context]:Detected layernorm nodes in FP16.
WARNING:torch_tensorrt [TensorRT Conversion Context]:Running layernorm after self-attention with FP16 Reduce or Pow may cause overflow. Forcing Reduce or Pow Layers in FP32 precision, or exporting the model to use INormalizationLayer (available with ONNX opset >= 17) can help preserving accuracy.
Describe the solution you'd like
Add a lowering pass to detect this case and properly route the subgraph to the right converters
Describe alternatives you've considered
Additional context
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
feature requestNew feature or requestNew feature or request