Arm backend: Add FP16 tests of models (mv3, ic3)

[martinlsm]

Add testing of the following models executed in FP16:

• MobileNetV3
• InceptionV3

This patch verifies that the Arm backend is able to lower full models in FP16 to valid TOSA, and execute them with acceptable numerical accuracy.

cc @digantdesai @SS-JIA @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17586

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures

As of commit 1376dd0 with merge base bd6a75d ():

NEW FAILURES - The following jobs have failed:

• pull / test-openvino-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 19831e850380fed51bbdb22ba04d523e6d18b9d03a072ed9905e7a4b9f4e9eed /exec failed with exit code 1
• pull / test-samsung-models-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 46cc4b43ccf4f52c7f5c835c609e057b3edc1de08ad6d70f98d653e0a91a2c9c /exec failed with exit code 1
• pull / test-samsung-quantmodels-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 404f33817ebc057ebfd48851971055f94dc080aaaf5755e0f001499df7bb7c2b /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-deps (test_pytest_models_tosa) / linux-job (gh)
  RuntimeError: Command docker exec -t fdcd550fb1409c07f9b688389999f57051fc8ba75802fe3f42ec78ab4db837d8 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[martinlsm]

@pytorchbot label ciflow/trunk

[martinlsm]

@pytorchbot label "partner: arm"

[martinlsm]

@pytorchbot label "release notes: none"

[Copilot]

Pull request overview

Adds FP16 end-to-end model tests for the Arm backend to validate FP16 lowering to TOSA and ensure outputs remain within acceptable numeric error.

Changes:

• Add an FP16 variant of the MobileNetV3 (small) TOSA FP pipeline test.
• Add an FP16 variant of the InceptionV3 TOSA FP pipeline test.
• Configure looser absolute tolerances for FP16 output comparisons.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
backends/arm/test/models/test_mobilenet_v3_arm.py	Adds a new slow TOSA FP16 model test using an FP16 MobileNetV3 module + FP16 inputs.
backends/arm/test/models/test_inception_v3_arm.py	Adds a new slow TOSA FP16 model test using an FP16 InceptionV3 module + FP16 inputs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

mv3_fp16 is instantiated and converted at import time, which forces a second model construction + weight load even when the FP16 test isn’t selected. Consider creating the FP16 model inside test_mv3_tosa_FP_fp16() (or via a cached pytest fixture) to reduce test import time and memory usage.

[Copilot]

This FP16 test relaxes atol but leaves rtol at the default (1e-3). For reduced-precision model tests elsewhere (e.g. bf16), both tolerances are typically relaxed; consider specifying an appropriate rtol here as well to avoid overly strict relative comparisons and potential flakiness.

Suggested change

	atol=2e-2,
	atol=2e-2,
	rtol=1e-2,

[Copilot]

ic3_fp16 is instantiated and converted at import time, which forces a second model construction + weight load even when the FP16 test isn’t selected. Consider creating the FP16 model inside test_ic3_tosa_FP_fp16() (or via a cached pytest fixture) to reduce test import time and memory usage.

[Copilot]

This FP16 test relaxes atol but leaves rtol at the default (1e-3). For reduced-precision model tests elsewhere (e.g. bf16), both tolerances are typically relaxed; consider specifying an appropriate rtol here as well to avoid overly strict relative comparisons and potential flakiness.

Suggested change

	atol=1e-2,
	atol=1e-2,
	rtol=1e-2,

[zingo]

OK to merge when atol/rtol is numped to tests pass.
e.g. This seem to get

FAILED backends/arm/test/models/test_inception_v3_arm.py::test_ic3_tosa_FP_fp16 - AssertionError: Output 0 does not match reference output.
	Given atol: 0.01, rtol: 0.001.
	Output tensor shape: torch.Size([1, 1000]), dtype: torch.float16
	Difference: max: 0.03515625, abs: 0.03515625, mean abs error: 0.005782970428466797.
	-- Model vs. Reference --
	 Numel: 1000, 1000
	Median: -0.06884765625, -0.06573486328125
	  Mean: -0.02605916690826416, -0.026028093814849853
	   Max: 2.703125, 2.67578125
	   Min: -2.623046875, -2.62109375
= 1 failed, 91 passed, 3 skipped, 7 xfailed, 952 warnings in 963.15s (0:16:03) =