Introduce real mode for benchmarks running #5529
Conversation
| } | ||
| if (Config.isModeEnabled(Mode.REAL)) { | ||
| NSApplication.sharedApplication() | ||
| val frameRate = 120 // TODO: get from device |
There was a problem hiding this comment.
NSScreen.mainScreen.maximumFramesPerSecond
| val duration = start.elapsedNow() | ||
| val stats = BenchmarkResult( | ||
| name = benchmark.name, | ||
| frameBudget = (1.seconds.inWholeNanoseconds / frameRate).nanoseconds, |
There was a problem hiding this comment.
Looks like frameRate is used only to calculate frameBudget.
What do you think about passing the correct frameBudget value in parameters and remove the calculation here?
It would make the function a bit more clear - when I saw the frameRate parameter first I thought the the BenchmarkRunner tries to enforce a certain frameRate.
There was a problem hiding this comment.
I would like to avoid the same calculation for all targets. What about renaming it to deviceFrameRate?
| return@launch | ||
| val composeRoot = document.getElementById("root")!! | ||
| if (Config.isModeEnabled(Mode.REAL)) { | ||
| val frameRate = 120 // can we get this from device? |
There was a problem hiding this comment.
Unfortunately, there is no direct API.
But we can get it by running requestAnimationFrame for 1-2 seconds - https://developer.mozilla.org/en-US/docs/Web/API/Window/requestAnimationFrame and then take the average.
There was a problem hiding this comment.
I guess, we can postpone this, until we really need it
There was a problem hiding this comment.
LGTM!
I tried it on a real device and some tests, such as Visual Effects, showed slight improvements for parallel rendering.
After running tests on iOS, I would suggest several improvements:
- Write the total summary for all tests at the end, including the initial config. Currently it's hard to collect all the data because it can be interrupted with other log messages.
- Write number of total vs dropped frames for every test
- Write Duration for every text
- Add ability to configure warmup, and also it's good to have at least 2-3 seconds pause between tests and also after warmup to make easy separate tests in instruments.
- in real mode benchmarks are run phisically on the underlying target screen reporting fps metric
- add FPSInfo metric for `real` mode
pjBooms
force-pushed
the
pjBooms/benchmarks-real-mode
branch
from
fa0c3bf to
8654cee
Compare
In
realmode benchmarks are run physically on the underlying target screen reporting fps metric.Fixes CMP-9696
Testing
Manually
Release Notes
N/A