Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/09e0f48c0a40df19c5f039d4a2fd99a3835a2902 (+ 351 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6-llvm-arm-master-O2-build/22/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0022 (llvmorg-17-init-01856-g09e0f48c0a40) 2023-02-07T20:19:29 0:00:00
Previous 0021 (llvmorg-17-init-01505-gb83caa32dc86) 2023-02-04T06:00:22 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 12 34
Performance Improvements 40 58
Added Tests 0 2
Unchanged Tests 692 650
Total Tests 744

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 1.76% 7275.000 7403.000 - 0.37% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (unexplained diff vs previous) 200.00% 3.000 9.000 - -18.18% -
433.milc (unexplained diff vs previous) (percentage) 166.67% 0.030 0.080 - -11.11% -
433.milc (max) 80.00% 5.000 9.000 - -10.00% -
471.omnetpp (unexplained diff vs previous) 70.67% 75.000 128.000 - 2460.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 67.96% 1.030 1.730 - 2371.43% -
433.milc (max) (percentage) 60.00% 0.050 0.080 - -11.11% -
471.omnetpp (max) 34.02% 97.000 130.000 - 293.94% -
471.omnetpp (max) (percentage) 31.34% 1.340 1.760 - 282.61% -
471.omnetpp 9.52% 42.000 46.000 - 43.75% -
471.omnetpp (avg) 9.52% 42.000 46.000 - 43.75% -
471.omnetpp (avg) (percentage) 8.62% 0.580 0.630 - 43.18% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) (percentage) -88.24% 0.170 0.020 - -60.00% -
429.mcf (unexplained diff vs previous) -87.50% 16.000 2.000 - -60.00% -
470.lbm (unexplained diff vs previous) -82.61% 69.000 12.000 - -20.00% -
470.lbm (unexplained diff vs previous) (percentage) -81.40% 0.430 0.080 - -11.11% -
462.libquantum (max) (percentage) -78.43% 0.510 0.110 - 83.33% -
462.libquantum (max) -78.38% 74.000 16.000 - 100.00% -
462.libquantum (unexplained diff vs previous) (percentage) -60.00% 0.150 0.060 - 0.00% -
462.libquantum (unexplained diff vs previous) -59.09% 22.000 9.000 - 0.00% -
429.mcf (max) -18.75% 16.000 13.000 - 30.00% -
429.mcf (max) (percentage) -17.65% 0.170 0.140 - 27.27% -
458.sjeng (max) -12.50% 48.000 42.000 - -17.65% -
458.sjeng (max) (percentage) -12.12% 0.330 0.290 - -17.14% -
473.astar (max) -11.11% 9.000 8.000 - -46.67% -
473.astar (max) (percentage) -10.00% 0.100 0.090 - -47.06% -
444.namd (max) -10.00% 10.000 9.000 - -64.00% -
456.hmmer (max) (percentage) -9.90% 1.010 0.910 - -76.18% -
464.h264ref (max) -9.38% 96.000 87.000 - 0.00% -
450.soplex (max) -9.09% 11.000 10.000 - -50.00% -
456.hmmer (max) -9.09% 77.000 70.000 - -75.44% -
403.gcc (max) (percentage) -9.09% 0.220 0.200 - -74.36% -
401.bzip2 (max) -8.33% 12.000 11.000 - 22.22% -
464.h264ref (max) (percentage) -8.16% 0.980 0.900 - 0.00% -
453.povray (max) -7.41% 27.000 25.000 - -13.79% -
450.soplex (max) (percentage) -7.14% 0.140 0.130 - -45.83% -
403.gcc (max) -7.14% 14.000 13.000 - -74.51% -
470.lbm (max) -7.04% 71.000 66.000 - 135.71% -
470.lbm (max) (percentage) -6.67% 0.450 0.420 - 133.33% -
453.povray (max) (percentage) -6.67% 0.600 0.560 - -11.11% -
482.sphinx3 (max) -5.83% 103.000 97.000 - 97.96% -
482.sphinx3 (max) (percentage) -5.26% 0.570 0.540 - 100.00% -
447.dealII (max) (percentage) -4.90% 1.020 0.970 - 46.97% -
470.lbm -4.88% 41.000 39.000 - -17.02% -
470.lbm (avg) -4.88% 41.000 39.000 - -17.02% -
470.lbm (avg) (percentage) -3.85% 0.260 0.250 - -16.67% -
447.dealII (max) -3.85% 52.000 50.000 - 47.06% -
445.gobmk (max) -3.03% 33.000 32.000 - 6.67% -
483.xalancbmk (max) -2.78% 36.000 35.000 - -44.44% -
464.h264ref (unexplained diff vs previous) -2.38% 42.000 41.000 - -53.93% -
464.h264ref (unexplained diff vs previous) (percentage) -2.33% 0.430 0.420 - -53.85% -
483.xalancbmk (max) (percentage) -1.61% 0.620 0.610 - -43.52% -

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
456.hmmer 3.69% 7481.000 7757.000 - 0.90% -
445.gobmk 2.92% 11199.000 11526.000 - 0.54% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
471.omnetpp (unexplained diff vs previous) 2460.00% 5.000 128.000 - 70.67% -
471.omnetpp (unexplained diff vs previous) (percentage) 2371.43% 0.070 1.730 - 67.96% -
471.omnetpp (max) 293.94% 33.000 130.000 - 34.02% -
471.omnetpp (max) (percentage) 282.61% 0.460 1.760 - 31.34% -
470.lbm (max) 135.71% 28.000 66.000 - -7.04% -
470.lbm (max) (percentage) 133.33% 0.180 0.420 - -6.67% -
462.libquantum (max) 100.00% 8.000 16.000 - -78.38% -
482.sphinx3 (max) (percentage) 100.00% 0.270 0.540 - -5.26% -
482.sphinx3 (max) 97.96% 49.000 97.000 - -5.83% -
462.libquantum (max) (percentage) 83.33% 0.060 0.110 - -78.43% -
447.dealII (max) 47.06% 34.000 50.000 - -3.85% -
447.dealII (max) (percentage) 46.97% 0.660 0.970 - -4.90% -
471.omnetpp 43.75% 32.000 46.000 - 9.52% -
471.omnetpp (avg) 43.75% 32.000 46.000 - 9.52% -
471.omnetpp (avg) (percentage) 43.18% 0.440 0.630 - 8.62% -
429.mcf (max) 30.00% 10.000 13.000 - -18.75% -
429.mcf (max) (percentage) 27.27% 0.110 0.140 - -17.65% -
401.bzip2 (max) (percentage) 25.00% 0.080 0.100 - -9.09% -
462.libquantum (avg) (percentage) 23.08% 0.130 0.160 - 0.00% -
401.bzip2 (max) 22.22% 9.000 11.000 - -8.33% -
462.libquantum 21.05% 19.000 23.000 - 0.00% -
462.libquantum (avg) 21.05% 19.000 23.000 - 0.00% -
456.hmmer 13.68% 95.000 108.000 - 0.93% -
456.hmmer (avg) 13.68% 95.000 108.000 - 0.93% -
401.bzip2 11.11% 9.000 10.000 - 0.00% -
401.bzip2 (avg) 11.11% 9.000 10.000 - 0.00% -
456.hmmer (avg) (percentage) 10.24% 1.270 1.400 - 0.00% -
445.gobmk (max) 6.67% 30.000 32.000 - -3.03% -
450.soplex 6.67% 15.000 16.000 - 0.00% -
450.soplex (avg) 6.67% 15.000 16.000 - 0.00% -
450.soplex (avg) (percentage) 5.26% 0.190 0.200 - 0.00% -
445.gobmk (max) (percentage) 3.70% 0.270 0.280 - -3.45% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk -1.44% 5921.000 5836.000 - 0.41% -
453.povray -1.41% 4614.000 4549.000 - -0.74% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
456.hmmer (max) (percentage) -76.18% 3.820 0.910 - -9.90% -
456.hmmer (max) -75.44% 285.000 70.000 - -9.09% -
403.gcc (max) -74.51% 51.000 13.000 - -7.14% -
403.gcc (max) (percentage) -74.36% 0.780 0.200 - -9.09% -
444.namd (max) -64.00% 25.000 9.000 - -10.00% -
444.namd (max) (percentage) -61.54% 0.260 0.100 - -9.09% -
429.mcf (unexplained diff vs previous) -60.00% 5.000 2.000 - -87.50% -
429.mcf (unexplained diff vs previous) (percentage) -60.00% 0.050 0.020 - -88.24% -
464.h264ref (unexplained diff vs previous) -53.93% 89.000 41.000 - -2.38% -
464.h264ref (unexplained diff vs previous) (percentage) -53.85% 0.910 0.420 - -2.33% -
450.soplex (max) -50.00% 20.000 10.000 - -9.09% -
473.astar (max) (percentage) -47.06% 0.170 0.090 - -10.00% -
473.astar (max) -46.67% 15.000 8.000 - -11.11% -
450.soplex (max) (percentage) -45.83% 0.240 0.130 - -7.14% -
483.xalancbmk (max) -44.44% 63.000 35.000 - -2.78% -
483.xalancbmk (max) (percentage) -43.52% 1.080 0.610 - -1.61% -
470.lbm (unexplained diff vs previous) -20.00% 15.000 12.000 - -82.61% -
444.namd -18.75% 16.000 13.000 - 0.00% -
444.namd (avg) -18.75% 16.000 13.000 - 0.00% -
433.milc (unexplained diff vs previous) -18.18% 11.000 9.000 - 200.00% -
473.astar -18.18% 11.000 9.000 - 0.00% -
473.astar (avg) -18.18% 11.000 9.000 - 0.00% -
458.sjeng (max) -17.65% 51.000 42.000 - -12.50% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
458.sjeng (avg) (percentage) -17.46% 0.630 0.520 - 0.00% -
458.sjeng -17.39% 92.000 76.000 - 0.00% -
458.sjeng (avg) -17.39% 92.000 76.000 - 0.00% -
458.sjeng (max) (percentage) -17.14% 0.350 0.290 - -12.12% -
470.lbm -17.02% 47.000 39.000 - -4.88% -
470.lbm (avg) -17.02% 47.000 39.000 - -4.88% -
470.lbm (avg) (percentage) -16.67% 0.300 0.250 - -3.85% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
453.povray (max) -13.79% 29.000 25.000 - -7.41% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
483.xalancbmk -12.96% 54.000 47.000 - 0.00% -
483.xalancbmk (avg) -12.96% 54.000 47.000 - 0.00% -
433.milc -12.50% 8.000 7.000 - 0.00% -
433.milc (avg) -12.50% 8.000 7.000 - 0.00% -
445.gobmk -12.50% 24.000 21.000 - 0.00% -
445.gobmk (avg) -12.50% 24.000 21.000 - 0.00% -
483.xalancbmk (avg) (percentage) -11.96% 0.920 0.810 - 0.00% -
453.povray (max) (percentage) -11.11% 0.630 0.560 - -6.67% -
433.milc (max) -10.00% 10.000 9.000 - 80.00% -
482.sphinx3 (avg) (percentage) -9.80% 0.510 0.460 - 0.00% -
482.sphinx3 -9.78% 92.000 83.000 - 0.00% -
482.sphinx3 (avg) -9.78% 92.000 83.000 - 0.00% -
447.dealII (avg) (percentage) -7.27% 0.550 0.510 - 0.00% -
447.dealII -7.14% 28.000 26.000 - 0.00% -
447.dealII (avg) -7.14% 28.000 26.000 - 0.00% -
464.h264ref -4.84% 62.000 59.000 - 0.00% -
464.h264ref (avg) -4.84% 62.000 59.000 - 0.00% -
464.h264ref (avg) (percentage) -4.69% 0.640 0.610 - 0.00% -
453.povray -4.55% 22.000 21.000 - 0.00% -
453.povray (avg) -4.55% 22.000 21.000 - 0.00% -
453.povray (avg) (percentage) -4.08% 0.490 0.470 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.23s
Execution (seconds)
Name Current %
401.bzip2 11313.000 -
403.gcc 6626.000 -
429.mcf 9635.000 -
433.milc 11965.000 -
444.namd 9963.000 -
445.gobmk 11526.000 -
447.dealII 5179.000 -
450.soplex 8385.000 -
453.povray 4549.000 -
456.hmmer 7757.000 -
458.sjeng 14807.000 -
462.libquantum 14714.000 -
464.h264ref 9768.000 -
470.lbm 15893.000 -
471.omnetpp 7403.000 1.76%
473.astar 9413.000 -
482.sphinx3 18146.000 -
483.xalancbmk 5836.000 -
Geometric Mean 9476.014 -
Code Size (bytes)
Name Current %
401.bzip2 73552.000 -
403.gcc 3217329.000 -
429.mcf 11019.000 -
433.milc 113863.000 -
444.namd 226158.000 -
445.gobmk 1596791.000 -
447.dealII 2623214.000 -
450.soplex 309969.000 -
453.povray 913294.000 -
456.hmmer 258686.000 -
458.sjeng 149501.000 -
462.libquantum 35357.000 -
464.h264ref 622502.000 -
470.lbm 11257.000 -
471.omnetpp 527703.000 -
473.astar 35868.000 -
482.sphinx3 165756.000 -
483.xalancbmk 3352472.000 -
Geometric Mean 241868.400 -
sample variability
Name Current %
401.bzip2 10.000 -
401.bzip2 (avg) 10.000 -
401.bzip2 (avg) (percentage) 0.090 -
401.bzip2 (max) 11.000 -8.33%
401.bzip2 (max) (percentage) 0.100 -
403.gcc 23.000 -
403.gcc (avg) 23.000 -
403.gcc (avg) (percentage) 0.360 -
403.gcc (max) 13.000 -7.14%
403.gcc (max) (percentage) 0.200 -9.09%
429.mcf 7.000 -
429.mcf (avg) 7.000 -
429.mcf (avg) (percentage) 0.080 -
429.mcf (max) 13.000 -18.75%
429.mcf (max) (percentage) 0.140 -17.65%
429.mcf (unexplained diff vs previous) 2.000 -87.50%
429.mcf (unexplained diff vs previous) (percentage) 0.020 -88.24%
433.milc 7.000 -
433.milc (avg) 7.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 9.000 80.00%
433.milc (max) (percentage) 0.080 60.00%
433.milc (unexplained diff vs previous) 9.000 200.00%
433.milc (unexplained diff vs previous) (percentage) 0.080 166.67%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) 9.000 -10.00%
444.namd (max) (percentage) 0.100 -
445.gobmk 21.000 -
445.gobmk (avg) 21.000 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) 32.000 -3.03%
445.gobmk (max) (percentage) 0.280 -
447.dealII 26.000 -
447.dealII (avg) 26.000 -
447.dealII (avg) (percentage) 0.510 -
447.dealII (max) 50.000 -3.85%
447.dealII (max) (percentage) 0.970 -4.90%
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 0.200 -
450.soplex (max) 10.000 -9.09%
450.soplex (max) (percentage) 0.130 -7.14%
453.povray 21.000 -
453.povray (avg) 21.000 -
453.povray (avg) (percentage) 0.470 -
453.povray (max) 25.000 -7.41%
453.povray (max) (percentage) 0.560 -6.67%
456.hmmer 108.000 -
456.hmmer (avg) 108.000 -
456.hmmer (avg) (percentage) 1.400 -
456.hmmer (max) 70.000 -9.09%
456.hmmer (max) (percentage) 0.910 -9.90%
458.sjeng 76.000 -
458.sjeng (avg) 76.000 -
458.sjeng (avg) (percentage) 0.520 -
458.sjeng (max) 42.000 -12.50%
458.sjeng (max) (percentage) 0.290 -12.12%
462.libquantum 23.000 -
462.libquantum (avg) 23.000 -
462.libquantum (avg) (percentage) 0.160 -
462.libquantum (max) 16.000 -78.38%
462.libquantum (max) (percentage) 0.110 -78.43%
462.libquantum (unexplained diff vs previous) 9.000 -59.09%
462.libquantum (unexplained diff vs previous) (percentage) 0.060 -60.00%
464.h264ref 59.000 -
464.h264ref (avg) 59.000 -
464.h264ref (avg) (percentage) 0.610 -
464.h264ref (max) 87.000 -9.38%
464.h264ref (max) (percentage) 0.900 -8.16%
464.h264ref (unexplained diff vs previous) 41.000 -2.38%
464.h264ref (unexplained diff vs previous) (percentage) 0.420 -2.33%
470.lbm 39.000 -4.88%
470.lbm (avg) 39.000 -4.88%
470.lbm (avg) (percentage) 0.250 -3.85%
470.lbm (max) 66.000 -7.04%
470.lbm (max) (percentage) 0.420 -6.67%
470.lbm (unexplained diff vs previous) 12.000 -82.61%
470.lbm (unexplained diff vs previous) (percentage) 0.080 -81.40%
471.omnetpp 46.000 9.52%
471.omnetpp (avg) 46.000 9.52%
471.omnetpp (avg) (percentage) 0.630 8.62%
471.omnetpp (max) 130.000 34.02%
471.omnetpp (max) (percentage) 1.760 31.34%
471.omnetpp (unexplained diff vs previous) 128.000 70.67%
471.omnetpp (unexplained diff vs previous) (percentage) 1.730 67.96%
473.astar 9.000 -
473.astar (avg) 9.000 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) 8.000 -11.11%
473.astar (max) (percentage) 0.090 -10.00%
482.sphinx3 83.000 -
482.sphinx3 (avg) 83.000 -
482.sphinx3 (avg) (percentage) 0.460 -
482.sphinx3 (max) 97.000 -5.83%
482.sphinx3 (max) (percentage) 0.540 -5.26%
483.xalancbmk 47.000 -
483.xalancbmk (avg) 47.000 -
483.xalancbmk (avg) (percentage) 0.810 -
483.xalancbmk (max) 35.000 -2.78%
483.xalancbmk (max) (percentage) 0.610 -1.61%
Geometric Mean 3.737 -9.57%