Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/8b301b4f6b3d7068fd81f88ac001916c2f138c33 (+ 521 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6-llvm-arm-master-O2-build/1/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0001 (llvmorg-16-init-16842-g8b301b4f6b3d) 2023-01-07T09:57:27 0:00:00
Previous 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00
No Baseline Run

Tests Summary

Status Group #
Performance Regressions 29
Performance Improvements 53
Added Tests 6
Unchanged Tests 644
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ
456.hmmer 2.37% 7481.000 7658.000 -

Performance Regressions - sample variability Δ Previous Current σ
471.omnetpp (unexplained diff vs previous) 1500.00% 5.000 80.000 -
471.omnetpp (unexplained diff vs previous) (percentage) 1471.43% 0.070 1.100 -
462.libquantum (max) 237.50% 8.000 27.000 -
462.libquantum (max) (percentage) 216.67% 0.060 0.190 -
429.mcf (unexplained diff vs previous) 140.00% 5.000 12.000 -
429.mcf (unexplained diff vs previous) (percentage) 140.00% 0.050 0.120 -
471.omnetpp (max) 139.39% 33.000 79.000 -
471.omnetpp (max) (percentage) 136.96% 0.460 1.090 -
482.sphinx3 (max) (percentage) 55.56% 0.270 0.420 -
482.sphinx3 (unexplained diff vs previous) (percentage) 55.56% 0.270 0.420 -
482.sphinx3 (max) 55.10% 49.000 76.000 -
482.sphinx3 (unexplained diff vs previous) 55.10% 49.000 76.000 -
445.gobmk (max) (percentage) 29.63% 0.270 0.350 -
445.gobmk (max) 26.67% 30.000 38.000 -
433.milc (max) (percentage) 11.11% 0.090 0.100 -
433.milc (unexplained diff vs previous) (percentage) 11.11% 0.090 0.100 -
429.mcf (max) 10.00% 10.000 11.000 -
433.milc (max) 10.00% 10.000 11.000 -
433.milc (unexplained diff vs previous) 9.09% 11.000 12.000 -
403.gcc 8.70% 23.000 25.000 -
403.gcc (avg) 8.70% 23.000 25.000 -
403.gcc (avg) (percentage) 5.56% 0.360 0.380 -
445.gobmk (avg) (percentage) 4.55% 0.220 0.230 -
445.gobmk 4.17% 24.000 25.000 -
445.gobmk (avg) 4.17% 24.000 25.000 -
471.omnetpp (avg) (percentage) 2.27% 0.440 0.450 -
456.hmmer 1.05% 95.000 96.000 -
456.hmmer (avg) 1.05% 95.000 96.000 -

Performance Improvements - Execution Δ Previous Current σ
471.omnetpp -1.17% 7376.000 7290.000 -

Performance Improvements - sample variability Δ Previous Current σ
473.astar (unexplained diff vs previous) (percentage) -94.12% 0.170 0.010 -
473.astar (unexplained diff vs previous) -93.75% 16.000 1.000 -
470.lbm (unexplained diff vs previous) (percentage) -88.89% 0.090 0.010 -
470.lbm (unexplained diff vs previous) -86.67% 15.000 2.000 -
456.hmmer (unexplained diff vs previous) (percentage) -77.58% 3.970 0.890 -
456.hmmer (unexplained diff vs previous) -77.10% 297.000 68.000 -
401.bzip2 (unexplained diff vs previous) -50.00% 2.000 1.000 -
401.bzip2 (unexplained diff vs previous) (percentage) -50.00% 0.020 0.010 -
450.soplex (unexplained diff vs previous) -50.00% 8.000 4.000 -
450.soplex (unexplained diff vs previous) (percentage) -50.00% 0.100 0.050 -
401.bzip2 (max) -44.44% 9.000 5.000 -
458.sjeng (max) -41.18% 51.000 30.000 -
458.sjeng (max) (percentage) -40.00% 0.350 0.210 -
401.bzip2 (max) (percentage) -37.50% 0.080 0.050 -
444.namd (max) -36.00% 25.000 16.000 -
444.namd (max) (percentage) -34.62% 0.260 0.170 -
456.hmmer (max) (percentage) -33.25% 3.820 2.550 -
456.hmmer (max) -31.58% 285.000 195.000 -
450.soplex (max) -25.00% 20.000 15.000 -
403.gcc (max) (percentage) -24.36% 0.780 0.590 -
403.gcc (max) -23.53% 51.000 39.000 -
450.soplex (max) (percentage) -20.83% 0.240 0.190 -
464.h264ref (unexplained diff vs previous) (percentage) -17.58% 0.910 0.750 -
464.h264ref (unexplained diff vs previous) -16.85% 89.000 74.000 -
464.h264ref (max) (percentage) -15.56% 0.900 0.760 -
464.h264ref (max) -14.94% 87.000 74.000 -
473.astar (max) (percentage) -11.76% 0.170 0.150 -
470.lbm (max) (percentage) -11.11% 0.180 0.160 -
470.lbm (max) -10.71% 28.000 25.000 -
470.lbm -10.64% 47.000 42.000 -
470.lbm (avg) -10.64% 47.000 42.000 -
453.povray (max) -10.34% 29.000 26.000 -
470.lbm (avg) (percentage) -10.00% 0.300 0.270 -
453.povray (max) (percentage) -7.94% 0.630 0.580 -
483.xalancbmk (max) -7.94% 63.000 58.000 -
483.xalancbmk (max) (percentage) -7.41% 1.080 1.000 -
473.astar (max) -6.67% 15.000 14.000 -
444.namd -6.25% 16.000 15.000 -
444.namd (avg) -6.25% 16.000 15.000 -
444.namd (avg) (percentage) -5.88% 0.170 0.160 -
447.dealII (max) -5.88% 34.000 32.000 -
450.soplex (avg) (percentage) -5.26% 0.190 0.180 -
458.sjeng (avg) (percentage) -4.76% 0.630 0.600 -
447.dealII (max) (percentage) -4.55% 0.660 0.630 -
458.sjeng -4.35% 92.000 88.000 -
458.sjeng (avg) -4.35% 92.000 88.000 -
464.h264ref (avg) (percentage) -3.13% 0.640 0.620 -
482.sphinx3 (avg) (percentage) -1.96% 0.510 0.500 -
464.h264ref -1.61% 62.000 61.000 -
464.h264ref (avg) -1.61% 62.000 61.000 -
482.sphinx3 -1.09% 92.000 91.000 -
482.sphinx3 (avg) -1.09% 92.000 91.000 -

Added Tests - sample variability
403.gcc (unexplained diff vs previous)
403.gcc (unexplained diff vs previous) (percentage)
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.20s
Execution (seconds)
Name Current %
401.bzip2 11359.000 -
403.gcc 6624.000 -
429.mcf 9622.000 -
433.milc 11970.000 -
444.namd 9974.000 -
445.gobmk 11094.000 -
447.dealII 5214.000 -
450.soplex 8385.000 -
453.povray 4651.000 -
456.hmmer 7658.000 2.37%
458.sjeng 14702.000 -
462.libquantum 14698.000 -
464.h264ref 9847.000 -
470.lbm 15913.000 -
471.omnetpp 7290.000 -1.17%
473.astar 9377.000 -
482.sphinx3 18312.000 -
483.xalancbmk 5892.000 -
Geometric Mean 9466.723 -
Code Size (bytes)
Name Current %
401.bzip2 73592.000 -
403.gcc 3216369.000 -
429.mcf 11019.000 -
433.milc 113879.000 -
444.namd 226798.000 -
445.gobmk 1588327.000 -
447.dealII 2623390.000 -
450.soplex 309981.000 -
453.povray 913326.000 -
456.hmmer 258958.000 -
458.sjeng 149537.000 -
462.libquantum 35357.000 -
464.h264ref 622774.000 -
470.lbm 11257.000 -
471.omnetpp 527703.000 -
473.astar 35888.000 -
482.sphinx3 165788.000 -
483.xalancbmk 3353320.000 -
Geometric Mean 241878.743 -
sample variability
Name Current %
401.bzip2 9.000 -
401.bzip2 (avg) 9.000 -
401.bzip2 (avg) (percentage) 0.080 -
401.bzip2 (max) 5.000 -44.44%
401.bzip2 (max) (percentage) 0.050 -37.50%
401.bzip2 (unexplained diff vs previous) 1.000 -50.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.010 -50.00%
403.gcc 25.000 8.70%
403.gcc (avg) 25.000 8.70%
403.gcc (avg) (percentage) 0.380 5.56%
403.gcc (max) 39.000 -23.53%
403.gcc (max) (percentage) 0.590 -24.36%
403.gcc (unexplained diff vs previous) 39.000 -
403.gcc (unexplained diff vs previous) (percentage) 0.590 -
429.mcf 7.000 -
429.mcf (avg) 7.000 -
429.mcf (avg) (percentage) 0.080 -
429.mcf (max) 11.000 10.00%
429.mcf (max) (percentage) 0.120 -
429.mcf (unexplained diff vs previous) 12.000 140.00%
429.mcf (unexplained diff vs previous) (percentage) 0.120 140.00%
433.milc 8.000 -
433.milc (avg) 8.000 -
433.milc (avg) (percentage) 0.070 -
433.milc (max) 11.000 10.00%
433.milc (max) (percentage) 0.100 11.11%
433.milc (unexplained diff vs previous) 12.000 9.09%
433.milc (unexplained diff vs previous) (percentage) 0.100 11.11%
444.namd 15.000 -6.25%
444.namd (avg) 15.000 -6.25%
444.namd (avg) (percentage) 0.160 -5.88%
444.namd (max) 16.000 -36.00%
444.namd (max) (percentage) 0.170 -34.62%
444.namd (unexplained diff vs previous) 0.000 -
444.namd (unexplained diff vs previous) (percentage) 0.000 -
445.gobmk 25.000 4.17%
445.gobmk (avg) 25.000 4.17%
445.gobmk (avg) (percentage) 0.230 4.55%
445.gobmk (max) 38.000 26.67%
445.gobmk (max) (percentage) 0.350 29.63%
447.dealII 28.000 -
447.dealII (avg) 28.000 -
447.dealII (avg) (percentage) 0.550 -
447.dealII (max) 32.000 -5.88%
447.dealII (max) (percentage) 0.630 -4.55%
450.soplex 15.000 -
450.soplex (avg) 15.000 -
450.soplex (avg) (percentage) 0.180 -5.26%
450.soplex (max) 15.000 -25.00%
450.soplex (max) (percentage) 0.190 -20.83%
450.soplex (unexplained diff vs previous) 4.000 -50.00%
450.soplex (unexplained diff vs previous) (percentage) 0.050 -50.00%
453.povray 22.000 -
453.povray (avg) 22.000 -
453.povray (avg) (percentage) 0.490 -
453.povray (max) 26.000 -10.34%
453.povray (max) (percentage) 0.580 -7.94%
456.hmmer 96.000 1.05%
456.hmmer (avg) 96.000 1.05%
456.hmmer (avg) (percentage) 1.260 -
456.hmmer (max) 195.000 -31.58%
456.hmmer (max) (percentage) 2.550 -33.25%
456.hmmer (unexplained diff vs previous) 68.000 -77.10%
456.hmmer (unexplained diff vs previous) (percentage) 0.890 -77.58%
458.sjeng 88.000 -4.35%
458.sjeng (avg) 88.000 -4.35%
458.sjeng (avg) (percentage) 0.600 -4.76%
458.sjeng (max) 30.000 -41.18%
458.sjeng (max) (percentage) 0.210 -40.00%
462.libquantum 19.000 -
462.libquantum (avg) 19.000 -
462.libquantum (avg) (percentage) 0.130 -
462.libquantum (max) 27.000 237.50%
462.libquantum (max) (percentage) 0.190 216.67%
462.libquantum (unexplained diff vs previous) 12.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.080 -
464.h264ref 61.000 -1.61%
464.h264ref (avg) 61.000 -1.61%
464.h264ref (avg) (percentage) 0.620 -3.13%
464.h264ref (max) 74.000 -14.94%
464.h264ref (max) (percentage) 0.760 -15.56%
464.h264ref (unexplained diff vs previous) 74.000 -16.85%
464.h264ref (unexplained diff vs previous) (percentage) 0.750 -17.58%
470.lbm 42.000 -10.64%
470.lbm (avg) 42.000 -10.64%
470.lbm (avg) (percentage) 0.270 -10.00%
470.lbm (max) 25.000 -10.71%
470.lbm (max) (percentage) 0.160 -11.11%
470.lbm (unexplained diff vs previous) 2.000 -86.67%
470.lbm (unexplained diff vs previous) (percentage) 0.010 -88.89%
471.omnetpp 32.000 -
471.omnetpp (avg) 32.000 -
471.omnetpp (avg) (percentage) 0.450 2.27%
471.omnetpp (max) 79.000 139.39%
471.omnetpp (max) (percentage) 1.090 136.96%
471.omnetpp (unexplained diff vs previous) 80.000 1500.00%
471.omnetpp (unexplained diff vs previous) (percentage) 1.100 1471.43%
473.astar 11.000 -
473.astar (avg) 11.000 -
473.astar (avg) (percentage) 0.120 -
473.astar (max) 14.000 -6.67%
473.astar (max) (percentage) 0.150 -11.76%
473.astar (unexplained diff vs previous) 1.000 -93.75%
473.astar (unexplained diff vs previous) (percentage) 0.010 -94.12%
482.sphinx3 91.000 -1.09%
482.sphinx3 (avg) 91.000 -1.09%
482.sphinx3 (avg) (percentage) 0.500 -1.96%
482.sphinx3 (max) 76.000 55.10%
482.sphinx3 (max) (percentage) 0.420 55.56%
482.sphinx3 (unexplained diff vs previous) 76.000 55.10%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.420 55.56%
483.xalancbmk 54.000 -
483.xalancbmk (avg) 54.000 -
483.xalancbmk (avg) (percentage) 0.920 -
483.xalancbmk (max) 58.000 -7.94%
483.xalancbmk (max) (percentage) 1.000 -7.41%
Geometric Mean 2.850 -23.72%