Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/f2f3b1a87ad294433986683864f75575c3a81779 (+ 525 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6-llvm-arm-master-Os-build/1/
View Options
Report
Filter

llvm-arm-master-Os test results

Run Order Start Time Duration
Current 0001 (llvmorg-16-init-16932-gf2f3b1a87ad2) 2023-01-08T01:15:25 0:00:00
Previous 0001 (llvmorg-16-init-16061-ge7790fbed32b) 2022-12-29T19:41:07 0:00:00
No Baseline Run

Tests Summary

Status Group #
Performance Regressions 90
Performance Improvements 48
Unchanged Tests 642
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ
429.mcf 20.20% 11999.000 14423.000 -
458.sjeng 6.16% 16406.000 17417.000 -

Performance Regressions - sample variability Δ Previous Current σ
462.libquantum (max) (percentage) 3990.00% 0.200 8.180 -
429.mcf (max) 3650.75% 67.000 2513.000 -
462.libquantum (max) 3576.92% 26.000 956.000 -
456.hmmer (max) (percentage) 3086.67% 0.300 9.560 -
429.mcf (max) (percentage) 3012.50% 0.560 17.430 -
456.hmmer (max) 2695.00% 40.000 1118.000 -
483.xalancbmk (max) (percentage) 2246.15% 0.650 15.250 -
483.xalancbmk (max) 1868.85% 61.000 1201.000 -
445.gobmk (max) (percentage) 1541.67% 0.240 3.940 -
445.gobmk (max) 1503.45% 29.000 465.000 -
464.h264ref (max) (percentage) 1328.21% 0.390 5.570 -
464.h264ref (max) 1215.00% 60.000 789.000 -
458.sjeng (max) 1164.62% 65.000 822.000 -
458.sjeng (max) (percentage) 1080.00% 0.400 4.720 -
401.bzip2 (max) (percentage) 1061.29% 0.310 3.600 -
401.bzip2 (max) 1025.00% 48.000 540.000 -
473.astar (max) (percentage) 733.75% 0.800 6.670 -
473.astar (max) 650.88% 114.000 856.000 -
462.libquantum (avg) (percentage) 500.00% 0.150 0.900 -
462.libquantum 452.63% 19.000 105.000 -
462.libquantum (avg) 452.63% 19.000 105.000 -
456.hmmer (unexplained diff vs previous) (percentage) 428.57% 0.070 0.370 -
471.omnetpp (max) (percentage) 409.09% 1.760 8.960 -
403.gcc (max) (percentage) 392.11% 0.380 1.870 -
403.gcc (max) 388.24% 34.000 166.000 -
429.mcf 383.02% 53.000 256.000 -
429.mcf (avg) 383.02% 53.000 256.000 -
456.hmmer (unexplained diff vs previous) 377.78% 9.000 43.000 -
471.omnetpp (max) 348.15% 189.000 847.000 -
483.xalancbmk (avg) (percentage) 320.00% 0.500 2.100 -
429.mcf (avg) (percentage) 295.56% 0.450 1.780 -
456.hmmer (avg) (percentage) 286.11% 0.360 1.390 -
483.xalancbmk 251.06% 47.000 165.000 -
483.xalancbmk (avg) 251.06% 47.000 165.000 -
456.hmmer 237.50% 48.000 162.000 -
456.hmmer (avg) 237.50% 48.000 162.000 -
445.gobmk (avg) (percentage) 200.00% 0.180 0.540 -
445.gobmk 186.36% 22.000 63.000 -
445.gobmk (avg) 186.36% 22.000 63.000 -
473.astar (avg) (percentage) 152.00% 0.500 1.260 -
464.h264ref (avg) (percentage) 142.42% 0.330 0.800 -
401.bzip2 (avg) (percentage) 136.36% 0.220 0.520 -
401.bzip2 129.41% 34.000 78.000 -
401.bzip2 (avg) 129.41% 34.000 78.000 -
473.astar 126.76% 71.000 161.000 -
473.astar (avg) 126.76% 71.000 161.000 -
464.h264ref 121.57% 51.000 113.000 -
464.h264ref (avg) 121.57% 51.000 113.000 -
450.soplex (max) (percentage) 105.26% 0.570 1.170 -
450.soplex (max) 102.44% 82.000 166.000 -
471.omnetpp (avg) (percentage) 96.15% 0.780 1.530 -
458.sjeng 90.53% 95.000 181.000 -
458.sjeng (avg) 90.53% 95.000 181.000 -
433.milc (max) (percentage) 88.32% 1.370 2.580 -
482.sphinx3 (max) (percentage) 84.94% 3.320 6.140 -
433.milc (max) 82.61% 184.000 336.000 -
403.gcc (unexplained diff vs previous) (percentage) 81.08% 0.370 0.670 -
470.lbm (unexplained diff vs previous) (percentage) 79.33% 3.580 6.420 -
458.sjeng (avg) (percentage) 79.31% 0.580 1.040 -
403.gcc (unexplained diff vs previous) 76.47% 34.000 60.000 -
471.omnetpp 73.49% 83.000 144.000 -
471.omnetpp (avg) 73.49% 83.000 144.000 -
482.sphinx3 (max) 72.51% 873.000 1506.000 -
447.dealII (max) (percentage) 68.99% 1.580 2.670 -
470.lbm (max) (percentage) 68.77% 5.540 9.350 -
447.dealII (max) 62.79% 129.000 210.000 -
470.lbm (avg) (percentage) 55.60% 2.320 3.610 -
470.lbm (unexplained diff vs previous) 55.33% 497.000 772.000 -
470.lbm (max) 46.16% 769.000 1124.000 -
471.omnetpp (unexplained diff vs previous) (percentage) 44.19% 0.860 1.240 -
403.gcc (avg) (percentage) 39.58% 0.480 0.670 -
403.gcc 37.21% 43.000 59.000 -
403.gcc (avg) 37.21% 43.000 59.000 -
470.lbm 34.78% 322.000 434.000 -
470.lbm (avg) 34.78% 322.000 434.000 -
482.sphinx3 (avg) (percentage) 29.03% 1.860 2.400 -
471.omnetpp (unexplained diff vs previous) 27.17% 92.000 117.000 -
444.namd (max) (percentage) 26.15% 6.540 8.250 -
473.astar (unexplained diff vs previous) (percentage) 21.05% 0.760 0.920 -
482.sphinx3 20.45% 489.000 589.000 -
482.sphinx3 (avg) 20.45% 489.000 589.000 -
444.namd (max) 14.06% 882.000 1006.000 -
473.astar (unexplained diff vs previous) 9.26% 108.000 118.000 -
450.soplex (avg) (percentage) 8.33% 0.360 0.390 -
450.soplex 7.84% 51.000 55.000 -
450.soplex (avg) 7.84% 51.000 55.000 -
433.milc (avg) (percentage) 2.97% 1.010 1.040 -
453.povray (max) (percentage) 1.55% 1.930 1.960 -

Performance Improvements - Execution Δ Previous Current σ
483.xalancbmk -16.40% 9427.000 7881.000 -
470.lbm -13.39% 13882.000 12023.000 -
456.hmmer -12.44% 13365.000 11703.000 -
471.omnetpp -11.98% 10746.000 9459.000 -
462.libquantum -10.92% 13126.000 11692.000 -
473.astar -9.98% 14262.000 12839.000 -
444.namd -9.61% 13495.000 12198.000 -
464.h264ref -8.47% 15493.000 14181.000 -
482.sphinx3 -6.72% 26310.000 24543.000 -
445.gobmk -4.80% 12400.000 11805.000 -
401.bzip2 -4.32% 15683.000 15005.000 -
447.dealII -3.73% 8196.000 7890.000 -
433.milc -3.18% 13482.000 13053.000 -
403.gcc -2.22% 9099.000 8897.000 -
453.povray -2.01% 6857.000 6719.000 -
450.soplex -1.62% 14429.000 14195.000 -

Performance Improvements - sample variability Δ Previous Current σ
444.namd (unexplained diff vs previous) -99.88% 828.000 1.000 -
444.namd (unexplained diff vs previous) (percentage) -99.84% 6.140 0.010 -
450.soplex (unexplained diff vs previous) -98.78% 82.000 1.000 -
450.soplex (unexplained diff vs previous) (percentage) -98.25% 0.570 0.010 -
447.dealII (unexplained diff vs previous) -96.69% 121.000 4.000 -
447.dealII (unexplained diff vs previous) (percentage) -96.62% 1.480 0.050 -
433.milc (unexplained diff vs previous) -88.46% 182.000 21.000 -
433.milc (unexplained diff vs previous) (percentage) -88.15% 1.350 0.160 -
401.bzip2 (unexplained diff vs previous) -87.76% 49.000 6.000 -
401.bzip2 (unexplained diff vs previous) (percentage) -87.10% 0.310 0.040 -
453.povray (unexplained diff vs previous) -86.15% 130.000 18.000 -
453.povray (unexplained diff vs previous) (percentage) -85.79% 1.900 0.270 -
429.mcf (unexplained diff vs previous) (percentage) -69.64% 0.560 0.170 -
429.mcf (unexplained diff vs previous) -64.18% 67.000 24.000 -
464.h264ref (unexplained diff vs previous) -63.33% 60.000 22.000 -
464.h264ref (unexplained diff vs previous) (percentage) -58.97% 0.390 0.160 -
445.gobmk (unexplained diff vs previous) -56.67% 30.000 13.000 -
445.gobmk (unexplained diff vs previous) (percentage) -54.17% 0.240 0.110 -
462.libquantum (unexplained diff vs previous) -25.00% 16.000 12.000 -
462.libquantum (unexplained diff vs previous) (percentage) -16.67% 0.120 0.100 -
444.namd -13.62% 514.000 444.000 -
444.namd (avg) -13.62% 514.000 444.000 -
458.sjeng (unexplained diff vs previous) (percentage) -8.00% 0.250 0.230 -
447.dealII -7.50% 120.000 111.000 -
447.dealII (avg) -7.50% 120.000 111.000 -
482.sphinx3 (unexplained diff vs previous) -6.75% 904.000 843.000 -
444.namd (avg) (percentage) -4.46% 3.810 3.640 -
447.dealII (avg) (percentage) -4.08% 1.470 1.410 -
453.povray -3.23% 93.000 90.000 -
453.povray (avg) -3.23% 93.000 90.000 -
458.sjeng (unexplained diff vs previous) -2.44% 41.000 40.000 -
453.povray (avg) (percentage) -1.46% 1.370 1.350 -

Report Time: 0.21s
Execution (seconds)
Name Current %
401.bzip2 15005.000 -4.32%
403.gcc 8897.000 -2.22%
429.mcf 14423.000 20.20%
433.milc 13053.000 -3.18%
444.namd 12198.000 -9.61%
445.gobmk 11805.000 -4.80%
447.dealII 7890.000 -3.73%
450.soplex 14195.000 -1.62%
453.povray 6719.000 -2.01%
456.hmmer 11703.000 -12.44%
458.sjeng 17417.000 6.16%
462.libquantum 11692.000 -10.92%
464.h264ref 14181.000 -8.47%
470.lbm 12023.000 -13.39%
471.omnetpp 9459.000 -11.98%
473.astar 12839.000 -9.98%
482.sphinx3 24543.000 -6.72%
483.xalancbmk 7881.000 -16.40%
Geometric Mean 11984.076 -5.63%
Code Size (bytes)
Name Current %
401.bzip2 39458.000 -
403.gcc 1960330.000 -
429.mcf 7915.000 -
433.milc 74971.000 -
444.namd 175160.000 -
445.gobmk 1312356.000 -
447.dealII 1777992.000 -
450.soplex 198708.000 -
453.povray 593161.000 -
456.hmmer 168072.000 -
458.sjeng 97694.000 -
462.libquantum 26756.000 -
464.h264ref 354474.000 -
470.lbm 9723.000 -
471.omnetpp 373579.000 -
473.astar 23216.000 -
482.sphinx3 115720.000 -
483.xalancbmk 2287149.000 -
Geometric Mean 164387.846 -
sample variability
Name Current %
401.bzip2 78.000 129.41%
401.bzip2 (avg) 78.000 129.41%
401.bzip2 (avg) (percentage) 0.520 136.36%
401.bzip2 (max) 540.000 1025.00%
401.bzip2 (max) (percentage) 3.600 1061.29%
401.bzip2 (unexplained diff vs previous) 6.000 -87.76%
401.bzip2 (unexplained diff vs previous) (percentage) 0.040 -87.10%
403.gcc 59.000 37.21%
403.gcc (avg) 59.000 37.21%
403.gcc (avg) (percentage) 0.670 39.58%
403.gcc (max) 166.000 388.24%
403.gcc (max) (percentage) 1.870 392.11%
403.gcc (unexplained diff vs previous) 60.000 76.47%
403.gcc (unexplained diff vs previous) (percentage) 0.670 81.08%
429.mcf 256.000 383.02%
429.mcf (avg) 256.000 383.02%
429.mcf (avg) (percentage) 1.780 295.56%
429.mcf (max) 2513.000 3650.75%
429.mcf (max) (percentage) 17.430 3012.50%
429.mcf (unexplained diff vs previous) 24.000 -64.18%
429.mcf (unexplained diff vs previous) (percentage) 0.170 -69.64%
433.milc 135.000 -
433.milc (avg) 135.000 -
433.milc (avg) (percentage) 1.040 2.97%
433.milc (max) 336.000 82.61%
433.milc (max) (percentage) 2.580 88.32%
433.milc (unexplained diff vs previous) 21.000 -88.46%
433.milc (unexplained diff vs previous) (percentage) 0.160 -88.15%
444.namd 444.000 -13.62%
444.namd (avg) 444.000 -13.62%
444.namd (avg) (percentage) 3.640 -4.46%
444.namd (max) 1006.000 14.06%
444.namd (max) (percentage) 8.250 26.15%
444.namd (unexplained diff vs previous) 1.000 -99.88%
444.namd (unexplained diff vs previous) (percentage) 0.010 -99.84%
445.gobmk 63.000 186.36%
445.gobmk (avg) 63.000 186.36%
445.gobmk (avg) (percentage) 0.540 200.00%
445.gobmk (max) 465.000 1503.45%
445.gobmk (max) (percentage) 3.940 1541.67%
445.gobmk (unexplained diff vs previous) 13.000 -56.67%
445.gobmk (unexplained diff vs previous) (percentage) 0.110 -54.17%
447.dealII 111.000 -7.50%
447.dealII (avg) 111.000 -7.50%
447.dealII (avg) (percentage) 1.410 -4.08%
447.dealII (max) 210.000 62.79%
447.dealII (max) (percentage) 2.670 68.99%
447.dealII (unexplained diff vs previous) 4.000 -96.69%
447.dealII (unexplained diff vs previous) (percentage) 0.050 -96.62%
450.soplex 55.000 7.84%
450.soplex (avg) 55.000 7.84%
450.soplex (avg) (percentage) 0.390 8.33%
450.soplex (max) 166.000 102.44%
450.soplex (max) (percentage) 1.170 105.26%
450.soplex (unexplained diff vs previous) 1.000 -98.78%
450.soplex (unexplained diff vs previous) (percentage) 0.010 -98.25%
453.povray 90.000 -3.23%
453.povray (avg) 90.000 -3.23%
453.povray (avg) (percentage) 1.350 -1.46%
453.povray (max) 131.000 -
453.povray (max) (percentage) 1.960 1.55%
453.povray (unexplained diff vs previous) 18.000 -86.15%
453.povray (unexplained diff vs previous) (percentage) 0.270 -85.79%
456.hmmer 162.000 237.50%
456.hmmer (avg) 162.000 237.50%
456.hmmer (avg) (percentage) 1.390 286.11%
456.hmmer (max) 1118.000 2695.00%
456.hmmer (max) (percentage) 9.560 3086.67%
456.hmmer (unexplained diff vs previous) 43.000 377.78%
456.hmmer (unexplained diff vs previous) (percentage) 0.370 428.57%
458.sjeng 181.000 90.53%
458.sjeng (avg) 181.000 90.53%
458.sjeng (avg) (percentage) 1.040 79.31%
458.sjeng (max) 822.000 1164.62%
458.sjeng (max) (percentage) 4.720 1080.00%
458.sjeng (unexplained diff vs previous) 40.000 -2.44%
458.sjeng (unexplained diff vs previous) (percentage) 0.230 -8.00%
462.libquantum 105.000 452.63%
462.libquantum (avg) 105.000 452.63%
462.libquantum (avg) (percentage) 0.900 500.00%
462.libquantum (max) 956.000 3576.92%
462.libquantum (max) (percentage) 8.180 3990.00%
462.libquantum (unexplained diff vs previous) 12.000 -25.00%
462.libquantum (unexplained diff vs previous) (percentage) 0.100 -16.67%
464.h264ref 113.000 121.57%
464.h264ref (avg) 113.000 121.57%
464.h264ref (avg) (percentage) 0.800 142.42%
464.h264ref (max) 789.000 1215.00%
464.h264ref (max) (percentage) 5.570 1328.21%
464.h264ref (unexplained diff vs previous) 22.000 -63.33%
464.h264ref (unexplained diff vs previous) (percentage) 0.160 -58.97%
470.lbm 434.000 34.78%
470.lbm (avg) 434.000 34.78%
470.lbm (avg) (percentage) 3.610 55.60%
470.lbm (max) 1124.000 46.16%
470.lbm (max) (percentage) 9.350 68.77%
470.lbm (unexplained diff vs previous) 772.000 55.33%
470.lbm (unexplained diff vs previous) (percentage) 6.420 79.33%
471.omnetpp 144.000 73.49%
471.omnetpp (avg) 144.000 73.49%
471.omnetpp (avg) (percentage) 1.530 96.15%
471.omnetpp (max) 847.000 348.15%
471.omnetpp (max) (percentage) 8.960 409.09%
471.omnetpp (unexplained diff vs previous) 117.000 27.17%
471.omnetpp (unexplained diff vs previous) (percentage) 1.240 44.19%
473.astar 161.000 126.76%
473.astar (avg) 161.000 126.76%
473.astar (avg) (percentage) 1.260 152.00%
473.astar (max) 856.000 650.88%
473.astar (max) (percentage) 6.670 733.75%
473.astar (unexplained diff vs previous) 118.000 9.26%
473.astar (unexplained diff vs previous) (percentage) 0.920 21.05%
482.sphinx3 589.000 20.45%
482.sphinx3 (avg) 589.000 20.45%
482.sphinx3 (avg) (percentage) 2.400 29.03%
482.sphinx3 (max) 1506.000 72.51%
482.sphinx3 (max) (percentage) 6.140 84.94%
482.sphinx3 (unexplained diff vs previous) 843.000 -6.75%
482.sphinx3 (unexplained diff vs previous) (percentage) 3.430 -
483.xalancbmk 165.000 251.06%
483.xalancbmk (avg) 165.000 251.06%
483.xalancbmk (avg) (percentage) 2.100 320.00%
483.xalancbmk (max) 1201.000 1868.85%
483.xalancbmk (max) (percentage) 15.250 2246.15%
Geometric Mean 17.726 59.47%