Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/dafebd5b5a08dde25f5f52f65cac54bd6ec0ecde (+ 414 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6-llvm-arm-master-Os-build/5/
View Options
Report
Filter

llvm-arm-master-Os test results

Run Order Start Time Duration
Current 0005 (llvmorg-16-init-18533-gdafebd5b5a08) 2023-01-21T16:48:12 0:00:00
Previous 0005 (llvmorg-16-init-18533-gdafebd5b5a08) 2023-01-21T16:48:12 0:00:00
Baseline 0001 (llvmorg-16-init-16061-ge7790fbed32b) 2022-12-29T19:41:07 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 0 77
Performance Improvements 0 56
Unchanged Tests 780 647
Total Tests 780

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 20.57% 11999.000 14467.000 - 0.00% -
458.sjeng 5.78% 16406.000 17355.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk (max) (percentage) 1890.77% 0.650 12.940 - 0.00% -
483.xalancbmk (max) 1532.79% 61.000 996.000 - 0.00% -
458.sjeng (max) 950.77% 65.000 683.000 - 0.00% -
458.sjeng (max) (percentage) 885.00% 0.400 3.940 - 0.00% -
401.bzip2 (max) (percentage) 874.19% 0.310 3.020 - 0.00% -
401.bzip2 (max) 847.92% 48.000 455.000 - 0.00% -
462.libquantum (avg) (percentage) 426.67% 0.150 0.790 - 0.00% -
462.libquantum 384.21% 19.000 92.000 - 0.00% -
462.libquantum (avg) 384.21% 19.000 92.000 - 0.00% -
403.gcc (max) (percentage) 310.53% 0.380 1.560 - 0.00% -
403.gcc (max) 308.82% 34.000 139.000 - 0.00% -
429.mcf 279.25% 53.000 201.000 - 0.00% -
429.mcf (avg) 279.25% 53.000 201.000 - 0.00% -
483.xalancbmk (avg) (percentage) 264.00% 0.500 1.820 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) 262.50% 0.240 0.870 - 0.00% -
445.gobmk (max) (percentage) 258.33% 0.240 0.860 - 0.00% -
445.gobmk (max) 244.83% 29.000 100.000 - 0.00% -
445.gobmk (unexplained diff vs previous) 236.67% 30.000 101.000 - 0.00% -
456.hmmer (avg) (percentage) 230.56% 0.360 1.190 - 0.00% -
429.mcf (avg) (percentage) 208.89% 0.450 1.390 - 0.00% -
483.xalancbmk 197.87% 47.000 140.000 - 0.00% -
483.xalancbmk (avg) 197.87% 47.000 140.000 - 0.00% -
456.hmmer 189.58% 48.000 139.000 - 0.00% -
456.hmmer (avg) 189.58% 48.000 139.000 - 0.00% -
445.gobmk (avg) (percentage) 177.78% 0.180 0.500 - 0.00% -
445.gobmk 163.64% 22.000 58.000 - 0.00% -
445.gobmk (avg) 163.64% 22.000 58.000 - 0.00% -
462.libquantum (max) (percentage) 135.00% 0.200 0.470 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) 116.67% 0.120 0.260 - 0.00% -
473.astar (avg) (percentage) 112.00% 0.500 1.060 - 0.00% -
462.libquantum (max) 111.54% 26.000 55.000 - 0.00% -
464.h264ref (avg) (percentage) 103.03% 0.330 0.670 - 0.00% -
458.sjeng (unexplained diff vs previous) 100.00% 41.000 82.000 - 0.00% -
401.bzip2 (avg) (percentage) 95.45% 0.220 0.430 - 0.00% -
473.astar 91.55% 71.000 136.000 - 0.00% -
473.astar (avg) 91.55% 71.000 136.000 - 0.00% -
401.bzip2 88.24% 34.000 64.000 - 0.00% -
401.bzip2 (avg) 88.24% 34.000 64.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) 88.00% 0.250 0.470 - 0.00% -
462.libquantum (unexplained diff vs previous) 87.50% 16.000 30.000 - 0.00% -
464.h264ref 86.27% 51.000 95.000 - 0.00% -
464.h264ref (avg) 86.27% 51.000 95.000 - 0.00% -
471.omnetpp (avg) (percentage) 74.36% 0.780 1.360 - 0.00% -
456.hmmer (max) (percentage) 73.33% 0.300 0.520 - 0.00% -
433.milc (max) (percentage) 56.93% 1.370 2.150 - 0.00% -
471.omnetpp 55.42% 83.000 129.000 - 0.00% -
471.omnetpp (avg) 55.42% 83.000 129.000 - 0.00% -
456.hmmer (max) 52.50% 40.000 61.000 - 0.00% -
433.milc (max) 51.63% 184.000 279.000 - 0.00% -
470.lbm (avg) (percentage) 47.84% 2.320 3.430 - 0.00% -
458.sjeng 45.26% 95.000 138.000 - 0.00% -
458.sjeng (avg) 45.26% 95.000 138.000 - 0.00% -
447.dealII (max) (percentage) 40.51% 1.580 2.220 - 0.00% -
447.dealII (max) 37.98% 129.000 178.000 - 0.00% -
458.sjeng (avg) (percentage) 37.93% 0.580 0.800 - 0.00% -
403.gcc (avg) (percentage) 33.33% 0.480 0.640 - 0.00% -
403.gcc 32.56% 43.000 57.000 - 0.00% -
403.gcc (avg) 32.56% 43.000 57.000 - 0.00% -
470.lbm 30.12% 322.000 419.000 - 0.00% -
470.lbm (avg) 30.12% 322.000 419.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) 18.08% 6.140 7.250 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 16.13% 0.310 0.360 - 0.00% -
482.sphinx3 (avg) (percentage) 14.52% 1.860 2.130 - 0.00% -
453.povray (max) (percentage) 10.36% 1.930 2.130 - 0.00% -
401.bzip2 (unexplained diff vs previous) 10.20% 49.000 54.000 - 0.00% -
453.povray (max) 9.85% 132.000 145.000 - 0.00% -
453.povray (unexplained diff vs previous) (percentage) 9.47% 1.900 2.080 - 0.00% -
453.povray (unexplained diff vs previous) 9.23% 130.000 142.000 - 0.00% -
482.sphinx3 6.54% 489.000 521.000 - 0.00% -
482.sphinx3 (avg) 6.54% 489.000 521.000 - 0.00% -
444.namd (avg) (percentage) 5.77% 3.810 4.030 - 0.00% -
444.namd (max) (percentage) 3.36% 6.540 6.760 - 0.00% -
450.soplex (avg) (percentage) 2.78% 0.360 0.370 - 0.00% -
450.soplex 1.96% 51.000 52.000 - 0.00% -
450.soplex (avg) 1.96% 51.000 52.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk -18.33% 9427.000 7699.000 - 0.00% -
444.namd -15.73% 13495.000 11372.000 - 0.00% -
456.hmmer -12.22% 13365.000 11732.000 - 0.00% -
470.lbm -11.99% 13882.000 12218.000 - 0.00% -
471.omnetpp -11.31% 10746.000 9531.000 - 0.00% -
462.libquantum -10.74% 13126.000 11716.000 - 0.00% -
473.astar -9.90% 14262.000 12850.000 - 0.00% -
464.h264ref -8.25% 15493.000 14215.000 - 0.00% -
482.sphinx3 -6.95% 26310.000 24482.000 - 0.00% -
445.gobmk -5.88% 12400.000 11671.000 - 0.00% -
401.bzip2 -3.91% 15683.000 15070.000 - 0.00% -
433.milc -3.57% 13482.000 13001.000 - 0.00% -
403.gcc -2.04% 9099.000 8913.000 - 0.00% -
447.dealII -1.95% 8196.000 8036.000 - 0.00% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
470.lbm (unexplained diff vs previous) -98.59% 497.000 7.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -98.32% 3.580 0.060 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -93.36% 904.000 60.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -92.73% 3.440 0.250 - 0.00% -
473.astar (unexplained diff vs previous) -79.63% 108.000 22.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) -78.26% 92.000 20.000 - 0.00% -
473.astar (unexplained diff vs previous) (percentage) -77.63% 0.760 0.170 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) -75.58% 0.860 0.210 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) -71.43% 0.560 0.160 - 0.00% -
429.mcf (max) (percentage) -66.07% 0.560 0.190 - 0.00% -
429.mcf (unexplained diff vs previous) -65.67% 67.000 23.000 - 0.00% -
433.milc (unexplained diff vs previous) -60.44% 182.000 72.000 - 0.00% -
429.mcf (max) -59.70% 67.000 27.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -59.26% 1.350 0.550 - 0.00% -
471.omnetpp (max) -44.44% 189.000 105.000 - 0.00% -
471.omnetpp (max) (percentage) -36.93% 1.760 1.110 - 0.00% -
464.h264ref (max) -35.00% 60.000 39.000 - 0.00% -
450.soplex (max) -34.15% 82.000 54.000 - 0.00% -
450.soplex (unexplained diff vs previous) -34.15% 82.000 54.000 - 0.00% -
470.lbm (max) -33.81% 769.000 509.000 - 0.00% -
464.h264ref (unexplained diff vs previous) -33.33% 60.000 40.000 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.570 0.380 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) -33.33% 0.570 0.380 - 0.00% -
482.sphinx3 (max) -28.29% 873.000 626.000 - 0.00% -
464.h264ref (max) (percentage) -28.21% 0.390 0.280 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) -28.21% 0.390 0.280 - 0.00% -
470.lbm (max) (percentage) -24.73% 5.540 4.170 - 0.00% -
482.sphinx3 (max) (percentage) -22.89% 3.320 2.560 - 0.00% -
473.astar (max) -22.81% 114.000 88.000 - 0.00% -
433.milc -15.44% 136.000 115.000 - 0.00% -
433.milc (avg) -15.44% 136.000 115.000 - 0.00% -
447.dealII -15.00% 120.000 102.000 - 0.00% -
447.dealII (avg) -15.00% 120.000 102.000 - 0.00% -
473.astar (max) (percentage) -13.75% 0.800 0.690 - 0.00% -
444.namd (max) -12.93% 882.000 768.000 - 0.00% -
447.dealII (avg) (percentage) -12.93% 1.470 1.280 - 0.00% -
433.milc (avg) (percentage) -11.88% 1.010 0.890 - 0.00% -
444.namd -10.89% 514.000 458.000 - 0.00% -
444.namd (avg) -10.89% 514.000 458.000 - 0.00% -
453.povray (avg) (percentage) -4.38% 1.370 1.310 - 0.00% -
453.povray -4.30% 93.000 89.000 - 0.00% -
453.povray (avg) -4.30% 93.000 89.000 - 0.00% -

Report Time: 0.33s
Execution (seconds)
Name Current %
401.bzip2 15070.000 -
403.gcc 8913.000 -
429.mcf 14467.000 -
433.milc 13001.000 -
444.namd 11372.000 -
445.gobmk 11671.000 -
447.dealII 8036.000 -
450.soplex 14286.000 -
453.povray 6816.000 -
456.hmmer 11732.000 -
458.sjeng 17355.000 -
462.libquantum 11716.000 -
464.h264ref 14215.000 -
470.lbm 12218.000 -
471.omnetpp 9531.000 -
473.astar 12850.000 -
482.sphinx3 24482.000 -
483.xalancbmk 7699.000 -
Geometric Mean 11960.615 -
Code Size (bytes)
Name Current %
401.bzip2 39458.000 -
403.gcc 1959970.000 -
429.mcf 7915.000 -
433.milc 74971.000 -
444.namd 175160.000 -
445.gobmk 1312180.000 -
447.dealII 1778600.000 -
450.soplex 198708.000 -
453.povray 593201.000 -
456.hmmer 168088.000 -
458.sjeng 97694.000 -
462.libquantum 26756.000 -
464.h264ref 354546.000 -
470.lbm 9723.000 -
471.omnetpp 373571.000 -
473.astar 23216.000 -
482.sphinx3 115784.000 -
483.xalancbmk 2287109.000 -
Geometric Mean 164396.101 -
sample variability
Name Current %
401.bzip2 64.000 -
401.bzip2 (avg) 64.000 -
401.bzip2 (avg) (percentage) 0.430 -
401.bzip2 (max) 455.000 -
401.bzip2 (max) (percentage) 3.020 -
401.bzip2 (unexplained diff vs previous) 54.000 -
401.bzip2 (unexplained diff vs previous) (percentage) 0.360 -
403.gcc 57.000 -
403.gcc (avg) 57.000 -
403.gcc (avg) (percentage) 0.640 -
403.gcc (max) 139.000 -
403.gcc (max) (percentage) 1.560 -
429.mcf 201.000 -
429.mcf (avg) 201.000 -
429.mcf (avg) (percentage) 1.390 -
429.mcf (max) 27.000 -
429.mcf (max) (percentage) 0.190 -
429.mcf (unexplained diff vs previous) 23.000 -
429.mcf (unexplained diff vs previous) (percentage) 0.160 -
433.milc 115.000 -
433.milc (avg) 115.000 -
433.milc (avg) (percentage) 0.890 -
433.milc (max) 279.000 -
433.milc (max) (percentage) 2.150 -
433.milc (unexplained diff vs previous) 72.000 -
433.milc (unexplained diff vs previous) (percentage) 0.550 -
444.namd 458.000 -
444.namd (avg) 458.000 -
444.namd (avg) (percentage) 4.030 -
444.namd (max) 768.000 -
444.namd (max) (percentage) 6.760 -
444.namd (unexplained diff vs previous) 824.000 -
444.namd (unexplained diff vs previous) (percentage) 7.250 -
445.gobmk 58.000 -
445.gobmk (avg) 58.000 -
445.gobmk (avg) (percentage) 0.500 -
445.gobmk (max) 100.000 -
445.gobmk (max) (percentage) 0.860 -
445.gobmk (unexplained diff vs previous) 101.000 -
445.gobmk (unexplained diff vs previous) (percentage) 0.870 -
447.dealII 102.000 -
447.dealII (avg) 102.000 -
447.dealII (avg) (percentage) 1.280 -
447.dealII (max) 178.000 -
447.dealII (max) (percentage) 2.220 -
450.soplex 52.000 -
450.soplex (avg) 52.000 -
450.soplex (avg) (percentage) 0.370 -
450.soplex (max) 54.000 -
450.soplex (max) (percentage) 0.380 -
450.soplex (unexplained diff vs previous) 54.000 -
450.soplex (unexplained diff vs previous) (percentage) 0.380 -
453.povray 89.000 -
453.povray (avg) 89.000 -
453.povray (avg) (percentage) 1.310 -
453.povray (max) 145.000 -
453.povray (max) (percentage) 2.130 -
453.povray (unexplained diff vs previous) 142.000 -
453.povray (unexplained diff vs previous) (percentage) 2.080 -
456.hmmer 139.000 -
456.hmmer (avg) 139.000 -
456.hmmer (avg) (percentage) 1.190 -
456.hmmer (max) 61.000 -
456.hmmer (max) (percentage) 0.520 -
458.sjeng 138.000 -
458.sjeng (avg) 138.000 -
458.sjeng (avg) (percentage) 0.800 -
458.sjeng (max) 683.000 -
458.sjeng (max) (percentage) 3.940 -
458.sjeng (unexplained diff vs previous) 82.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.470 -
462.libquantum 92.000 -
462.libquantum (avg) 92.000 -
462.libquantum (avg) (percentage) 0.790 -
462.libquantum (max) 55.000 -
462.libquantum (max) (percentage) 0.470 -
462.libquantum (unexplained diff vs previous) 30.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.260 -
464.h264ref 95.000 -
464.h264ref (avg) 95.000 -
464.h264ref (avg) (percentage) 0.670 -
464.h264ref (max) 39.000 -
464.h264ref (max) (percentage) 0.280 -
464.h264ref (unexplained diff vs previous) 40.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.280 -
470.lbm 419.000 -
470.lbm (avg) 419.000 -
470.lbm (avg) (percentage) 3.430 -
470.lbm (max) 509.000 -
470.lbm (max) (percentage) 4.170 -
470.lbm (unexplained diff vs previous) 7.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.060 -
471.omnetpp 129.000 -
471.omnetpp (avg) 129.000 -
471.omnetpp (avg) (percentage) 1.360 -
471.omnetpp (max) 105.000 -
471.omnetpp (max) (percentage) 1.110 -
471.omnetpp (unexplained diff vs previous) 20.000 -
471.omnetpp (unexplained diff vs previous) (percentage) 0.210 -
473.astar 136.000 -
473.astar (avg) 136.000 -
473.astar (avg) (percentage) 1.060 -
473.astar (max) 88.000 -
473.astar (max) (percentage) 0.690 -
473.astar (unexplained diff vs previous) 22.000 -
473.astar (unexplained diff vs previous) (percentage) 0.170 -
482.sphinx3 521.000 -
482.sphinx3 (avg) 521.000 -
482.sphinx3 (avg) (percentage) 2.130 -
482.sphinx3 (max) 626.000 -
482.sphinx3 (max) (percentage) 2.560 -
482.sphinx3 (unexplained diff vs previous) 60.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.250 -
483.xalancbmk 140.000 -
483.xalancbmk (avg) 140.000 -
483.xalancbmk (avg) (percentage) 1.820 -
483.xalancbmk (max) 996.000 -
483.xalancbmk (max) (percentage) 12.940 -
Geometric Mean 14.755 -