Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/dd5c4da5521930682b3f9c473004a25814f1dd6f (+ 2017 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Os-build/51/
View Options
Report
Filter

llvm-arm-master-Os test results

Run Order Start Time Duration
Current 0051 (llvmorg-17-init-18093-gdd5c4da55219) 2023-06-28T19:59:45 0:00:00
Previous 0045 (llvmorg-17-init-16076-g7075f9d926c9) 2023-06-12T01:29:55 0:00:00
Baseline 0001 (llvmorg-16-init-16061-ge7790fbed32b) 2022-12-29T19:41:07 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 9 36
Performance Improvements 9 76
Added Tests 2 2
Unchanged Tests 760 666
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex 3.09% 518.000 534.000 - -96.30% -
483.xalancbmk 1.70% 3598.000 3659.000 - -61.19% -
447.dealII 1.65% 1762.000 1791.000 - -78.15% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
447.dealII 5.88% 17.000 18.000 - -85.00% -
447.dealII (avg) 5.88% 17.000 18.000 - -85.00% -
483.xalancbmk 1.78% 169.000 172.000 - 265.96% -
483.xalancbmk (avg) 1.78% 169.000 172.000 - 265.96% -
464.h264ref 1.45% 69.000 70.000 - 37.25% -
464.h264ref (avg) 1.45% 69.000 70.000 - 37.25% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -3.28% 61.000 59.000 - -99.35% -
445.gobmk -2.20% 5274.000 5158.000 - -58.40% -
482.sphinx3 -1.79% 892.000 876.000 - -96.67% -
471.omnetpp -1.35% 2813.000 2775.000 - -74.18% -
429.mcf -1.18% 1188.000 1174.000 - -90.22% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
403.gcc (max) -33.33% 3.000 2.000 - -94.12% -
471.omnetpp (max) -3.12% 32.000 31.000 - -83.60% -
445.gobmk -2.08% 48.000 47.000 - 113.64% -
445.gobmk (avg) -2.08% 48.000 47.000 - 113.64% -

Added Tests - sample variability
403.gcc
403.gcc (avg)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.71% 198708.000 202098.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) 1215.79% 0.380 5.000 - 0.00% -
462.libquantum (avg) (percentage) 1040.00% 0.150 1.710 - 0.00% -
429.mcf (avg) (percentage) 891.11% 0.450 4.460 - 0.00% -
483.xalancbmk (avg) (percentage) 842.00% 0.500 4.710 - 0.00% -
473.astar (avg) (percentage) 666.00% 0.500 3.830 - 0.00% -
462.libquantum (max) (percentage) 650.00% 0.200 1.500 - 0.00% -
450.soplex (max) (percentage) 478.95% 0.570 3.300 - 0.00% -
471.omnetpp (avg) (percentage) 417.95% 0.780 4.040 - 0.00% -
445.gobmk (avg) (percentage) 411.11% 0.180 0.920 - 0.00% -
456.hmmer (avg) (percentage) 402.78% 0.360 1.810 - 0.00% -
464.h264ref (avg) (percentage) 284.85% 0.330 1.270 - 0.00% -
483.xalancbmk 265.96% 47.000 172.000 - 1.78% -
483.xalancbmk (avg) 265.96% 47.000 172.000 - 1.78% -
401.bzip2 (avg) (percentage) 250.00% 0.220 0.770 - 0.00% -
429.mcf (max) (percentage) 242.86% 0.560 1.920 - 0.00% -
403.gcc (avg) (percentage) 206.25% 0.480 1.470 - 0.00% -
458.sjeng (max) (percentage) 160.00% 0.400 1.040 - 0.00% -
473.astar 154.93% 71.000 181.000 - -0.55% -
473.astar (avg) 154.93% 71.000 181.000 - -0.55% -
445.gobmk 113.64% 22.000 47.000 - -2.08% -
445.gobmk (avg) 113.64% 22.000 47.000 - -2.08% -
450.soplex (avg) (percentage) 100.00% 0.360 0.720 - 0.00% -
458.sjeng (avg) (percentage) 94.83% 0.580 1.130 - 0.00% -
401.bzip2 (max) (percentage) 77.42% 0.310 0.550 - 0.00% -
483.xalancbmk (max) (percentage) 38.46% 0.650 0.900 - 0.00% -
464.h264ref 37.25% 51.000 70.000 - 1.45% -
464.h264ref (avg) 37.25% 51.000 70.000 - 1.45% -
471.omnetpp 34.94% 83.000 112.000 - -0.88% -
471.omnetpp (avg) 34.94% 83.000 112.000 - -0.88% -
456.hmmer 33.33% 48.000 64.000 - 0.00% -
456.hmmer (avg) 33.33% 48.000 64.000 - 0.00% -
458.sjeng (max) 29.23% 65.000 84.000 - 0.00% -
453.povray (avg) (percentage) 5.84% 1.370 1.450 - 0.00% -
470.lbm (avg) (percentage) 4.31% 2.320 2.420 - 0.00% -
482.sphinx3 (avg) (percentage) 1.08% 1.860 1.880 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -5.65% 26756.000 25243.000 - -0.09% -
453.povray -2.69% 593097.000 577118.000 - 0.02% -
433.milc -2.38% 74971.000 73190.000 - 0.10% -
401.bzip2 -2.17% 39458.000 38601.000 - -0.04% -
447.dealII -1.62% 1777992.000 1749195.000 - -0.03% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.35% 9099.000 59.000 - -3.28% -
462.libquantum -99.25% 13126.000 98.000 - 0.00% -
482.sphinx3 -96.67% 26310.000 876.000 - -1.79% -
450.soplex -96.30% 14429.000 534.000 - 3.09% -
444.namd -94.72% 13495.000 712.000 - 0.42% -
453.povray -93.19% 6857.000 467.000 - -0.43% -
433.milc -91.40% 13482.000 1159.000 - -0.77% -
429.mcf -90.22% 11999.000 1174.000 - -1.18% -
401.bzip2 -82.52% 15683.000 2741.000 - -0.58% -
447.dealII -78.15% 8196.000 1791.000 - 1.65% -
470.lbm -77.37% 13882.000 3141.000 - -0.16% -
471.omnetpp -74.18% 10746.000 2775.000 - -1.35% -
456.hmmer -73.33% 13365.000 3565.000 - 0.76% -
473.astar -66.74% 14262.000 4743.000 - -0.50% -
464.h264ref -64.30% 15493.000 5531.000 - 0.38% -
483.xalancbmk -61.19% 9427.000 3659.000 - 1.70% -
445.gobmk -58.40% 12400.000 5158.000 - -2.20% -
458.sjeng -50.25% 16406.000 8162.000 - -0.10% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
482.sphinx3 (max) -99.20% 873.000 7.000 - 0.00% -
444.namd (max) -98.98% 882.000 9.000 - 0.00% -
482.sphinx3 -96.73% 489.000 16.000 - 0.00% -
482.sphinx3 (avg) -96.73% 489.000 16.000 - 0.00% -
470.lbm (max) -96.36% 769.000 28.000 - 0.00% -
444.namd -96.30% 514.000 19.000 - 0.00% -
444.namd (avg) -96.30% 514.000 19.000 - 0.00% -
453.povray (max) -96.21% 132.000 5.000 - 0.00% -
462.libquantum (max) -96.15% 26.000 1.000 - 0.00% -
433.milc (max) -95.11% 184.000 9.000 - 0.00% -
462.libquantum -94.74% 19.000 1.000 - 0.00% -
462.libquantum (avg) -94.74% 19.000 1.000 - 0.00% -
403.gcc (max) -94.12% 34.000 2.000 - -33.33% -
450.soplex -94.12% 51.000 3.000 - 0.00% -
450.soplex (avg) -94.12% 51.000 3.000 - 0.00% -
453.povray -93.55% 93.000 6.000 - 0.00% -
453.povray (avg) -93.55% 93.000 6.000 - 0.00% -
433.milc -93.38% 136.000 9.000 - 0.00% -
433.milc (avg) -93.38% 136.000 9.000 - 0.00% -
447.dealII (max) -93.02% 129.000 9.000 - 0.00% -
473.astar (max) -87.72% 114.000 14.000 - 0.00% -
447.dealII -85.00% 120.000 18.000 - 5.88% -
447.dealII (avg) -85.00% 120.000 18.000 - 5.88% -
471.omnetpp (max) -83.60% 189.000 31.000 - -3.12% -
470.lbm (max) (percentage) -83.57% 5.540 0.910 - 0.00% -
456.hmmer (max) -80.00% 40.000 8.000 - 0.00% -
444.namd (max) (percentage) -79.51% 6.540 1.340 - 0.00% -
450.soplex (max) -79.27% 82.000 17.000 - 0.00% -
464.h264ref (max) -78.33% 60.000 13.000 - 0.00% -
470.lbm -76.40% 322.000 76.000 - 0.00% -
470.lbm (avg) -76.40% 322.000 76.000 - 0.00% -
482.sphinx3 (max) (percentage) -75.30% 3.320 0.820 - 0.00% -
401.bzip2 (max) -68.75% 48.000 15.000 - 0.00% -
429.mcf (max) -67.16% 67.000 22.000 - 0.00% -
447.dealII (max) (percentage) -67.09% 1.580 0.520 - 0.00% -
445.gobmk (max) -62.07% 29.000 11.000 - 0.00% -
473.astar (max) (percentage) -61.25% 0.800 0.310 - 0.00% -
483.xalancbmk (max) -47.54% 61.000 32.000 - 0.00% -
433.milc (max) (percentage) -43.07% 1.370 0.780 - 0.00% -
464.h264ref (max) (percentage) -38.46% 0.390 0.240 - 0.00% -
401.bzip2 -38.24% 34.000 21.000 - 0.00% -
401.bzip2 (avg) -38.24% 34.000 21.000 - 0.00% -
471.omnetpp (max) (percentage) -34.66% 1.760 1.150 - 0.00% -
453.povray (max) (percentage) -34.20% 1.930 1.270 - 0.00% -
447.dealII (avg) (percentage) -31.29% 1.470 1.010 - 0.00% -
444.namd (avg) (percentage) -29.40% 3.810 2.690 - 0.00% -
433.milc (avg) (percentage) -19.80% 1.010 0.810 - 0.00% -
456.hmmer (max) (percentage) -16.67% 0.300 0.250 - 0.00% -
445.gobmk (max) (percentage) -8.33% 0.240 0.220 - 0.00% -
458.sjeng -3.16% 95.000 92.000 - 0.00% -
458.sjeng (avg) -3.16% 95.000 92.000 - 0.00% -
429.mcf -1.89% 53.000 52.000 - 0.00% -
429.mcf (avg) -1.89% 53.000 52.000 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)

Report Time: 0.35s
Execution (seconds)
Name Current %
401.bzip2 2741.000 -
403.gcc 59.000 -3.28%
429.mcf 1174.000 -1.18%
433.milc 1159.000 -
444.namd 712.000 -
445.gobmk 5158.000 -2.20%
447.dealII 1791.000 1.65%
450.soplex 534.000 3.09%
453.povray 467.000 -
456.hmmer 3565.000 -
458.sjeng 8162.000 -
462.libquantum 98.000 -
464.h264ref 5531.000 -
470.lbm 3141.000 -
471.omnetpp 2775.000 -1.35%
473.astar 4743.000 -
482.sphinx3 876.000 -1.79%
483.xalancbmk 3659.000 1.70%
Geometric Mean 1444.990 -
Code Size (bytes)
Name Current %
401.bzip2 38601.000 -
403.gcc 1960265.000 -
429.mcf 7910.000 -
433.milc 73190.000 -
444.namd 175681.000 -
445.gobmk 1310832.000 -
447.dealII 1749195.000 -
450.soplex 202098.000 -
453.povray 577118.000 -
456.hmmer 166398.000 -
458.sjeng 97029.000 -
462.libquantum 25243.000 -
464.h264ref 352814.000 -
470.lbm 9734.000 -
471.omnetpp 376659.000 -
473.astar 23115.000 -
482.sphinx3 114671.000 -
483.xalancbmk 2286516.000 -
Geometric Mean 162970.796 -
sample variability
Name Current %
401.bzip2 21.000 -
401.bzip2 (avg) 21.000 -
401.bzip2 (avg) (percentage) 0.770 -
401.bzip2 (max) 15.000 -
401.bzip2 (max) (percentage) 0.550 -
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 1.470 -
403.gcc (max) 2.000 -33.33%
403.gcc (max) (percentage) 5.000 -
429.mcf 52.000 -
429.mcf (avg) 52.000 -
429.mcf (avg) (percentage) 4.460 -
429.mcf (max) 22.000 -
429.mcf (max) (percentage) 1.920 -
433.milc 9.000 -
433.milc (avg) 9.000 -
433.milc (avg) (percentage) 0.810 -
433.milc (max) 9.000 -
433.milc (max) (percentage) 0.780 -
444.namd 19.000 -
444.namd (avg) 19.000 -
444.namd (avg) (percentage) 2.690 -
444.namd (max) 9.000 -
444.namd (max) (percentage) 1.340 -
445.gobmk 47.000 -2.08%
445.gobmk (avg) 47.000 -2.08%
445.gobmk (avg) (percentage) 0.920 -
445.gobmk (max) 11.000 -
445.gobmk (max) (percentage) 0.220 -
447.dealII 18.000 5.88%
447.dealII (avg) 18.000 5.88%
447.dealII (avg) (percentage) 1.010 -
447.dealII (max) 9.000 -
447.dealII (max) (percentage) 0.520 -
450.soplex 3.000 -
450.soplex (avg) 3.000 -
450.soplex (avg) (percentage) 0.720 -
450.soplex (max) 17.000 -
450.soplex (max) (percentage) 3.300 -
453.povray 6.000 -
453.povray (avg) 6.000 -
453.povray (avg) (percentage) 1.450 -
453.povray (max) 5.000 -
453.povray (max) (percentage) 1.270 -
456.hmmer 64.000 -
456.hmmer (avg) 64.000 -
456.hmmer (avg) (percentage) 1.810 -
456.hmmer (max) 8.000 -
456.hmmer (max) (percentage) 0.250 -
458.sjeng 92.000 -
458.sjeng (avg) 92.000 -
458.sjeng (avg) (percentage) 1.130 -
458.sjeng (max) 84.000 -
458.sjeng (max) (percentage) 1.040 -
462.libquantum 1.000 -
462.libquantum (avg) 1.000 -
462.libquantum (avg) (percentage) 1.710 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.500 -
464.h264ref 70.000 1.45%
464.h264ref (avg) 70.000 1.45%
464.h264ref (avg) (percentage) 1.270 -
464.h264ref (max) 13.000 -
464.h264ref (max) (percentage) 0.240 -
470.lbm 76.000 -
470.lbm (avg) 76.000 -
470.lbm (avg) (percentage) 2.420 -
470.lbm (max) 28.000 -
470.lbm (max) (percentage) 0.910 -
471.omnetpp 112.000 -
471.omnetpp (avg) 112.000 -
471.omnetpp (avg) (percentage) 4.040 -
471.omnetpp (max) 31.000 -3.12%
471.omnetpp (max) (percentage) 1.150 -
473.astar 181.000 -
473.astar (avg) 181.000 -
473.astar (avg) (percentage) 3.830 -
473.astar (max) 14.000 -
473.astar (max) (percentage) 0.310 -
482.sphinx3 16.000 -
482.sphinx3 (avg) 16.000 -
482.sphinx3 (avg) (percentage) 1.880 -
482.sphinx3 (max) 7.000 -
482.sphinx3 (max) (percentage) 0.820 -
483.xalancbmk 172.000 1.78%
483.xalancbmk (avg) 172.000 1.78%
483.xalancbmk (avg) (percentage) 4.710 -
483.xalancbmk (max) 32.000 -
483.xalancbmk (max) (percentage) 0.900 -
Geometric Mean 5.062 -22.12%