Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/5f1448fe1585b5677d5f0064e4eeac3b493d8a18 (+ 1 commits)
regression LLVM-887
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6-llvm-arm-master-Oz_LTO-build/10/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0010 (llvmorg-17-init-00114-g5f1448fe1585) 2023-01-25T07:50:33 0:00:00
Previous 0009 (llvmorg-17-init-00113-g287508cd9c43) 2023-01-24T19:56:06 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 11 16
Performance Improvements 45 87
Unchanged Tests 724 677
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
470.lbm 1.76% 9338.000 9502.000 - 1.76% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (unexplained diff vs previous) 159.38% 32.000 83.000 - 80.43% -
456.hmmer (unexplained diff vs previous) (percentage) 159.26% 0.270 0.700 - 79.49% -
456.hmmer (max) (percentage) 48.94% 0.470 0.700 - -93.32% -
456.hmmer (max) 46.43% 56.000 82.000 - -93.40% -
429.mcf (max) 15.38% 13.000 15.000 - -99.34% -
429.mcf (unexplained diff vs previous) 7.14% 14.000 15.000 - -31.82% -
473.astar 1.39% 72.000 73.000 - -34.82% -
473.astar (avg) 1.39% 72.000 73.000 - -34.82% -
462.libquantum 1.19% 84.000 85.000 - -22.73% -
462.libquantum (avg) 1.19% 84.000 85.000 - -22.73% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (unexplained diff vs previous) (percentage) -89.77% 0.880 0.090 - -80.43% -
450.soplex (unexplained diff vs previous) -89.55% 134.000 14.000 - -79.71% -
471.omnetpp (unexplained diff vs previous) -51.11% 45.000 22.000 - -78.64% -
471.omnetpp (unexplained diff vs previous) (percentage) -50.00% 0.420 0.210 - -78.57% -
450.soplex (max) -25.19% 135.000 101.000 - -59.44% -
450.soplex (max) (percentage) -24.72% 0.890 0.670 - -59.39% -
473.astar (max) -11.11% 9.000 8.000 - -99.03% -
458.sjeng (max) -8.60% 93.000 85.000 - -90.30% -
444.namd (max) -7.81% 679.000 626.000 - -37.59% -
458.sjeng (max) (percentage) -7.27% 0.550 0.510 - -90.08% -
444.namd (max) (percentage) -7.22% 5.820 5.400 - -33.25% -
447.dealII (max) (percentage) -7.09% 1.270 1.180 - -50.42% -
447.dealII (max) -6.72% 119.000 111.000 - -50.22% -
429.mcf (avg) (percentage) -6.72% 1.340 1.250 - -41.04% -
429.mcf -6.42% 187.000 175.000 - -40.88% -
429.mcf (avg) -6.42% 187.000 175.000 - -40.88% -
453.povray (max) (percentage) -6.10% 2.130 2.000 - 455.56% -
453.povray (max) -5.92% 152.000 143.000 - 472.00% -
403.gcc (max) -5.88% 51.000 48.000 - -78.28% -
471.omnetpp -5.71% 140.000 132.000 - -20.96% -
471.omnetpp (avg) -5.71% 140.000 132.000 - -20.96% -
403.gcc (max) (percentage) -5.56% 0.540 0.510 - -78.11% -
471.omnetpp (avg) (percentage) -5.30% 1.320 1.250 - -21.88% -
433.milc (max) -4.86% 185.000 176.000 - 62.96% -
450.soplex (avg) (percentage) -4.35% 0.460 0.440 - -12.00% -
450.soplex -4.35% 69.000 66.000 - -12.00% -
450.soplex (avg) -4.35% 69.000 66.000 - -12.00% -
462.libquantum (max) -4.35% 69.000 66.000 - -92.82% -
433.milc (max) (percentage) -4.35% 1.380 1.320 - 65.00% -
464.h264ref (max) (percentage) -4.23% 0.710 0.680 - 61.90% -
462.libquantum (max) (percentage) -4.17% 0.480 0.460 - -92.79% -
483.xalancbmk (max) -4.08% 49.000 47.000 - -96.11% -
464.h264ref (max) -3.88% 103.000 99.000 - 65.00% -
483.xalancbmk (max) (percentage) -3.77% 0.530 0.510 - -96.05% -
482.sphinx3 (max) -3.68% 734.000 707.000 - -19.93% -
482.sphinx3 (max) (percentage) -3.64% 2.750 2.650 - -19.94% -
445.gobmk (max) -2.56% 312.000 304.000 - -26.39% -
456.hmmer -2.50% 120.000 117.000 - -28.66% -
456.hmmer (avg) -2.50% 120.000 117.000 - -28.66% -
445.gobmk (max) (percentage) -2.13% 2.820 2.760 - -26.01% -
456.hmmer (avg) (percentage) -1.98% 1.010 0.990 - -28.78% -
401.bzip2 -1.52% 66.000 65.000 - -37.50% -
401.bzip2 (avg) -1.52% 66.000 65.000 - -37.50% -
458.sjeng -1.00% 100.000 99.000 - -27.21% -
458.sjeng (avg) -1.00% 100.000 99.000 - -27.21% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm 1.76% 9338.000 9502.000 - 1.76% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
453.povray 2.48% 6987.000 7160.000 - 0.07% -
471.omnetpp 1.40% 10477.000 10624.000 - -0.21% -
464.h264ref 1.21% 14498.000 14673.000 - 0.87% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
453.povray (max) 472.00% 25.000 143.000 - -5.92% -
453.povray (max) (percentage) 455.56% 0.360 2.000 - -6.10% -
456.hmmer (unexplained diff vs previous) 80.43% 46.000 83.000 - 159.38% -
456.hmmer (unexplained diff vs previous) (percentage) 79.49% 0.390 0.700 - 159.26% -
433.milc (max) (percentage) 65.00% 0.800 1.320 - -4.35% -
464.h264ref (max) 65.00% 60.000 99.000 - -3.88% -
433.milc (max) 62.96% 108.000 176.000 - -4.86% -
464.h264ref (max) (percentage) 61.90% 0.420 0.680 - -4.23% -
464.h264ref (avg) (percentage) 17.24% 0.290 0.340 - 0.00% -
464.h264ref 16.67% 42.000 49.000 - 0.00% -
464.h264ref (avg) 16.67% 42.000 49.000 - 0.00% -
444.namd (avg) (percentage) 7.39% 2.570 2.760 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
444.namd -6.49% 12400.000 11595.000 - -0.71% -
433.milc -1.50% 13601.000 13397.000 - -0.28% -
458.sjeng -1.18% 17055.000 16853.000 - -0.84% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (max) -99.34% 2281.000 15.000 - 15.38% -
429.mcf (max) (percentage) -99.33% 16.310 0.110 - 10.00% -
473.astar (max) -99.03% 822.000 8.000 - -11.11% -
473.astar (max) (percentage) -98.96% 6.700 0.070 - -12.50% -
401.bzip2 (max) -97.37% 609.000 16.000 - 0.00% -
401.bzip2 (max) (percentage) -97.35% 3.780 0.100 - 0.00% -
483.xalancbmk (max) -96.11% 1209.000 47.000 - -4.08% -
483.xalancbmk (max) (percentage) -96.05% 12.900 0.510 - -3.77% -
456.hmmer (max) -93.40% 1242.000 82.000 - 46.43% -
456.hmmer (max) (percentage) -93.32% 10.480 0.700 - 48.94% -
462.libquantum (max) -92.82% 919.000 66.000 - -4.35% -
462.libquantum (max) (percentage) -92.79% 6.380 0.460 - -4.17% -
471.omnetpp (max) (percentage) -92.67% 9.280 0.680 - 0.00% -
471.omnetpp (max) -92.59% 972.000 72.000 - 0.00% -
458.sjeng (max) -90.30% 876.000 85.000 - -8.60% -
458.sjeng (max) (percentage) -90.08% 5.140 0.510 - -7.27% -
450.soplex (unexplained diff vs previous) (percentage) -80.43% 0.460 0.090 - -89.77% -
450.soplex (unexplained diff vs previous) -79.71% 69.000 14.000 - -89.55% -
471.omnetpp (unexplained diff vs previous) -78.64% 103.000 22.000 - -51.11% -
471.omnetpp (unexplained diff vs previous) (percentage) -78.57% 0.980 0.210 - -50.00% -
403.gcc (max) -78.28% 221.000 48.000 - -5.88% -
403.gcc (max) (percentage) -78.11% 2.330 0.510 - -5.56% -
450.soplex (max) -59.44% 249.000 101.000 - -25.19% -
450.soplex (max) (percentage) -59.39% 1.650 0.670 - -24.72% -
447.dealII (max) (percentage) -50.42% 2.380 1.180 - -7.09% -
447.dealII (max) -50.22% 223.000 111.000 - -6.72% -
470.lbm (max) -49.47% 758.000 383.000 - -0.78% -
470.lbm (max) (percentage) -49.30% 5.720 2.900 - 0.00% -
429.mcf (avg) (percentage) -41.04% 2.120 1.250 - -6.72% -
429.mcf -40.88% 296.000 175.000 - -6.42% -
429.mcf (avg) -40.88% 296.000 175.000 - -6.42% -
444.namd (max) -37.59% 1003.000 626.000 - -7.81% -
401.bzip2 -37.50% 104.000 65.000 - -1.52% -
401.bzip2 (avg) -37.50% 104.000 65.000 - -1.52% -
401.bzip2 (avg) (percentage) -36.92% 0.650 0.410 - 0.00% -
473.astar (avg) (percentage) -35.87% 0.920 0.590 - 0.00% -
483.xalancbmk (avg) (percentage) -34.91% 2.120 1.380 - 0.00% -
483.xalancbmk -34.85% 198.000 129.000 - 0.00% -
483.xalancbmk (avg) -34.85% 198.000 129.000 - 0.00% -
473.astar -34.82% 112.000 73.000 - 1.39% -
473.astar (avg) -34.82% 112.000 73.000 - 1.39% -
444.namd (max) (percentage) -33.25% 8.090 5.400 - -7.22% -
429.mcf (unexplained diff vs previous) -31.82% 22.000 15.000 - 7.14% -
429.mcf (unexplained diff vs previous) (percentage) -31.25% 0.160 0.110 - 10.00% -
456.hmmer (avg) (percentage) -28.78% 1.390 0.990 - -1.98% -
456.hmmer -28.66% 164.000 117.000 - -2.50% -
456.hmmer (avg) -28.66% 164.000 117.000 - -2.50% -
458.sjeng -27.21% 136.000 99.000 - -1.00% -
458.sjeng (avg) -27.21% 136.000 99.000 - -1.00% -
445.gobmk (max) -26.39% 413.000 304.000 - -2.56% -
458.sjeng (avg) (percentage) -26.25% 0.800 0.590 - 0.00% -
445.gobmk (max) (percentage) -26.01% 3.730 2.760 - -2.13% -
447.dealII (avg) (percentage) -23.81% 1.050 0.800 - 0.00% -
447.dealII -23.47% 98.000 75.000 - 0.00% -
447.dealII (avg) -23.47% 98.000 75.000 - 0.00% -
462.libquantum (avg) (percentage) -23.38% 0.770 0.590 - 0.00% -
445.gobmk -22.97% 74.000 57.000 - 0.00% -
445.gobmk (avg) -22.97% 74.000 57.000 - 0.00% -
462.libquantum -22.73% 110.000 85.000 - 1.19% -
462.libquantum (avg) -22.73% 110.000 85.000 - 1.19% -
445.gobmk (avg) (percentage) -22.39% 0.670 0.520 - 0.00% -
471.omnetpp (avg) (percentage) -21.88% 1.600 1.250 - -5.30% -
471.omnetpp -20.96% 167.000 132.000 - -5.71% -
471.omnetpp (avg) -20.96% 167.000 132.000 - -5.71% -
482.sphinx3 (max) (percentage) -19.94% 3.310 2.650 - -3.64% -
482.sphinx3 (max) -19.93% 883.000 707.000 - -3.68% -
470.lbm -15.19% 441.000 374.000 - -0.53% -
470.lbm (avg) -15.19% 441.000 374.000 - -0.53% -
470.lbm (avg) (percentage) -15.02% 3.330 2.830 - 0.00% -
482.sphinx3 -14.44% 464.000 397.000 - 0.00% -
482.sphinx3 (avg) -14.44% 464.000 397.000 - 0.00% -
482.sphinx3 (avg) (percentage) -14.37% 1.740 1.490 - 0.00% -
450.soplex -12.00% 75.000 66.000 - -4.35% -
450.soplex (avg) -12.00% 75.000 66.000 - -4.35% -
450.soplex (avg) (percentage) -12.00% 0.500 0.440 - -4.35% -
403.gcc -9.62% 52.000 47.000 - 0.00% -
403.gcc (avg) -9.62% 52.000 47.000 - 0.00% -
433.milc -9.48% 116.000 105.000 - -0.94% -
433.milc (avg) -9.48% 116.000 105.000 - -0.94% -
403.gcc (avg) (percentage) -9.09% 0.550 0.500 - 0.00% -
433.milc (avg) (percentage) -8.14% 0.860 0.790 - 0.00% -
453.povray (avg) (percentage) -6.25% 0.800 0.750 - 0.00% -
453.povray -3.64% 55.000 53.000 - 0.00% -
453.povray (avg) -3.64% 55.000 53.000 - 0.00% -

Report Time: 0.33s
Execution (seconds)
Name Current %
401.bzip2 16085.000 -
403.gcc 9474.000 -
429.mcf 14013.000 -
433.milc 13397.000 -
444.namd 11595.000 -
445.gobmk 11041.000 -
447.dealII 9428.000 -
450.soplex 15219.000 -
453.povray 7160.000 -
456.hmmer 11840.000 -
458.sjeng 16853.000 -
462.libquantum 14429.000 -
464.h264ref 14673.000 -
470.lbm 13236.000 -
471.omnetpp 10624.000 -
473.astar 12379.000 -
482.sphinx3 26688.000 -
483.xalancbmk 9366.000 -
Geometric Mean 12653.635 -
Code Size (bytes)
Name Current %
401.bzip2 33686.000 -
403.gcc 1669056.000 -
429.mcf 7239.000 -
433.milc 47001.000 -
444.namd 125187.000 -
445.gobmk 1260507.000 -
447.dealII 195784.000 -
450.soplex 146757.000 -
453.povray 524493.000 -
456.hmmer 80837.000 -
458.sjeng 80249.000 -
462.libquantum 12496.000 -
464.h264ref 303759.000 -
470.lbm 9502.000 1.76%
471.omnetpp 316648.000 -
473.astar 15482.000 -
482.sphinx3 87927.000 -
483.xalancbmk 1615640.000 -
Geometric Mean 111771.536 -
sample variability
Name Current %
401.bzip2 65.000 -1.52%
401.bzip2 (avg) 65.000 -1.52%
401.bzip2 (avg) (percentage) 0.410 -
401.bzip2 (max) 16.000 -
401.bzip2 (max) (percentage) 0.100 -
403.gcc 47.000 -
403.gcc (avg) 47.000 -
403.gcc (avg) (percentage) 0.500 -
403.gcc (max) 48.000 -5.88%
403.gcc (max) (percentage) 0.510 -5.56%
429.mcf 175.000 -6.42%
429.mcf (avg) 175.000 -6.42%
429.mcf (avg) (percentage) 1.250 -6.72%
429.mcf (max) 15.000 15.38%
429.mcf (max) (percentage) 0.110 -
429.mcf (unexplained diff vs previous) 15.000 7.14%
429.mcf (unexplained diff vs previous) (percentage) 0.110 -
433.milc 105.000 -
433.milc (avg) 105.000 -
433.milc (avg) (percentage) 0.790 -
433.milc (max) 176.000 -4.86%
433.milc (max) (percentage) 1.320 -4.35%
444.namd 320.000 -
444.namd (avg) 320.000 -
444.namd (avg) (percentage) 2.760 -
444.namd (max) 626.000 -7.81%
444.namd (max) (percentage) 5.400 -7.22%
445.gobmk 57.000 -
445.gobmk (avg) 57.000 -
445.gobmk (avg) (percentage) 0.520 -
445.gobmk (max) 304.000 -2.56%
445.gobmk (max) (percentage) 2.760 -2.13%
447.dealII 75.000 -
447.dealII (avg) 75.000 -
447.dealII (avg) (percentage) 0.800 -
447.dealII (max) 111.000 -6.72%
447.dealII (max) (percentage) 1.180 -7.09%
450.soplex 66.000 -4.35%
450.soplex (avg) 66.000 -4.35%
450.soplex (avg) (percentage) 0.440 -4.35%
450.soplex (max) 101.000 -25.19%
450.soplex (max) (percentage) 0.670 -24.72%
450.soplex (unexplained diff vs previous) 14.000 -89.55%
450.soplex (unexplained diff vs previous) (percentage) 0.090 -89.77%
453.povray 53.000 -
453.povray (avg) 53.000 -
453.povray (avg) (percentage) 0.750 -
453.povray (max) 143.000 -5.92%
453.povray (max) (percentage) 2.000 -6.10%
456.hmmer 117.000 -2.50%
456.hmmer (avg) 117.000 -2.50%
456.hmmer (avg) (percentage) 0.990 -1.98%
456.hmmer (max) 82.000 46.43%
456.hmmer (max) (percentage) 0.700 48.94%
456.hmmer (unexplained diff vs previous) 83.000 159.38%
456.hmmer (unexplained diff vs previous) (percentage) 0.700 159.26%
458.sjeng 99.000 -1.00%
458.sjeng (avg) 99.000 -1.00%
458.sjeng (avg) (percentage) 0.590 -
458.sjeng (max) 85.000 -8.60%
458.sjeng (max) (percentage) 0.510 -7.27%
462.libquantum 85.000 1.19%
462.libquantum (avg) 85.000 1.19%
462.libquantum (avg) (percentage) 0.590 -
462.libquantum (max) 66.000 -4.35%
462.libquantum (max) (percentage) 0.460 -4.17%
464.h264ref 49.000 -
464.h264ref (avg) 49.000 -
464.h264ref (avg) (percentage) 0.340 -
464.h264ref (max) 99.000 -3.88%
464.h264ref (max) (percentage) 0.680 -4.23%
470.lbm 374.000 -
470.lbm (avg) 374.000 -
470.lbm (avg) (percentage) 2.830 -
470.lbm (max) 383.000 -
470.lbm (max) (percentage) 2.900 -
471.omnetpp 132.000 -5.71%
471.omnetpp (avg) 132.000 -5.71%
471.omnetpp (avg) (percentage) 1.250 -5.30%
471.omnetpp (max) 72.000 -
471.omnetpp (max) (percentage) 0.680 -
471.omnetpp (unexplained diff vs previous) 22.000 -51.11%
471.omnetpp (unexplained diff vs previous) (percentage) 0.210 -50.00%
473.astar 73.000 1.39%
473.astar (avg) 73.000 1.39%
473.astar (avg) (percentage) 0.590 -
473.astar (max) 8.000 -11.11%
473.astar (max) (percentage) 0.070 -
482.sphinx3 397.000 -
482.sphinx3 (avg) 397.000 -
482.sphinx3 (avg) (percentage) 1.490 -
482.sphinx3 (max) 707.000 -3.68%
482.sphinx3 (max) (percentage) 2.650 -3.64%
483.xalancbmk 129.000 -
483.xalancbmk (avg) 129.000 -
483.xalancbmk (avg) (percentage) 1.380 -
483.xalancbmk (max) 47.000 -4.08%
483.xalancbmk (max) (percentage) 0.510 -3.77%
Geometric Mean 12.346 -5.44%