Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/0992e6ee115e701ab9034844060581cf105bd3b5 (+ 385 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6-llvm-arm-master-Oz_LTO-build/16/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0016 (llvmorg-17-init-02284-g0992e6ee115e) 2023-02-12T10:00:27 0:00:00
Previous 0016 (llvmorg-17-init-02284-g0992e6ee115e) 2023-02-12T10:00:27 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 0 114
Performance Improvements 0 15
Added Tests 0 6
Unchanged Tests 780 645
Total Tests 780

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk 15.44% 9376.000 10824.000 - 0.00% -
471.omnetpp 13.73% 10477.000 11915.000 - 0.00% -
470.lbm 10.79% 13259.000 14689.000 - 0.00% -
456.hmmer 10.22% 11854.000 13065.000 - 0.00% -
473.astar 9.81% 12272.000 13476.000 - 0.00% -
462.libquantum 7.65% 14409.000 15512.000 - 0.00% -
464.h264ref 7.24% 14498.000 15547.000 - 0.00% -
482.sphinx3 6.70% 26682.000 28469.000 - 0.00% -
445.gobmk 4.72% 11093.000 11617.000 - 0.00% -
401.bzip2 3.36% 16137.000 16680.000 - 0.00% -
447.dealII 3.32% 9380.000 9691.000 - 0.00% -
403.gcc 2.25% 9509.000 9723.000 - 0.00% -
433.milc 1.37% 13601.000 13787.000 - 0.00% -
450.soplex 1.09% 15112.000 15277.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) 112500.00% 1.000 1126.000 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) 72500.00% 0.010 7.260 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) 13937.50% 0.160 22.460 - 0.00% -
429.mcf (unexplained diff vs previous) 11563.64% 22.000 2566.000 - 0.00% -
473.astar (unexplained diff vs previous) 4556.00% 25.000 1164.000 - 0.00% -
473.astar (unexplained diff vs previous) (percentage) 4220.00% 0.200 8.640 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) 4183.33% 0.060 2.570 - 0.00% -
444.namd (unexplained diff vs previous) 3787.50% 8.000 311.000 - 0.00% -
456.hmmer (unexplained diff vs previous) 2565.22% 46.000 1226.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) 2305.13% 0.390 9.380 - 0.00% -
464.h264ref (max) 1733.33% 60.000 1100.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) 1590.91% 33.000 558.000 - 0.00% -
464.h264ref (max) (percentage) 1585.71% 0.420 7.080 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 1575.00% 0.200 3.350 - 0.00% -
433.milc (unexplained diff vs previous) 1323.08% 13.000 185.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) 1240.00% 0.100 1.340 - 0.00% -
471.omnetpp (unexplained diff vs previous) 1118.45% 103.000 1255.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 974.49% 0.980 10.530 - 0.00% -
447.dealII (unexplained diff vs previous) 555.56% 54.000 354.000 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) 529.31% 0.580 3.650 - 0.00% -
464.h264ref 490.48% 42.000 248.000 - 0.00% -
464.h264ref (avg) 490.48% 42.000 248.000 - 0.00% -
464.h264ref (avg) (percentage) 451.72% 0.290 1.600 - 0.00% -
450.soplex (unexplained diff vs previous) 308.70% 69.000 282.000 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 302.17% 0.460 1.850 - 0.00% -
453.povray (max) (percentage) 261.11% 0.360 1.300 - 0.00% -
453.povray (max) 260.00% 25.000 90.000 - 0.00% -
473.astar 154.46% 112.000 285.000 - 0.00% -
473.astar (avg) 154.46% 112.000 285.000 - 0.00% -
462.libquantum 150.91% 110.000 276.000 - 0.00% -
462.libquantum (avg) 150.91% 110.000 276.000 - 0.00% -
462.libquantum (avg) (percentage) 131.17% 0.770 1.780 - 0.00% -
473.astar (avg) (percentage) 130.43% 0.920 2.120 - 0.00% -
456.hmmer 117.07% 164.000 356.000 - 0.00% -
456.hmmer (avg) 117.07% 164.000 356.000 - 0.00% -
483.xalancbmk 110.61% 198.000 417.000 - 0.00% -
483.xalancbmk (avg) 110.61% 198.000 417.000 - 0.00% -
482.sphinx3 (max) 99.55% 883.000 1762.000 - 0.00% -
470.lbm (max) 99.47% 758.000 1512.000 - 0.00% -
429.mcf (avg) (percentage) 97.64% 2.120 4.190 - 0.00% -
471.omnetpp 96.41% 167.000 328.000 - 0.00% -
471.omnetpp (avg) 96.41% 167.000 328.000 - 0.00% -
456.hmmer (avg) (percentage) 96.40% 1.390 2.730 - 0.00% -
482.sphinx3 (max) (percentage) 87.01% 3.310 6.190 - 0.00% -
458.sjeng (avg) (percentage) 85.00% 0.800 1.480 - 0.00% -
483.xalancbmk (avg) (percentage) 82.08% 2.120 3.860 - 0.00% -
470.lbm (max) (percentage) 80.07% 5.720 10.300 - 0.00% -
458.sjeng 75.74% 136.000 239.000 - 0.00% -
458.sjeng (avg) 75.74% 136.000 239.000 - 0.00% -
433.milc (max) 73.15% 108.000 187.000 - 0.00% -
471.omnetpp (avg) (percentage) 72.50% 1.600 2.760 - 0.00% -
470.lbm (unexplained diff vs previous) 70.65% 804.000 1372.000 - 0.00% -
433.milc (max) (percentage) 70.00% 0.800 1.360 - 0.00% -
445.gobmk 67.57% 74.000 124.000 - 0.00% -
445.gobmk (avg) 67.57% 74.000 124.000 - 0.00% -
447.dealII (max) 64.57% 223.000 367.000 - 0.00% -
429.mcf 61.49% 296.000 478.000 - 0.00% -
429.mcf (avg) 61.49% 296.000 478.000 - 0.00% -
445.gobmk (avg) (percentage) 59.70% 0.670 1.070 - 0.00% -
447.dealII (max) (percentage) 59.24% 2.380 3.790 - 0.00% -
473.astar (max) 54.87% 822.000 1273.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) 54.13% 6.060 9.340 - 0.00% -
401.bzip2 49.04% 104.000 155.000 - 0.00% -
401.bzip2 (avg) 49.04% 104.000 155.000 - 0.00% -
482.sphinx3 44.61% 464.000 671.000 - 0.00% -
482.sphinx3 (avg) 44.61% 464.000 671.000 - 0.00% -
471.omnetpp (max) 44.24% 972.000 1402.000 - 0.00% -
401.bzip2 (avg) (percentage) 43.08% 0.650 0.930 - 0.00% -
473.astar (max) (percentage) 41.04% 6.700 9.450 - 0.00% -
403.gcc 38.46% 52.000 72.000 - 0.00% -
403.gcc (avg) 38.46% 52.000 72.000 - 0.00% -
403.gcc (avg) (percentage) 36.36% 0.550 0.750 - 0.00% -
482.sphinx3 (avg) (percentage) 35.63% 1.740 2.360 - 0.00% -
462.libquantum (max) 32.10% 919.000 1214.000 - 0.00% -
483.xalancbmk (max) 27.63% 1209.000 1543.000 - 0.00% -
471.omnetpp (max) (percentage) 26.83% 9.280 11.770 - 0.00% -
450.soplex 25.33% 75.000 94.000 - 0.00% -
450.soplex (avg) 25.33% 75.000 94.000 - 0.00% -
450.soplex (avg) (percentage) 24.00% 0.500 0.620 - 0.00% -
470.lbm 23.81% 441.000 546.000 - 0.00% -
470.lbm (avg) 23.81% 441.000 546.000 - 0.00% -
462.libquantum (max) (percentage) 22.73% 6.380 7.830 - 0.00% -
447.dealII 22.45% 98.000 120.000 - 0.00% -
447.dealII (avg) 22.45% 98.000 120.000 - 0.00% -
447.dealII (avg) (percentage) 18.10% 1.050 1.240 - 0.00% -
450.soplex (max) 15.26% 249.000 287.000 - 0.00% -
444.namd (avg) (percentage) 14.01% 2.570 2.930 - 0.00% -
450.soplex (max) (percentage) 13.94% 1.650 1.880 - 0.00% -
401.bzip2 (max) 13.79% 609.000 693.000 - 0.00% -
429.mcf (max) (percentage) 12.45% 16.310 18.340 - 0.00% -
470.lbm (avg) (percentage) 11.71% 3.330 3.720 - 0.00% -
444.namd 11.32% 318.000 354.000 - 0.00% -
444.namd (avg) 11.32% 318.000 354.000 - 0.00% -
483.xalancbmk (max) (percentage) 10.54% 12.900 14.260 - 0.00% -
403.gcc (max) 10.41% 221.000 244.000 - 0.00% -
401.bzip2 (max) (percentage) 10.05% 3.780 4.160 - 0.00% -
456.hmmer (max) 8.94% 1242.000 1353.000 - 0.00% -
445.gobmk (max) 8.47% 413.000 448.000 - 0.00% -
403.gcc (max) (percentage) 7.73% 2.330 2.510 - 0.00% -
445.gobmk (max) (percentage) 3.49% 3.730 3.860 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf -18.31% 13988.000 11427.000 - 0.00% -
458.sjeng -5.24% 17055.000 16161.000 - 0.00% -
444.namd -2.47% 12400.000 12094.000 - 0.00% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -45.66% 1003.000 545.000 - 0.00% -
444.namd (max) (percentage) -44.25% 8.090 4.510 - 0.00% -
458.sjeng (max) -15.53% 876.000 740.000 - 0.00% -
458.sjeng (max) (percentage) -10.89% 5.140 4.580 - 0.00% -
433.milc (avg) (percentage) -9.30% 0.860 0.780 - 0.00% -
429.mcf (max) -8.15% 2281.000 2095.000 - 0.00% -
433.milc -7.76% 116.000 107.000 - 0.00% -
433.milc (avg) -7.76% 116.000 107.000 - 0.00% -
453.povray (avg) (percentage) -7.50% 0.800 0.740 - 0.00% -
453.povray -7.27% 55.000 51.000 - 0.00% -
453.povray (avg) -7.27% 55.000 51.000 - 0.00% -
456.hmmer (max) (percentage) -1.15% 10.480 10.360 - 0.00% -

Added Tests - sample variability
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)
483.xalancbmk (unexplained diff vs previous)
483.xalancbmk (unexplained diff vs previous) (percentage)

Report Time: 0.34s
Execution (seconds)
Name Current %
401.bzip2 16680.000 -
403.gcc 9723.000 -
429.mcf 11427.000 -
433.milc 13787.000 -
444.namd 12094.000 -
445.gobmk 11617.000 -
447.dealII 9691.000 -
450.soplex 15277.000 -
453.povray 6990.000 -
456.hmmer 13065.000 -
458.sjeng 16161.000 -
462.libquantum 15512.000 -
464.h264ref 15547.000 -
470.lbm 14689.000 -
471.omnetpp 11915.000 -
473.astar 13476.000 -
482.sphinx3 28469.000 -
483.xalancbmk 10824.000 -
Geometric Mean 13145.933 -
Code Size (bytes)
Name Current %
401.bzip2 33698.000 -
403.gcc 1669436.000 -
429.mcf 7239.000 -
433.milc 47001.000 -
444.namd 124963.000 -
445.gobmk 1260795.000 -
447.dealII 195792.000 -
450.soplex 146773.000 -
453.povray 524437.000 -
456.hmmer 80877.000 -
458.sjeng 80185.000 -
462.libquantum 12504.000 -
464.h264ref 303799.000 -
470.lbm 9338.000 -
471.omnetpp 316668.000 -
473.astar 15470.000 -
482.sphinx3 87927.000 -
483.xalancbmk 1615520.000 -
Geometric Mean 111655.703 -
sample variability
Name Current %
401.bzip2 155.000 -
401.bzip2 (avg) 155.000 -
401.bzip2 (avg) (percentage) 0.930 -
401.bzip2 (max) 693.000 -
401.bzip2 (max) (percentage) 4.160 -
401.bzip2 (unexplained diff vs previous) 558.000 -
401.bzip2 (unexplained diff vs previous) (percentage) 3.350 -
403.gcc 72.000 -
403.gcc (avg) 72.000 -
403.gcc (avg) (percentage) 0.750 -
403.gcc (max) 244.000 -
403.gcc (max) (percentage) 2.510 -
429.mcf 478.000 -
429.mcf (avg) 478.000 -
429.mcf (avg) (percentage) 4.190 -
429.mcf (max) 2095.000 -
429.mcf (max) (percentage) 18.340 -
429.mcf (unexplained diff vs previous) 2566.000 -
429.mcf (unexplained diff vs previous) (percentage) 22.460 -
433.milc 107.000 -
433.milc (avg) 107.000 -
433.milc (avg) (percentage) 0.780 -
433.milc (max) 187.000 -
433.milc (max) (percentage) 1.360 -
433.milc (unexplained diff vs previous) 185.000 -
433.milc (unexplained diff vs previous) (percentage) 1.340 -
444.namd 354.000 -
444.namd (avg) 354.000 -
444.namd (avg) (percentage) 2.930 -
444.namd (max) 545.000 -
444.namd (max) (percentage) 4.510 -
444.namd (unexplained diff vs previous) 311.000 -
444.namd (unexplained diff vs previous) (percentage) 2.570 -
445.gobmk 124.000 -
445.gobmk (avg) 124.000 -
445.gobmk (avg) (percentage) 1.070 -
445.gobmk (max) 448.000 -
445.gobmk (max) (percentage) 3.860 -
447.dealII 120.000 -
447.dealII (avg) 120.000 -
447.dealII (avg) (percentage) 1.240 -
447.dealII (max) 367.000 -
447.dealII (max) (percentage) 3.790 -
447.dealII (unexplained diff vs previous) 354.000 -
447.dealII (unexplained diff vs previous) (percentage) 3.650 -
450.soplex 94.000 -
450.soplex (avg) 94.000 -
450.soplex (avg) (percentage) 0.620 -
450.soplex (max) 287.000 -
450.soplex (max) (percentage) 1.880 -
450.soplex (unexplained diff vs previous) 282.000 -
450.soplex (unexplained diff vs previous) (percentage) 1.850 -
453.povray 51.000 -
453.povray (avg) 51.000 -
453.povray (avg) (percentage) 0.740 -
453.povray (max) 90.000 -
453.povray (max) (percentage) 1.300 -
453.povray (unexplained diff vs previous) 52.000 -
453.povray (unexplained diff vs previous) (percentage) 0.740 -
456.hmmer 356.000 -
456.hmmer (avg) 356.000 -
456.hmmer (avg) (percentage) 2.730 -
456.hmmer (max) 1353.000 -
456.hmmer (max) (percentage) 10.360 -
456.hmmer (unexplained diff vs previous) 1226.000 -
456.hmmer (unexplained diff vs previous) (percentage) 9.380 -
458.sjeng 239.000 -
458.sjeng (avg) 239.000 -
458.sjeng (avg) (percentage) 1.480 -
458.sjeng (max) 740.000 -
458.sjeng (max) (percentage) 4.580 -
458.sjeng (unexplained diff vs previous) 697.000 -
458.sjeng (unexplained diff vs previous) (percentage) 4.310 -
462.libquantum 276.000 -
462.libquantum (avg) 276.000 -
462.libquantum (avg) (percentage) 1.780 -
462.libquantum (max) 1214.000 -
462.libquantum (max) (percentage) 7.830 -
462.libquantum (unexplained diff vs previous) 1126.000 -
462.libquantum (unexplained diff vs previous) (percentage) 7.260 -
464.h264ref 248.000 -
464.h264ref (avg) 248.000 -
464.h264ref (avg) (percentage) 1.600 -
464.h264ref (max) 1100.000 -
464.h264ref (max) (percentage) 7.080 -
464.h264ref (unexplained diff vs previous) 953.000 -
464.h264ref (unexplained diff vs previous) (percentage) 6.130 -
470.lbm 546.000 -
470.lbm (avg) 546.000 -
470.lbm (avg) (percentage) 3.720 -
470.lbm (max) 1512.000 -
470.lbm (max) (percentage) 10.300 -
470.lbm (unexplained diff vs previous) 1372.000 -
470.lbm (unexplained diff vs previous) (percentage) 9.340 -
471.omnetpp 328.000 -
471.omnetpp (avg) 328.000 -
471.omnetpp (avg) (percentage) 2.760 -
471.omnetpp (max) 1402.000 -
471.omnetpp (max) (percentage) 11.770 -
471.omnetpp (unexplained diff vs previous) 1255.000 -
471.omnetpp (unexplained diff vs previous) (percentage) 10.530 -
473.astar 285.000 -
473.astar (avg) 285.000 -
473.astar (avg) (percentage) 2.120 -
473.astar (max) 1273.000 -
473.astar (max) (percentage) 9.450 -
473.astar (unexplained diff vs previous) 1164.000 -
473.astar (unexplained diff vs previous) (percentage) 8.640 -
482.sphinx3 671.000 -
482.sphinx3 (avg) 671.000 -
482.sphinx3 (avg) (percentage) 2.360 -
482.sphinx3 (max) 1762.000 -
482.sphinx3 (max) (percentage) 6.190 -
483.xalancbmk 417.000 -
483.xalancbmk (avg) 417.000 -
483.xalancbmk (avg) (percentage) 3.860 -
483.xalancbmk (max) 1543.000 -
483.xalancbmk (max) (percentage) 14.260 -
483.xalancbmk (unexplained diff vs previous) 1351.000 -
483.xalancbmk (unexplained diff vs previous) (percentage) 12.480 -
Geometric Mean 50.865 -