Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/fa1f88cdeca245497843c507167b84f5522e88cc (+ 295 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-aarch64-master-Oz_LTO-build/31/
View Options
Report
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0031 (llvmorg-17-init-10786-gfa1f88cdeca2) 2023-05-04T11:50:38 0:00:00
Previous 0030 (llvmorg-17-init-10491-g728b8a139804) 2023-04-28T15:57:24 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 8 41
Performance Improvements 31 74
Added Tests 5 7
Unchanged Tests 700 622
Total Tests 744

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 5.80% 69.000 73.000 - -99.28% -
400.perlbench 1.63% 1835.000 1865.000 - -83.18% -
453.povray 1.52% 527.000 535.000 - -92.67% -
483.xalancbmk 1.05% 4961.000 5013.000 - -47.96% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (max) 100.00% 4.000 8.000 - -97.03% -
433.milc (max) (percentage) 85.00% 0.400 0.740 - -60.00% -
429.mcf (unexplained diff vs previous) 80.00% 5.000 9.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) 77.78% 0.270 0.480 - 0.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -1.46% 137.000 135.000 - -98.60% -
471.omnetpp -1.35% 2961.000 2921.000 - -75.49% -
444.namd -1.01% 694.000 687.000 - -92.67% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (max) -9.52% 21.000 19.000 - -45.71% -
447.dealII (avg) (percentage) -8.99% 0.890 0.810 - -10.99% -
458.sjeng (max) (percentage) -6.90% 0.290 0.270 - 17.39% -
429.mcf (max) -6.67% 15.000 14.000 - -79.41% -
447.dealII -6.25% 16.000 15.000 - -81.71% -
447.dealII (avg) -6.25% 16.000 15.000 - -81.71% -
401.bzip2 (max) (percentage) -6.10% 0.820 0.770 - 1440.00% -
429.mcf -5.63% 71.000 67.000 - -2.90% -
429.mcf (avg) -5.63% 71.000 67.000 - -2.90% -
473.astar (max) -5.56% 18.000 17.000 - 70.00% -
473.astar (max) (percentage) -5.56% 0.360 0.340 - 325.00% -
429.mcf (avg) (percentage) -5.25% 3.810 3.610 - 875.68% -
471.omnetpp (max) -5.17% 58.000 55.000 - -53.78% -
470.lbm (max) (percentage) -5.00% 0.400 0.380 - 533.33% -
401.bzip2 (max) -5.00% 20.000 19.000 - 216.67% -
429.mcf (max) (percentage) -4.82% 0.830 0.790 - 119.44% -
471.omnetpp (max) (percentage) -3.55% 1.970 1.900 - 90.00% -
482.sphinx3 (max) (percentage) -2.75% 2.180 2.120 - 207.25% -
445.gobmk (max) (percentage) -2.63% 1.520 1.480 - 146.67% -
456.hmmer (max) (percentage) -2.63% 0.380 0.370 - -50.67% -
464.h264ref (max) -2.56% 39.000 38.000 - 18.75% -
444.namd (max) (percentage) -2.50% 0.400 0.390 - -92.47% -
483.xalancbmk (max) (percentage) -2.50% 0.400 0.390 - -23.53% -
445.gobmk (max) -2.47% 81.000 79.000 - 12.86% -
462.libquantum (max) (percentage) -2.29% 1.310 1.280 - 3100.00% -
433.milc (avg) (percentage) -2.08% 0.960 0.940 - 13.25% -
450.soplex (max) (percentage) -1.30% 3.070 3.030 - 175.45% -
464.h264ref (max) (percentage) -1.12% 0.890 0.880 - 225.93% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 3.12% 224407.000 231402.000 - 0.02% -
447.dealII 1.44% 325219.000 329891.000 - 0.00% -
471.omnetpp 1.18% 490147.000 495909.000 - -0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
433.milc (unexplained diff vs previous) (percentage) 3650.00% 0.020 0.750 - 0.00% -
462.libquantum (max) (percentage) 3100.00% 0.040 1.280 - -2.29% -
473.astar (avg) (percentage) 1871.43% 0.070 1.380 - 0.00% -
401.bzip2 (max) (percentage) 1440.00% 0.050 0.770 - -6.10% -
401.bzip2 (avg) (percentage) 1320.00% 0.100 1.420 - 0.00% -
462.libquantum (avg) (percentage) 1172.73% 0.110 1.400 - 0.00% -
429.mcf (avg) (percentage) 875.68% 0.370 3.610 - -5.25% -
450.soplex (avg) (percentage) 681.48% 0.540 4.220 - 0.00% -
473.astar 666.67% 9.000 69.000 - 0.00% -
473.astar (avg) 666.67% 9.000 69.000 - 0.00% -
470.lbm (max) (percentage) 533.33% 0.060 0.380 - -5.00% -
470.lbm (avg) (percentage) 432.76% 0.580 3.090 - 0.00% -
473.astar (max) (percentage) 325.00% 0.080 0.340 - -5.56% -
482.sphinx3 (avg) (percentage) 257.14% 0.910 3.250 - 0.00% -
464.h264ref (max) (percentage) 225.93% 0.270 0.880 - -1.12% -
401.bzip2 (max) 216.67% 6.000 19.000 - -5.00% -
482.sphinx3 (max) (percentage) 207.25% 0.690 2.120 - -2.75% -
433.milc (unexplained diff vs previous) 200.00% 3.000 9.000 - 0.00% -
450.soplex (max) (percentage) 175.45% 1.100 3.030 - -1.30% -
401.bzip2 169.23% 13.000 35.000 - 0.00% -
401.bzip2 (avg) 169.23% 13.000 35.000 - 0.00% -
445.gobmk (max) (percentage) 146.67% 0.600 1.480 - -2.63% -
429.mcf (max) (percentage) 119.44% 0.360 0.790 - -4.82% -
470.lbm (max) 100.00% 6.000 12.000 - 0.00% -
471.omnetpp (max) (percentage) 90.00% 1.000 1.900 - -3.55% -
473.astar (max) 70.00% 10.000 17.000 - -5.56% -
458.sjeng (avg) (percentage) 69.57% 0.230 0.390 - 0.00% -
445.gobmk (avg) (percentage) 62.50% 0.320 0.520 - 0.00% -
470.lbm 53.12% 64.000 98.000 - 0.00% -
470.lbm (avg) 53.12% 64.000 98.000 - 0.00% -
471.omnetpp (avg) (percentage) 31.67% 0.600 0.790 - 0.00% -
453.povray (max) (percentage) 21.70% 1.060 1.290 - -0.77% -
453.povray (avg) (percentage) 20.55% 0.730 0.880 - 0.00% -
464.h264ref (avg) (percentage) 18.92% 0.370 0.440 - 0.00% -
464.h264ref (max) 18.75% 32.000 38.000 - -2.56% -
458.sjeng (max) (percentage) 17.39% 0.230 0.270 - -6.90% -
433.milc (avg) (percentage) 13.25% 0.830 0.940 - -2.08% -
445.gobmk (max) 12.86% 70.000 79.000 - -2.47% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 -1.17% 44357.000 43840.000 - -0.01% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.28% 10150.000 73.000 - 5.80% -
462.libquantum -98.60% 9659.000 135.000 - -1.46% -
482.sphinx3 -96.60% 23779.000 808.000 - 0.62% -
450.soplex -96.19% 13988.000 533.000 - 0.76% -
444.namd -92.67% 9377.000 687.000 - -1.01% -
453.povray -92.67% 7301.000 535.000 - 1.52% -
433.milc -91.72% 14590.000 1208.000 - -0.74% -
429.mcf -90.14% 18905.000 1864.000 - -0.48% -
400.perlbench -83.18% 11085.000 1865.000 - 1.63% -
401.bzip2 -81.87% 13962.000 2531.000 - 0.60% -
447.dealII -79.27% 9102.000 1887.000 - -0.11% -
471.omnetpp -75.49% 11917.000 2921.000 - -1.35% -
470.lbm -71.09% 11046.000 3193.000 - 0.19% -
456.hmmer -69.56% 9122.000 2777.000 - 0.51% -
464.h264ref -63.62% 12142.000 4417.000 - 0.00% -
473.astar -61.87% 13301.000 5072.000 - 0.20% -
445.gobmk -54.03% 11668.000 5364.000 - -0.19% -
458.sjeng -51.77% 15282.000 7371.000 - 0.41% -
483.xalancbmk -47.96% 9633.000 5013.000 - 1.05% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.59% 485.000 2.000 - 0.00% -
433.milc (max) -97.03% 269.000 8.000 - 100.00% -
444.namd -96.71% 213.000 7.000 - 0.00% -
444.namd (avg) -96.71% 213.000 7.000 - 0.00% -
444.namd (max) (percentage) -92.47% 5.180 0.390 - -2.50% -
453.povray -92.45% 53.000 4.000 - 0.00% -
453.povray (avg) -92.45% 53.000 4.000 - 0.00% -
453.povray (max) -92.21% 77.000 6.000 - 0.00% -
433.milc -90.91% 121.000 11.000 - 0.00% -
433.milc (avg) -90.91% 121.000 11.000 - 0.00% -
462.libquantum -90.00% 10.000 1.000 - 0.00% -
462.libquantum (avg) -90.00% 10.000 1.000 - 0.00% -
482.sphinx3 (max) -89.63% 164.000 17.000 - 0.00% -
450.soplex (max) -89.54% 153.000 16.000 - 0.00% -
400.perlbench (max) -88.31% 77.000 9.000 - 0.00% -
482.sphinx3 -87.96% 216.000 26.000 - 0.00% -
482.sphinx3 (avg) -87.96% 216.000 26.000 - 0.00% -
447.dealII (max) -86.51% 126.000 17.000 - 0.00% -
456.hmmer (max) -85.29% 68.000 10.000 - 0.00% -
400.perlbench -83.93% 56.000 9.000 - 0.00% -
400.perlbench (avg) -83.93% 56.000 9.000 - 0.00% -
447.dealII -81.71% 82.000 15.000 - -6.25% -
447.dealII (avg) -81.71% 82.000 15.000 - -6.25% -
429.mcf (max) -79.41% 68.000 14.000 - -6.67% -
456.hmmer -77.50% 80.000 18.000 - 0.00% -
456.hmmer (avg) -77.50% 80.000 18.000 - 0.00% -
450.soplex -70.67% 75.000 22.000 - 0.00% -
450.soplex (avg) -70.67% 75.000 22.000 - 0.00% -
471.omnetpp -67.61% 71.000 23.000 - 0.00% -
471.omnetpp (avg) -67.61% 71.000 23.000 - 0.00% -
462.libquantum (max) -66.67% 3.000 1.000 - 0.00% -
483.xalancbmk (max) -61.22% 49.000 19.000 - 0.00% -
433.milc (max) (percentage) -60.00% 1.850 0.740 - 85.00% -
464.h264ref -56.82% 44.000 19.000 - 0.00% -
464.h264ref (avg) -56.82% 44.000 19.000 - 0.00% -
471.omnetpp (max) -53.78% 119.000 55.000 - -5.17% -
444.namd (avg) (percentage) -52.19% 2.280 1.090 - 0.00% -
456.hmmer (max) (percentage) -50.67% 0.750 0.370 - -2.63% -
483.xalancbmk -48.78% 41.000 21.000 - 0.00% -
483.xalancbmk (avg) -48.78% 41.000 21.000 - 0.00% -
458.sjeng (max) -45.71% 35.000 19.000 - -9.52% -
447.dealII (max) (percentage) -33.81% 1.390 0.920 - 0.00% -
400.perlbench (max) (percentage) -27.14% 0.700 0.510 - 0.00% -
445.gobmk -27.03% 37.000 27.000 - 0.00% -
445.gobmk (avg) -27.03% 37.000 27.000 - 0.00% -
456.hmmer (avg) (percentage) -23.86% 0.880 0.670 - 0.00% -
483.xalancbmk (max) (percentage) -23.53% 0.510 0.390 - -2.50% -
458.sjeng -20.00% 35.000 28.000 - 0.00% -
458.sjeng (avg) -20.00% 35.000 28.000 - 0.00% -
403.gcc (max) (percentage) -17.50% 0.400 0.330 - 0.00% -
447.dealII (avg) (percentage) -10.99% 0.910 0.810 - -8.99% -
429.mcf -2.90% 69.000 67.000 - -5.63% -
429.mcf (avg) -2.90% 69.000 67.000 - -5.63% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 52.63% 19.000 29.000 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)

Report Time: 0.37s
Execution (seconds)
Name Current %
400.perlbench 1865.000 1.63%
401.bzip2 2531.000 -
403.gcc 73.000 5.80%
429.mcf 1864.000 -
433.milc 1208.000 -
444.namd 687.000 -1.01%
445.gobmk 5364.000 -
447.dealII 1887.000 -
450.soplex 533.000 -
453.povray 535.000 1.52%
456.hmmer 2777.000 -
458.sjeng 7371.000 -
462.libquantum 135.000 -1.46%
464.h264ref 4417.000 -
470.lbm 3193.000 -
471.omnetpp 2921.000 -1.35%
473.astar 5072.000 -
482.sphinx3 808.000 -
483.xalancbmk 5013.000 1.05%
Geometric Mean 1538.730 -
Code Size (bytes)
Name Current %
400.perlbench 898398.000 -
401.bzip2 43840.000 -
403.gcc 2485232.000 -
429.mcf 9240.000 -
433.milc 62502.000 -
444.namd 142552.000 -
445.gobmk 1851818.000 -
447.dealII 329891.000 -
450.soplex 231402.000 -
453.povray 684701.000 -
456.hmmer 113308.000 -
458.sjeng 106627.000 -
462.libquantum 16828.000 -
464.h264ref 398457.000 -
470.lbm 10268.000 -
471.omnetpp 495909.000 -
473.astar 23716.000 -
482.sphinx3 119733.000 -
483.xalancbmk 2534126.000 -
Geometric Mean 169713.171 -
sample variability
Name Current %
400.perlbench 9.000 -
400.perlbench (avg) 9.000 -
400.perlbench (avg) (percentage) 0.510 -
400.perlbench (max) 9.000 -
400.perlbench (max) (percentage) 0.510 -
401.bzip2 35.000 -
401.bzip2 (avg) 35.000 -
401.bzip2 (avg) (percentage) 1.420 -
401.bzip2 (max) 19.000 -5.00%
401.bzip2 (max) (percentage) 0.770 -6.10%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.310 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.330 -
429.mcf 67.000 -5.63%
429.mcf (avg) 67.000 -5.63%
429.mcf (avg) (percentage) 3.610 -5.25%
429.mcf (max) 14.000 -6.67%
429.mcf (max) (percentage) 0.790 -4.82%
429.mcf (unexplained diff vs previous) 9.000 80.00%
429.mcf (unexplained diff vs previous) (percentage) 0.480 77.78%
433.milc 11.000 -
433.milc (avg) 11.000 -
433.milc (avg) (percentage) 0.940 -2.08%
433.milc (max) 8.000 100.00%
433.milc (max) (percentage) 0.740 85.00%
433.milc (unexplained diff vs previous) 9.000 -
433.milc (unexplained diff vs previous) (percentage) 0.750 -
444.namd 7.000 -
444.namd (avg) 7.000 -
444.namd (avg) (percentage) 1.090 -
444.namd (max) 2.000 -
444.namd (max) (percentage) 0.390 -2.50%
445.gobmk 27.000 -
445.gobmk (avg) 27.000 -
445.gobmk (avg) (percentage) 0.520 -
445.gobmk (max) 79.000 -2.47%
445.gobmk (max) (percentage) 1.480 -2.63%
447.dealII 15.000 -6.25%
447.dealII (avg) 15.000 -6.25%
447.dealII (avg) (percentage) 0.810 -8.99%
447.dealII (max) 17.000 -
447.dealII (max) (percentage) 0.920 -
447.dealII (unexplained diff vs previous) 2.000 -
447.dealII (unexplained diff vs previous) (percentage) 0.110 -
450.soplex 22.000 -
450.soplex (avg) 22.000 -
450.soplex (avg) (percentage) 4.220 -
450.soplex (max) 16.000 -
450.soplex (max) (percentage) 3.030 -1.30%
453.povray 4.000 -
453.povray (avg) 4.000 -
453.povray (avg) (percentage) 0.880 -
453.povray (max) 6.000 -
453.povray (max) (percentage) 1.290 -
456.hmmer 18.000 -
456.hmmer (avg) 18.000 -
456.hmmer (avg) (percentage) 0.670 -
456.hmmer (max) 10.000 -
456.hmmer (max) (percentage) 0.370 -2.63%
458.sjeng 28.000 -
458.sjeng (avg) 28.000 -
458.sjeng (avg) (percentage) 0.390 -
458.sjeng (max) 19.000 -9.52%
458.sjeng (max) (percentage) 0.270 -6.90%
462.libquantum 1.000 -
462.libquantum (avg) 1.000 -
462.libquantum (avg) (percentage) 1.400 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.280 -2.29%
464.h264ref 19.000 -
464.h264ref (avg) 19.000 -
464.h264ref (avg) (percentage) 0.440 -
464.h264ref (max) 38.000 -2.56%
464.h264ref (max) (percentage) 0.880 -1.12%
470.lbm 98.000 -
470.lbm (avg) 98.000 -
470.lbm (avg) (percentage) 3.090 -
470.lbm (max) 12.000 -
470.lbm (max) (percentage) 0.380 -5.00%
471.omnetpp 23.000 -
471.omnetpp (avg) 23.000 -
471.omnetpp (avg) (percentage) 0.790 -
471.omnetpp (max) 55.000 -5.17%
471.omnetpp (max) (percentage) 1.900 -3.55%
473.astar 69.000 -
473.astar (avg) 69.000 -
473.astar (avg) (percentage) 1.380 -
473.astar (max) 17.000 -5.56%
473.astar (max) (percentage) 0.340 -5.56%
482.sphinx3 26.000 -
482.sphinx3 (avg) 26.000 -
482.sphinx3 (avg) (percentage) 3.250 -
482.sphinx3 (max) 17.000 -
482.sphinx3 (max) (percentage) 2.120 -2.75%
483.xalancbmk 21.000 -
483.xalancbmk (avg) 21.000 -
483.xalancbmk (avg) (percentage) 0.430 -
483.xalancbmk (max) 19.000 -
483.xalancbmk (max) (percentage) 0.390 -2.50%
Geometric Mean 3.259 -13.17%