Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/51b9e336619b09305357e5ec43f64890d4965e20 (+ 4331 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-aarch64-master-Oz_LTO-build/52/
View Options
Report
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0052 (llvmorg-17-init-17811-g51b9e336619b) 2023-06-22T09:44:21 0:00:00
Previous 0039 (llvmorg-17-init-13480-g01cc38843d76) 2023-06-01T11:36:51 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 13 32
Performance Improvements 10 84
Added Tests 3 3
Unchanged Tests 778 685
Total Tests 804

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
483.xalancbmk 3.69% 4960.000 5143.000 - -46.61% -
403.gcc 2.86% 70.000 72.000 - -99.29% -
400.perlbench 1.57% 1850.000 1879.000 - -83.05% -
471.omnetpp 1.57% 2999.000 3046.000 - -74.44% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench 25.00% 4.000 5.000 - -91.07% -
400.perlbench (avg) 25.00% 4.000 5.000 - -91.07% -
400.perlbench (max) 12.50% 8.000 9.000 - -88.31% -
483.xalancbmk (max) 5.26% 19.000 20.000 - -59.18% -
483.xalancbmk 4.76% 21.000 22.000 - -46.34% -
483.xalancbmk (avg) 4.76% 21.000 22.000 - -46.34% -
471.omnetpp 3.23% 31.000 32.000 - -54.93% -
471.omnetpp (avg) 3.23% 31.000 32.000 - -54.93% -
471.omnetpp (max) 1.85% 54.000 55.000 - -53.78% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
453.povray -2.08% 528.000 517.000 - -92.92% -
470.lbm -1.97% 3195.000 3132.000 - -71.65% -
444.namd -1.43% 697.000 687.000 - -92.67% -
473.astar -1.38% 5058.000 4988.000 - -62.50% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
444.namd (max) -12.50% 8.000 7.000 - -98.56% -
470.lbm (max) -9.09% 11.000 10.000 - 66.67% -
473.astar -1.92% 52.000 51.000 - 466.67% -
473.astar (avg) -1.92% 52.000 51.000 - 466.67% -
470.lbm -1.56% 64.000 63.000 - -1.56% -
470.lbm (avg) -1.56% 64.000 63.000 - -1.56% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 2.38% 224407.000 229742.000 - -0.76% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 2850.00% 0.040 1.180 - 0.00% -
473.astar (avg) (percentage) 1371.43% 0.070 1.030 - 0.00% -
462.libquantum (avg) (percentage) 972.73% 0.110 1.180 - 0.00% -
401.bzip2 (avg) (percentage) 840.00% 0.100 0.940 - 0.00% -
429.mcf (avg) (percentage) 613.51% 0.370 2.640 - 0.00% -
473.astar (max) (percentage) 562.50% 0.080 0.530 - 0.00% -
401.bzip2 (max) (percentage) 560.00% 0.050 0.330 - 0.00% -
470.lbm (max) (percentage) 483.33% 0.060 0.350 - 0.00% -
473.astar 466.67% 9.000 51.000 - -1.92% -
473.astar (avg) 466.67% 9.000 51.000 - -1.92% -
450.soplex (avg) (percentage) 464.81% 0.540 3.050 - 0.00% -
470.lbm (avg) (percentage) 250.00% 0.580 2.030 - 0.00% -
473.astar (max) 160.00% 10.000 26.000 - 0.00% -
464.h264ref (max) (percentage) 144.44% 0.270 0.660 - 0.00% -
445.gobmk (max) (percentage) 116.67% 0.600 1.300 - 0.00% -
429.mcf (max) (percentage) 111.11% 0.360 0.760 - 0.00% -
482.sphinx3 (avg) (percentage) 110.99% 0.910 1.920 - 0.00% -
471.omnetpp (max) (percentage) 83.00% 1.000 1.830 - 0.00% -
458.sjeng (max) (percentage) 78.26% 0.230 0.410 - 0.00% -
401.bzip2 76.92% 13.000 23.000 - 0.00% -
401.bzip2 (avg) 76.92% 13.000 23.000 - 0.00% -
471.omnetpp (avg) (percentage) 76.67% 0.600 1.060 - 0.00% -
470.lbm (max) 66.67% 6.000 10.000 - -9.09% -
458.sjeng (avg) (percentage) 39.13% 0.230 0.320 - 0.00% -
456.hmmer (max) (percentage) 38.67% 0.750 1.040 - 0.00% -
401.bzip2 (max) 33.33% 6.000 8.000 - 0.00% -
450.soplex (max) (percentage) 29.09% 1.100 1.420 - 0.00% -
445.gobmk (avg) (percentage) 12.50% 0.320 0.360 - 0.00% -
464.h264ref (avg) (percentage) 10.81% 0.370 0.410 - 0.00% -
482.sphinx3 (max) (percentage) 7.25% 0.690 0.740 - 0.00% -
453.povray (avg) (percentage) 2.74% 0.730 0.750 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
453.povray -1.65% 691522.000 680137.000 - -0.66% -
401.bzip2 -1.43% 44357.000 43724.000 - -0.26% -
462.libquantum -1.40% 16989.000 16752.000 - -0.45% -
444.namd -1.16% 143421.000 141756.000 - -0.39% -
433.milc -1.13% 62863.000 62154.000 - -0.56% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.29% 10150.000 72.000 - 2.86% -
462.libquantum -98.59% 9659.000 136.000 - 0.00% -
482.sphinx3 -96.62% 23779.000 803.000 - 0.38% -
450.soplex -96.20% 13988.000 531.000 - 0.00% -
453.povray -92.92% 7301.000 517.000 - -2.08% -
444.namd -92.67% 9377.000 687.000 - -1.43% -
433.milc -91.69% 14590.000 1212.000 - 0.33% -
429.mcf -90.06% 18905.000 1879.000 - 0.27% -
400.perlbench -83.05% 11085.000 1879.000 - 1.57% -
401.bzip2 -81.92% 13962.000 2524.000 - -0.24% -
447.dealII -79.15% 9102.000 1898.000 - -0.16% -
471.omnetpp -74.44% 11917.000 3046.000 - 1.57% -
470.lbm -71.65% 11046.000 3132.000 - -1.97% -
456.hmmer -69.90% 9122.000 2746.000 - -0.79% -
464.h264ref -64.26% 12142.000 4340.000 - 0.07% -
473.astar -62.50% 13301.000 4988.000 - -1.38% -
445.gobmk -54.25% 11668.000 5338.000 - -0.48% -
458.sjeng -51.91% 15282.000 7349.000 - -0.57% -
483.xalancbmk -46.61% 9633.000 5143.000 - 3.69% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -98.56% 485.000 7.000 - -12.50% -
433.milc (max) -97.40% 269.000 7.000 - 0.00% -
444.namd -97.18% 213.000 6.000 - 0.00% -
444.namd (avg) -97.18% 213.000 6.000 - 0.00% -
482.sphinx3 (max) -96.95% 164.000 5.000 - 0.00% -
450.soplex (max) -95.42% 153.000 7.000 - 0.00% -
453.povray (max) -94.81% 77.000 4.000 - 0.00% -
453.povray -94.34% 53.000 3.000 - 0.00% -
453.povray (avg) -94.34% 53.000 3.000 - 0.00% -
482.sphinx3 -93.06% 216.000 15.000 - 0.00% -
482.sphinx3 (avg) -93.06% 216.000 15.000 - 0.00% -
433.milc -92.56% 121.000 9.000 - 0.00% -
433.milc (avg) -92.56% 121.000 9.000 - 0.00% -
400.perlbench -91.07% 56.000 5.000 - 25.00% -
400.perlbench (avg) -91.07% 56.000 5.000 - 25.00% -
462.libquantum -90.00% 10.000 1.000 - 0.00% -
462.libquantum (avg) -90.00% 10.000 1.000 - 0.00% -
447.dealII (max) -89.68% 126.000 13.000 - 0.00% -
400.perlbench (max) -88.31% 77.000 9.000 - 12.50% -
447.dealII -84.15% 82.000 13.000 - 0.00% -
447.dealII (avg) -84.15% 82.000 13.000 - 0.00% -
456.hmmer -80.00% 80.000 16.000 - 0.00% -
456.hmmer (avg) -80.00% 80.000 16.000 - 0.00% -
429.mcf (max) -79.41% 68.000 14.000 - 0.00% -
450.soplex -78.67% 75.000 16.000 - 0.00% -
450.soplex (avg) -78.67% 75.000 16.000 - 0.00% -
444.namd (max) (percentage) -77.61% 5.180 1.160 - 0.00% -
433.milc (max) (percentage) -67.03% 1.850 0.610 - 0.00% -
462.libquantum (max) -66.67% 3.000 1.000 - 0.00% -
464.h264ref -61.36% 44.000 17.000 - 0.00% -
464.h264ref (avg) -61.36% 44.000 17.000 - 0.00% -
444.namd (avg) (percentage) -60.09% 2.280 0.910 - 0.00% -
483.xalancbmk (max) -59.18% 49.000 20.000 - 5.26% -
456.hmmer (max) -58.82% 68.000 28.000 - 0.00% -
471.omnetpp -54.93% 71.000 32.000 - 3.23% -
471.omnetpp (avg) -54.93% 71.000 32.000 - 3.23% -
471.omnetpp (max) -53.78% 119.000 55.000 - 1.85% -
447.dealII (max) (percentage) -49.64% 1.390 0.700 - 0.00% -
445.gobmk -48.65% 37.000 19.000 - 0.00% -
445.gobmk (avg) -48.65% 37.000 19.000 - 0.00% -
400.perlbench (avg) (percentage) -47.06% 0.510 0.270 - 0.00% -
483.xalancbmk -46.34% 41.000 22.000 - 4.76% -
483.xalancbmk (avg) -46.34% 41.000 22.000 - 4.76% -
458.sjeng -34.29% 35.000 23.000 - 0.00% -
458.sjeng (avg) -34.29% 35.000 23.000 - 0.00% -
400.perlbench (max) (percentage) -31.43% 0.700 0.480 - 0.00% -
456.hmmer (avg) (percentage) -30.68% 0.880 0.610 - 0.00% -
429.mcf -28.99% 69.000 49.000 - 0.00% -
429.mcf (avg) -28.99% 69.000 49.000 - 0.00% -
483.xalancbmk (max) (percentage) -23.53% 0.510 0.390 - 0.00% -
447.dealII (avg) (percentage) -23.08% 0.910 0.700 - 0.00% -
453.povray (max) (percentage) -18.87% 1.060 0.860 - 0.00% -
403.gcc (max) (percentage) -17.50% 0.400 0.330 - 0.00% -
458.sjeng (max) -14.29% 35.000 30.000 - 0.00% -
464.h264ref (max) -12.50% 32.000 28.000 - 0.00% -
433.milc (avg) (percentage) -7.23% 0.830 0.770 - 0.00% -
470.lbm -1.56% 64.000 63.000 - -1.56% -
470.lbm (avg) -1.56% 64.000 63.000 - -1.56% -
445.gobmk (max) -1.43% 70.000 69.000 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 52.63% 19.000 29.000 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)

Report Time: 0.49s
Execution (seconds)
Name Current %
400.perlbench 1879.000 1.57%
401.bzip2 2524.000 -
403.gcc 72.000 2.86%
429.mcf 1879.000 -
433.milc 1212.000 -
444.namd 687.000 -1.43%
445.gobmk 5338.000 -
447.dealII 1898.000 -
450.soplex 531.000 -
453.povray 517.000 -2.08%
456.hmmer 2746.000 -
458.sjeng 7349.000 -
462.libquantum 136.000 -
464.h264ref 4340.000 -
470.lbm 3132.000 -1.97%
471.omnetpp 3046.000 1.57%
473.astar 4988.000 -1.38%
482.sphinx3 803.000 -
483.xalancbmk 5143.000 3.69%
Geometric Mean 1535.987 -
Code Size (bytes)
Name Current %
400.perlbench 893606.000 -
401.bzip2 43724.000 -
403.gcc 2477472.000 -
429.mcf 9192.000 -
433.milc 62154.000 -
444.namd 141756.000 -
445.gobmk 1847586.000 -
447.dealII 328047.000 -
450.soplex 229742.000 -
453.povray 680137.000 -
456.hmmer 112748.000 -
458.sjeng 106831.000 -
462.libquantum 16752.000 -
464.h264ref 396029.000 -
470.lbm 10216.000 -
471.omnetpp 493285.000 -
473.astar 23608.000 -
482.sphinx3 119581.000 -
483.xalancbmk 2523170.000 -
Geometric Mean 168968.706 -
sample variability
Name Current %
400.perlbench 5.000 25.00%
400.perlbench (avg) 5.000 25.00%
400.perlbench (avg) (percentage) 0.270 -
400.perlbench (max) 9.000 12.50%
400.perlbench (max) (percentage) 0.480 -
401.bzip2 23.000 -
401.bzip2 (avg) 23.000 -
401.bzip2 (avg) (percentage) 0.940 -
401.bzip2 (max) 8.000 -
401.bzip2 (max) (percentage) 0.330 -
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.310 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.330 -
429.mcf 49.000 -
429.mcf (avg) 49.000 -
429.mcf (avg) (percentage) 2.640 -
429.mcf (max) 14.000 -
429.mcf (max) (percentage) 0.760 -
433.milc 9.000 -
433.milc (avg) 9.000 -
433.milc (avg) (percentage) 0.770 -
433.milc (max) 7.000 -
433.milc (max) (percentage) 0.610 -
444.namd 6.000 -
444.namd (avg) 6.000 -
444.namd (avg) (percentage) 0.910 -
444.namd (max) 7.000 -12.50%
444.namd (max) (percentage) 1.160 -
445.gobmk 19.000 -
445.gobmk (avg) 19.000 -
445.gobmk (avg) (percentage) 0.360 -
445.gobmk (max) 69.000 -
445.gobmk (max) (percentage) 1.300 -
447.dealII 13.000 -
447.dealII (avg) 13.000 -
447.dealII (avg) (percentage) 0.700 -
447.dealII (max) 13.000 -
447.dealII (max) (percentage) 0.700 -
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 3.050 -
450.soplex (max) 7.000 -
450.soplex (max) (percentage) 1.420 -
453.povray 3.000 -
453.povray (avg) 3.000 -
453.povray (avg) (percentage) 0.750 -
453.povray (max) 4.000 -
453.povray (max) (percentage) 0.860 -
456.hmmer 16.000 -
456.hmmer (avg) 16.000 -
456.hmmer (avg) (percentage) 0.610 -
456.hmmer (max) 28.000 -
456.hmmer (max) (percentage) 1.040 -
458.sjeng 23.000 -
458.sjeng (avg) 23.000 -
458.sjeng (avg) (percentage) 0.320 -
458.sjeng (max) 30.000 -
458.sjeng (max) (percentage) 0.410 -
462.libquantum 1.000 -
462.libquantum (avg) 1.000 -
462.libquantum (avg) (percentage) 1.180 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.180 -
464.h264ref 17.000 -
464.h264ref (avg) 17.000 -
464.h264ref (avg) (percentage) 0.410 -
464.h264ref (max) 28.000 -
464.h264ref (max) (percentage) 0.660 -
470.lbm 63.000 -1.56%
470.lbm (avg) 63.000 -1.56%
470.lbm (avg) (percentage) 2.030 -
470.lbm (max) 10.000 -9.09%
470.lbm (max) (percentage) 0.350 -
471.omnetpp 32.000 3.23%
471.omnetpp (avg) 32.000 3.23%
471.omnetpp (avg) (percentage) 1.060 -
471.omnetpp (max) 55.000 1.85%
471.omnetpp (max) (percentage) 1.830 -
473.astar 51.000 -1.92%
473.astar (avg) 51.000 -1.92%
473.astar (avg) (percentage) 1.030 -
473.astar (max) 26.000 -
473.astar (max) (percentage) 0.530 -
482.sphinx3 15.000 -
482.sphinx3 (avg) 15.000 -
482.sphinx3 (avg) (percentage) 1.920 -
482.sphinx3 (max) 5.000 -
482.sphinx3 (max) (percentage) 0.740 -
483.xalancbmk 22.000 4.76%
483.xalancbmk (avg) 22.000 4.76%
483.xalancbmk (avg) (percentage) 0.430 -
483.xalancbmk (max) 20.000 5.26%
483.xalancbmk (max) (percentage) 0.390 -
Geometric Mean 2.945 -28.12%