Fields

hardware None
os None

Parameters

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/b629414ae13d6dcd641e92d0389c5d4b7638a644 (+ 837 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-aarch64-master-Oz_LTO-build/123/
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0123 (llvmorg-19-init-01119-gb629414ae13d) 2024-01-15T18:19:44 0:00:00
Previous 0122 (llvmorg-19-init-00282-ga3bbe627d2d8) 2023-12-28T20:17:59 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00


Tests Summary

Status Group # # (B)
Performance Regressions 17 30
Performance Improvements 42 97
Added Tests 0 25
Unchanged Tests 817 724
Total Tests 876

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
447.dealII 3.84% 1876.000 1948.000 - -78.60% -
450.soplex 2.55% 548.000 562.000 - -95.98% -
462.libquantum 1.54% 130.000 132.000 - -98.63% -
453.povray 1.33% 526.000 533.000 - -92.70% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
473.astar (unexplained diff vs previous) (percentage) 254.55% 0.330 1.170 - 0.00% -
473.astar (unexplained diff vs previous) 252.94% 17.000 60.000 - 0.00% -
473.astar (max) 126.92% 26.000 59.000 - 490.00% -
473.astar (max) (percentage) 123.08% 0.520 1.160 - 1350.00% -
401.bzip2 (max) 18.75% 16.000 19.000 - 216.67% -
401.bzip2 (max) (percentage) 12.70% 0.630 0.710 - 1320.00% -
401.bzip2 (unexplained diff vs previous) 11.76% 17.000 19.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 11.11% 0.630 0.700 - 0.00% -
464.h264ref 6.25% 16.000 17.000 - -61.36% -
464.h264ref (avg) 6.25% 16.000 17.000 - -61.36% -
401.bzip2 4.55% 22.000 23.000 - 76.92% -
401.bzip2 (avg) 4.55% 22.000 23.000 - 76.92% -
473.astar (avg) (percentage) 1.04% 0.960 0.970 - 1285.71% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp -1.25% 3043.000 3005.000 - -74.78% -
473.astar -1.16% 5185.000 5125.000 - -61.47% -
459.GemsFDTD -1.12% 6690.000 6615.000 - 0.00% -
400.perlbench -1.03% 1942.000 1922.000 - -82.66% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) -50.65% 3.080 1.520 - 0.00% -
410.bwaves (unexplained diff vs previous) -50.00% 88.000 44.000 - 0.00% -
456.hmmer (max) -50.00% 22.000 11.000 - -83.82% -
456.hmmer (max) (percentage) -50.00% 0.760 0.380 - -49.33% -
462.libquantum (unexplained diff vs previous) -50.00% 4.000 2.000 - 0.00% -
410.bwaves (unexplained diff vs previous) (percentage) -49.45% 1.820 0.920 - 0.00% -
471.omnetpp (unexplained diff vs previous) -35.59% 59.000 38.000 - -68.33% -
471.omnetpp (unexplained diff vs previous) (percentage) -35.05% 1.940 1.260 - 24.75% -
456.hmmer (unexplained diff vs previous) -33.33% 12.000 8.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -31.71% 0.410 0.280 - 0.00% -
410.bwaves -25.84% 89.000 66.000 - 0.00% -
410.bwaves (avg) -25.84% 89.000 66.000 - 0.00% -
410.bwaves (avg) (percentage) -25.41% 1.850 1.380 - 0.00% -
433.milc (max) -10.00% 10.000 9.000 - -96.65% -
470.lbm (max) -7.14% 14.000 13.000 - 116.67% -
462.libquantum (max) (percentage) -6.35% 2.990 2.800 - 6900.00% -
456.hmmer -6.25% 16.000 15.000 - -81.25% -
456.hmmer (avg) -6.25% 16.000 15.000 - -81.25% -
482.sphinx3 (max) (percentage) -6.25% 0.480 0.450 - -34.78% -
471.omnetpp (max) -5.97% 67.000 63.000 - -47.06% -
471.omnetpp (max) (percentage) -5.38% 2.230 2.110 - 111.00% -
433.milc (max) (percentage) -4.88% 0.820 0.780 - -57.84% -
429.mcf (max) (percentage) -4.73% 1.690 1.610 - 347.22% -
470.lbm (max) (percentage) -4.65% 0.430 0.410 - 583.33% -
483.xalancbmk -4.35% 23.000 22.000 - -46.34% -
483.xalancbmk (avg) -4.35% 23.000 22.000 - -46.34% -
458.sjeng (max) -4.00% 25.000 24.000 - -31.43% -
453.povray (max) (percentage) -3.77% 0.530 0.510 - -51.89% -
456.hmmer (avg) (percentage) -3.57% 0.560 0.540 - -38.64% -
447.dealII (max) (percentage) -3.33% 0.900 0.870 - -37.41% -
429.mcf (max) -3.03% 33.000 32.000 - -52.94% -
458.sjeng (max) (percentage) -2.94% 0.340 0.330 - 43.48% -
471.omnetpp -2.70% 37.000 36.000 - -49.30% -
471.omnetpp (avg) -2.70% 37.000 36.000 - -49.30% -
444.namd (max) (percentage) -2.38% 0.840 0.820 - -84.17% -
410.bwaves (max) -2.25% 89.000 87.000 - 0.00% -
403.gcc (max) (percentage) -1.15% 7.860 7.770 - 1842.50% -
410.bwaves (max) (percentage) -1.08% 1.850 1.830 - 0.00% -

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.70% 224407.000 228230.000 - -0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 6900.00% 0.040 2.800 - -6.35% -
403.gcc (max) (percentage) 1842.50% 0.400 7.770 - -1.15% -
473.astar (max) (percentage) 1350.00% 0.080 1.160 - 123.08% -
401.bzip2 (max) (percentage) 1320.00% 0.050 0.710 - 12.70% -
473.astar (avg) (percentage) 1285.71% 0.070 0.970 - 1.04% -
462.libquantum (avg) (percentage) 1018.18% 0.110 1.230 - 0.82% -
403.gcc (avg) (percentage) 829.03% 0.310 2.880 - 0.00% -
401.bzip2 (avg) (percentage) 750.00% 0.100 0.850 - 0.00% -
470.lbm (max) (percentage) 583.33% 0.060 0.410 - -4.65% -
429.mcf (avg) (percentage) 543.24% 0.370 2.380 - 0.00% -
473.astar (max) 490.00% 10.000 59.000 - 126.92% -
473.astar 444.44% 9.000 49.000 - 0.00% -
473.astar (avg) 444.44% 9.000 49.000 - 0.00% -
450.soplex (avg) (percentage) 407.41% 0.540 2.740 - 0.00% -
429.mcf (max) (percentage) 347.22% 0.360 1.610 - -4.73% -
470.lbm (avg) (percentage) 220.69% 0.580 1.860 - 0.00% -
401.bzip2 (max) 216.67% 6.000 19.000 - 18.75% -
470.lbm (max) 116.67% 6.000 13.000 - -7.14% -
471.omnetpp (max) (percentage) 111.00% 1.000 2.110 - -5.38% -
445.gobmk (max) (percentage) 105.00% 0.600 1.230 - 0.00% -
471.omnetpp (avg) (percentage) 105.00% 0.600 1.230 - 0.82% -
482.sphinx3 (avg) (percentage) 90.11% 0.910 1.730 - 0.00% -
401.bzip2 76.92% 13.000 23.000 - 4.55% -
401.bzip2 (avg) 76.92% 13.000 23.000 - 4.55% -
458.sjeng (max) (percentage) 43.48% 0.230 0.330 - -2.94% -
464.h264ref (max) (percentage) 40.74% 0.270 0.380 - 0.00% -
458.sjeng (avg) (percentage) 26.09% 0.230 0.290 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 24.75% 1.010 1.260 - -35.05% -
445.gobmk (avg) (percentage) 15.62% 0.320 0.370 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
444.namd -5.18% 143421.000 135996.000 - 0.01% -
464.h264ref -2.79% 399974.000 388809.000 - 0.07% -
453.povray -1.87% 691522.000 678577.000 - -0.01% -
433.milc -1.85% 62863.000 61698.000 - 0.04% -
401.bzip2 -1.76% 44357.000 43576.000 - 0.00% -
429.mcf -1.65% 9249.000 9096.000 - -0.04% -
462.libquantum -1.51% 16989.000 16732.000 - 0.00% -
473.astar -1.47% 23789.000 23440.000 - 0.00% -
456.hmmer -1.30% 113613.000 112136.000 - 0.00% -
482.sphinx3 -1.07% 120254.000 118969.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.30% 10150.000 71.000 - 0.00% -
462.libquantum -98.63% 9659.000 132.000 - 1.54% -
482.sphinx3 -96.51% 23779.000 831.000 - -0.24% -
450.soplex -95.98% 13988.000 562.000 - 2.55% -
444.namd -92.81% 9377.000 674.000 - 0.75% -
453.povray -92.70% 7301.000 533.000 - 1.33% -
433.milc -91.36% 14590.000 1260.000 - 0.48% -
429.mcf -89.46% 18905.000 1993.000 - 0.61% -
400.perlbench -82.66% 11085.000 1922.000 - -1.03% -
401.bzip2 -80.59% 13962.000 2710.000 - 0.71% -
447.dealII -78.60% 9102.000 1948.000 - 3.84% -
471.omnetpp -74.78% 11917.000 3005.000 - -1.25% -
470.lbm -70.71% 11046.000 3235.000 - -0.92% -
456.hmmer -68.24% 9122.000 2897.000 - -0.28% -
464.h264ref -62.08% 12142.000 4604.000 - 0.50% -
473.astar -61.47% 13301.000 5125.000 - -1.16% -
445.gobmk -54.77% 11668.000 5277.000 - 0.29% -
458.sjeng -51.11% 15282.000 7472.000 - -0.20% -
483.xalancbmk -44.84% 9633.000 5314.000 - -0.97% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -98.97% 485.000 5.000 - 0.00% -
482.sphinx3 (max) -98.17% 164.000 3.000 - 0.00% -
444.namd -97.65% 213.000 5.000 - 0.00% -
444.namd (avg) -97.65% 213.000 5.000 - 0.00% -
453.povray (max) -97.40% 77.000 2.000 - 0.00% -
450.soplex (max) -96.73% 153.000 5.000 - 0.00% -
433.milc (max) -96.65% 269.000 9.000 - -10.00% -
453.povray -94.34% 53.000 3.000 - 0.00% -
453.povray (avg) -94.34% 53.000 3.000 - 0.00% -
403.gcc -93.55% 31.000 2.000 - 0.00% -
403.gcc (avg) -93.55% 31.000 2.000 - 0.00% -
482.sphinx3 -93.52% 216.000 14.000 - 0.00% -
482.sphinx3 (avg) -93.52% 216.000 14.000 - 0.00% -
433.milc -92.56% 121.000 9.000 - 0.00% -
433.milc (avg) -92.56% 121.000 9.000 - 0.00% -
400.perlbench -91.07% 56.000 5.000 - 0.00% -
400.perlbench (avg) -91.07% 56.000 5.000 - 0.00% -
462.libquantum -90.00% 10.000 1.000 - 0.00% -
462.libquantum (avg) -90.00% 10.000 1.000 - 0.00% -
400.perlbench (max) -88.31% 77.000 9.000 - 0.00% -
403.gcc (max) -87.50% 40.000 5.000 - 0.00% -
447.dealII (max) -87.30% 126.000 16.000 - 0.00% -
444.namd (max) (percentage) -84.17% 5.180 0.820 - -2.38% -
447.dealII -84.15% 82.000 13.000 - 0.00% -
447.dealII (avg) -84.15% 82.000 13.000 - 0.00% -
456.hmmer (max) -83.82% 68.000 11.000 - -50.00% -
456.hmmer -81.25% 80.000 15.000 - -6.25% -
456.hmmer (avg) -81.25% 80.000 15.000 - -6.25% -
450.soplex -80.00% 75.000 15.000 - 0.00% -
450.soplex (avg) -80.00% 75.000 15.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) -68.33% 120.000 38.000 - -35.59% -
444.namd (avg) (percentage) -64.04% 2.280 0.820 - 0.00% -
464.h264ref -61.36% 44.000 17.000 - 6.25% -
464.h264ref (avg) -61.36% 44.000 17.000 - 6.25% -
483.xalancbmk (max) -59.18% 49.000 20.000 - 0.00% -
433.milc (max) (percentage) -57.84% 1.850 0.780 - -4.88% -
429.mcf (max) -52.94% 68.000 32.000 - -3.03% -
453.povray (max) (percentage) -51.89% 1.060 0.510 - -3.77% -
456.hmmer (max) (percentage) -49.33% 0.750 0.380 - -50.00% -
471.omnetpp -49.30% 71.000 36.000 - -2.70% -
471.omnetpp (avg) -49.30% 71.000 36.000 - -2.70% -
445.gobmk -48.65% 37.000 19.000 - 0.00% -
445.gobmk (avg) -48.65% 37.000 19.000 - 0.00% -
400.perlbench (avg) (percentage) -47.06% 0.510 0.270 - 0.00% -
471.omnetpp (max) -47.06% 119.000 63.000 - -5.97% -
464.h264ref (max) -46.88% 32.000 17.000 - 0.00% -
483.xalancbmk -46.34% 41.000 22.000 - -4.35% -
483.xalancbmk (avg) -46.34% 41.000 22.000 - -4.35% -
458.sjeng -40.00% 35.000 21.000 - 0.00% -
458.sjeng (avg) -40.00% 35.000 21.000 - 0.00% -
456.hmmer (avg) (percentage) -38.64% 0.880 0.540 - -3.57% -
447.dealII (max) (percentage) -37.41% 1.390 0.870 - -3.33% -
482.sphinx3 (max) (percentage) -34.78% 0.690 0.450 - -6.25% -
400.perlbench (max) (percentage) -32.86% 0.700 0.470 - 0.00% -
429.mcf -31.88% 69.000 47.000 - 0.00% -
429.mcf (avg) -31.88% 69.000 47.000 - 0.00% -
458.sjeng (max) -31.43% 35.000 24.000 - -4.00% -
483.xalancbmk (max) (percentage) -25.49% 0.510 0.380 - 0.00% -
447.dealII (avg) (percentage) -21.98% 0.910 0.710 - 0.00% -
445.gobmk (max) -8.57% 70.000 64.000 - 0.00% -
450.soplex (max) (percentage) -6.36% 1.100 1.030 - -0.96% -
470.lbm -6.25% 64.000 60.000 - 0.00% -
470.lbm (avg) -6.25% 64.000 60.000 - 0.00% -
433.milc (avg) (percentage) -6.02% 0.830 0.780 - 0.00% -
453.povray (avg) (percentage) -5.48% 0.730 0.690 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 78.95% 19.000 34.000 - 0.00% -
nb_compile_successful 26.32% 19.000 24.000 - 0.00% -
nb_execution_successful 26.32% 19.000 24.000 - 0.00% -

Added Tests - Code Size
410.bwaves
434.zeusmp
437.leslie3d
459.GemsFDTD
465.tonto

Added Tests - Execution
410.bwaves
434.zeusmp
437.leslie3d
459.GemsFDTD
465.tonto

Added Tests - sample variability
401.bzip2 (unexplained diff vs previous)
401.bzip2 (unexplained diff vs previous) (percentage)
410.bwaves
410.bwaves (avg)
410.bwaves (avg) (percentage)
410.bwaves (max)
410.bwaves (max) (percentage)
410.bwaves (unexplained diff vs previous)
410.bwaves (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
473.astar (unexplained diff vs previous)
473.astar (unexplained diff vs previous) (percentage)


Report Time: 0.36s
Name Current %
400.perlbench 1922.000 -1.03%
401.bzip2 2710.000 -
403.gcc 71.000 -
410.bwaves 4793.000 -
429.mcf 1993.000 -
433.milc 1260.000 -
434.zeusmp 2045.000 -
437.leslie3d 16978.000 -
444.namd 674.000 -
445.gobmk 5277.000 -
447.dealII 1948.000 3.84%
450.soplex 562.000 2.55%
453.povray 533.000 1.33%
456.hmmer 2897.000 -
458.sjeng 7472.000 -
459.GemsFDTD 6615.000 -1.12%
462.libquantum 132.000 1.54%
464.h264ref 4604.000 -
465.tonto 27290.000 -
470.lbm 3235.000 -
471.omnetpp 3005.000 -1.25%
473.astar 5125.000 -1.16%
482.sphinx3 831.000 -
483.xalancbmk 5314.000 -
Geometric Mean 2201.281 -
Name Current %
400.perlbench 892955.000 -
401.bzip2 43576.000 -
403.gcc 2464912.000 -
410.bwaves 619312.000 -
429.mcf 9096.000 -
433.milc 61698.000 -
434.zeusmp 1011683.000 -
437.leslie3d 902492.000 -
444.namd 135996.000 -
445.gobmk 1843730.000 -
447.dealII 325875.000 -
450.soplex 228230.000 -
453.povray 678577.000 -
456.hmmer 112136.000 -
458.sjeng 105999.000 -
459.GemsFDTD 1656895.000 -
462.libquantum 16732.000 -
464.h264ref 388809.000 -
465.tonto 7841478.000 -
470.lbm 10224.000 -
471.omnetpp 490197.000 -
473.astar 23440.000 -
482.sphinx3 118969.000 -
483.xalancbmk 2515338.000 -
Geometric Mean 264384.068 -
Name Current %
400.perlbench 5.000 -
400.perlbench (avg) 5.000 -
400.perlbench (avg) (percentage) 0.270 -
400.perlbench (max) 9.000 -
400.perlbench (max) (percentage) 0.470 -
401.bzip2 23.000 4.55%
401.bzip2 (avg) 23.000 4.55%
401.bzip2 (avg) (percentage) 0.850 -
401.bzip2 (max) 19.000 18.75%
401.bzip2 (max) (percentage) 0.710 12.70%
401.bzip2 (unexplained diff vs previous) 19.000 11.76%
401.bzip2 (unexplained diff vs previous) (percentage) 0.700 11.11%
403.gcc 2.000 -
403.gcc (avg) 2.000 -
403.gcc (avg) (percentage) 2.880 -
403.gcc (max) 5.000 -
403.gcc (max) (percentage) 7.770 -1.15%
410.bwaves 66.000 -25.84%
410.bwaves (avg) 66.000 -25.84%
410.bwaves (avg) (percentage) 1.380 -25.41%
410.bwaves (max) 87.000 -2.25%
410.bwaves (max) (percentage) 1.830 -1.08%
410.bwaves (unexplained diff vs previous) 44.000 -50.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.920 -49.45%
429.mcf 47.000 -
429.mcf (avg) 47.000 -
429.mcf (avg) (percentage) 2.380 -
429.mcf (max) 32.000 -3.03%
429.mcf (max) (percentage) 1.610 -4.73%
433.milc 9.000 -
433.milc (avg) 9.000 -
433.milc (avg) (percentage) 0.780 -
433.milc (max) 9.000 -10.00%
433.milc (max) (percentage) 0.780 -4.88%
444.namd 5.000 -
444.namd (avg) 5.000 -
444.namd (avg) (percentage) 0.820 -
444.namd (max) 5.000 -
444.namd (max) (percentage) 0.820 -2.38%
445.gobmk 19.000 -
445.gobmk (avg) 19.000 -
445.gobmk (avg) (percentage) 0.370 -
445.gobmk (max) 64.000 -
445.gobmk (max) (percentage) 1.230 -
447.dealII 13.000 -
447.dealII (avg) 13.000 -
447.dealII (avg) (percentage) 0.710 -
447.dealII (max) 16.000 -
447.dealII (max) (percentage) 0.870 -3.33%
450.soplex 15.000 -
450.soplex (avg) 15.000 -
450.soplex (avg) (percentage) 2.740 -
450.soplex (max) 5.000 -
450.soplex (max) (percentage) 1.030 -
453.povray 3.000 -
453.povray (avg) 3.000 -
453.povray (avg) (percentage) 0.690 -
453.povray (max) 2.000 -
453.povray (max) (percentage) 0.510 -3.77%
456.hmmer 15.000 -6.25%
456.hmmer (avg) 15.000 -6.25%
456.hmmer (avg) (percentage) 0.540 -3.57%
456.hmmer (max) 11.000 -50.00%
456.hmmer (max) (percentage) 0.380 -50.00%
456.hmmer (unexplained diff vs previous) 8.000 -33.33%
456.hmmer (unexplained diff vs previous) (percentage) 0.280 -31.71%
458.sjeng 21.000 -
458.sjeng (avg) 21.000 -
458.sjeng (avg) (percentage) 0.290 -
458.sjeng (max) 24.000 -4.00%
458.sjeng (max) (percentage) 0.330 -2.94%
462.libquantum 1.000 -
462.libquantum (avg) 1.000 -
462.libquantum (avg) (percentage) 1.230 -
462.libquantum (max) 3.000 -
462.libquantum (max) (percentage) 2.800 -6.35%
462.libquantum (unexplained diff vs previous) 2.000 -50.00%
462.libquantum (unexplained diff vs previous) (percentage) 1.520 -50.65%
464.h264ref 17.000 6.25%
464.h264ref (avg) 17.000 6.25%
464.h264ref (avg) (percentage) 0.370 -
464.h264ref (max) 17.000 -
464.h264ref (max) (percentage) 0.380 -
470.lbm 60.000 -
470.lbm (avg) 60.000 -
470.lbm (avg) (percentage) 1.860 -
470.lbm (max) 13.000 -7.14%
470.lbm (max) (percentage) 0.410 -4.65%
471.omnetpp 36.000 -2.70%
471.omnetpp (avg) 36.000 -2.70%
471.omnetpp (avg) (percentage) 1.230 -
471.omnetpp (max) 63.000 -5.97%
471.omnetpp (max) (percentage) 2.110 -5.38%
471.omnetpp (unexplained diff vs previous) 38.000 -35.59%
471.omnetpp (unexplained diff vs previous) (percentage) 1.260 -35.05%
473.astar 49.000 -
473.astar (avg) 49.000 -
473.astar (avg) (percentage) 0.970 1.04%
473.astar (max) 59.000 126.92%
473.astar (max) (percentage) 1.160 123.08%
473.astar (unexplained diff vs previous) 60.000 252.94%
473.astar (unexplained diff vs previous) (percentage) 1.170 254.55%
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.730 -
482.sphinx3 (max) 3.000 -
482.sphinx3 (max) (percentage) 0.450 -6.25%
483.xalancbmk 22.000 -4.35%
483.xalancbmk (avg) 22.000 -4.35%
483.xalancbmk (avg) (percentage) 0.430 -
483.xalancbmk (max) 20.000 -
483.xalancbmk (max) (percentage) 0.380 -
Geometric Mean 4.487 -2.60%