Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/178cdbf746bbc49ad18b5e83a95514ddd42f764c (+ 999 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/250/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0250 (llvmorg-21-init-09580-g178cdbf746bb) 2025-04-12T16:38:48 0:00:00
Previous 0249 (llvmorg-21-init-08581-g83344da69145) 2025-04-01T07:49:48 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 23 14
Performance Improvements 22 118
Added Tests 8 4
Unchanged Tests 703 620
Total Tests 756

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 2.04% 2788.000 2845.000 - -72.85% -
450.soplex 1.86% 539.000 549.000 - -96.37% -
403.gcc 1.79% 56.000 57.000 - -99.40% -
401.bzip2 1.31% 2822.000 2859.000 - -82.28% -
429.mcf 1.06% 1135.000 1147.000 - -91.80% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (max) 400.00% 2.000 10.000 - -95.98% -
482.sphinx3 (max) 350.00% 2.000 9.000 - -98.98% -
471.omnetpp (unexplained diff vs previous) 338.46% 13.000 57.000 - -44.66% -
450.soplex (max) (percentage) 332.56% 0.430 1.860 - 12.73% -
471.omnetpp (unexplained diff vs previous) (percentage) 325.53% 0.470 2.000 - 104.08% -
473.astar (unexplained diff vs previous) 325.00% 4.000 17.000 - -32.00% -
473.astar (unexplained diff vs previous) (percentage) 311.11% 0.090 0.370 - 85.00% -
482.sphinx3 (max) (percentage) 264.00% 0.250 0.910 - -72.51% -
429.mcf (unexplained diff vs previous) (percentage) 200.00% 0.350 1.050 - 556.25% -
429.mcf (unexplained diff vs previous) 200.00% 4.000 12.000 - -45.45% -
471.omnetpp (max) 93.33% 30.000 58.000 - -94.03% -
471.omnetpp (max) (percentage) 83.78% 1.110 2.040 - -78.02% -
473.astar (max) 41.67% 12.000 17.000 - -97.93% -
473.astar (max) (percentage) 32.14% 0.280 0.370 - -94.48% -
470.lbm (max) (percentage) 8.70% 0.460 0.500 - -91.26% -
470.lbm (max) 5.56% 18.000 19.000 - -97.49% -
471.omnetpp 2.94% 34.000 35.000 - -79.04% -
471.omnetpp (avg) 2.94% 34.000 35.000 - -79.04% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
453.povray -1.40% 499.000 492.000 - -92.96% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (max) -40.00% 15.000 9.000 - -99.28% -
456.hmmer (max) (percentage) -35.00% 0.400 0.260 - -97.52% -
456.hmmer (unexplained diff vs previous) -25.00% 8.000 6.000 - -86.96% -
456.hmmer (unexplained diff vs previous) (percentage) -23.81% 0.210 0.160 - -58.97% -
462.libquantum (max) -20.00% 5.000 4.000 - -99.56% -
462.libquantum (max) (percentage) -16.72% 3.230 2.690 - -57.84% -
453.povray (max) -16.67% 6.000 5.000 - -80.00% -
401.bzip2 (max) -14.29% 7.000 6.000 - -99.01% -
433.milc (max) -8.33% 12.000 11.000 - -89.81% -
401.bzip2 (max) (percentage) -7.69% 0.260 0.240 - -93.65% -
464.h264ref (max) -7.69% 13.000 12.000 - -80.00% -
433.milc (max) (percentage) -7.34% 1.090 1.010 - 26.25% -
464.h264ref (max) (percentage) -4.55% 0.220 0.210 - -50.00% -
429.mcf (max) (percentage) -4.37% 1.830 1.750 - -89.27% -
453.povray (max) (percentage) -3.20% 1.250 1.210 - 236.11% -
444.namd (max) (percentage) -3.03% 0.330 0.320 - -96.04% -
403.gcc (max) (percentage) -2.30% 4.790 4.680 - 100.86% -
445.gobmk (max) (percentage) -1.96% 0.510 0.500 - -86.60% -
456.hmmer (avg) (percentage) -1.69% 0.590 0.580 - -58.27% -
462.libquantum (avg) (percentage) -1.27% 1.570 1.550 - 101.30% -
447.dealII (max) (percentage) -1.23% 0.810 0.800 - -66.39% -

Added Tests - sample variability
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (unexplained diff vs previous) (percentage) 556.25% 0.160 1.050 - 200.00% -
403.gcc (avg) (percentage) 536.36% 0.550 3.500 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 295.65% 0.460 1.820 - 0.00% -
453.povray (max) (percentage) 236.11% 0.360 1.210 - -3.20% -
450.soplex (avg) (percentage) 160.00% 0.500 1.300 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 104.08% 0.980 2.000 - 325.53% -
462.libquantum (avg) (percentage) 101.30% 0.770 1.550 - -1.27% -
403.gcc (max) (percentage) 100.86% 2.330 4.680 - -2.30% -
473.astar (unexplained diff vs previous) (percentage) 85.00% 0.200 0.370 - 311.11% -
433.milc (max) (percentage) 26.25% 0.800 1.010 - -7.34% -
453.povray (avg) (percentage) 23.75% 0.800 0.990 - 0.00% -
450.soplex (max) (percentage) 12.73% 1.650 1.860 - 332.56% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - 0.00% -
nb_compile_successful -5.26% 19.000 18.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.48% 7239.000 6915.000 - 0.00% -
462.libquantum -2.95% 12504.000 12135.000 - 0.00% -
433.milc -2.41% 47001.000 45869.000 - 0.02% -
453.povray -2.33% 524493.000 512254.000 - 0.01% -
447.dealII -2.22% 195792.000 191446.000 - -0.00% -
444.namd -1.86% 124963.000 122635.000 - 0.16% -
464.h264ref -1.80% 303767.000 298297.000 - 0.01% -
401.bzip2 -1.59% 33698.000 33161.000 - 0.01% -
456.hmmer -1.35% 80861.000 79769.000 - 0.00% -
470.lbm -1.35% 9338.000 9212.000 - 0.00% -
458.sjeng -1.23% 80297.000 79307.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.40% 9509.000 57.000 - 1.79% -
462.libquantum -98.75% 14409.000 180.000 - 0.00% -
450.soplex -96.37% 15112.000 549.000 - 1.86% -
482.sphinx3 -96.28% 26682.000 993.000 - 0.91% -
444.namd -93.44% 12400.000 814.000 - 0.74% -
453.povray -92.96% 6987.000 492.000 - -1.40% -
429.mcf -91.80% 13988.000 1147.000 - 1.06% -
433.milc -91.35% 13601.000 1176.000 - -0.25% -
401.bzip2 -82.28% 16137.000 2859.000 - 1.31% -
447.dealII -79.25% 9380.000 1946.000 - -0.31% -
471.omnetpp -72.85% 10477.000 2845.000 - 2.04% -
470.lbm -70.04% 13259.000 3973.000 - -0.50% -
456.hmmer -67.80% 11854.000 3817.000 - 0.16% -
473.astar -62.42% 12272.000 4612.000 - 0.37% -
464.h264ref -58.58% 14498.000 6005.000 - -0.02% -
445.gobmk -56.32% 11093.000 4845.000 - -0.14% -
458.sjeng -52.21% 17055.000 8151.000 - -0.16% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.80% 1003.000 2.000 - 0.00% -
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - 0.00% -
462.libquantum (max) -99.56% 919.000 4.000 - -20.00% -
456.hmmer (max) -99.28% 1242.000 9.000 - -40.00% -
429.mcf (max) -99.12% 2281.000 20.000 - 0.00% -
403.gcc (max) -99.10% 221.000 2.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -99.02% 914.000 9.000 - 0.00% -
401.bzip2 (max) -99.01% 609.000 6.000 - -14.29% -
482.sphinx3 (max) -98.98% 883.000 9.000 - 350.00% -
482.sphinx3 -98.49% 464.000 7.000 - 0.00% -
482.sphinx3 (avg) -98.49% 464.000 7.000 - 0.00% -
444.namd -98.43% 318.000 5.000 - 0.00% -
444.namd (avg) -98.43% 318.000 5.000 - 0.00% -
458.sjeng (max) -98.29% 876.000 15.000 - 0.00% -
462.libquantum -98.18% 110.000 2.000 - 0.00% -
462.libquantum (avg) -98.18% 110.000 2.000 - 0.00% -
403.gcc -98.08% 52.000 1.000 - 0.00% -
403.gcc (avg) -98.08% 52.000 1.000 - 0.00% -
473.astar (max) -97.93% 822.000 17.000 - 41.67% -
456.hmmer (max) (percentage) -97.52% 10.480 0.260 - -35.00% -
470.lbm (unexplained diff vs previous) -97.51% 804.000 20.000 - 0.00% -
470.lbm (max) -97.49% 758.000 19.000 - 5.56% -
458.sjeng (max) (percentage) -96.30% 5.140 0.190 - 0.00% -
444.namd (max) (percentage) -96.04% 8.090 0.320 - -3.03% -
450.soplex (max) -95.98% 249.000 10.000 - 400.00% -
429.mcf -95.27% 296.000 14.000 - 0.00% -
429.mcf (avg) -95.27% 296.000 14.000 - 0.00% -
470.lbm -95.01% 441.000 22.000 - 0.00% -
470.lbm (avg) -95.01% 441.000 22.000 - 0.00% -
473.astar (max) (percentage) -94.48% 6.700 0.370 - 32.14% -
445.gobmk (max) -94.19% 413.000 24.000 - 0.00% -
471.omnetpp (max) -94.03% 972.000 58.000 - 93.33% -
401.bzip2 (max) (percentage) -93.65% 3.780 0.240 - -7.69% -
447.dealII (max) -93.27% 223.000 15.000 - 0.00% -
433.milc -93.10% 116.000 8.000 - 0.00% -
433.milc (avg) -93.10% 116.000 8.000 - 0.00% -
453.povray -92.73% 55.000 4.000 - 0.00% -
453.povray (avg) -92.73% 55.000 4.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -91.75% 6.060 0.500 - 0.00% -
470.lbm (max) (percentage) -91.26% 5.720 0.500 - 8.70% -
450.soplex -90.67% 75.000 7.000 - 0.00% -
450.soplex (avg) -90.67% 75.000 7.000 - 0.00% -
433.milc (max) -89.81% 108.000 11.000 - -8.33% -
429.mcf (max) (percentage) -89.27% 16.310 1.750 - -4.37% -
401.bzip2 -88.46% 104.000 12.000 - 0.00% -
401.bzip2 (avg) -88.46% 104.000 12.000 - 0.00% -
456.hmmer (unexplained diff vs previous) -86.96% 46.000 6.000 - -25.00% -
445.gobmk (max) (percentage) -86.60% 3.730 0.500 - -1.96% -
456.hmmer -86.59% 164.000 22.000 - 0.00% -
456.hmmer (avg) -86.59% 164.000 22.000 - 0.00% -
447.dealII -85.71% 98.000 14.000 - 0.00% -
447.dealII (avg) -85.71% 98.000 14.000 - 0.00% -
450.soplex (unexplained diff vs previous) -85.51% 69.000 10.000 - 0.00% -
445.gobmk -83.78% 74.000 12.000 - 0.00% -
445.gobmk (avg) -83.78% 74.000 12.000 - 0.00% -
470.lbm (avg) (percentage) -82.88% 3.330 0.570 - 0.00% -
458.sjeng -82.35% 136.000 24.000 - 0.00% -
458.sjeng (avg) -82.35% 136.000 24.000 - 0.00% -
473.astar -81.25% 112.000 21.000 - 0.00% -
473.astar (avg) -81.25% 112.000 21.000 - 0.00% -
453.povray (max) -80.00% 25.000 5.000 - -16.67% -
464.h264ref (max) -80.00% 60.000 12.000 - -7.69% -
471.omnetpp -79.04% 167.000 35.000 - 2.94% -
471.omnetpp (avg) -79.04% 167.000 35.000 - 2.94% -
471.omnetpp (max) (percentage) -78.02% 9.280 2.040 - 83.78% -
444.namd (avg) (percentage) -74.32% 2.570 0.660 - 0.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -73.47% 3.430 0.910 - 0.00% -
482.sphinx3 (max) (percentage) -72.51% 3.310 0.910 - 264.00% -
464.h264ref -71.43% 42.000 12.000 - 0.00% -
464.h264ref (avg) -71.43% 42.000 12.000 - 0.00% -
447.dealII (max) (percentage) -66.39% 2.380 0.800 - -1.23% -
458.sjeng (avg) (percentage) -62.50% 0.800 0.300 - 0.00% -
445.gobmk (avg) (percentage) -61.19% 0.670 0.260 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -58.97% 0.390 0.160 - -23.81% -
456.hmmer (avg) (percentage) -58.27% 1.390 0.580 - -1.69% -
462.libquantum (max) (percentage) -57.84% 6.380 2.690 - -16.72% -
482.sphinx3 (avg) (percentage) -55.17% 1.740 0.780 - 0.00% -
464.h264ref (max) (percentage) -50.00% 0.420 0.210 - -4.55% -
473.astar (avg) (percentage) -50.00% 0.920 0.460 - -2.13% -
429.mcf (unexplained diff vs previous) -45.45% 22.000 12.000 - 200.00% -
471.omnetpp (unexplained diff vs previous) -44.66% 103.000 57.000 - 338.46% -
429.mcf (avg) (percentage) -40.57% 2.120 1.260 - -0.79% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 0.00% -
473.astar (unexplained diff vs previous) -32.00% 25.000 17.000 - 325.00% -
401.bzip2 (avg) (percentage) -30.77% 0.650 0.450 - 0.00% -
447.dealII (avg) (percentage) -28.57% 1.050 0.750 - 0.00% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - 0.00% -
471.omnetpp (avg) (percentage) -22.50% 1.600 1.240 - 0.81% -
433.milc (avg) (percentage) -12.79% 0.860 0.750 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 0.00% -

Added Tests - sample variability
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
401.bzip2 2859.000 1.31%
403.gcc 57.000 1.79%
429.mcf 1147.000 1.06%
433.milc 1176.000 -
444.namd 814.000 -
445.gobmk 4845.000 -
447.dealII 1946.000 -
450.soplex 549.000 1.86%
453.povray 492.000 -1.40%
456.hmmer 3817.000 -
458.sjeng 8151.000 -
462.libquantum 180.000 -
464.h264ref 6005.000 -
470.lbm 3973.000 -
471.omnetpp 2845.000 2.04%
473.astar 4612.000 -
482.sphinx3 993.000 -
Geometric Mean 1480.996 -
Code Size (bytes)
Name Current %
401.bzip2 33161.000 -
403.gcc 1653476.000 -
429.mcf 6915.000 -
433.milc 45869.000 -
444.namd 122635.000 -
445.gobmk 1255977.000 -
447.dealII 191446.000 -
450.soplex 146436.000 -
453.povray 512254.000 -
456.hmmer 79769.000 -
458.sjeng 79307.000 -
462.libquantum 12135.000 -
464.h264ref 298297.000 -
470.lbm 9212.000 -
471.omnetpp 317466.000 -
473.astar 15583.000 -
482.sphinx3 87400.000 -
Geometric Mean 94025.303 -
sample variability
Name Current %
401.bzip2 12.000 -
401.bzip2 (avg) 12.000 -
401.bzip2 (avg) (percentage) 0.450 -
401.bzip2 (max) 6.000 -14.29%
401.bzip2 (max) (percentage) 0.240 -7.69%
403.gcc 1.000 -
403.gcc (avg) 1.000 -
403.gcc (avg) (percentage) 3.500 -
403.gcc (max) 2.000 -
403.gcc (max) (percentage) 4.680 -2.30%
429.mcf 14.000 -
429.mcf (avg) 14.000 -
429.mcf (avg) (percentage) 1.260 -
429.mcf (max) 20.000 -
429.mcf (max) (percentage) 1.750 -4.37%
429.mcf (unexplained diff vs previous) 12.000 200.00%
429.mcf (unexplained diff vs previous) (percentage) 1.050 200.00%
433.milc 8.000 -
433.milc (avg) 8.000 -
433.milc (avg) (percentage) 0.750 -
433.milc (max) 11.000 -8.33%
433.milc (max) (percentage) 1.010 -7.34%
444.namd 5.000 -
444.namd (avg) 5.000 -
444.namd (avg) (percentage) 0.660 -
444.namd (max) 2.000 -
444.namd (max) (percentage) 0.320 -3.03%
445.gobmk 12.000 -
445.gobmk (avg) 12.000 -
445.gobmk (avg) (percentage) 0.260 -
445.gobmk (max) 24.000 -
445.gobmk (max) (percentage) 0.500 -1.96%
447.dealII 14.000 -
447.dealII (avg) 14.000 -
447.dealII (avg) (percentage) 0.750 -
447.dealII (max) 15.000 -
447.dealII (max) (percentage) 0.800 -1.23%
450.soplex 7.000 -
450.soplex (avg) 7.000 -
450.soplex (avg) (percentage) 1.300 -
450.soplex (max) 10.000 400.00%
450.soplex (max) (percentage) 1.860 332.56%
450.soplex (unexplained diff vs previous) 10.000 -
450.soplex (unexplained diff vs previous) (percentage) 1.820 -
453.povray 4.000 -
453.povray (avg) 4.000 -
453.povray (avg) (percentage) 0.990 -
453.povray (max) 5.000 -16.67%
453.povray (max) (percentage) 1.210 -3.20%
456.hmmer 22.000 -
456.hmmer (avg) 22.000 -
456.hmmer (avg) (percentage) 0.580 -1.69%
456.hmmer (max) 9.000 -40.00%
456.hmmer (max) (percentage) 0.260 -35.00%
456.hmmer (unexplained diff vs previous) 6.000 -25.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.160 -23.81%
458.sjeng 24.000 -
458.sjeng (avg) 24.000 -
458.sjeng (avg) (percentage) 0.300 -
458.sjeng (max) 15.000 -
458.sjeng (max) (percentage) 0.190 -
458.sjeng (unexplained diff vs previous) 13.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.160 -
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.550 -1.27%
462.libquantum (max) 4.000 -20.00%
462.libquantum (max) (percentage) 2.690 -16.72%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 12.000 -
464.h264ref (avg) 12.000 -
464.h264ref (avg) (percentage) 0.210 -
464.h264ref (max) 12.000 -7.69%
464.h264ref (max) (percentage) 0.210 -4.55%
470.lbm 22.000 -
470.lbm (avg) 22.000 -
470.lbm (avg) (percentage) 0.570 -
470.lbm (max) 19.000 5.56%
470.lbm (max) (percentage) 0.500 8.70%
470.lbm (unexplained diff vs previous) 20.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.500 -
471.omnetpp 35.000 2.94%
471.omnetpp (avg) 35.000 2.94%
471.omnetpp (avg) (percentage) 1.240 -
471.omnetpp (max) 58.000 93.33%
471.omnetpp (max) (percentage) 2.040 83.78%
471.omnetpp (unexplained diff vs previous) 57.000 338.46%
471.omnetpp (unexplained diff vs previous) (percentage) 2.000 325.53%
473.astar 21.000 -
473.astar (avg) 21.000 -
473.astar (avg) (percentage) 0.460 -
473.astar (max) 17.000 41.67%
473.astar (max) (percentage) 0.370 32.14%
473.astar (unexplained diff vs previous) 17.000 325.00%
473.astar (unexplained diff vs previous) (percentage) 0.370 311.11%
482.sphinx3 7.000 -
482.sphinx3 (avg) 7.000 -
482.sphinx3 (avg) (percentage) 0.780 -
482.sphinx3 (max) 9.000 350.00%
482.sphinx3 (max) (percentage) 0.910 264.00%
482.sphinx3 (unexplained diff vs previous) 9.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.910 -
483.xalancbmk (avg) (percentage) 1.440 -
483.xalancbmk (max) (percentage) 0.050 -
Geometric Mean 2.619 14.88%