Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/d9e4309b451c1b24d4e0a6304057663b877e5266 (+ 1914 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/171/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0171 (llvmorg-19-init-03206-gd9e4309b451c) 2024-02-14T06:18:14 0:00:00
Previous 0170 (no-change) 2024-02-05T00:45:55 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 0 27
Performance Improvements 18 39
Added Tests 12 12
Unchanged Tests 702 654
Total Tests 732

Run-Over-Run Changes Detail

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -14.29% 14.000 12.000 - -99.86% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (unexplained diff vs previous) (percentage) -79.61% 1.030 0.210 - 950.00% -
401.bzip2 (unexplained diff vs previous) -79.31% 116.000 24.000 - 1100.00% -
429.mcf (unexplained diff vs previous) -66.67% 3.000 1.000 - -80.00% -
429.mcf (unexplained diff vs previous) (percentage) -66.67% 0.030 0.010 - -80.00% -
429.mcf (max) -11.54% 26.000 23.000 - 130.00% -
429.mcf (max) (percentage) -11.11% 0.270 0.240 - 118.18% -
401.bzip2 (max) (percentage) -9.80% 1.020 0.920 - 1050.00% -
401.bzip2 (max) -9.65% 114.000 103.000 - 1044.44% -
458.sjeng (max) -8.84% 249.000 227.000 - 345.10% -
458.sjeng (max) (percentage) -8.05% 1.740 1.600 - 357.14% -
462.libquantum (max) -7.69% 39.000 36.000 - 350.00% -
462.libquantum (max) (percentage) -7.41% 0.270 0.250 - 316.67% -
429.mcf -6.25% 16.000 15.000 - 114.29% -
429.mcf (avg) -6.25% 16.000 15.000 - 114.29% -
429.mcf (avg) (percentage) -5.88% 0.170 0.160 - 100.00% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful 11.76% 17.000 19.000 - 0.00% -
nb_total_tests 11.76% 17.000 19.000 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 3.29% 73592.000 76011.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (unexplained diff vs previous) 1100.00% 2.000 24.000 - -79.31% -
401.bzip2 (max) (percentage) 1050.00% 0.080 0.920 - -9.80% -
401.bzip2 (max) 1044.44% 9.000 103.000 - -9.65% -
401.bzip2 (unexplained diff vs previous) (percentage) 950.00% 0.020 0.210 - -79.61% -
401.bzip2 (avg) (percentage) 637.50% 0.080 0.590 - 0.00% -
401.bzip2 633.33% 9.000 66.000 - 0.00% -
401.bzip2 (avg) 633.33% 9.000 66.000 - 0.00% -
458.sjeng (max) (percentage) 357.14% 0.350 1.600 - -8.05% -
462.libquantum (max) 350.00% 8.000 36.000 - -7.69% -
458.sjeng (max) 345.10% 51.000 227.000 - -8.84% -
462.libquantum (max) (percentage) 316.67% 0.060 0.250 - -7.41% -
429.mcf (max) 130.00% 10.000 23.000 - -11.54% -
429.mcf (max) (percentage) 118.18% 0.110 0.240 - -11.11% -
429.mcf 114.29% 7.000 15.000 - -6.25% -
429.mcf (avg) 114.29% 7.000 15.000 - -6.25% -
429.mcf (avg) (percentage) 100.00% 0.080 0.160 - -5.88% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (avg) (percentage) 46.15% 0.130 0.190 - 0.00% -
462.libquantum 42.11% 19.000 27.000 - 0.00% -
462.libquantum (avg) 42.11% 19.000 27.000 - 0.00% -
482.sphinx3 (max) (percentage) 29.63% 0.270 0.350 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.69% 35357.000 32992.000 - 0.05% -
429.mcf -1.93% 11019.000 10806.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
482.sphinx3 -99.99% 18232.000 2.000 - 0.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.86% 8399.000 12.000 - -14.29% -
403.gcc -97.90% 6613.000 139.000 - 0.72% -
483.xalancbmk -55.56% 5921.000 2631.000 - 0.00% -
458.sjeng -2.98% 14662.000 14225.000 - -0.88% -
401.bzip2 -1.07% 11351.000 11230.000 - -0.21% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
429.mcf (unexplained diff vs previous) -80.00% 5.000 1.000 - -66.67% -
429.mcf (unexplained diff vs previous) (percentage) -80.00% 0.050 0.010 - -66.67% -
456.hmmer (max) (percentage) -72.51% 3.820 1.050 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
483.xalancbmk (max) -61.90% 63.000 24.000 - 0.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
447.dealII (max) (percentage) -13.64% 0.660 0.570 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
483.xalancbmk (max) (percentage) -12.04% 1.080 0.950 - 0.00% -
458.sjeng -11.96% 92.000 81.000 - 0.00% -
458.sjeng (avg) -11.96% 92.000 81.000 - 0.00% -
458.sjeng (avg) (percentage) -9.52% 0.630 0.570 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11230.000 -
403.gcc 139.000 -
429.mcf 9686.000 -
433.milc 5.000 -
450.soplex 12.000 -14.29%
458.sjeng 14225.000 -
462.libquantum 14672.000 -
482.sphinx3 2.000 -
483.xalancbmk 2631.000 -
Geometric Mean 463.971 -1.73%
Code Size (bytes)
Name Current %
401.bzip2 76011.000 -
429.mcf 10806.000 -
458.sjeng 149196.000 -
462.libquantum 32992.000 -
Geometric Mean 44841.157 -
Score
Name Current %
nb_compile_successful 19.000 11.76%
nb_execution_successful 4.000 -
nb_total_tests 19.000 11.76%
Geometric Mean 11.303 7.70%
sample variability
Name Current %
401.bzip2 66.000 -
401.bzip2 (avg) 66.000 -
401.bzip2 (avg) (percentage) 0.590 -
401.bzip2 (max) 103.000 -9.65%
401.bzip2 (max) (percentage) 0.920 -9.80%
401.bzip2 (unexplained diff vs previous) 24.000 -79.31%
401.bzip2 (unexplained diff vs previous) (percentage) 0.210 -79.61%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 15.000 -6.25%
429.mcf (avg) 15.000 -6.25%
429.mcf (avg) (percentage) 0.160 -5.88%
429.mcf (max) 23.000 -11.54%
429.mcf (max) (percentage) 0.240 -11.11%
429.mcf (unexplained diff vs previous) 1.000 -66.67%
429.mcf (unexplained diff vs previous) (percentage) 0.010 -66.67%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.130 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.570 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.050 -
458.sjeng 81.000 -
458.sjeng (avg) 81.000 -
458.sjeng (avg) (percentage) 0.570 -
458.sjeng (max) 227.000 -8.84%
458.sjeng (max) (percentage) 1.600 -8.05%
462.libquantum 27.000 -
462.libquantum (avg) 27.000 -
462.libquantum (avg) (percentage) 0.190 -
462.libquantum (max) 36.000 -7.69%
462.libquantum (max) (percentage) 0.250 -7.41%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.580 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.840 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 0.000 -
482.sphinx3 (avg) 0.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 0.000 -
482.sphinx3 (max) (percentage) 0.350 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 24.000 -
483.xalancbmk (max) (percentage) 0.950 -
Geometric Mean 0.225 -83.47%