Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/2a0314f153b3fdc6d8b51d35c2e4a4bb67dfcaef (+ 272 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/124/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0124 (llvmorg-18-init-14175-g2a0314f153b3) 2023-12-07T10:00:20 0:00:00
Previous 0122 (no-change) 2023-12-06T07:19:18 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 7 29
Performance Improvements 17 33
Added Tests 15 19
Unchanged Tests 693 651
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 100.00% 1.000 2.000 - -99.99% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) 225.00% 0.040 0.130 - 0.00% -
462.libquantum (unexplained diff vs previous) 216.67% 6.000 19.000 - 0.00% -
429.mcf (unexplained diff vs previous) 166.67% 3.000 8.000 - 60.00% -
429.mcf (unexplained diff vs previous) (percentage) 166.67% 0.030 0.080 - 60.00% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -
nb_total_tests -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -7.14% 14.000 13.000 - -99.85% -
403.gcc -3.47% 144.000 139.000 - -97.90% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (unexplained diff vs previous) -71.88% 96.000 27.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) -71.21% 0.660 0.190 - 0.00% -
458.sjeng (max) -38.46% 117.000 72.000 - 41.18% -
458.sjeng (max) (percentage) -38.27% 0.810 0.500 - 42.86% -
462.libquantum (max) -5.88% 51.000 48.000 - 500.00% -
429.mcf (max) -5.80% 69.000 65.000 - 550.00% -
462.libquantum (max) (percentage) -5.71% 0.350 0.330 - 450.00% -
429.mcf (max) (percentage) -5.56% 0.720 0.680 - 518.18% -
458.sjeng -2.67% 75.000 73.000 - -20.65% -
458.sjeng (avg) -2.67% 75.000 73.000 - -20.65% -
401.bzip2 (max) -2.19% 137.000 134.000 - 1388.89% -
458.sjeng (avg) (percentage) -1.92% 0.520 0.510 - -19.05% -
401.bzip2 (max) (percentage) -1.65% 1.210 1.190 - 1387.50% -
401.bzip2 -1.45% 69.000 68.000 - 655.56% -
401.bzip2 (avg) -1.45% 69.000 68.000 - 655.56% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 1.64% 73592.000 74799.000 - -0.09% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) 1388.89% 9.000 134.000 - -2.19% -
401.bzip2 (max) (percentage) 1387.50% 0.080 1.190 - -1.65% -
401.bzip2 (avg) (percentage) 662.50% 0.080 0.610 - 0.00% -
401.bzip2 655.56% 9.000 68.000 - -1.45% -
401.bzip2 (avg) 655.56% 9.000 68.000 - -1.45% -
429.mcf (max) 550.00% 10.000 65.000 - -5.80% -
429.mcf (max) (percentage) 518.18% 0.110 0.680 - -5.56% -
462.libquantum (max) 500.00% 8.000 48.000 - -5.88% -
462.libquantum (max) (percentage) 450.00% 0.060 0.330 - -5.71% -
429.mcf 114.29% 7.000 15.000 - 0.00% -
429.mcf (avg) 114.29% 7.000 15.000 - 0.00% -
429.mcf (avg) (percentage) 100.00% 0.080 0.160 - 0.00% -
471.omnetpp (max) (percentage) 84.78% 0.460 0.850 - 0.00% -
462.libquantum (avg) (percentage) 61.54% 0.130 0.210 - 0.00% -
429.mcf (unexplained diff vs previous) 60.00% 5.000 8.000 - 166.67% -
429.mcf (unexplained diff vs previous) (percentage) 60.00% 0.050 0.080 - 166.67% -
462.libquantum 57.89% 19.000 30.000 - 0.00% -
462.libquantum (avg) 57.89% 19.000 30.000 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
458.sjeng (max) (percentage) 42.86% 0.350 0.500 - -38.27% -
458.sjeng (max) 41.18% 51.000 72.000 - -38.46% -
482.sphinx3 (max) (percentage) 33.33% 0.270 0.360 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 0.00% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -
nb_total_tests -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.60% 35357.000 33024.000 - 0.00% -
429.mcf -1.64% 11019.000 10838.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm -99.99% 15896.000 1.000 - 0.00% -
482.sphinx3 -99.99% 18232.000 2.000 - 100.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.85% 8399.000 13.000 - -7.14% -
403.gcc -97.90% 6613.000 139.000 - -3.47% -
458.sjeng -1.08% 14662.000 14504.000 - -0.19% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -72.25% 3.820 1.060 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
445.gobmk (max) (percentage) -48.15% 0.270 0.140 - 0.00% -
473.astar (max) (percentage) -47.06% 0.170 0.090 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -34.44% 0.900 0.590 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
458.sjeng -20.65% 92.000 73.000 - -2.67% -
458.sjeng (avg) -20.65% 92.000 73.000 - -2.67% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
458.sjeng (avg) (percentage) -19.05% 0.630 0.510 - -1.92% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
447.dealII (max) (percentage) -12.12% 0.660 0.580 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
483.xalancbmk (max) (percentage) -7.41% 1.080 1.000 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Report Time: 0.23s
Execution (seconds)
Name Current %
401.bzip2 11302.000 -
403.gcc 139.000 -3.47%
429.mcf 9668.000 -
433.milc 5.000 -
450.soplex 13.000 -7.14%
458.sjeng 14504.000 -
462.libquantum 14700.000 -
470.lbm 1.000 -
482.sphinx3 2.000 100.00%
Geometric Mean 195.701 6.66%
Code Size (bytes)
Name Current %
401.bzip2 74799.000 -
429.mcf 10838.000 -
458.sjeng 148668.000 -
462.libquantum 33024.000 -
Geometric Mean 44665.585 -
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 4.000 -
nb_total_tests 18.000 -5.26%
Geometric Mean 10.903 -3.54%
sample variability
Name Current %
401.bzip2 68.000 -1.45%
401.bzip2 (avg) 68.000 -1.45%
401.bzip2 (avg) (percentage) 0.610 -
401.bzip2 (max) 134.000 -2.19%
401.bzip2 (max) (percentage) 1.190 -1.65%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 15.000 -
429.mcf (avg) 15.000 -
429.mcf (avg) (percentage) 0.160 -
429.mcf (max) 65.000 -5.80%
429.mcf (max) (percentage) 0.680 -5.56%
429.mcf (unexplained diff vs previous) 8.000 166.67%
429.mcf (unexplained diff vs previous) (percentage) 0.080 166.67%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.140 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.580 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.060 -
458.sjeng 73.000 -2.67%
458.sjeng (avg) 73.000 -2.67%
458.sjeng (avg) (percentage) 0.510 -1.92%
458.sjeng (max) 72.000 -38.46%
458.sjeng (max) (percentage) 0.500 -38.27%
458.sjeng (unexplained diff vs previous) 27.000 -71.88%
458.sjeng (unexplained diff vs previous) (percentage) 0.190 -71.21%
462.libquantum 30.000 -
462.libquantum (avg) 30.000 -
462.libquantum (avg) (percentage) 0.210 -
462.libquantum (max) 48.000 -5.88%
462.libquantum (max) (percentage) 0.330 -5.71%
462.libquantum (unexplained diff vs previous) 19.000 216.67%
462.libquantum (unexplained diff vs previous) (percentage) 0.130 225.00%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.590 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.850 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.090 -
482.sphinx3 0.000 -
482.sphinx3 (avg) 0.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 0.000 -
482.sphinx3 (max) (percentage) 0.360 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) (percentage) 1.000 -
Geometric Mean 0.151 -86.81%