Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/1c8c365de2ac59198e6ef9d61a5557b846ce0bca (+ 252 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/64/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0064 (llvmorg-18-init-05921-g1c8c365de2ac) 2023-09-11T22:22:16 0:00:00
Previous 0063 (llvmorg-18-init-05669-g942ce3198591) 2023-09-06T08:58:32 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 2 21
Performance Improvements 10 30
Added Tests 5 3
Unchanged Tests 715 678
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng 1.33% 75.000 76.000 - -17.39% -
458.sjeng (avg) 1.33% 75.000 76.000 - -17.39% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) -10.00% 10.000 9.000 - -10.00% -
462.libquantum (max) -6.25% 48.000 45.000 - 462.50% -
462.libquantum (max) (percentage) -6.06% 0.330 0.310 - 416.67% -
401.bzip2 (max) -5.77% 104.000 98.000 - 988.89% -
401.bzip2 (max) (percentage) -5.43% 0.920 0.870 - 987.50% -
458.sjeng (max) (percentage) -4.71% 0.850 0.810 - 131.43% -
458.sjeng (max) -4.13% 121.000 116.000 - 127.45% -
483.xalancbmk (max) (percentage) -1.57% 1.270 1.250 - 15.74% -
464.h264ref (max) (percentage) -1.54% 0.650 0.640 - -28.89% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful 125.00% 4.000 9.000 - -50.00% -

Added Tests - Code Size
450.soplex

Added Tests - Execution
450.soplex

Added Tests - sample variability
450.soplex
450.soplex (avg)
450.soplex (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.44% 309981.000 314460.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) 988.89% 9.000 98.000 - -5.77% -
401.bzip2 (max) (percentage) 987.50% 0.080 0.870 - -5.43% -
462.libquantum (max) 462.50% 8.000 45.000 - -6.25% -
462.libquantum (max) (percentage) 416.67% 0.060 0.310 - -6.06% -
401.bzip2 (avg) (percentage) 137.50% 0.080 0.190 - 0.00% -
401.bzip2 133.33% 9.000 21.000 - 0.00% -
401.bzip2 (avg) 133.33% 9.000 21.000 - 0.00% -
458.sjeng (max) (percentage) 131.43% 0.350 0.810 - -4.71% -
458.sjeng (max) 127.45% 51.000 116.000 - -4.13% -
471.omnetpp (max) (percentage) 93.48% 0.460 0.890 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
482.sphinx3 (max) (percentage) 51.85% 0.270 0.410 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
462.libquantum 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) 15.79% 19.000 22.000 - 0.00% -
483.xalancbmk (max) (percentage) 15.74% 1.080 1.250 - -1.57% -
462.libquantum (avg) (percentage) 15.38% 0.130 0.150 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -50.00% 18.000 9.000 - 125.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -4.43% 35357.000 33790.000 - 0.33% -
401.bzip2 -1.94% 73592.000 72163.000 - -0.69% -
429.mcf -1.21% 11019.000 10886.000 - -0.29% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
450.soplex -99.83% 8399.000 14.000 - 0.00% -
458.sjeng -1.75% 14662.000 14405.000 - 0.82% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -80.77% 0.780 0.150 - 0.00% -
456.hmmer (max) (percentage) -69.90% 3.820 1.150 - -0.86% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
445.gobmk (max) (percentage) -40.74% 0.270 0.160 - 0.00% -
444.namd (max) (percentage) -38.46% 0.260 0.160 - 0.00% -
473.astar (max) (percentage) -35.29% 0.170 0.110 - 0.00% -
453.povray (max) (percentage) -31.75% 0.630 0.430 - 0.00% -
464.h264ref (max) (percentage) -28.89% 0.900 0.640 - -1.54% -
450.soplex (max) (percentage) -20.83% 0.240 0.190 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
458.sjeng -17.39% 92.000 76.000 - 1.33% -
458.sjeng (avg) -17.39% 92.000 76.000 - 1.33% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
458.sjeng (avg) (percentage) -15.87% 0.630 0.530 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
429.mcf (max) -10.00% 10.000 9.000 - -10.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -
447.dealII (max) (percentage) -1.52% 0.660 0.650 - 0.00% -

Added Tests - sample variability
450.soplex
450.soplex (avg)
450.soplex (max)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11362.000 -
429.mcf 9677.000 -
450.soplex 14.000 -
458.sjeng 14405.000 -
462.libquantum 14719.000 -
Geometric Mean 3182.318 -74.16%
Code Size (bytes)
Name Current %
401.bzip2 72163.000 -
429.mcf 10886.000 -
450.soplex 314460.000 -
458.sjeng 148148.000 -
462.libquantum 33790.000 -
Geometric Mean 65833.427 47.60%
Score
Name Current %
nb_compile_successful 19.000 -
nb_execution_successful 9.000 125.00%
nb_total_tests 19.000 -
Geometric Mean 14.811 31.04%
sample variability
Name Current %
401.bzip2 21.000 -
401.bzip2 (avg) 21.000 -
401.bzip2 (avg) (percentage) 0.190 -
401.bzip2 (max) 98.000 -5.77%
401.bzip2 (max) (percentage) 0.870 -5.43%
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) (percentage) 0.150 -
429.mcf 7.000 -
429.mcf (avg) 7.000 -
429.mcf (avg) (percentage) 0.080 -
429.mcf (max) 9.000 -10.00%
429.mcf (max) (percentage) 0.100 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.160 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.160 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.650 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.190 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.430 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.150 -
458.sjeng 76.000 1.33%
458.sjeng (avg) 76.000 1.33%
458.sjeng (avg) (percentage) 0.530 -
458.sjeng (max) 116.000 -4.13%
458.sjeng (max) (percentage) 0.810 -4.71%
462.libquantum 22.000 -
462.libquantum (avg) 22.000 -
462.libquantum (avg) (percentage) 0.150 -
462.libquantum (max) 45.000 -6.25%
462.libquantum (max) (percentage) 0.310 -6.06%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.640 -1.54%
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.890 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.110 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) (percentage) 0.410 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) (percentage) 1.250 -1.57%
Geometric Mean 0.546 -42.27%