Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/e06fc2b2e0651b60e8d21b46f9708bc74d8e4f16 (+ 766 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/67/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0067 (llvmorg-18-init-06687-ge06fc2b2e065) 2023-09-19T01:46:18 0:00:00
Previous 0066 (no-change) 2023-09-14T17:04:08 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 0 26
Performance Improvements 18 28
Added Tests 0 2
Unchanged Tests 714 676
Total Tests 732

Run-Over-Run Changes Detail

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
462.libquantum -2.17% 33790.000 33056.000 - -6.51% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (unexplained diff vs previous) (percentage) -95.00% 1.600 0.080 - 0.00% -
458.sjeng (unexplained diff vs previous) -94.71% 227.000 12.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) -81.82% 0.440 0.080 - 60.00% -
429.mcf (unexplained diff vs previous) -80.95% 42.000 8.000 - 60.00% -
458.sjeng (max) -16.96% 224.000 186.000 - 264.71% -
458.sjeng (max) (percentage) -16.46% 1.580 1.320 - 277.14% -
429.mcf (max) -7.32% 41.000 38.000 - 280.00% -
429.mcf (max) (percentage) -6.98% 0.430 0.400 - 263.64% -
462.libquantum (max) -5.26% 38.000 36.000 - 350.00% -
462.libquantum (max) (percentage) -3.85% 0.260 0.250 - 316.67% -
458.sjeng -3.75% 80.000 77.000 - -16.30% -
458.sjeng (avg) -3.75% 80.000 77.000 - -16.30% -
401.bzip2 (max) (percentage) -3.61% 0.830 0.800 - 900.00% -
458.sjeng (avg) (percentage) -3.51% 0.570 0.550 - -12.70% -
401.bzip2 (max) -3.19% 94.000 91.000 - 911.11% -
453.povray (max) (percentage) -2.33% 0.430 0.420 - -33.33% -
471.omnetpp (max) (percentage) -1.12% 0.890 0.880 - 91.30% -

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) 911.11% 9.000 91.000 - -3.19% -
401.bzip2 (max) (percentage) 900.00% 0.080 0.800 - -3.61% -
462.libquantum (max) 350.00% 8.000 36.000 - -5.26% -
462.libquantum (max) (percentage) 316.67% 0.060 0.250 - -3.85% -
429.mcf (max) 280.00% 10.000 38.000 - -7.32% -
458.sjeng (max) (percentage) 277.14% 0.350 1.320 - -16.46% -
458.sjeng (max) 264.71% 51.000 186.000 - -16.96% -
429.mcf (max) (percentage) 263.64% 0.110 0.400 - -6.98% -
401.bzip2 (avg) (percentage) 125.00% 0.080 0.180 - 0.00% -
401.bzip2 122.22% 9.000 20.000 - 0.00% -
401.bzip2 (avg) 122.22% 9.000 20.000 - 0.00% -
471.omnetpp (max) (percentage) 91.30% 0.460 0.880 - -1.12% -
429.mcf (unexplained diff vs previous) 60.00% 5.000 8.000 - -80.95% -
429.mcf (unexplained diff vs previous) (percentage) 60.00% 0.050 0.080 - -81.82% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
482.sphinx3 (max) (percentage) 48.15% 0.270 0.400 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
462.libquantum 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) (percentage) 15.38% 0.130 0.150 - 0.00% -
429.mcf 14.29% 7.000 8.000 - 0.00% -
429.mcf (avg) 14.29% 7.000 8.000 - 0.00% -
483.xalancbmk (max) (percentage) 12.96% 1.080 1.220 - -0.81% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.51% 35357.000 33056.000 - -2.17% -
401.bzip2 -2.01% 73592.000 72115.000 - -0.07% -
429.mcf -1.21% 11019.000 10886.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -3.38% 14662.000 14166.000 - -0.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -80.77% 0.780 0.150 - 0.00% -
456.hmmer (max) (percentage) -70.16% 3.820 1.140 - -0.87% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
473.astar (max) (percentage) -41.18% 0.170 0.100 - -9.09% -
445.gobmk (max) (percentage) -40.74% 0.270 0.160 - 0.00% -
444.namd (max) (percentage) -38.46% 0.260 0.160 - 0.00% -
453.povray (max) (percentage) -33.33% 0.630 0.420 - -2.33% -
464.h264ref (max) (percentage) -28.89% 0.900 0.640 - 0.00% -
450.soplex (max) (percentage) -25.00% 0.240 0.180 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
458.sjeng -16.30% 92.000 77.000 - -3.75% -
458.sjeng (avg) -16.30% 92.000 77.000 - -3.75% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
458.sjeng (avg) (percentage) -12.70% 0.630 0.550 - -3.51% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -
447.dealII (max) (percentage) -3.03% 0.660 0.640 - 0.00% -

Added Tests - sample variability
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11463.000 -
429.mcf 9627.000 -
458.sjeng 14166.000 -
462.libquantum 14731.000 -
Geometric Mean 12318.765 -
Code Size (bytes)
Name Current %
401.bzip2 72115.000 -
429.mcf 10886.000 -
458.sjeng 148148.000 -
462.libquantum 33056.000 -2.17%
Geometric Mean 44280.244 -
sample variability
Name Current %
401.bzip2 20.000 -
401.bzip2 (avg) 20.000 -
401.bzip2 (avg) (percentage) 0.180 -
401.bzip2 (max) 91.000 -3.19%
401.bzip2 (max) (percentage) 0.800 -3.61%
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) (percentage) 0.150 -
429.mcf 8.000 -
429.mcf (avg) 8.000 -
429.mcf (avg) (percentage) 0.090 -
429.mcf (max) 38.000 -7.32%
429.mcf (max) (percentage) 0.400 -6.98%
429.mcf (unexplained diff vs previous) 8.000 -80.95%
429.mcf (unexplained diff vs previous) (percentage) 0.080 -81.82%
433.milc (avg) (percentage) 0.060 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.160 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.160 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.640 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) (percentage) 0.180 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.420 -2.33%
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.140 -
458.sjeng 77.000 -3.75%
458.sjeng (avg) 77.000 -3.75%
458.sjeng (avg) (percentage) 0.550 -3.51%
458.sjeng (max) 186.000 -16.96%
458.sjeng (max) (percentage) 1.320 -16.46%
458.sjeng (unexplained diff vs previous) 12.000 -94.71%
458.sjeng (unexplained diff vs previous) (percentage) 0.080 -95.00%
462.libquantum 22.000 -
462.libquantum (avg) 22.000 -
462.libquantum (avg) (percentage) 0.150 -
462.libquantum (max) 36.000 -5.26%
462.libquantum (max) (percentage) 0.250 -3.85%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.640 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.880 -1.12%
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.100 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) (percentage) 0.400 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) (percentage) 1.220 -
Geometric Mean 0.992 -17.86%