Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/ed59b8a11c899a4beb9145d31d95dfe2855facfd (+ 9203 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/30/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0030 (llvmorg-17-init-11932-ged59b8a11c89) 2023-04-05T18:16:08 0:00:00
Previous 0025 (llvmorg-17-init-02729-gdf016a9525e5) 2023-02-17T09:03:40 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 7 25
Performance Improvements 22 26
Unchanged Tests 727 705
Total Tests 756

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) 233.33% 6.000 20.000 - 100.00% -
429.mcf (max) (percentage) 200.00% 0.070 0.210 - 90.91% -
462.libquantum (max) (percentage) 123.08% 0.130 0.290 - 383.33% -
462.libquantum (max) 121.05% 19.000 42.000 - 425.00% -
429.mcf 14.29% 7.000 8.000 - 14.29% -
429.mcf (avg) 14.29% 7.000 8.000 - 14.29% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -77.78% 18.000 4.000 - -77.78% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
462.libquantum -4.22% 35357.000 33864.000 - -4.22% -
401.bzip2 -1.28% 73552.000 72611.000 - -1.33% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
458.sjeng -4.22% 14739.000 14117.000 - -3.72% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (max) (percentage) -29.73% 1.480 1.040 - 126.09% -
401.bzip2 (max) (percentage) -25.71% 0.700 0.520 - 550.00% -
458.sjeng (max) -25.32% 154.000 115.000 - 125.49% -
401.bzip2 (max) -25.32% 79.000 59.000 - 555.56% -
433.milc (max) (percentage) -25.00% 0.040 0.030 - -66.67% -
458.sjeng (max) (percentage) -21.90% 1.050 0.820 - 134.29% -
456.hmmer (max) (percentage) -19.02% 1.630 1.320 - -65.45% -
450.soplex (max) (percentage) -18.52% 0.270 0.220 - -8.33% -
445.gobmk (max) (percentage) -17.39% 0.230 0.190 - -29.63% -
473.astar (max) (percentage) -13.33% 0.150 0.130 - -23.53% -
444.namd (max) (percentage) -13.04% 0.230 0.200 - -23.08% -
483.xalancbmk (max) (percentage) -10.65% 1.690 1.510 - 39.81% -
482.sphinx3 (max) (percentage) -9.80% 0.510 0.460 - 70.37% -
464.h264ref (max) (percentage) -7.79% 0.770 0.710 - -21.11% -
447.dealII (max) (percentage) -7.59% 0.790 0.730 - 10.61% -
458.sjeng -6.58% 76.000 71.000 - -22.83% -
458.sjeng (avg) -6.58% 76.000 71.000 - -22.83% -
453.povray (max) (percentage) -6.00% 0.500 0.470 - -25.40% -
458.sjeng (avg) (percentage) -1.92% 0.520 0.510 - -19.05% -

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) 555.56% 9.000 59.000 - -25.32% -
401.bzip2 (max) (percentage) 550.00% 0.080 0.520 - -25.71% -
462.libquantum (max) 425.00% 8.000 42.000 - 121.05% -
462.libquantum (max) (percentage) 383.33% 0.060 0.290 - 123.08% -
458.sjeng (max) (percentage) 134.29% 0.350 0.820 - -21.90% -
471.omnetpp (max) (percentage) 126.09% 0.460 1.040 - -29.73% -
458.sjeng (max) 125.49% 51.000 115.000 - -25.32% -
429.mcf (max) 100.00% 10.000 20.000 - 233.33% -
429.mcf (max) (percentage) 90.91% 0.110 0.210 - 200.00% -
482.sphinx3 (max) (percentage) 70.37% 0.270 0.460 - -9.80% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
483.xalancbmk (max) (percentage) 39.81% 1.080 1.510 - -10.65% -
401.bzip2 (avg) (percentage) 37.50% 0.080 0.110 - 0.00% -
401.bzip2 33.33% 9.000 12.000 - 0.00% -
401.bzip2 (avg) 33.33% 9.000 12.000 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
462.libquantum 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) (percentage) 15.38% 0.130 0.150 - 0.00% -
429.mcf 14.29% 7.000 8.000 - 14.29% -
429.mcf (avg) 14.29% 7.000 8.000 - 14.29% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
447.dealII (max) (percentage) 10.61% 0.660 0.730 - -7.59% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - -77.78% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -4.22% 35357.000 33864.000 - -4.22% -
401.bzip2 -1.33% 73592.000 72611.000 - -1.28% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -3.72% 14662.000 14117.000 - -4.22% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -78.21% 0.780 0.170 - -5.56% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - -25.00% -
456.hmmer (max) (percentage) -65.45% 3.820 1.320 - -19.02% -
445.gobmk (max) (percentage) -29.63% 0.270 0.190 - -17.39% -
453.povray (max) (percentage) -25.40% 0.630 0.470 - -6.00% -
473.astar (max) (percentage) -23.53% 0.170 0.130 - -13.33% -
444.namd (max) (percentage) -23.08% 0.260 0.200 - -13.04% -
458.sjeng -22.83% 92.000 71.000 - -6.58% -
458.sjeng (avg) -22.83% 92.000 71.000 - -6.58% -
464.h264ref (max) (percentage) -21.11% 0.900 0.710 - -7.79% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
458.sjeng (avg) (percentage) -19.05% 0.630 0.510 - -1.92% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
450.soplex (max) (percentage) -8.33% 0.240 0.220 - -18.52% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Report Time: 0.23s
Execution (seconds)
Name Current %
401.bzip2 11372.000 -
429.mcf 9646.000 -
458.sjeng 14117.000 -4.22%
462.libquantum 14697.000 -
Geometric Mean 12282.557 -1.07%
Code Size (bytes)
Name Current %
401.bzip2 72611.000 -1.28%
429.mcf 10910.000 -
458.sjeng 148836.000 -
462.libquantum 33864.000 -4.22%
Geometric Mean 44701.115 -1.75%
Score
Name Current %
nb_compile_successful 19.000 -
nb_execution_successful 4.000 -77.78%
nb_total_tests 19.000 -
Geometric Mean 11.303 -39.43%
sample variability
Name Current %
401.bzip2 12.000 -
401.bzip2 (avg) 12.000 -
401.bzip2 (avg) (percentage) 0.110 -
401.bzip2 (max) 59.000 -25.32%
401.bzip2 (max) (percentage) 0.520 -25.71%
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) (percentage) 0.170 -
429.mcf 8.000 14.29%
429.mcf (avg) 8.000 14.29%
429.mcf (avg) (percentage) 0.090 -
429.mcf (max) 20.000 233.33%
429.mcf (max) (percentage) 0.210 200.00%
433.milc (avg) (percentage) 0.060 -
433.milc (max) (percentage) 0.030 -25.00%
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.200 -13.04%
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.190 -17.39%
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.730 -7.59%
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) (percentage) 0.220 -18.52%
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.470 -6.00%
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.320 -19.02%
458.sjeng 71.000 -6.58%
458.sjeng (avg) 71.000 -6.58%
458.sjeng (avg) (percentage) 0.510 -1.92%
458.sjeng (max) 115.000 -25.32%
458.sjeng (max) (percentage) 0.820 -21.90%
462.libquantum 22.000 -
462.libquantum (avg) 22.000 -
462.libquantum (avg) (percentage) 0.150 -
462.libquantum (max) 42.000 121.05%
462.libquantum (max) (percentage) 0.290 123.08%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.710 -7.79%
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 1.040 -29.73%
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.130 -13.33%
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) (percentage) 0.460 -9.80%
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) (percentage) 1.510 -10.65%
Geometric Mean 0.949 2.05%