Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/f7669ba3d9443bc95dd63fa25beea13e6265fdc5 (+ 1680 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/157/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0157 (llvmorg-18-init-18311-gf7669ba3d944) 2024-01-16T07:57:32 0:00:00
Previous 0155 (no-change) 2024-01-08T20:16:49 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 8 27
Performance Improvements 14 35
Added Tests 13 16
Unchanged Tests 697 654
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
458.sjeng 1.64% 14363.000 14598.000 - -0.44% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (max) 14.90% 208.000 239.000 - 368.63% -
458.sjeng (max) (percentage) 13.10% 1.450 1.640 - 368.57% -
458.sjeng (unexplained diff vs previous) 10.85% 212.000 235.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) 8.78% 1.480 1.610 - 0.00% -
458.sjeng 5.48% 73.000 77.000 - -16.30% -
458.sjeng (avg) 5.48% 73.000 77.000 - -16.30% -
458.sjeng (avg) (percentage) 3.92% 0.510 0.530 - -15.87% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -18.75% 16.000 13.000 - -99.85% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) (percentage) -77.55% 0.490 0.110 - 120.00% -
429.mcf (unexplained diff vs previous) -76.60% 47.000 11.000 - 120.00% -
462.libquantum (unexplained diff vs previous) -66.67% 18.000 6.000 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) -66.67% 0.120 0.040 - 0.00% -
462.libquantum (max) -50.00% 26.000 13.000 - 62.50% -
462.libquantum (max) (percentage) -50.00% 0.180 0.090 - 50.00% -
429.mcf (max) -13.04% 46.000 40.000 - 300.00% -
429.mcf (max) (percentage) -12.50% 0.480 0.420 - 281.82% -
401.bzip2 (max) (percentage) -5.17% 0.580 0.550 - 587.50% -
401.bzip2 (max) -4.62% 65.000 62.000 - 588.89% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful 137.50% 8.000 19.000 - 0.00% -
nb_total_tests 137.50% 8.000 19.000 - 0.00% -
nb_execution_successful 33.33% 3.000 4.000 - -77.78% -

Added Tests - Code Size
401.bzip2

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 1.57% 73592.000 74751.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (avg) (percentage) 637.50% 0.080 0.590 - 0.00% -
401.bzip2 633.33% 9.000 66.000 - 0.00% -
401.bzip2 (avg) 633.33% 9.000 66.000 - 0.00% -
401.bzip2 (max) 588.89% 9.000 62.000 - -4.62% -
401.bzip2 (max) (percentage) 587.50% 0.080 0.550 - -5.17% -
458.sjeng (max) 368.63% 51.000 239.000 - 14.90% -
458.sjeng (max) (percentage) 368.57% 0.350 1.640 - 13.10% -
429.mcf (max) 300.00% 10.000 40.000 - -13.04% -
429.mcf (max) (percentage) 281.82% 0.110 0.420 - -12.50% -
429.mcf 128.57% 7.000 16.000 - 0.00% -
429.mcf (avg) 128.57% 7.000 16.000 - 0.00% -
429.mcf (unexplained diff vs previous) 120.00% 5.000 11.000 - -76.60% -
429.mcf (unexplained diff vs previous) (percentage) 120.00% 0.050 0.110 - -77.55% -
429.mcf (avg) (percentage) 112.50% 0.080 0.170 - 0.00% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
462.libquantum (max) 62.50% 8.000 13.000 - -50.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (max) (percentage) 50.00% 0.060 0.090 - -50.00% -
462.libquantum (avg) (percentage) 46.15% 0.130 0.190 - 0.00% -
462.libquantum 42.11% 19.000 27.000 - 0.00% -
462.libquantum (avg) 42.11% 19.000 27.000 - 0.00% -
482.sphinx3 (max) (percentage) 33.33% 0.270 0.360 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 33.33% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.69% 35357.000 32992.000 - -0.10% -
429.mcf -1.93% 11019.000 10806.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm -99.99% 15896.000 1.000 - 0.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.85% 8399.000 13.000 - -18.75% -
403.gcc -97.90% 6613.000 139.000 - 0.72% -
483.xalancbmk -55.40% 5921.000 2641.000 - 0.11% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -72.51% 3.820 1.050 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
483.xalancbmk (max) -60.32% 63.000 25.000 - 0.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
458.sjeng -16.30% 92.000 77.000 - 5.48% -
458.sjeng (avg) -16.30% 92.000 77.000 - 5.48% -
458.sjeng (avg) (percentage) -15.87% 0.630 0.530 - 3.92% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
447.dealII (max) (percentage) -13.64% 0.660 0.570 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
483.xalancbmk (max) (percentage) -11.11% 1.080 0.960 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11299.000 -
403.gcc 139.000 -
429.mcf 9690.000 -
433.milc 5.000 -
450.soplex 13.000 -18.75%
458.sjeng 14598.000 1.64%
462.libquantum 14707.000 -
470.lbm 1.000 -
483.xalancbmk 2641.000 -
Geometric Mean 435.282 -2.01%
Code Size (bytes)
Name Current %
401.bzip2 74751.000 -
429.mcf 10806.000 -
458.sjeng 148764.000 -
462.libquantum 32992.000 -
Geometric Mean 44621.803 18.73%
Score
Name Current %
nb_compile_successful 19.000 137.50%
nb_execution_successful 4.000 33.33%
nb_total_tests 19.000 137.50%
Geometric Mean 11.303 95.92%
sample variability
Name Current %
401.bzip2 66.000 -
401.bzip2 (avg) 66.000 -
401.bzip2 (avg) (percentage) 0.590 -
401.bzip2 (max) 62.000 -4.62%
401.bzip2 (max) (percentage) 0.550 -5.17%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 16.000 -
429.mcf (avg) 16.000 -
429.mcf (avg) (percentage) 0.170 -
429.mcf (max) 40.000 -13.04%
429.mcf (max) (percentage) 0.420 -12.50%
429.mcf (unexplained diff vs previous) 11.000 -76.60%
429.mcf (unexplained diff vs previous) (percentage) 0.110 -77.55%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.130 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.570 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.050 -
458.sjeng 77.000 5.48%
458.sjeng (avg) 77.000 5.48%
458.sjeng (avg) (percentage) 0.530 3.92%
458.sjeng (max) 239.000 14.90%
458.sjeng (max) (percentage) 1.640 13.10%
458.sjeng (unexplained diff vs previous) 235.000 10.85%
458.sjeng (unexplained diff vs previous) (percentage) 1.610 8.78%
462.libquantum 27.000 -
462.libquantum (avg) 27.000 -
462.libquantum (avg) (percentage) 0.190 -
462.libquantum (max) 13.000 -50.00%
462.libquantum (max) (percentage) 0.090 -50.00%
462.libquantum (unexplained diff vs previous) 6.000 -66.67%
462.libquantum (unexplained diff vs previous) (percentage) 0.040 -66.67%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.580 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.840 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) (percentage) 0.360 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 25.000 -
483.xalancbmk (max) (percentage) 0.960 -
Geometric Mean 0.256 -82.85%