Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/2cd19df792056bbac38ed64c028e335d0c7ef05d (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/182/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0182 (no-change) 2024-03-13T10:45:01 0:00:00
Previous 0180 (no-change) 2024-03-12T06:50:35 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 15 27
Performance Improvements 10 35
Added Tests 12 16
Unchanged Tests 695 654
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) 933.33% 0.030 0.310 - 0.00% -
462.libquantum (unexplained diff vs previous) 820.00% 5.000 46.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) 94.55% 0.550 1.070 - 0.00% -
458.sjeng (unexplained diff vs previous) 92.41% 79.000 152.000 - 0.00% -
458.sjeng (max) (percentage) 49.30% 0.710 1.060 - 202.86% -
458.sjeng (max) 47.52% 101.000 149.000 - 192.16% -
462.libquantum 7.41% 27.000 29.000 - 52.63% -
462.libquantum (avg) 7.41% 27.000 29.000 - 52.63% -
462.libquantum (avg) (percentage) 5.26% 0.190 0.200 - 53.85% -
458.sjeng (avg) (percentage) 1.75% 0.570 0.580 - -7.94% -
458.sjeng 1.23% 81.000 82.000 - -10.87% -
458.sjeng (avg) 1.23% 81.000 82.000 - -10.87% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -50.00% 4.000 2.000 - -88.89% -
nb_compile_successful -42.11% 19.000 11.000 - -42.11% -
nb_total_tests -42.11% 19.000 11.000 - -42.11% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
401.bzip2 -2.02% 11506.000 11274.000 - -0.68% -
458.sjeng -1.06% 14296.000 14144.000 - -3.53% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (max) -10.37% 164.000 147.000 - 1533.33% -
401.bzip2 (max) (percentage) -8.39% 1.430 1.310 - 1537.50% -
429.mcf (max) -6.25% 48.000 45.000 - 350.00% -
429.mcf (max) (percentage) -6.00% 0.500 0.470 - 327.27% -
462.libquantum (max) -5.88% 51.000 48.000 - 500.00% -
462.libquantum (max) (percentage) -5.71% 0.350 0.330 - 450.00% -
401.bzip2 -2.90% 69.000 67.000 - 644.44% -
401.bzip2 (avg) -2.90% 69.000 67.000 - 644.44% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 1.24% 9606.000 9725.000 - 0.40% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) (percentage) 1537.50% 0.080 1.310 - -8.39% -
401.bzip2 (max) 1533.33% 9.000 147.000 - -10.37% -
401.bzip2 (avg) (percentage) 650.00% 0.080 0.600 - 0.00% -
401.bzip2 644.44% 9.000 67.000 - -2.90% -
401.bzip2 (avg) 644.44% 9.000 67.000 - -2.90% -
462.libquantum (max) 500.00% 8.000 48.000 - -5.88% -
462.libquantum (max) (percentage) 450.00% 0.060 0.330 - -5.71% -
429.mcf (max) 350.00% 10.000 45.000 - -6.25% -
429.mcf (max) (percentage) 327.27% 0.110 0.470 - -6.00% -
458.sjeng (max) (percentage) 202.86% 0.350 1.060 - 49.30% -
458.sjeng (max) 192.16% 51.000 149.000 - 47.52% -
429.mcf 142.86% 7.000 17.000 - 0.00% -
429.mcf (avg) 142.86% 7.000 17.000 - 0.00% -
429.mcf (avg) (percentage) 125.00% 0.080 0.180 - 0.00% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (avg) (percentage) 53.85% 0.130 0.200 - 5.26% -
462.libquantum 52.63% 19.000 29.000 - 7.41% -
462.libquantum (avg) 52.63% 19.000 29.000 - 7.41% -
482.sphinx3 (max) (percentage) 29.63% 0.270 0.350 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -88.89% 18.000 2.000 - -50.00% -
nb_compile_successful -42.11% 19.000 11.000 - -42.11% -
nb_total_tests -42.11% 19.000 11.000 - -42.11% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -7.48% 35357.000 32712.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm -99.99% 15896.000 1.000 - 0.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.80% 8399.000 17.000 - 0.00% -
403.gcc -97.85% 6613.000 142.000 - 0.00% -
483.xalancbmk -55.53% 5921.000 2633.000 - 0.00% -
458.sjeng -3.53% 14662.000 14144.000 - -1.06% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -72.51% 3.820 1.050 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
483.xalancbmk (max) -60.32% 63.000 25.000 - 0.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
447.dealII (max) (percentage) -13.64% 0.660 0.570 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
483.xalancbmk (max) (percentage) -12.04% 1.080 0.950 - 0.00% -
458.sjeng -10.87% 92.000 82.000 - 1.23% -
458.sjeng (avg) -10.87% 92.000 82.000 - 1.23% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
458.sjeng (avg) (percentage) -7.94% 0.630 0.580 - 1.75% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)

Report Time: 0.23s
Execution (seconds)
Name Current %
401.bzip2 11274.000 -2.02%
403.gcc 142.000 -
429.mcf 9725.000 -
433.milc 5.000 -
450.soplex 17.000 -
458.sjeng 14144.000 -1.06%
462.libquantum 14722.000 -
470.lbm 1.000 -
483.xalancbmk 2633.000 -
Geometric Mean 447.910 -
Code Size (bytes)
Name Current %
458.sjeng 149196.000 -
462.libquantum 32712.000 -
Geometric Mean 69860.572 -
Score
Name Current %
nb_compile_successful 11.000 -42.11%
nb_execution_successful 2.000 -50.00%
nb_total_tests 11.000 -42.11%
Geometric Mean 6.232 -44.87%
sample variability
Name Current %
401.bzip2 67.000 -2.90%
401.bzip2 (avg) 67.000 -2.90%
401.bzip2 (avg) (percentage) 0.600 -
401.bzip2 (max) 147.000 -10.37%
401.bzip2 (max) (percentage) 1.310 -8.39%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 17.000 -
429.mcf (avg) 17.000 -
429.mcf (avg) (percentage) 0.180 -
429.mcf (max) 45.000 -6.25%
429.mcf (max) (percentage) 0.470 -6.00%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.130 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.570 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.050 -
458.sjeng 82.000 1.23%
458.sjeng (avg) 82.000 1.23%
458.sjeng (avg) (percentage) 0.580 1.75%
458.sjeng (max) 149.000 47.52%
458.sjeng (max) (percentage) 1.060 49.30%
458.sjeng (unexplained diff vs previous) 152.000 92.41%
458.sjeng (unexplained diff vs previous) (percentage) 1.070 94.55%
462.libquantum 29.000 7.41%
462.libquantum (avg) 29.000 7.41%
462.libquantum (avg) (percentage) 0.200 5.26%
462.libquantum (max) 48.000 -5.88%
462.libquantum (max) (percentage) 0.330 -5.71%
462.libquantum (unexplained diff vs previous) 46.000 820.00%
462.libquantum (unexplained diff vs previous) (percentage) 0.310 933.33%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.580 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.840 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) (percentage) 0.350 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 25.000 -
483.xalancbmk (max) (percentage) 0.950 -
Geometric Mean 0.274 -80.04%