Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/0ec4b82cfddbe0c7c7eb4b3486f55139f56d5f16 (+ 861 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/101/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0101 (llvmorg-18-init-13317-g0ec4b82cfddb) 2023-11-26T08:31:49 0:00:00
Previous 0099 (no-change) 2023-11-21T19:15:00 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 8 23
Performance Improvements 15 38
Added Tests 15 17
Unchanged Tests 694 654
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex 21.43% 14.000 17.000 - -99.80% -
403.gcc 2.17% 138.000 141.000 - -97.87% -
458.sjeng 1.19% 14366.000 14537.000 - -0.85% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) 57.89% 19.000 30.000 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) 53.85% 0.130 0.200 - 0.00% -
483.xalancbmk (max) 3.70% 27.000 28.000 - -55.56% -
458.sjeng 1.49% 67.000 68.000 - -26.09% -
458.sjeng (avg) 1.49% 67.000 68.000 - -26.09% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 -66.67% 3.000 1.000 - -99.99% -
401.bzip2 -1.73% 11498.000 11299.000 - -0.46% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) -80.00% 20.000 4.000 - -60.00% -
429.mcf (max) (percentage) -76.19% 0.210 0.050 - -54.55% -
429.mcf (unexplained diff vs previous) (percentage) -28.57% 0.070 0.050 - 0.00% -
429.mcf (unexplained diff vs previous) -28.57% 7.000 5.000 - 0.00% -
458.sjeng (max) (percentage) -9.26% 0.540 0.490 - 40.00% -
458.sjeng (max) -7.79% 77.000 71.000 - 39.22% -
450.soplex (max) (percentage) -5.88% 0.170 0.160 - -33.33% -
401.bzip2 (max) -4.57% 175.000 167.000 - 1755.56% -
462.libquantum (max) -3.33% 30.000 29.000 - 262.50% -
401.bzip2 (max) (percentage) -3.27% 1.530 1.480 - 1750.00% -
447.dealII (max) (percentage) -1.67% 0.600 0.590 - -10.61% -
401.bzip2 -1.39% 72.000 71.000 - 688.89% -
401.bzip2 (avg) -1.39% 72.000 71.000 - 688.89% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 1.92% 73592.000 75007.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) 1755.56% 9.000 167.000 - -4.57% -
401.bzip2 (max) (percentage) 1750.00% 0.080 1.480 - -3.27% -
401.bzip2 688.89% 9.000 71.000 - -1.39% -
401.bzip2 (avg) 688.89% 9.000 71.000 - -1.39% -
401.bzip2 (avg) (percentage) 687.50% 0.080 0.630 - 0.00% -
462.libquantum (max) 262.50% 8.000 29.000 - -3.33% -
462.libquantum (max) (percentage) 233.33% 0.060 0.200 - -4.76% -
471.omnetpp (max) (percentage) 84.78% 0.460 0.850 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (avg) (percentage) 53.85% 0.130 0.200 - 0.00% -
462.libquantum 52.63% 19.000 29.000 - 0.00% -
462.libquantum (avg) 52.63% 19.000 29.000 - 0.00% -
458.sjeng (max) (percentage) 40.00% 0.350 0.490 - -9.26% -
458.sjeng (max) 39.22% 51.000 71.000 - -7.79% -
482.sphinx3 (max) (percentage) 37.04% 0.270 0.370 - 0.00% -
429.mcf 28.57% 7.000 9.000 - 0.00% -
429.mcf (avg) 28.57% 7.000 9.000 - 0.00% -
429.mcf (avg) (percentage) 25.00% 0.080 0.100 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.60% 35357.000 33024.000 - 0.00% -
429.mcf -1.64% 11019.000 10838.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
482.sphinx3 -99.99% 18232.000 1.000 - -66.67% -
470.lbm -99.99% 15896.000 1.000 - 0.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.80% 8399.000 17.000 - 21.43% -
403.gcc -97.87% 6613.000 141.000 - 2.17% -
483.xalancbmk -54.91% 5921.000 2670.000 - 0.23% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -71.73% 3.820 1.080 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
429.mcf (max) -60.00% 10.000 4.000 - -80.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (max) -55.56% 63.000 28.000 - 3.70% -
429.mcf (max) (percentage) -54.55% 0.110 0.050 - -76.19% -
445.gobmk (max) (percentage) -48.15% 0.270 0.140 - 0.00% -
473.astar (max) (percentage) -47.06% 0.170 0.090 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -36.51% 0.630 0.400 - 0.00% -
464.h264ref (max) (percentage) -33.33% 0.900 0.600 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - -5.88% -
458.sjeng -26.09% 92.000 68.000 - 1.49% -
458.sjeng (avg) -26.09% 92.000 68.000 - 1.49% -
458.sjeng (avg) (percentage) -25.40% 0.630 0.470 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
447.dealII (max) (percentage) -10.61% 0.660 0.590 - -1.67% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -
483.xalancbmk (max) (percentage) -2.78% 1.080 1.050 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11299.000 -1.73%
403.gcc 141.000 2.17%
429.mcf 9679.000 -
433.milc 5.000 -
450.soplex 17.000 21.43%
458.sjeng 14537.000 1.19%
462.libquantum 14669.000 -
470.lbm 1.000 -
482.sphinx3 1.000 -66.67%
483.xalancbmk 2670.000 -
Geometric Mean 243.949 -8.50%
Code Size (bytes)
Name Current %
401.bzip2 75007.000 -
429.mcf 10838.000 -
458.sjeng 148668.000 -
462.libquantum 33024.000 -
Geometric Mean 44696.604 -
sample variability
Name Current %
401.bzip2 71.000 -1.39%
401.bzip2 (avg) 71.000 -1.39%
401.bzip2 (avg) (percentage) 0.630 -
401.bzip2 (max) 167.000 -4.57%
401.bzip2 (max) (percentage) 1.480 -3.27%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 9.000 -
429.mcf (avg) 9.000 -
429.mcf (avg) (percentage) 0.100 -
429.mcf (max) 4.000 -80.00%
429.mcf (max) (percentage) 0.050 -76.19%
429.mcf (unexplained diff vs previous) 5.000 -28.57%
429.mcf (unexplained diff vs previous) (percentage) 0.050 -28.57%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.140 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.590 -1.67%
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -5.88%
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.400 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.080 -
458.sjeng 68.000 1.49%
458.sjeng (avg) 68.000 1.49%
458.sjeng (avg) (percentage) 0.470 -
458.sjeng (max) 71.000 -7.79%
458.sjeng (max) (percentage) 0.490 -9.26%
462.libquantum 29.000 -
462.libquantum (avg) 29.000 -
462.libquantum (avg) (percentage) 0.200 -
462.libquantum (max) 29.000 -3.33%
462.libquantum (max) (percentage) 0.200 -
462.libquantum (unexplained diff vs previous) 30.000 57.89%
462.libquantum (unexplained diff vs previous) (percentage) 0.200 53.85%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.600 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.850 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.090 -
482.sphinx3 0.000 -
482.sphinx3 (avg) 0.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 0.000 -
482.sphinx3 (max) (percentage) 0.370 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 28.000 3.70%
483.xalancbmk (max) (percentage) 1.050 -
Geometric Mean 0.156 -87.30%