Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/d7c672834ec863b458af8ca493157e1e31aaf480 (+ 1002 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/183/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0183 (llvmorg-19-init-06285-gd7c672834ec8) 2024-03-18T10:53:17 0:00:00
Previous 0182 (no-change) 2024-03-13T10:45:01 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 5 25
Performance Improvements 15 37
Added Tests 18 17
Unchanged Tests 694 653
Total Tests 732

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
458.sjeng 1.19% 14144.000 14313.000 - -2.38% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) 3.23% 0.310 0.320 - 0.00% -
462.libquantum (unexplained diff vs previous) 2.17% 46.000 47.000 - 0.00% -
458.sjeng 1.22% 82.000 83.000 - -9.78% -
458.sjeng (avg) 1.22% 82.000 83.000 - -9.78% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -11.76% 17.000 15.000 - -99.82% -
403.gcc -1.41% 142.000 140.000 - -97.88% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (max) (percentage) -7.55% 1.060 0.980 - 180.00% -
401.bzip2 (max) -6.12% 147.000 138.000 - 1433.33% -
401.bzip2 (max) (percentage) -6.11% 1.310 1.230 - 1437.50% -
458.sjeng (max) -6.04% 149.000 140.000 - 174.51% -
429.mcf (max) -4.44% 45.000 43.000 - 330.00% -
429.mcf (max) (percentage) -4.26% 0.470 0.450 - 309.09% -
462.libquantum (max) -4.17% 48.000 46.000 - 475.00% -
483.xalancbmk (max) -4.00% 25.000 24.000 - -61.90% -
462.libquantum (max) (percentage) -3.03% 0.330 0.320 - 433.33% -
483.xalancbmk (max) (percentage) -1.05% 0.950 0.940 - -12.96% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful 100.00% 2.000 4.000 - -77.78% -
nb_compile_successful 72.73% 11.000 19.000 - 0.00% -
nb_total_tests 72.73% 11.000 19.000 - 0.00% -

Added Tests - Code Size
401.bzip2
429.mcf

Added Tests - Execution
482.sphinx3

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 3.31% 73592.000 76027.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (max) (percentage) 1437.50% 0.080 1.230 - -6.11% -
401.bzip2 (max) 1433.33% 9.000 138.000 - -6.12% -
401.bzip2 (avg) (percentage) 650.00% 0.080 0.600 - 0.00% -
401.bzip2 644.44% 9.000 67.000 - 0.00% -
401.bzip2 (avg) 644.44% 9.000 67.000 - 0.00% -
462.libquantum (max) 475.00% 8.000 46.000 - -4.17% -
462.libquantum (max) (percentage) 433.33% 0.060 0.320 - -3.03% -
429.mcf (max) 330.00% 10.000 43.000 - -4.44% -
429.mcf (max) (percentage) 309.09% 0.110 0.450 - -4.26% -
458.sjeng (max) (percentage) 180.00% 0.350 0.980 - -7.55% -
458.sjeng (max) 174.51% 51.000 140.000 - -6.04% -
429.mcf 142.86% 7.000 17.000 - 0.00% -
429.mcf (avg) 142.86% 7.000 17.000 - 0.00% -
429.mcf (avg) (percentage) 125.00% 0.080 0.180 - 0.00% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (avg) (percentage) 53.85% 0.130 0.200 - 0.00% -
462.libquantum 52.63% 19.000 29.000 - 0.00% -
462.libquantum (avg) 52.63% 19.000 29.000 - 0.00% -
482.sphinx3 (max) (percentage) 29.63% 0.270 0.350 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 100.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -7.48% 35357.000 32712.000 - 0.00% -
429.mcf -1.93% 11019.000 10806.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm -99.99% 15896.000 1.000 - 0.00% -
482.sphinx3 -99.99% 18232.000 2.000 - 0.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.82% 8399.000 15.000 - -11.76% -
403.gcc -97.88% 6613.000 140.000 - -1.41% -
483.xalancbmk -55.51% 5921.000 2634.000 - 0.04% -
458.sjeng -2.38% 14662.000 14313.000 - 1.19% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -72.51% 3.820 1.050 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
483.xalancbmk (max) -61.90% 63.000 24.000 - -4.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
447.dealII (max) (percentage) -13.64% 0.660 0.570 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
483.xalancbmk (max) (percentage) -12.96% 1.080 0.940 - -1.05% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
458.sjeng -9.78% 92.000 83.000 - 1.22% -
458.sjeng (avg) -9.78% 92.000 83.000 - 1.22% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
458.sjeng (avg) (percentage) -7.94% 0.630 0.580 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11270.000 -
403.gcc 140.000 -1.41%
429.mcf 9677.000 -
433.milc 5.000 -
450.soplex 15.000 -11.76%
458.sjeng 14313.000 1.19%
462.libquantum 14675.000 -
470.lbm 1.000 -
482.sphinx3 2.000 -
483.xalancbmk 2634.000 -
Geometric Mean 257.210 -42.58%
Code Size (bytes)
Name Current %
401.bzip2 76027.000 -
429.mcf 10806.000 -
458.sjeng 148944.000 -
462.libquantum 32712.000 -
Geometric Mean 44729.159 -35.97%
Score
Name Current %
nb_compile_successful 19.000 72.73%
nb_execution_successful 4.000 100.00%
nb_total_tests 19.000 72.73%
Geometric Mean 11.303 81.38%
sample variability
Name Current %
401.bzip2 67.000 -
401.bzip2 (avg) 67.000 -
401.bzip2 (avg) (percentage) 0.600 -
401.bzip2 (max) 138.000 -6.12%
401.bzip2 (max) (percentage) 1.230 -6.11%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 17.000 -
429.mcf (avg) 17.000 -
429.mcf (avg) (percentage) 0.180 -
429.mcf (max) 43.000 -4.44%
429.mcf (max) (percentage) 0.450 -4.26%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.130 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.570 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.050 -
458.sjeng 83.000 1.22%
458.sjeng (avg) 83.000 1.22%
458.sjeng (avg) (percentage) 0.580 -
458.sjeng (max) 140.000 -6.04%
458.sjeng (max) (percentage) 0.980 -7.55%
462.libquantum 29.000 -
462.libquantum (avg) 29.000 -
462.libquantum (avg) (percentage) 0.200 -
462.libquantum (max) 46.000 -4.17%
462.libquantum (max) (percentage) 0.320 -3.03%
462.libquantum (unexplained diff vs previous) 47.000 2.17%
462.libquantum (unexplained diff vs previous) (percentage) 0.320 3.23%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.580 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.840 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 0.000 -
482.sphinx3 (avg) 0.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 0.000 -
482.sphinx3 (max) (percentage) 0.350 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 24.000 -4.00%
483.xalancbmk (max) (percentage) 0.940 -1.05%
Geometric Mean 0.172 -87.95%