Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/e8b6fa5f301de4688b7a4bd6c41d30f29f0e2ddd (+ 1 commits)
regression LLVM-1079
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/148/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0148 (llvmorg-18-init-15732-ge8b6fa5f301d) 2023-12-26T01:45:45 0:00:00
Previous 0146 (no-change) 2023-12-25T05:59:17 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 9 29
Performance Improvements 15 37
Added Tests 16 17
Unchanged Tests 680 637
Total Tests 720

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 50.00% 2.000 3.000 - -99.98% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (unexplained diff vs previous) (percentage) 1600.00% 0.050 0.850 - 4150.00% -
401.bzip2 (unexplained diff vs previous) 1516.67% 6.000 97.000 - 4750.00% -
429.mcf (unexplained diff vs previous) 40.74% 27.000 38.000 - 660.00% -
429.mcf (max) (percentage) 39.29% 0.280 0.390 - 254.55% -
429.mcf (unexplained diff vs previous) (percentage) 39.29% 0.280 0.390 - 680.00% -
429.mcf (max) 37.04% 27.000 37.000 - 270.00% -
401.bzip2 1.49% 67.000 68.000 - 655.56% -
401.bzip2 (avg) 1.49% 67.000 68.000 - 655.56% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -6.67% 15.000 14.000 - -99.83% -
403.gcc -4.23% 142.000 136.000 - -97.94% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) -91.43% 0.350 0.030 - 0.00% -
462.libquantum (unexplained diff vs previous) -90.20% 51.000 5.000 - 0.00% -
462.libquantum (max) -11.76% 51.000 45.000 - 462.50% -
462.libquantum (max) (percentage) -11.43% 0.350 0.310 - 416.67% -
401.bzip2 (max) (percentage) -6.58% 1.520 1.420 - 1675.00% -
458.sjeng (max) -5.71% 105.000 99.000 - 94.12% -
458.sjeng (max) (percentage) -5.48% 0.730 0.690 - 97.14% -
401.bzip2 (max) -5.26% 171.000 162.000 - 1700.00% -
458.sjeng -1.39% 72.000 71.000 - -22.83% -
458.sjeng (avg) -1.39% 72.000 71.000 - -22.83% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful 72.73% 11.000 19.000 - 0.00% -
nb_total_tests 72.73% 11.000 19.000 - 0.00% -
nb_execution_successful 33.33% 3.000 4.000 - -77.78% -

Added Tests - Code Size
458.sjeng

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 1.55% 73592.000 74735.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (unexplained diff vs previous) 4750.00% 2.000 97.000 - 1516.67% -
401.bzip2 (unexplained diff vs previous) (percentage) 4150.00% 0.020 0.850 - 1600.00% -
401.bzip2 (max) 1700.00% 9.000 162.000 - -5.26% -
401.bzip2 (max) (percentage) 1675.00% 0.080 1.420 - -6.58% -
429.mcf (unexplained diff vs previous) (percentage) 680.00% 0.050 0.390 - 39.29% -
429.mcf (unexplained diff vs previous) 660.00% 5.000 38.000 - 40.74% -
401.bzip2 655.56% 9.000 68.000 - 1.49% -
401.bzip2 (avg) 655.56% 9.000 68.000 - 1.49% -
401.bzip2 (avg) (percentage) 650.00% 0.080 0.600 - 0.00% -
462.libquantum (max) 462.50% 8.000 45.000 - -11.76% -
462.libquantum (max) (percentage) 416.67% 0.060 0.310 - -11.43% -
429.mcf (max) 270.00% 10.000 37.000 - 37.04% -
429.mcf (max) (percentage) 254.55% 0.110 0.390 - 39.29% -
429.mcf 114.29% 7.000 15.000 - 0.00% -
429.mcf (avg) 114.29% 7.000 15.000 - 0.00% -
429.mcf (avg) (percentage) 100.00% 0.080 0.160 - 0.00% -
458.sjeng (max) (percentage) 97.14% 0.350 0.690 - -5.48% -
458.sjeng (max) 94.12% 51.000 99.000 - -5.71% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
462.libquantum (avg) (percentage) 53.85% 0.130 0.200 - 0.00% -
462.libquantum 52.63% 19.000 29.000 - 0.00% -
462.libquantum (avg) 52.63% 19.000 29.000 - 0.00% -
482.sphinx3 (max) (percentage) 33.33% 0.270 0.360 - 0.00% -
470.lbm (max) (percentage) 22.22% 0.180 0.220 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -
450.soplex (avg) (percentage) 10.53% 0.190 0.210 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -77.78% 18.000 4.000 - 33.33% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.60% 35357.000 33024.000 - 0.00% -
429.mcf -1.64% 11019.000 10838.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm -99.99% 15896.000 1.000 - 0.00% -
482.sphinx3 -99.98% 18232.000 3.000 - 50.00% -
433.milc -99.96% 11954.000 5.000 - 0.00% -
450.soplex -99.83% 8399.000 14.000 - -6.67% -
403.gcc -97.94% 6613.000 136.000 - -4.23% -
483.xalancbmk -55.56% 5921.000 2631.000 - -0.15% -
458.sjeng -1.87% 14662.000 14388.000 - -0.17% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) -82.05% 0.780 0.140 - 0.00% -
456.hmmer (max) (percentage) -72.25% 3.820 1.060 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
483.xalancbmk (max) -60.32% 63.000 25.000 - 0.00% -
483.xalancbmk -59.26% 54.000 22.000 - 0.00% -
483.xalancbmk (avg) -59.26% 54.000 22.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
450.soplex (max) (percentage) -33.33% 0.240 0.160 - 0.00% -
458.sjeng -22.83% 92.000 71.000 - -1.39% -
458.sjeng (avg) -22.83% 92.000 71.000 - -1.39% -
458.sjeng (avg) (percentage) -20.63% 0.630 0.500 - 0.00% -
470.lbm (avg) (percentage) -20.00% 0.300 0.240 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
433.milc (avg) (percentage) -14.29% 0.070 0.060 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
447.dealII (max) (percentage) -12.12% 0.660 0.580 - 0.00% -
483.xalancbmk (max) (percentage) -10.19% 1.080 0.970 - 0.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
403.gcc (avg) (percentage) -8.33% 0.360 0.330 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
403.gcc
403.gcc (avg)
403.gcc (max)
433.milc
433.milc (avg)
433.milc (max)
450.soplex
450.soplex (avg)
450.soplex (max)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm
470.lbm (avg)
470.lbm (max)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (max)

Report Time: 0.23s
Execution (seconds)
Name Current %
401.bzip2 11410.000 -
403.gcc 136.000 -4.23%
429.mcf 9667.000 -
433.milc 5.000 -
450.soplex 14.000 -6.67%
458.sjeng 14388.000 -
462.libquantum 14702.000 -
470.lbm 1.000 -
482.sphinx3 3.000 50.00%
483.xalancbmk 2631.000 -
Geometric Mean 265.699 2.99%
Code Size (bytes)
Name Current %
401.bzip2 74735.000 -
429.mcf 10838.000 -
458.sjeng 148668.000 -
462.libquantum 33024.000 -
Geometric Mean 44656.028 49.32%
Score
Name Current %
nb_compile_successful 19.000 72.73%
nb_execution_successful 4.000 33.33%
nb_total_tests 19.000 72.73%
Geometric Mean 11.303 58.45%
sample variability
Name Current %
401.bzip2 68.000 1.49%
401.bzip2 (avg) 68.000 1.49%
401.bzip2 (avg) (percentage) 0.600 -
401.bzip2 (max) 162.000 -5.26%
401.bzip2 (max) (percentage) 1.420 -6.58%
401.bzip2 (unexplained diff vs previous) 97.000 1516.67%
401.bzip2 (unexplained diff vs previous) (percentage) 0.850 1600.00%
403.gcc 0.000 -
403.gcc (avg) 0.000 -
403.gcc (avg) (percentage) 0.330 -
403.gcc (max) 0.000 -
403.gcc (max) (percentage) 0.140 -
429.mcf 15.000 -
429.mcf (avg) 15.000 -
429.mcf (avg) (percentage) 0.160 -
429.mcf (max) 37.000 37.04%
429.mcf (max) (percentage) 0.390 39.29%
429.mcf (unexplained diff vs previous) 38.000 40.74%
429.mcf (unexplained diff vs previous) (percentage) 0.390 39.29%
433.milc 0.000 -
433.milc (avg) 0.000 -
433.milc (avg) (percentage) 0.060 -
433.milc (max) 0.000 -
433.milc (max) (percentage) 0.030 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) (percentage) 0.140 -
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) (percentage) 0.130 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) (percentage) 0.580 -
450.soplex 0.000 -
450.soplex (avg) 0.000 -
450.soplex (avg) (percentage) 0.210 -
450.soplex (max) 0.000 -
450.soplex (max) (percentage) 0.160 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) (percentage) 0.390 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) (percentage) 1.060 -
458.sjeng 71.000 -1.39%
458.sjeng (avg) 71.000 -1.39%
458.sjeng (avg) (percentage) 0.500 -
458.sjeng (max) 99.000 -5.71%
458.sjeng (max) (percentage) 0.690 -5.48%
462.libquantum 29.000 -
462.libquantum (avg) 29.000 -
462.libquantum (avg) (percentage) 0.200 -
462.libquantum (max) 45.000 -11.76%
462.libquantum (max) (percentage) 0.310 -11.43%
462.libquantum (unexplained diff vs previous) 5.000 -90.20%
462.libquantum (unexplained diff vs previous) (percentage) 0.030 -91.43%
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) (percentage) 0.580 -
470.lbm 0.000 -
470.lbm (avg) 0.000 -
470.lbm (avg) (percentage) 0.240 -
470.lbm (max) 0.000 -
470.lbm (max) (percentage) 0.220 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) (percentage) 0.840 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 0.000 -
482.sphinx3 (avg) 0.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 0.000 -
482.sphinx3 (max) (percentage) 0.360 -
483.xalancbmk 22.000 -
483.xalancbmk (avg) 22.000 -
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 25.000 -
483.xalancbmk (max) (percentage) 0.970 -
Geometric Mean 0.192 -85.93%