Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/344b5346a0e899d59ff52f60421788eef4e2c864 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_vect-spec2k6--llvm-arm-master-O3-build/91/
View Options
Report
Filter

llvm-arm-master-O3 test results

Run Order Start Time Duration
Current 0091 (no-change) 2023-11-26T03:43:04 0:00:00
Previous 0089 (no-change) 2023-11-24T20:41:25 0:00:00
Baseline 0006 (llvmorg-17-init-05342-g7ba9c060d48f) 2023-03-18T09:13:31 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 34 3
Performance Improvements 65 16
Added Tests 2 92
Unchanged Tests 667 657
Total Tests 768

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
453.povray 1.99% 452.000 461.000 - -1.71% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
464.h264ref (unexplained diff vs previous) 1500.00% 1.000 16.000 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) 1400.00% 0.020 0.300 - 0.00% -
445.gobmk (unexplained diff vs previous) 450.00% 4.000 22.000 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) 412.50% 0.080 0.410 - 0.00% -
464.h264ref (max) 220.00% 5.000 16.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) 183.33% 0.120 0.340 - 0.00% -
464.h264ref (max) (percentage) 181.82% 0.110 0.310 - 0.00% -
456.hmmer (unexplained diff vs previous) 175.00% 4.000 11.000 - 0.00% -
445.gobmk (max) 100.00% 11.000 22.000 - 0.00% -
445.gobmk (max) (percentage) 90.91% 0.220 0.420 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) 75.68% 0.370 0.650 - 0.00% -
433.milc (unexplained diff vs previous) 75.00% 4.000 7.000 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 60.08% 2.480 3.970 - 0.00% -
450.soplex (unexplained diff vs previous) 53.85% 13.000 20.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) 38.46% 13.000 18.000 0.000 0.00% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 36.00% 0.500 0.680 0.000 0.00% 0.000
453.povray (unexplained diff vs previous) 28.57% 7.000 9.000 0.000 0.00% 0.000
453.povray (unexplained diff vs previous) (percentage) 25.81% 1.550 1.950 0.000 0.00% 0.000
464.h264ref 20.00% 5.000 6.000 - 0.00% -
464.h264ref (avg) 20.00% 5.000 6.000 - 0.00% -
464.h264ref (avg) (percentage) 20.00% 0.100 0.120 - 0.00% -
450.soplex 14.29% 7.000 8.000 - 0.00% -
450.soplex (avg) 14.29% 7.000 8.000 - 0.00% -
450.soplex (avg) (percentage) 13.10% 1.450 1.640 - 0.00% -
473.astar (unexplained diff vs previous) 11.11% 9.000 10.000 - 0.00% -
473.astar (unexplained diff vs previous) (percentage) 10.53% 0.190 0.210 - 0.00% -
450.soplex (max) (percentage) 10.40% 3.460 3.820 - 0.00% -
445.gobmk 7.69% 13.000 14.000 - 0.00% -
445.gobmk (avg) 7.69% 13.000 14.000 - 0.00% -
450.soplex (max) 5.56% 18.000 19.000 - 0.00% -
456.hmmer (avg) (percentage) 4.00% 0.250 0.260 - 0.00% -
445.gobmk (avg) (percentage) 3.85% 0.260 0.270 - 0.00% -
453.povray (avg) (percentage) 1.09% 1.830 1.850 - 0.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -3.82% 524.000 504.000 - -12.80% -
403.gcc -1.69% 59.000 58.000 - -6.45% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (unexplained diff vs previous) (percentage) -83.82% 0.680 0.110 0.000 0.00% 0.000
482.sphinx3 (unexplained diff vs previous) -83.33% 6.000 1.000 0.000 0.00% 0.000
470.lbm (unexplained diff vs previous) (percentage) -82.05% 0.390 0.070 0.000 0.00% 0.000
470.lbm (unexplained diff vs previous) -81.82% 11.000 2.000 0.000 0.00% 0.000
483.xalancbmk (unexplained diff vs previous) (percentage) -68.00% 0.250 0.080 - 0.00% -
483.xalancbmk (unexplained diff vs previous) -66.67% 9.000 3.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) -57.26% 1.170 0.500 0.000 0.00% 0.000
429.mcf (unexplained diff vs previous) -57.14% 14.000 6.000 0.000 0.00% 0.000
462.libquantum (unexplained diff vs previous) -50.00% 2.000 1.000 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) -49.49% 1.980 1.000 - 0.00% -
444.namd (max) (percentage) -27.91% 0.860 0.620 - 0.00% -
462.libquantum (max) (percentage) -24.74% 1.940 1.460 - 0.00% -
429.mcf (max) -21.43% 14.000 11.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) -20.83% 0.720 0.570 - 0.00% -
444.namd (max) -20.00% 5.000 4.000 - 0.00% -
444.namd (unexplained diff vs previous) -20.00% 5.000 4.000 - 0.00% -
470.lbm (max) -18.18% 11.000 9.000 - 0.00% -
470.lbm (max) (percentage) -17.95% 0.390 0.320 - 0.00% -
429.mcf (max) (percentage) -16.81% 1.190 0.990 - 0.00% -
433.milc -16.67% 6.000 5.000 - 0.00% -
433.milc (avg) -16.67% 6.000 5.000 - 0.00% -
482.sphinx3 (max) -16.67% 6.000 5.000 - 0.00% -
447.dealII -14.29% 14.000 12.000 - 0.00% -
447.dealII (avg) -14.29% 14.000 12.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) -13.33% 0.300 0.260 - 0.00% -
429.mcf -12.50% 8.000 7.000 - 0.00% -
429.mcf (avg) -12.50% 8.000 7.000 - 0.00% -
458.sjeng (unexplained diff vs previous) -12.00% 25.000 22.000 - 0.00% -
433.milc (max) -11.11% 9.000 8.000 - 0.00% -
482.sphinx3 (max) (percentage) -10.29% 0.680 0.610 - 0.00% -
447.dealII (avg) (percentage) -8.65% 1.040 0.950 - 0.00% -
401.bzip2 -7.69% 13.000 12.000 - 0.00% -
401.bzip2 (avg) -7.69% 13.000 12.000 - 0.00% -
456.hmmer (max) -7.14% 14.000 13.000 - 0.00% -
471.omnetpp (max) (percentage) -6.67% 0.750 0.700 - 0.00% -
483.xalancbmk (max) (percentage) -6.67% 0.900 0.840 - 0.00% -
483.xalancbmk (max) -6.25% 32.000 30.000 - 0.00% -
401.bzip2 (max) -5.88% 17.000 16.000 - 0.00% -
483.xalancbmk (avg) (percentage) -5.62% 0.890 0.840 - 0.00% -
471.omnetpp (max) -5.26% 19.000 18.000 - 0.00% -
482.sphinx3 (avg) (percentage) -5.00% 0.400 0.380 - 0.00% -
473.astar (max) -5.00% 20.000 19.000 - 0.00% -
401.bzip2 (max) (percentage) -4.76% 0.630 0.600 - 0.00% -
473.astar (max) (percentage) -4.76% 0.420 0.400 - 0.00% -
456.hmmer (max) (percentage) -4.44% 0.450 0.430 - 0.00% -
458.sjeng (avg) (percentage) -4.17% 0.480 0.460 - 0.00% -
401.bzip2 (avg) (percentage) -4.08% 0.490 0.470 - 0.00% -
458.sjeng (max) -4.00% 25.000 24.000 - 0.00% -
447.dealII (max) -3.70% 27.000 26.000 - 0.00% -
471.omnetpp -3.45% 29.000 28.000 - 0.00% -
471.omnetpp (avg) -3.45% 29.000 28.000 - 0.00% -
458.sjeng (max) (percentage) -3.23% 0.310 0.300 - 0.00% -
483.xalancbmk -3.23% 31.000 30.000 - 0.00% -
483.xalancbmk (avg) -3.23% 31.000 30.000 - 0.00% -
471.omnetpp (avg) (percentage) -2.70% 1.110 1.080 - 0.00% -
470.lbm (avg) (percentage) -2.63% 0.380 0.370 - 0.00% -
458.sjeng -2.56% 39.000 38.000 - 0.00% -
458.sjeng (avg) -2.56% 39.000 38.000 - 0.00% -
447.dealII (max) (percentage) -2.48% 2.020 1.970 - 0.00% -
433.milc (max) (percentage) -2.41% 0.830 0.810 - 0.00% -
453.povray (max) (percentage) -1.99% 3.020 2.960 - 0.00% -
429.mcf (avg) (percentage) -1.49% 0.670 0.660 - 0.00% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 28.57% 21.000 27.000 - 42.11% -

Added Tests - sample variability
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 3.36% 76651.000 79227.000 - 0.00% -
433.milc 1.04% 128358.000 129698.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
447.dealII 1.04% 1352.000 1366.000 - -0.73% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -2.18% 34560.000 33808.000 - 0.00% -
444.namd -1.70% 224547.000 220723.000 - 0.00% -
464.h264ref -1.51% 767882.000 756258.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
450.soplex -12.80% 578.000 504.000 - -3.82% -
433.milc -8.98% 1191.000 1084.000 - -0.64% -
429.mcf -6.97% 1291.000 1201.000 - 0.50% -
403.gcc -6.45% 62.000 58.000 - -1.69% -
470.lbm -5.04% 2978.000 2828.000 - 0.07% -
462.libquantum -3.85% 104.000 100.000 - -0.99% -
482.sphinx3 -3.28% 914.000 884.000 - -0.11% -
483.xalancbmk -2.88% 3679.000 3573.000 - -0.08% -
401.bzip2 -1.97% 2794.000 2739.000 - 0.22% -
453.povray -1.71% 469.000 461.000 - 1.99% -
445.gobmk -1.26% 5384.000 5316.000 - 0.42% -
471.omnetpp -1.05% 2666.000 2638.000 - 0.69% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 42.11% 19.000 27.000 - 28.57% -

Added Tests - sample variability
401.bzip2
401.bzip2 (avg)
401.bzip2 (avg) (percentage)
401.bzip2 (max)
401.bzip2 (max) (percentage)
403.gcc
403.gcc (avg)
403.gcc (avg) (percentage)
403.gcc (max)
403.gcc (max) (percentage)
429.mcf
429.mcf (avg)
429.mcf (avg) (percentage)
429.mcf (max)
429.mcf (max) (percentage)
433.milc
433.milc (avg)
433.milc (avg) (percentage)
433.milc (max)
433.milc (max) (percentage)
444.namd
444.namd (avg)
444.namd (avg) (percentage)
444.namd (max)
444.namd (max) (percentage)
445.gobmk
445.gobmk (avg)
445.gobmk (avg) (percentage)
445.gobmk (max)
445.gobmk (max) (percentage)
447.dealII
447.dealII (avg)
447.dealII (avg) (percentage)
447.dealII (max)
447.dealII (max) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
450.soplex
450.soplex (avg)
450.soplex (avg) (percentage)
450.soplex (max)
450.soplex (max) (percentage)
453.povray
453.povray (avg)
453.povray (avg) (percentage)
453.povray (max)
453.povray (max) (percentage)
456.hmmer
456.hmmer (avg)
456.hmmer (avg) (percentage)
456.hmmer (max)
456.hmmer (max) (percentage)
458.sjeng
458.sjeng (avg)
458.sjeng (avg) (percentage)
458.sjeng (max)
458.sjeng (max) (percentage)
462.libquantum
462.libquantum (avg)
462.libquantum (avg) (percentage)
462.libquantum (max)
462.libquantum (max) (percentage)
464.h264ref
464.h264ref (avg)
464.h264ref (avg) (percentage)
464.h264ref (max)
464.h264ref (max) (percentage)
470.lbm
470.lbm (avg)
470.lbm (avg) (percentage)
470.lbm (max)
470.lbm (max) (percentage)
471.omnetpp
471.omnetpp (avg)
471.omnetpp (avg) (percentage)
471.omnetpp (max)
471.omnetpp (max) (percentage)
473.astar
473.astar (avg)
473.astar (avg) (percentage)
473.astar (max)
473.astar (max) (percentage)
482.sphinx3
482.sphinx3 (avg)
482.sphinx3 (avg) (percentage)
482.sphinx3 (max)
482.sphinx3 (max) (percentage)
483.xalancbmk
483.xalancbmk (avg)
483.xalancbmk (avg) (percentage)
483.xalancbmk (max)
483.xalancbmk (max) (percentage)

Report Time: 0.27s
Execution (seconds)
Name Current %
401.bzip2 2739.000 -
403.gcc 58.000 -1.69%
429.mcf 1201.000 -
433.milc 1084.000 -
444.namd 700.000 -
445.gobmk 5316.000 -
447.dealII 1366.000 -
450.soplex 504.000 -3.82%
453.povray 461.000 1.99%
456.hmmer 3238.000 -
458.sjeng 8314.000 -
462.libquantum 100.000 -
464.h264ref 5252.000 -
470.lbm 2828.000 -
471.omnetpp 2638.000 -
473.astar 4871.000 -
482.sphinx3 884.000 -
483.xalancbmk 3573.000 -
Geometric Mean 1394.249 -
Code Size (bytes)
Name Current %
401.bzip2 79227.000 -
403.gcc 3293376.000 -
429.mcf 11826.000 -
433.milc 129698.000 -
444.namd 220723.000 -
445.gobmk 1653483.000 -
447.dealII 2751248.000 -
450.soplex 320763.000 -
453.povray 923675.000 -
456.hmmer 268792.000 -
458.sjeng 145844.000 -
462.libquantum 33808.000 -
464.h264ref 756258.000 -
470.lbm 11392.000 -
471.omnetpp 540296.000 -
473.astar 36847.000 -
482.sphinx3 169439.000 -
483.xalancbmk 3451559.000 -
Geometric Mean 251170.107 -
Score
Name Current %
nb_compile_successful 19.000 -
nb_execution_successful 18.000 -
nb_total_tests 27.000 28.57%
Geometric Mean 20.980 8.74%
sample variability
Name Current %
401.bzip2 12.000 -7.69%
401.bzip2 (avg) 12.000 -7.69%
401.bzip2 (avg) (percentage) 0.470 -4.08%
401.bzip2 (max) 16.000 -5.88%
401.bzip2 (max) (percentage) 0.600 -4.76%
401.bzip2 (unexplained diff vs previous) 6.000 -
401.bzip2 (unexplained diff vs previous) (percentage) 0.220 -
403.gcc 1.000 -
403.gcc (avg) 1.000 -
403.gcc (avg) (percentage) 2.970 -
403.gcc (max) 2.000 -
403.gcc (max) (percentage) 3.470 -
429.mcf 7.000 -12.50%
429.mcf (avg) 7.000 -12.50%
429.mcf (avg) (percentage) 0.660 -1.49%
429.mcf (max) 11.000 -21.43%
429.mcf (max) (percentage) 0.990 -16.81%
429.mcf (unexplained diff vs previous) 6.000 -57.14%
429.mcf (unexplained diff vs previous) (percentage) 0.500 -57.26%
433.milc 5.000 -16.67%
433.milc (avg) 5.000 -16.67%
433.milc (avg) (percentage) 0.550 -
433.milc (max) 8.000 -11.11%
433.milc (max) (percentage) 0.810 -2.41%
433.milc (unexplained diff vs previous) 7.000 75.00%
433.milc (unexplained diff vs previous) (percentage) 0.650 75.68%
444.namd 3.000 -
444.namd (avg) 3.000 -
444.namd (avg) (percentage) 0.560 -
444.namd (max) 4.000 -20.00%
444.namd (max) (percentage) 0.620 -27.91%
444.namd (unexplained diff vs previous) 4.000 -20.00%
444.namd (unexplained diff vs previous) (percentage) 0.570 -20.83%
445.gobmk 14.000 7.69%
445.gobmk (avg) 14.000 7.69%
445.gobmk (avg) (percentage) 0.270 3.85%
445.gobmk (max) 22.000 100.00%
445.gobmk (max) (percentage) 0.420 90.91%
445.gobmk (unexplained diff vs previous) 22.000 450.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.410 412.50%
447.dealII 12.000 -14.29%
447.dealII (avg) 12.000 -14.29%
447.dealII (avg) (percentage) 0.950 -8.65%
447.dealII (max) 26.000 -3.70%
447.dealII (max) (percentage) 1.970 -2.48%
447.dealII (unexplained diff vs previous) 0.000 -
447.dealII (unexplained diff vs previous) (percentage) 0.000 -
450.soplex 8.000 14.29%
450.soplex (avg) 8.000 14.29%
450.soplex (avg) (percentage) 1.640 13.10%
450.soplex (max) 19.000 5.56%
450.soplex (max) (percentage) 3.820 10.40%
450.soplex (unexplained diff vs previous) 20.000 53.85%
450.soplex (unexplained diff vs previous) (percentage) 3.970 60.08%
453.povray 8.000 -
453.povray (avg) 8.000 -
453.povray (avg) (percentage) 1.850 1.09%
453.povray (max) 13.000 -
453.povray (max) (percentage) 2.960 -1.99%
453.povray (unexplained diff vs previous) 9.000 28.57%
453.povray (unexplained diff vs previous) (percentage) 1.950 25.81%
456.hmmer 8.000 -
456.hmmer (avg) 8.000 -
456.hmmer (avg) (percentage) 0.260 4.00%
456.hmmer (max) 13.000 -7.14%
456.hmmer (max) (percentage) 0.430 -4.44%
456.hmmer (unexplained diff vs previous) 11.000 175.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.340 183.33%
458.sjeng 38.000 -2.56%
458.sjeng (avg) 38.000 -2.56%
458.sjeng (avg) (percentage) 0.460 -4.17%
458.sjeng (max) 24.000 -4.00%
458.sjeng (max) (percentage) 0.300 -3.23%
458.sjeng (unexplained diff vs previous) 22.000 -12.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.260 -13.33%
462.libquantum 1.000 -
462.libquantum (avg) 1.000 -
462.libquantum (avg) (percentage) 1.010 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.460 -24.74%
462.libquantum (unexplained diff vs previous) 1.000 -50.00%
462.libquantum (unexplained diff vs previous) (percentage) 1.000 -49.49%
464.h264ref 6.000 20.00%
464.h264ref (avg) 6.000 20.00%
464.h264ref (avg) (percentage) 0.120 20.00%
464.h264ref (max) 16.000 220.00%
464.h264ref (max) (percentage) 0.310 181.82%
464.h264ref (unexplained diff vs previous) 16.000 1500.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.300 1400.00%
470.lbm 10.000 -
470.lbm (avg) 10.000 -
470.lbm (avg) (percentage) 0.370 -2.63%
470.lbm (max) 9.000 -18.18%
470.lbm (max) (percentage) 0.320 -17.95%
470.lbm (unexplained diff vs previous) 2.000 -81.82%
470.lbm (unexplained diff vs previous) (percentage) 0.070 -82.05%
471.omnetpp 28.000 -3.45%
471.omnetpp (avg) 28.000 -3.45%
471.omnetpp (avg) (percentage) 1.080 -2.70%
471.omnetpp (max) 18.000 -5.26%
471.omnetpp (max) (percentage) 0.700 -6.67%
471.omnetpp (unexplained diff vs previous) 18.000 38.46%
471.omnetpp (unexplained diff vs previous) (percentage) 0.680 36.00%
473.astar 11.000 -
473.astar (avg) 11.000 -
473.astar (avg) (percentage) 0.230 -
473.astar (max) 19.000 -5.00%
473.astar (max) (percentage) 0.400 -4.76%
473.astar (unexplained diff vs previous) 10.000 11.11%
473.astar (unexplained diff vs previous) (percentage) 0.210 10.53%
482.sphinx3 3.000 -
482.sphinx3 (avg) 3.000 -
482.sphinx3 (avg) (percentage) 0.380 -5.00%
482.sphinx3 (max) 5.000 -16.67%
482.sphinx3 (max) (percentage) 0.610 -10.29%
482.sphinx3 (unexplained diff vs previous) 1.000 -83.33%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.110 -83.82%
483.xalancbmk 30.000 -3.23%
483.xalancbmk (avg) 30.000 -3.23%
483.xalancbmk (avg) (percentage) 0.840 -5.62%
483.xalancbmk (max) 30.000 -6.25%
483.xalancbmk (max) (percentage) 0.840 -6.67%
483.xalancbmk (unexplained diff vs previous) 3.000 -66.67%
483.xalancbmk (unexplained diff vs previous) (percentage) 0.080 -68.00%
Geometric Mean 2.236 -