Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/e1bd759ea56710a9e7b9a4068af0ee95f1ad217e (+ 867 commits)
test_url https://ci.linaro.org/job/tcwg_bmk_ci_llvm-build-tcwg_bmk_tk1-llvm-master-arm-spec2k6-O2/383/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0383 (llvmorg-16-init-08433-ge1bd759ea567) 2022-10-15T21:50:25 0:00:00
Previous 0380 (llvmorg-16-init-07566-g3f62314c235b) 2022-10-09T07:25:00 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 21 62
Performance Improvements 55 41
Added Tests 0 2
Unchanged Tests 704 675
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
456.hmmer 2.17% 7633.000 7799.000 - 4.25% -
483.xalancbmk 1.59% 5669.000 5759.000 - -2.74% -
482.sphinx3 1.36% 18283.000 18532.000 - 1.65% -
447.dealII 1.01% 5158.000 5210.000 - 0.75% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (unexplained diff vs previous) 1560.00% 10.000 166.000 - -44.11% -
456.hmmer (unexplained diff vs previous) (percentage) 1538.46% 0.130 2.130 - -46.35% -
470.lbm (unexplained diff vs previous) 400.00% 4.000 20.000 - 33.33% -
470.lbm (unexplained diff vs previous) (percentage) 333.33% 0.030 0.130 - 44.44% -
456.hmmer (max) 231.37% 51.000 169.000 - -40.70% -
456.hmmer (max) (percentage) 223.88% 0.670 2.170 - -43.19% -
456.hmmer 44.83% 58.000 84.000 - -11.58% -
456.hmmer (avg) 44.83% 58.000 84.000 - -11.58% -
456.hmmer (avg) (percentage) 40.26% 0.770 1.080 - -14.96% -
429.mcf 11.11% 9.000 10.000 - 42.86% -
429.mcf (avg) 11.11% 9.000 10.000 - 42.86% -
401.bzip2 (max) 6.25% 16.000 17.000 - 88.89% -
483.xalancbmk 2.08% 48.000 49.000 - -9.26% -
483.xalancbmk (avg) 2.08% 48.000 49.000 - -9.26% -
471.omnetpp (avg) (percentage) 1.72% 0.580 0.590 - 34.09% -
482.sphinx3 1.54% 65.000 66.000 - -28.26% -
482.sphinx3 (avg) 1.54% 65.000 66.000 - -28.26% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
458.sjeng -1.66% 14829.000 14583.000 - -0.54% -
445.gobmk -1.64% 11050.000 10869.000 - -2.95% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (unexplained diff vs previous) (percentage) -81.25% 0.160 0.030 - 0.00% -
462.libquantum (unexplained diff vs previous) -79.17% 24.000 5.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -66.67% 0.120 0.040 - -55.56% -
433.milc (unexplained diff vs previous) -64.29% 14.000 5.000 - -54.55% -
450.soplex (unexplained diff vs previous) (percentage) -45.83% 0.240 0.130 - 30.00% -
450.soplex (unexplained diff vs previous) -45.00% 20.000 11.000 - 37.50% -
462.libquantum (max) (percentage) -18.75% 0.160 0.130 - 116.67% -
450.soplex (max) -18.18% 22.000 18.000 - -10.00% -
462.libquantum (max) -17.39% 23.000 19.000 - 137.50% -
433.milc (max) -15.79% 19.000 16.000 - 60.00% -
462.libquantum -15.79% 19.000 16.000 - -15.79% -
462.libquantum (avg) -15.79% 19.000 16.000 - -15.79% -
450.soplex (max) (percentage) -15.38% 0.260 0.220 - -8.33% -
462.libquantum (avg) (percentage) -15.38% 0.130 0.110 - -15.38% -
433.milc -15.38% 13.000 11.000 - 37.50% -
433.milc (avg) -15.38% 13.000 11.000 - 37.50% -
483.xalancbmk (max) (percentage) -12.69% 1.340 1.170 - 8.33% -
433.milc (max) (percentage) -12.50% 0.160 0.140 - 55.56% -
429.mcf (max) -11.11% 18.000 16.000 - 60.00% -
429.mcf (unexplained diff vs previous) -11.11% 18.000 16.000 - 220.00% -
483.xalancbmk (max) -10.67% 75.000 67.000 - 6.35% -
471.omnetpp (max) -10.64% 47.000 42.000 - 27.27% -
429.mcf (max) (percentage) -10.53% 0.190 0.170 - 54.55% -
429.mcf (unexplained diff vs previous) (percentage) -10.53% 0.190 0.170 - 240.00% -
471.omnetpp (max) (percentage) -9.23% 0.650 0.590 - 28.26% -
445.gobmk (max) -8.89% 45.000 41.000 - 36.67% -
470.lbm (max) (percentage) -8.70% 0.230 0.210 - 16.67% -
471.omnetpp (unexplained diff vs previous) -8.51% 47.000 43.000 - 760.00% -
470.lbm (max) -8.33% 36.000 33.000 - 17.86% -
471.omnetpp (unexplained diff vs previous) (percentage) -7.69% 0.650 0.600 - 757.14% -
458.sjeng (max) -7.39% 203.000 188.000 - 268.63% -
445.gobmk (max) (percentage) -7.32% 0.410 0.380 - 40.74% -
464.h264ref (max) -7.27% 165.000 153.000 - 75.86% -
470.lbm -6.67% 30.000 28.000 - -40.43% -
470.lbm (avg) -6.67% 30.000 28.000 - -40.43% -
473.astar (max) -6.67% 30.000 28.000 - 86.67% -
464.h264ref (max) (percentage) -6.51% 1.690 1.580 - 75.56% -
473.astar (max) (percentage) -6.06% 0.330 0.310 - 82.35% -
458.sjeng (max) (percentage) -5.84% 1.370 1.290 - 268.57% -
482.sphinx3 (max) (percentage) -5.45% 0.550 0.520 - 92.59% -
470.lbm (avg) (percentage) -5.26% 0.190 0.180 - -40.00% -
453.povray (max) (percentage) -4.92% 0.610 0.580 - -7.94% -
447.dealII (max) (percentage) -4.88% 1.230 1.170 - 77.27% -
447.dealII (max) -4.76% 63.000 60.000 - 76.47% -
444.namd (max) -4.55% 22.000 21.000 - -16.00% -
444.namd (max) (percentage) -4.35% 0.230 0.220 - -15.38% -
482.sphinx3 (max) -4.00% 100.000 96.000 - 95.92% -
453.povray (max) -3.70% 27.000 26.000 - -10.34% -
445.gobmk -2.94% 34.000 33.000 - 37.50% -
445.gobmk (avg) -2.94% 34.000 33.000 - 37.50% -
403.gcc (max) (percentage) -2.38% 0.420 0.410 - -47.44% -
458.sjeng -1.61% 124.000 122.000 - 32.61% -
458.sjeng (avg) -1.61% 124.000 122.000 - 32.61% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.79% 309981.000 315533.000 - 0.01% -
401.bzip2 1.58% 73592.000 74756.000 - -0.02% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
456.hmmer 4.25% 7481.000 7799.000 - 2.17% -
482.sphinx3 1.65% 18232.000 18532.000 - 1.36% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
471.omnetpp (unexplained diff vs previous) 760.00% 5.000 43.000 - -8.51% -
471.omnetpp (unexplained diff vs previous) (percentage) 757.14% 0.070 0.600 - -7.69% -
458.sjeng (max) 268.63% 51.000 188.000 - -7.39% -
458.sjeng (max) (percentage) 268.57% 0.350 1.290 - -5.84% -
429.mcf (unexplained diff vs previous) (percentage) 240.00% 0.050 0.170 - -10.53% -
429.mcf (unexplained diff vs previous) 220.00% 5.000 16.000 - -11.11% -
462.libquantum (max) 137.50% 8.000 19.000 - -17.39% -
462.libquantum (max) (percentage) 116.67% 0.060 0.130 - -18.75% -
482.sphinx3 (max) 95.92% 49.000 96.000 - -4.00% -
482.sphinx3 (max) (percentage) 92.59% 0.270 0.520 - -5.45% -
401.bzip2 (max) 88.89% 9.000 17.000 - 6.25% -
401.bzip2 (max) (percentage) 87.50% 0.080 0.150 - 0.00% -
473.astar (max) 86.67% 15.000 28.000 - -6.67% -
473.astar (max) (percentage) 82.35% 0.170 0.310 - -6.06% -
447.dealII (max) (percentage) 77.27% 0.660 1.170 - -4.88% -
447.dealII (max) 76.47% 34.000 60.000 - -4.76% -
464.h264ref (max) 75.86% 87.000 153.000 - -7.27% -
464.h264ref (max) (percentage) 75.56% 0.900 1.580 - -6.51% -
429.mcf (max) 60.00% 10.000 16.000 - -11.11% -
433.milc (max) 60.00% 10.000 16.000 - -15.79% -
433.milc (max) (percentage) 55.56% 0.090 0.140 - -12.50% -
429.mcf (max) (percentage) 54.55% 0.110 0.170 - -10.53% -
470.lbm (unexplained diff vs previous) (percentage) 44.44% 0.090 0.130 - 333.33% -
429.mcf 42.86% 7.000 10.000 - 11.11% -
429.mcf (avg) 42.86% 7.000 10.000 - 11.11% -
447.dealII 42.86% 28.000 40.000 - 0.00% -
447.dealII (avg) 42.86% 28.000 40.000 - 0.00% -
433.milc (avg) (percentage) 42.86% 0.070 0.100 - -9.09% -
447.dealII (avg) (percentage) 41.82% 0.550 0.780 - 0.00% -
445.gobmk (avg) (percentage) 40.91% 0.220 0.310 - 0.00% -
445.gobmk (max) (percentage) 40.74% 0.270 0.380 - -7.32% -
429.mcf (avg) (percentage) 37.50% 0.080 0.110 - 10.00% -
433.milc 37.50% 8.000 11.000 - -15.38% -
433.milc (avg) 37.50% 8.000 11.000 - -15.38% -
445.gobmk 37.50% 24.000 33.000 - -2.94% -
445.gobmk (avg) 37.50% 24.000 33.000 - -2.94% -
450.soplex (unexplained diff vs previous) 37.50% 8.000 11.000 - -45.00% -
445.gobmk (max) 36.67% 30.000 41.000 - -8.89% -
471.omnetpp (avg) (percentage) 34.09% 0.440 0.590 - 1.72% -
470.lbm (unexplained diff vs previous) 33.33% 15.000 20.000 - 400.00% -
458.sjeng (avg) (percentage) 33.33% 0.630 0.840 - 0.00% -
458.sjeng 32.61% 92.000 122.000 - -1.61% -
458.sjeng (avg) 32.61% 92.000 122.000 - -1.61% -
471.omnetpp 31.25% 32.000 42.000 - 0.00% -
471.omnetpp (avg) 31.25% 32.000 42.000 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 30.00% 0.100 0.130 - -45.83% -
471.omnetpp (max) (percentage) 28.26% 0.460 0.590 - -9.23% -
471.omnetpp (max) 27.27% 33.000 42.000 - -10.64% -
473.astar 18.18% 11.000 13.000 - 0.00% -
473.astar (avg) 18.18% 11.000 13.000 - 0.00% -
470.lbm (max) 17.86% 28.000 33.000 - -8.33% -
473.astar (avg) (percentage) 16.67% 0.120 0.140 - 0.00% -
470.lbm (max) (percentage) 16.67% 0.180 0.210 - -8.70% -
401.bzip2 11.11% 9.000 10.000 - 0.00% -
401.bzip2 (avg) 11.11% 9.000 10.000 - 0.00% -
483.xalancbmk (max) (percentage) 8.33% 1.080 1.170 - -12.69% -
483.xalancbmk (max) 6.35% 63.000 67.000 - -10.67% -
453.povray (avg) (percentage) 2.04% 0.490 0.500 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
445.gobmk -2.95% 11199.000 10869.000 - -1.64% -
483.xalancbmk -2.74% 5921.000 5759.000 - 1.59% -
471.omnetpp -2.16% 7376.000 7217.000 - -0.59% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
433.milc (unexplained diff vs previous) (percentage) -55.56% 0.090 0.040 - -66.67% -
433.milc (unexplained diff vs previous) -54.55% 11.000 5.000 - -64.29% -
403.gcc (max) (percentage) -47.44% 0.780 0.410 - -2.38% -
403.gcc (max) -47.06% 51.000 27.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -46.35% 3.970 2.130 - 1538.46% -
456.hmmer (unexplained diff vs previous) -44.11% 297.000 166.000 - 1560.00% -
456.hmmer (max) (percentage) -43.19% 3.820 2.170 - 223.88% -
456.hmmer (max) -40.70% 285.000 169.000 - 231.37% -
470.lbm -40.43% 47.000 28.000 - -6.67% -
470.lbm (avg) -40.43% 47.000 28.000 - -6.67% -
470.lbm (avg) (percentage) -40.00% 0.300 0.180 - -5.26% -
444.namd -31.25% 16.000 11.000 - 0.00% -
444.namd (avg) -31.25% 16.000 11.000 - 0.00% -
444.namd (avg) (percentage) -29.41% 0.170 0.120 - 0.00% -
482.sphinx3 (avg) (percentage) -29.41% 0.510 0.360 - 0.00% -
482.sphinx3 -28.26% 92.000 66.000 - 1.54% -
482.sphinx3 (avg) -28.26% 92.000 66.000 - 1.54% -
403.gcc (avg) (percentage) -27.78% 0.360 0.260 - 0.00% -
403.gcc -26.09% 23.000 17.000 - 0.00% -
403.gcc (avg) -26.09% 23.000 17.000 - 0.00% -
444.namd (max) -16.00% 25.000 21.000 - -4.55% -
450.soplex (avg) (percentage) -15.79% 0.190 0.160 - 0.00% -
462.libquantum -15.79% 19.000 16.000 - -15.79% -
462.libquantum (avg) -15.79% 19.000 16.000 - -15.79% -
444.namd (max) (percentage) -15.38% 0.260 0.220 - -4.35% -
462.libquantum (avg) (percentage) -15.38% 0.130 0.110 - -15.38% -
456.hmmer (avg) (percentage) -14.96% 1.270 1.080 - 40.26% -
450.soplex -13.33% 15.000 13.000 - 0.00% -
450.soplex (avg) -13.33% 15.000 13.000 - 0.00% -
456.hmmer -11.58% 95.000 84.000 - 44.83% -
456.hmmer (avg) -11.58% 95.000 84.000 - 44.83% -
453.povray (max) -10.34% 29.000 26.000 - -3.70% -
450.soplex (max) -10.00% 20.000 18.000 - -18.18% -
483.xalancbmk -9.26% 54.000 49.000 - 2.08% -
483.xalancbmk (avg) -9.26% 54.000 49.000 - 2.08% -
450.soplex (max) (percentage) -8.33% 0.240 0.220 - -15.38% -
453.povray (max) (percentage) -7.94% 0.630 0.580 - -4.92% -
483.xalancbmk (avg) (percentage) -6.52% 0.920 0.860 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
401.bzip2 11339.000 -
403.gcc 6650.000 -
429.mcf 9624.000 -
433.milc 11905.000 -
444.namd 9968.000 -
445.gobmk 10869.000 -1.64%
447.dealII 5210.000 1.01%
450.soplex 8474.000 -
453.povray 4579.000 -
456.hmmer 7799.000 2.17%
458.sjeng 14583.000 -1.66%
462.libquantum 14717.000 -
464.h264ref 9740.000 -
470.lbm 15834.000 -
471.omnetpp 7217.000 -
473.astar 9333.000 -
482.sphinx3 18532.000 1.36%
483.xalancbmk 5759.000 1.59%
Geometric Mean 9435.149 -
Code Size (bytes)
Name Current %
401.bzip2 74756.000 -
403.gcc 3222929.000 -
429.mcf 11067.000 -
433.milc 113899.000 -
444.namd 226810.000 -
445.gobmk 1593319.000 -
447.dealII 2633455.000 -
450.soplex 315533.000 -
453.povray 913782.000 -
456.hmmer 259466.000 -
458.sjeng 149677.000 -
462.libquantum 35305.000 -
464.h264ref 623742.000 -
470.lbm 11281.000 -
471.omnetpp 532075.000 -
473.astar 35940.000 -
482.sphinx3 166092.000 -
483.xalancbmk 3355952.000 -
Geometric Mean 242753.051 -
sample variability
Name Current %
401.bzip2 10.000 -
401.bzip2 (avg) 10.000 -
401.bzip2 (avg) (percentage) 0.090 -
401.bzip2 (max) 17.000 6.25%
401.bzip2 (max) (percentage) 0.150 -
403.gcc 17.000 -
403.gcc (avg) 17.000 -
403.gcc (avg) (percentage) 0.260 -
403.gcc (max) 27.000 -
403.gcc (max) (percentage) 0.410 -2.38%
429.mcf 10.000 11.11%
429.mcf (avg) 10.000 11.11%
429.mcf (avg) (percentage) 0.110 -
429.mcf (max) 16.000 -11.11%
429.mcf (max) (percentage) 0.170 -10.53%
429.mcf (unexplained diff vs previous) 16.000 -11.11%
429.mcf (unexplained diff vs previous) (percentage) 0.170 -10.53%
433.milc 11.000 -15.38%
433.milc (avg) 11.000 -15.38%
433.milc (avg) (percentage) 0.100 -
433.milc (max) 16.000 -15.79%
433.milc (max) (percentage) 0.140 -12.50%
433.milc (unexplained diff vs previous) 5.000 -64.29%
433.milc (unexplained diff vs previous) (percentage) 0.040 -66.67%
444.namd 11.000 -
444.namd (avg) 11.000 -
444.namd (avg) (percentage) 0.120 -
444.namd (max) 21.000 -4.55%
444.namd (max) (percentage) 0.220 -4.35%
445.gobmk 33.000 -2.94%
445.gobmk (avg) 33.000 -2.94%
445.gobmk (avg) (percentage) 0.310 -
445.gobmk (max) 41.000 -8.89%
445.gobmk (max) (percentage) 0.380 -7.32%
447.dealII 40.000 -
447.dealII (avg) 40.000 -
447.dealII (avg) (percentage) 0.780 -
447.dealII (max) 60.000 -4.76%
447.dealII (max) (percentage) 1.170 -4.88%
450.soplex 13.000 -
450.soplex (avg) 13.000 -
450.soplex (avg) (percentage) 0.160 -
450.soplex (max) 18.000 -18.18%
450.soplex (max) (percentage) 0.220 -15.38%
450.soplex (unexplained diff vs previous) 11.000 -45.00%
450.soplex (unexplained diff vs previous) (percentage) 0.130 -45.83%
453.povray 22.000 -
453.povray (avg) 22.000 -
453.povray (avg) (percentage) 0.500 -
453.povray (max) 26.000 -3.70%
453.povray (max) (percentage) 0.580 -4.92%
456.hmmer 84.000 44.83%
456.hmmer (avg) 84.000 44.83%
456.hmmer (avg) (percentage) 1.080 40.26%
456.hmmer (max) 169.000 231.37%
456.hmmer (max) (percentage) 2.170 223.88%
456.hmmer (unexplained diff vs previous) 166.000 1560.00%
456.hmmer (unexplained diff vs previous) (percentage) 2.130 1538.46%
458.sjeng 122.000 -1.61%
458.sjeng (avg) 122.000 -1.61%
458.sjeng (avg) (percentage) 0.840 -
458.sjeng (max) 188.000 -7.39%
458.sjeng (max) (percentage) 1.290 -5.84%
462.libquantum 16.000 -15.79%
462.libquantum (avg) 16.000 -15.79%
462.libquantum (avg) (percentage) 0.110 -15.38%
462.libquantum (max) 19.000 -17.39%
462.libquantum (max) (percentage) 0.130 -18.75%
462.libquantum (unexplained diff vs previous) 5.000 -79.17%
462.libquantum (unexplained diff vs previous) (percentage) 0.030 -81.25%
464.h264ref 62.000 -
464.h264ref (avg) 62.000 -
464.h264ref (avg) (percentage) 0.640 -
464.h264ref (max) 153.000 -7.27%
464.h264ref (max) (percentage) 1.580 -6.51%
470.lbm 28.000 -6.67%
470.lbm (avg) 28.000 -6.67%
470.lbm (avg) (percentage) 0.180 -5.26%
470.lbm (max) 33.000 -8.33%
470.lbm (max) (percentage) 0.210 -8.70%
470.lbm (unexplained diff vs previous) 20.000 400.00%
470.lbm (unexplained diff vs previous) (percentage) 0.130 333.33%
471.omnetpp 42.000 -
471.omnetpp (avg) 42.000 -
471.omnetpp (avg) (percentage) 0.590 1.72%
471.omnetpp (max) 42.000 -10.64%
471.omnetpp (max) (percentage) 0.590 -9.23%
471.omnetpp (unexplained diff vs previous) 43.000 -8.51%
471.omnetpp (unexplained diff vs previous) (percentage) 0.600 -7.69%
473.astar 13.000 -
473.astar (avg) 13.000 -
473.astar (avg) (percentage) 0.140 -
473.astar (max) 28.000 -6.67%
473.astar (max) (percentage) 0.310 -6.06%
482.sphinx3 66.000 1.54%
482.sphinx3 (avg) 66.000 1.54%
482.sphinx3 (avg) (percentage) 0.360 -
482.sphinx3 (max) 96.000 -4.00%
482.sphinx3 (max) (percentage) 0.520 -5.45%
483.xalancbmk 49.000 2.08%
483.xalancbmk (avg) 49.000 2.08%
483.xalancbmk (avg) (percentage) 0.860 -
483.xalancbmk (max) 67.000 -10.67%
483.xalancbmk (max) (percentage) 1.170 -12.69%
Geometric Mean 4.416 1.42%