Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/ce8d6826997048cf4ebc104b744896a9a3923230 (+ 297 commits)
test_url https://ci.linaro.org/job/tcwg_bmk_code_speed_spec2k6-llvm_arm_master_tk1_O2-build/2/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0002 (llvmorg-16-init-16321-gce8d68269970) 2023-01-03T08:53:23 0:00:00
Previous 0001 (llvmorg-16-init-16842-g8b301b4f6b3d) 2023-01-07T09:57:27 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 55 22
Performance Improvements 31 51
Added Tests 4 6
Unchanged Tests 666 677
Total Tests 756

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 1.10% 7290.000 7370.000 - -0.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
473.astar (unexplained diff vs previous) (percentage) 1500.00% 0.010 0.160 - -5.88% -
473.astar (unexplained diff vs previous) 1400.00% 1.000 15.000 - -6.25% -
470.lbm (unexplained diff vs previous) (percentage) 800.00% 0.010 0.090 - 0.00% -
470.lbm (unexplained diff vs previous) 650.00% 2.000 15.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) 600.00% 1.000 7.000 - 250.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 500.00% 0.010 0.060 - 200.00% -
450.soplex (unexplained diff vs previous) 350.00% 4.000 18.000 - 125.00% -
450.soplex (unexplained diff vs previous) (percentage) 320.00% 0.050 0.210 - 110.00% -
462.libquantum (unexplained diff vs previous) (percentage) 162.50% 0.080 0.210 - 0.00% -
462.libquantum (unexplained diff vs previous) 158.33% 12.000 31.000 - 0.00% -
401.bzip2 (max) 80.00% 5.000 9.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) 61.80% 0.890 1.440 - -63.73% -
456.hmmer (unexplained diff vs previous) 60.29% 68.000 109.000 - -63.30% -
401.bzip2 (max) (percentage) 60.00% 0.050 0.080 - 0.00% -
444.namd (max) 50.00% 16.000 24.000 - -4.00% -
444.namd (max) (percentage) 47.06% 0.170 0.250 - -3.85% -
403.gcc (max) (percentage) 27.12% 0.590 0.750 - -3.85% -
450.soplex (max) 26.67% 15.000 19.000 - -5.00% -
403.gcc (max) 25.64% 39.000 49.000 - -3.92% -
450.soplex (max) (percentage) 21.05% 0.190 0.230 - -4.17% -
458.sjeng (max) 16.67% 30.000 35.000 - -31.37% -
458.sjeng (max) (percentage) 14.29% 0.210 0.240 - -31.43% -
456.hmmer (max) (percentage) 12.16% 2.550 2.860 - -25.13% -
456.hmmer (max) 11.28% 195.000 217.000 - -23.86% -
462.libquantum (max) 11.11% 27.000 30.000 - 275.00% -
462.libquantum (max) (percentage) 10.53% 0.190 0.210 - 250.00% -
470.lbm 9.52% 42.000 46.000 - -2.13% -
470.lbm (avg) 9.52% 42.000 46.000 - -2.13% -
470.lbm (max) 8.00% 25.000 27.000 - -3.57% -
462.libquantum (avg) (percentage) 7.69% 0.130 0.140 - 7.69% -
470.lbm (avg) (percentage) 7.41% 0.270 0.290 - -3.33% -
473.astar (max) 7.14% 14.000 15.000 - 0.00% -
473.astar (max) (percentage) 6.67% 0.150 0.160 - -5.88% -
444.namd 6.67% 15.000 16.000 - 0.00% -
444.namd (avg) 6.67% 15.000 16.000 - 0.00% -
444.namd (avg) (percentage) 6.25% 0.160 0.170 - 0.00% -
470.lbm (max) (percentage) 6.25% 0.160 0.170 - -5.56% -
450.soplex (avg) (percentage) 5.56% 0.180 0.190 - 0.00% -
445.gobmk (max) 5.26% 38.000 40.000 - 33.33% -
462.libquantum 5.26% 19.000 20.000 - 5.26% -
462.libquantum (avg) 5.26% 19.000 20.000 - 5.26% -
483.xalancbmk (max) (percentage) 4.00% 1.000 1.040 - -3.70% -
453.povray (max) 3.85% 26.000 27.000 - -6.90% -
453.povray (max) (percentage) 3.45% 0.580 0.600 - -4.76% -
483.xalancbmk (max) 3.45% 58.000 60.000 - -4.76% -
447.dealII (max) (percentage) 3.17% 0.630 0.650 - -1.52% -
447.dealII (max) 3.12% 32.000 33.000 - -2.94% -
445.gobmk (max) (percentage) 2.86% 0.350 0.360 - 33.33% -
482.sphinx3 2.20% 91.000 93.000 - 1.09% -
482.sphinx3 (avg) 2.20% 91.000 93.000 - 1.09% -
482.sphinx3 (avg) (percentage) 2.00% 0.500 0.510 - 0.00% -
456.hmmer (avg) (percentage) 1.59% 1.260 1.280 - 0.79% -
456.hmmer 1.04% 96.000 97.000 - 2.11% -
456.hmmer (avg) 1.04% 96.000 97.000 - 2.11% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
483.xalancbmk -1.31% 5892.000 5815.000 - -1.79% -
453.povray -1.16% 4651.000 4597.000 - -0.37% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (unexplained diff vs previous) (percentage) -92.73% 1.100 0.080 - 14.29% -
471.omnetpp (unexplained diff vs previous) -92.50% 80.000 6.000 - 20.00% -
464.h264ref (unexplained diff vs previous) (percentage) -78.67% 0.750 0.160 - -82.42% -
464.h264ref (unexplained diff vs previous) -78.38% 74.000 16.000 - -82.02% -
433.milc (unexplained diff vs previous) (percentage) -70.00% 0.100 0.030 - -66.67% -
429.mcf (unexplained diff vs previous) -66.67% 12.000 4.000 - -20.00% -
429.mcf (unexplained diff vs previous) (percentage) -66.67% 0.120 0.040 - -20.00% -
433.milc (unexplained diff vs previous) -66.67% 12.000 4.000 - -63.64% -
471.omnetpp (max) (percentage) -62.39% 1.090 0.410 - -10.87% -
471.omnetpp (max) -62.03% 79.000 30.000 - -9.09% -
482.sphinx3 (max) -40.79% 76.000 45.000 - -8.16% -
482.sphinx3 (max) (percentage) -40.48% 0.420 0.250 - -7.41% -
433.milc (max) (percentage) -30.00% 0.100 0.070 - -22.22% -
433.milc (max) -27.27% 11.000 8.000 - -20.00% -
429.mcf (max) -18.18% 11.000 9.000 - -10.00% -
429.mcf (max) (percentage) -16.67% 0.120 0.100 - -9.09% -
464.h264ref (max) -10.81% 74.000 66.000 - -24.14% -
464.h264ref (max) (percentage) -10.53% 0.760 0.680 - -24.44% -
403.gcc -8.00% 25.000 23.000 - 0.00% -
403.gcc (avg) -8.00% 25.000 23.000 - 0.00% -
471.omnetpp (avg) (percentage) -6.67% 0.450 0.420 - -4.55% -
471.omnetpp -6.25% 32.000 30.000 - -6.25% -
471.omnetpp (avg) -6.25% 32.000 30.000 - -6.25% -
403.gcc (avg) (percentage) -5.26% 0.380 0.360 - 0.00% -
464.h264ref -3.28% 61.000 59.000 - -4.84% -
464.h264ref (avg) -3.28% 61.000 59.000 - -4.84% -
483.xalancbmk -1.85% 54.000 53.000 - -1.85% -
483.xalancbmk (avg) -1.85% 54.000 53.000 - -1.85% -
464.h264ref (avg) (percentage) -1.61% 0.620 0.610 - -4.69% -

Added Tests - sample variability
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
456.hmmer 1.46% 7481.000 7590.000 - -0.89% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) 275.00% 8.000 30.000 - 11.11% -
401.bzip2 (unexplained diff vs previous) 250.00% 2.000 7.000 - 600.00% -
462.libquantum (max) (percentage) 250.00% 0.060 0.210 - 10.53% -
444.namd (unexplained diff vs previous) (percentage) 216.67% 0.060 0.190 - 0.00% -
444.namd (unexplained diff vs previous) 216.67% 6.000 19.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 200.00% 0.020 0.060 - 500.00% -
450.soplex (unexplained diff vs previous) 125.00% 8.000 18.000 - 350.00% -
450.soplex (unexplained diff vs previous) (percentage) 110.00% 0.100 0.210 - 320.00% -
445.gobmk (max) 33.33% 30.000 40.000 - 5.26% -
445.gobmk (max) (percentage) 33.33% 0.270 0.360 - 2.86% -
471.omnetpp (unexplained diff vs previous) 20.00% 5.000 6.000 - -92.50% -
462.libquantum (avg) (percentage) 7.69% 0.130 0.140 - 7.69% -
462.libquantum 5.26% 19.000 20.000 - 5.26% -
462.libquantum (avg) 5.26% 19.000 20.000 - 5.26% -
445.gobmk (avg) (percentage) 4.55% 0.220 0.230 - 0.00% -
445.gobmk 4.17% 24.000 25.000 - 0.00% -
445.gobmk (avg) 4.17% 24.000 25.000 - 0.00% -
456.hmmer 2.11% 95.000 97.000 - 1.04% -
456.hmmer (avg) 2.11% 95.000 97.000 - 1.04% -
482.sphinx3 1.09% 92.000 93.000 - 2.20% -
482.sphinx3 (avg) 1.09% 92.000 93.000 - 2.20% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk -1.79% 5921.000 5815.000 - -1.31% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
464.h264ref (unexplained diff vs previous) (percentage) -82.42% 0.910 0.160 - -78.67% -
464.h264ref (unexplained diff vs previous) -82.02% 89.000 16.000 - -78.38% -
433.milc (unexplained diff vs previous) (percentage) -66.67% 0.090 0.030 - -70.00% -
456.hmmer (unexplained diff vs previous) (percentage) -63.73% 3.970 1.440 - 61.80% -
433.milc (unexplained diff vs previous) -63.64% 11.000 4.000 - -66.67% -
456.hmmer (unexplained diff vs previous) -63.30% 297.000 109.000 - 60.29% -
458.sjeng (max) (percentage) -31.43% 0.350 0.240 - 14.29% -
458.sjeng (max) -31.37% 51.000 35.000 - 16.67% -
456.hmmer (max) (percentage) -25.13% 3.820 2.860 - 12.16% -
464.h264ref (max) (percentage) -24.44% 0.900 0.680 - -10.53% -
464.h264ref (max) -24.14% 87.000 66.000 - -10.81% -
456.hmmer (max) -23.86% 285.000 217.000 - 11.28% -
433.milc (max) (percentage) -22.22% 0.090 0.070 - -30.00% -
429.mcf (unexplained diff vs previous) (percentage) -20.00% 0.050 0.040 - -66.67% -
429.mcf (unexplained diff vs previous) -20.00% 5.000 4.000 - -66.67% -
433.milc (max) -20.00% 10.000 8.000 - -27.27% -
471.omnetpp (max) (percentage) -10.87% 0.460 0.410 - -62.39% -
429.mcf (max) -10.00% 10.000 9.000 - -18.18% -
471.omnetpp (max) -9.09% 33.000 30.000 - -62.03% -
482.sphinx3 (max) -8.16% 49.000 45.000 - -40.79% -
482.sphinx3 (max) (percentage) -7.41% 0.270 0.250 - -40.48% -
453.povray (max) -6.90% 29.000 27.000 - 3.85% -
471.omnetpp -6.25% 32.000 30.000 - -6.25% -
471.omnetpp (avg) -6.25% 32.000 30.000 - -6.25% -
473.astar (unexplained diff vs previous) -6.25% 16.000 15.000 - 1400.00% -
473.astar (max) (percentage) -5.88% 0.170 0.160 - 6.67% -
473.astar (unexplained diff vs previous) (percentage) -5.88% 0.170 0.160 - 1500.00% -
450.soplex (max) -5.00% 20.000 19.000 - 26.67% -
464.h264ref -4.84% 62.000 59.000 - -3.28% -
464.h264ref (avg) -4.84% 62.000 59.000 - -3.28% -
453.povray (max) (percentage) -4.76% 0.630 0.600 - 3.45% -
458.sjeng (avg) (percentage) -4.76% 0.630 0.600 - 0.00% -
483.xalancbmk (max) -4.76% 63.000 60.000 - 3.45% -
464.h264ref (avg) (percentage) -4.69% 0.640 0.610 - -1.61% -
471.omnetpp (avg) (percentage) -4.55% 0.440 0.420 - -6.67% -
458.sjeng -4.35% 92.000 88.000 - 0.00% -
458.sjeng (avg) -4.35% 92.000 88.000 - 0.00% -
444.namd (max) -4.00% 25.000 24.000 - 50.00% -
403.gcc (max) -3.92% 51.000 49.000 - 25.64% -
403.gcc (max) (percentage) -3.85% 0.780 0.750 - 27.12% -
444.namd (max) (percentage) -3.85% 0.260 0.250 - 47.06% -
483.xalancbmk (max) (percentage) -3.70% 1.080 1.040 - 4.00% -
470.lbm (max) -3.57% 28.000 27.000 - 8.00% -
470.lbm (avg) (percentage) -3.33% 0.300 0.290 - 7.41% -
447.dealII (max) -2.94% 34.000 33.000 - 3.12% -
470.lbm -2.13% 47.000 46.000 - 9.52% -
470.lbm (avg) -2.13% 47.000 46.000 - 9.52% -
483.xalancbmk -1.85% 54.000 53.000 - -1.85% -
483.xalancbmk (avg) -1.85% 54.000 53.000 - -1.85% -
447.dealII (max) (percentage) -1.52% 0.660 0.650 - 3.17% -

Added Tests - sample variability
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.34s
Execution (seconds)
Name Current %
401.bzip2 11358.000 -
403.gcc 6585.000 -
429.mcf 9610.000 -
433.milc 11958.000 -
444.namd 9974.000 -
445.gobmk 11159.000 -
447.dealII 5178.000 -
450.soplex 8381.000 -
453.povray 4597.000 -1.16%
456.hmmer 7590.000 -
458.sjeng 14697.000 -
462.libquantum 14710.000 -
464.h264ref 9773.000 -
470.lbm 15911.000 -
471.omnetpp 7370.000 1.10%
473.astar 9378.000 -
482.sphinx3 18236.000 -
483.xalancbmk 5815.000 -1.31%
Geometric Mean 9443.667 -
Code Size (bytes)
Name Current %
401.bzip2 73592.000 -
403.gcc 3215041.000 -
429.mcf 11019.000 -
433.milc 113879.000 -
444.namd 226798.000 -
445.gobmk 1588647.000 -
447.dealII 2623454.000 -
450.soplex 309981.000 -
453.povray 913182.000 -
456.hmmer 258958.000 -
458.sjeng 149489.000 -
462.libquantum 35357.000 -
464.h264ref 622678.000 -
470.lbm 11257.000 -
471.omnetpp 527703.000 -
473.astar 35888.000 -
482.sphinx3 165788.000 -
483.xalancbmk 3353272.000 -
Geometric Mean 241867.532 -
sample variability
Name Current %
401.bzip2 9.000 -
401.bzip2 (avg) 9.000 -
401.bzip2 (avg) (percentage) 0.080 -
401.bzip2 (max) 9.000 80.00%
401.bzip2 (max) (percentage) 0.080 60.00%
401.bzip2 (unexplained diff vs previous) 7.000 600.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.060 500.00%
403.gcc 23.000 -8.00%
403.gcc (avg) 23.000 -8.00%
403.gcc (avg) (percentage) 0.360 -5.26%
403.gcc (max) 49.000 25.64%
403.gcc (max) (percentage) 0.750 27.12%
429.mcf 7.000 -
429.mcf (avg) 7.000 -
429.mcf (avg) (percentage) 0.080 -
429.mcf (max) 9.000 -18.18%
429.mcf (max) (percentage) 0.100 -16.67%
429.mcf (unexplained diff vs previous) 4.000 -66.67%
429.mcf (unexplained diff vs previous) (percentage) 0.040 -66.67%
433.milc 8.000 -
433.milc (avg) 8.000 -
433.milc (avg) (percentage) 0.070 -
433.milc (max) 8.000 -27.27%
433.milc (max) (percentage) 0.070 -30.00%
433.milc (unexplained diff vs previous) 4.000 -66.67%
433.milc (unexplained diff vs previous) (percentage) 0.030 -70.00%
444.namd 16.000 6.67%
444.namd (avg) 16.000 6.67%
444.namd (avg) (percentage) 0.170 6.25%
444.namd (max) 24.000 50.00%
444.namd (max) (percentage) 0.250 47.06%
444.namd (unexplained diff vs previous) 19.000 -
444.namd (unexplained diff vs previous) (percentage) 0.190 -
445.gobmk 25.000 -
445.gobmk (avg) 25.000 -
445.gobmk (avg) (percentage) 0.230 -
445.gobmk (max) 40.000 5.26%
445.gobmk (max) (percentage) 0.360 2.86%
445.gobmk (unexplained diff vs previous) 40.000 -
445.gobmk (unexplained diff vs previous) (percentage) 0.360 -
447.dealII 28.000 -
447.dealII (avg) 28.000 -
447.dealII (avg) (percentage) 0.550 -
447.dealII (max) 33.000 3.12%
447.dealII (max) (percentage) 0.650 3.17%
450.soplex 15.000 -
450.soplex (avg) 15.000 -
450.soplex (avg) (percentage) 0.190 5.56%
450.soplex (max) 19.000 26.67%
450.soplex (max) (percentage) 0.230 21.05%
450.soplex (unexplained diff vs previous) 18.000 350.00%
450.soplex (unexplained diff vs previous) (percentage) 0.210 320.00%
453.povray 22.000 -
453.povray (avg) 22.000 -
453.povray (avg) (percentage) 0.490 -
453.povray (max) 27.000 3.85%
453.povray (max) (percentage) 0.600 3.45%
456.hmmer 97.000 1.04%
456.hmmer (avg) 97.000 1.04%
456.hmmer (avg) (percentage) 1.280 1.59%
456.hmmer (max) 217.000 11.28%
456.hmmer (max) (percentage) 2.860 12.16%
456.hmmer (unexplained diff vs previous) 109.000 60.29%
456.hmmer (unexplained diff vs previous) (percentage) 1.440 61.80%
458.sjeng 88.000 -
458.sjeng (avg) 88.000 -
458.sjeng (avg) (percentage) 0.600 -
458.sjeng (max) 35.000 16.67%
458.sjeng (max) (percentage) 0.240 14.29%
458.sjeng (unexplained diff vs previous) 35.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.240 -
462.libquantum 20.000 5.26%
462.libquantum (avg) 20.000 5.26%
462.libquantum (avg) (percentage) 0.140 7.69%
462.libquantum (max) 30.000 11.11%
462.libquantum (max) (percentage) 0.210 10.53%
462.libquantum (unexplained diff vs previous) 31.000 158.33%
462.libquantum (unexplained diff vs previous) (percentage) 0.210 162.50%
464.h264ref 59.000 -3.28%
464.h264ref (avg) 59.000 -3.28%
464.h264ref (avg) (percentage) 0.610 -1.61%
464.h264ref (max) 66.000 -10.81%
464.h264ref (max) (percentage) 0.680 -10.53%
464.h264ref (unexplained diff vs previous) 16.000 -78.38%
464.h264ref (unexplained diff vs previous) (percentage) 0.160 -78.67%
470.lbm 46.000 9.52%
470.lbm (avg) 46.000 9.52%
470.lbm (avg) (percentage) 0.290 7.41%
470.lbm (max) 27.000 8.00%
470.lbm (max) (percentage) 0.170 6.25%
470.lbm (unexplained diff vs previous) 15.000 650.00%
470.lbm (unexplained diff vs previous) (percentage) 0.090 800.00%
471.omnetpp 30.000 -6.25%
471.omnetpp (avg) 30.000 -6.25%
471.omnetpp (avg) (percentage) 0.420 -6.67%
471.omnetpp (max) 30.000 -62.03%
471.omnetpp (max) (percentage) 0.410 -62.39%
471.omnetpp (unexplained diff vs previous) 6.000 -92.50%
471.omnetpp (unexplained diff vs previous) (percentage) 0.080 -92.73%
473.astar 11.000 -
473.astar (avg) 11.000 -
473.astar (avg) (percentage) 0.120 -
473.astar (max) 15.000 7.14%
473.astar (max) (percentage) 0.160 6.67%
473.astar (unexplained diff vs previous) 15.000 1400.00%
473.astar (unexplained diff vs previous) (percentage) 0.160 1500.00%
482.sphinx3 93.000 2.20%
482.sphinx3 (avg) 93.000 2.20%
482.sphinx3 (avg) (percentage) 0.510 2.00%
482.sphinx3 (max) 45.000 -40.79%
482.sphinx3 (max) (percentage) 0.250 -40.48%
483.xalancbmk 53.000 -1.85%
483.xalancbmk (avg) 53.000 -1.85%
483.xalancbmk (avg) (percentage) 0.920 -
483.xalancbmk (max) 60.000 3.45%
483.xalancbmk (max) (percentage) 1.040 4.00%
Geometric Mean 3.514 25.99%