Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/a8105026ff548412967bc68ffb57f4fb6aecb652 (+ 2661 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_speed-spec2k6--llvm-arm-master-O2-build/195/
View Options
Report
Filter

llvm-arm-master-O2 test results

Run Order Start Time Duration
Current 0195 (llvmorg-19-init-08946-ga8105026ff54) 2024-04-06T11:28:12 0:00:00
Previous 0194 (no-change) 2024-03-23T07:16:17 0:00:00
Baseline 0001 (llvmorg-16-init-16024-ga455c91601a8) 2022-12-31T07:27:27 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 13 44
Performance Improvements 9 66
Added Tests 6 2
Unchanged Tests 692 608
Total Tests 720

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
401.bzip2 1.40% 76731.000 77803.000 - 5.72% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
483.xalancbmk 3.13% 5503.000 5675.000 - -4.15% -
445.gobmk 1.41% 10905.000 11059.000 - -1.25% -
401.bzip2 1.11% 11245.000 11370.000 - 0.17% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) 50.00% 6.000 9.000 - -10.00% -
429.mcf (max) (percentage) 42.86% 0.070 0.100 - -9.09% -
445.gobmk 5.00% 20.000 21.000 - -12.50% -
445.gobmk (avg) 5.00% 20.000 21.000 - -12.50% -
483.xalancbmk (max) 3.92% 51.000 53.000 - -15.87% -
483.xalancbmk 2.17% 46.000 47.000 - -12.96% -
483.xalancbmk (avg) 2.17% 46.000 47.000 - -12.96% -
401.bzip2 1.61% 62.000 63.000 - 600.00% -
401.bzip2 (avg) 1.61% 62.000 63.000 - 600.00% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
473.astar -1.78% 38207.000 37527.000 - 4.57% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp -1.17% 7340.000 7254.000 - -1.65% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) (percentage) -7.69% 0.130 0.120 - 100.00% -
401.bzip2 (max) (percentage) -7.41% 0.270 0.250 - 212.50% -
401.bzip2 (max) -6.67% 30.000 28.000 - 211.11% -
462.libquantum (max) -6.67% 15.000 14.000 - 75.00% -
458.sjeng (max) -5.84% 137.000 129.000 - 152.94% -
458.sjeng (max) (percentage) -5.10% 0.980 0.930 - 165.71% -
471.omnetpp (max) -1.64% 61.000 60.000 - 81.82% -

Added Tests - sample variability
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf 5.91% 11019.000 11670.000 - 0.00% -
401.bzip2 5.72% 73592.000 77803.000 - 1.40% -
473.astar 4.57% 35888.000 37527.000 - -1.78% -
450.soplex 2.65% 309981.000 318195.000 - -0.01% -
483.xalancbmk 1.62% 3353660.000 3407922.000 - 0.14% -
471.omnetpp 1.43% 527703.000 535256.000 - 0.02% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
470.lbm 1.09% 15896.000 16069.000 - 0.04% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
450.soplex (max) (percentage) 15704.17% 0.240 37.930 - -0.39% -
450.soplex (max) 15680.00% 20.000 3156.000 - -0.66% -
450.soplex 2566.67% 15.000 400.000 - -0.25% -
450.soplex (avg) 2566.67% 15.000 400.000 - -0.25% -
450.soplex (avg) (percentage) 2431.58% 0.190 4.810 - 0.00% -
401.bzip2 600.00% 9.000 63.000 - 1.61% -
401.bzip2 (avg) 600.00% 9.000 63.000 - 1.61% -
401.bzip2 (avg) (percentage) 600.00% 0.080 0.560 - 0.00% -
401.bzip2 (max) (percentage) 212.50% 0.080 0.250 - -7.41% -
401.bzip2 (max) 211.11% 9.000 28.000 - -6.67% -
458.sjeng (max) (percentage) 165.71% 0.350 0.930 - -5.10% -
429.mcf 157.14% 7.000 18.000 - 0.00% -
429.mcf (avg) 157.14% 7.000 18.000 - 0.00% -
458.sjeng (max) 152.94% 51.000 129.000 - -5.84% -
429.mcf (avg) (percentage) 137.50% 0.080 0.190 - 0.00% -
403.gcc (max) 100.00% 51.000 102.000 - -0.97% -
403.gcc (max) (percentage) 100.00% 0.780 1.560 - -0.64% -
429.mcf (unexplained diff vs previous) 100.00% 5.000 10.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) 100.00% 0.050 0.100 - 0.00% -
462.libquantum (max) (percentage) 100.00% 0.060 0.120 - -7.69% -
471.omnetpp (max) (percentage) 82.61% 0.460 0.840 - 0.00% -
471.omnetpp (max) 81.82% 33.000 60.000 - -1.64% -
462.libquantum (max) 75.00% 8.000 14.000 - -6.67% -
471.omnetpp (avg) (percentage) 56.82% 0.440 0.690 - 0.00% -
471.omnetpp 56.25% 32.000 50.000 - 0.00% -
471.omnetpp (avg) 56.25% 32.000 50.000 - 0.00% -
462.libquantum (avg) (percentage) 46.15% 0.130 0.190 - 0.00% -
482.sphinx3 (max) 30.61% 49.000 64.000 - 0.00% -
482.sphinx3 (max) (percentage) 29.63% 0.270 0.350 - 0.00% -
403.gcc 21.74% 23.000 28.000 - 0.00% -
403.gcc (avg) 21.74% 23.000 28.000 - 0.00% -
403.gcc (avg) (percentage) 19.44% 0.360 0.430 - 0.00% -
462.libquantum 15.79% 19.000 22.000 - 0.00% -
462.libquantum (avg) 15.79% 19.000 22.000 - 0.00% -
456.hmmer 12.63% 95.000 107.000 - 0.00% -
456.hmmer (avg) 12.63% 95.000 107.000 - 0.00% -
456.hmmer (avg) (percentage) 11.81% 1.270 1.420 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -6.15% 35357.000 33184.000 - 0.00% -
444.namd -2.56% 226798.000 220982.000 - -0.08% -
453.povray -1.33% 913198.000 901024.000 - -0.06% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -19.29% 14679.000 11847.000 - -0.07% -
458.sjeng -4.94% 14662.000 13937.000 - -0.50% -
483.xalancbmk -4.15% 5921.000 5675.000 - 3.13% -
453.povray -2.21% 4614.000 4512.000 - 0.85% -
471.omnetpp -1.65% 7376.000 7254.000 - -1.17% -
445.gobmk -1.25% 11199.000 11059.000 - 1.41% -
447.dealII -1.16% 5171.000 5111.000 - -0.10% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
470.lbm (avg) (percentage) -76.67% 0.300 0.070 - 0.00% -
470.lbm -76.60% 47.000 11.000 - 0.00% -
470.lbm (avg) -76.60% 47.000 11.000 - 0.00% -
456.hmmer (max) (percentage) -72.51% 3.820 1.050 - 0.00% -
456.hmmer (max) -72.28% 285.000 79.000 - 0.00% -
433.milc (max) -70.00% 10.000 3.000 - 0.00% -
433.milc (max) (percentage) -66.67% 0.090 0.030 - 0.00% -
470.lbm (max) (percentage) -61.11% 0.180 0.070 - 0.00% -
470.lbm (max) -60.71% 28.000 11.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -55.56% 0.090 0.040 - 0.00% -
445.gobmk (max) -53.33% 30.000 14.000 - 0.00% -
470.lbm (unexplained diff vs previous) -53.33% 15.000 7.000 - 0.00% -
473.astar (max) -53.33% 15.000 7.000 - 0.00% -
473.astar (max) (percentage) -52.94% 0.170 0.080 - 0.00% -
445.gobmk (max) (percentage) -51.85% 0.270 0.130 - 0.00% -
444.namd (max) -48.00% 25.000 13.000 - 0.00% -
444.namd (max) (percentage) -46.15% 0.260 0.140 - 0.00% -
453.povray (max) -41.38% 29.000 17.000 - 0.00% -
453.povray (max) (percentage) -38.10% 0.630 0.390 - 0.00% -
433.milc -37.50% 8.000 5.000 - 0.00% -
433.milc (avg) -37.50% 8.000 5.000 - 0.00% -
464.h264ref (max) -35.63% 87.000 56.000 - 0.00% -
464.h264ref (max) (percentage) -35.56% 0.900 0.580 - 0.00% -
433.milc (avg) (percentage) -28.57% 0.070 0.050 - 0.00% -
444.namd -18.75% 16.000 13.000 - 0.00% -
444.namd (avg) -18.75% 16.000 13.000 - 0.00% -
453.povray -18.18% 22.000 18.000 - 0.00% -
453.povray (avg) -18.18% 22.000 18.000 - 0.00% -
473.astar -18.18% 11.000 9.000 - 0.00% -
473.astar (avg) -18.18% 11.000 9.000 - 0.00% -
444.namd (avg) (percentage) -17.65% 0.170 0.140 - 0.00% -
473.astar (avg) (percentage) -16.67% 0.120 0.100 - 0.00% -
453.povray (avg) (percentage) -16.33% 0.490 0.410 - 0.00% -
483.xalancbmk (max) -15.87% 63.000 53.000 - 3.92% -
482.sphinx3 (avg) (percentage) -15.69% 0.510 0.430 - 0.00% -
458.sjeng -15.22% 92.000 78.000 - 0.00% -
458.sjeng (avg) -15.22% 92.000 78.000 - 0.00% -
447.dealII (max) -14.71% 34.000 29.000 - 0.00% -
482.sphinx3 -14.13% 92.000 79.000 - 0.00% -
482.sphinx3 (avg) -14.13% 92.000 79.000 - 0.00% -
447.dealII (max) (percentage) -13.64% 0.660 0.570 - 0.00% -
445.gobmk (avg) (percentage) -13.64% 0.220 0.190 - 0.00% -
483.xalancbmk (max) (percentage) -12.96% 1.080 0.940 - 0.00% -
483.xalancbmk -12.96% 54.000 47.000 - 2.17% -
483.xalancbmk (avg) -12.96% 54.000 47.000 - 2.17% -
464.h264ref -12.90% 62.000 54.000 - 0.00% -
464.h264ref (avg) -12.90% 62.000 54.000 - 0.00% -
445.gobmk -12.50% 24.000 21.000 - 5.00% -
445.gobmk (avg) -12.50% 24.000 21.000 - 5.00% -
464.h264ref (avg) (percentage) -12.50% 0.640 0.560 - 0.00% -
458.sjeng (avg) (percentage) -11.11% 0.630 0.560 - 0.00% -
429.mcf (max) -10.00% 10.000 9.000 - 50.00% -
483.xalancbmk (avg) (percentage) -8.70% 0.920 0.840 - 0.00% -
447.dealII -7.14% 28.000 26.000 - 0.00% -
447.dealII (avg) -7.14% 28.000 26.000 - 0.00% -
447.dealII (avg) (percentage) -5.45% 0.550 0.520 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.22s
Execution (seconds)
Name Current %
401.bzip2 11370.000 1.11%
403.gcc 6576.000 -
429.mcf 9702.000 -
433.milc 11847.000 -
444.namd 9900.000 -
445.gobmk 11059.000 1.41%
447.dealII 5111.000 -
450.soplex 8321.000 -
453.povray 4512.000 -
456.hmmer 7541.000 -
458.sjeng 13937.000 -
462.libquantum 11847.000 -
464.h264ref 9764.000 -
470.lbm 16069.000 -
471.omnetpp 7254.000 -1.17%
473.astar 9450.000 -
482.sphinx3 18398.000 -
483.xalancbmk 5675.000 3.13%
Geometric Mean 9263.750 -
Code Size (bytes)
Name Current %
401.bzip2 77803.000 1.40%
403.gcc 3240948.000 -
429.mcf 11670.000 -
433.milc 113066.000 -
444.namd 220982.000 -
445.gobmk 1599550.000 -
447.dealII 2624662.000 -
450.soplex 318195.000 -
453.povray 901024.000 -
456.hmmer 258565.000 -
458.sjeng 149280.000 -
462.libquantum 33184.000 -
464.h264ref 617865.000 -
470.lbm 11172.000 -
471.omnetpp 535256.000 -
473.astar 37527.000 -1.78%
482.sphinx3 165899.000 -
483.xalancbmk 3407922.000 -
Geometric Mean 243241.949 -
sample variability
Name Current %
401.bzip2 63.000 1.61%
401.bzip2 (avg) 63.000 1.61%
401.bzip2 (avg) (percentage) 0.560 -
401.bzip2 (max) 28.000 -6.67%
401.bzip2 (max) (percentage) 0.250 -7.41%
403.gcc 28.000 -
403.gcc (avg) 28.000 -
403.gcc (avg) (percentage) 0.430 -
403.gcc (max) 102.000 -
403.gcc (max) (percentage) 1.560 -
429.mcf 18.000 -
429.mcf (avg) 18.000 -
429.mcf (avg) (percentage) 0.190 -
429.mcf (max) 9.000 50.00%
429.mcf (max) (percentage) 0.100 42.86%
429.mcf (unexplained diff vs previous) 10.000 -
429.mcf (unexplained diff vs previous) (percentage) 0.100 -
433.milc 5.000 -
433.milc (avg) 5.000 -
433.milc (avg) (percentage) 0.050 -
433.milc (max) 3.000 -
433.milc (max) (percentage) 0.030 -
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 0.140 -
444.namd (max) 13.000 -
444.namd (max) (percentage) 0.140 -
445.gobmk 21.000 5.00%
445.gobmk (avg) 21.000 5.00%
445.gobmk (avg) (percentage) 0.190 -
445.gobmk (max) 14.000 -
445.gobmk (max) (percentage) 0.130 -
447.dealII 26.000 -
447.dealII (avg) 26.000 -
447.dealII (avg) (percentage) 0.520 -
447.dealII (max) 29.000 -
447.dealII (max) (percentage) 0.570 -
450.soplex 400.000 -
450.soplex (avg) 400.000 -
450.soplex (avg) (percentage) 4.810 -
450.soplex (max) 3156.000 -
450.soplex (max) (percentage) 37.930 -
453.povray 18.000 -
453.povray (avg) 18.000 -
453.povray (avg) (percentage) 0.410 -
453.povray (max) 17.000 -
453.povray (max) (percentage) 0.390 -
456.hmmer 107.000 -
456.hmmer (avg) 107.000 -
456.hmmer (avg) (percentage) 1.420 -
456.hmmer (max) 79.000 -
456.hmmer (max) (percentage) 1.050 -
458.sjeng 78.000 -
458.sjeng (avg) 78.000 -
458.sjeng (avg) (percentage) 0.560 -
458.sjeng (max) 129.000 -5.84%
458.sjeng (max) (percentage) 0.930 -5.10%
462.libquantum 22.000 -
462.libquantum (avg) 22.000 -
462.libquantum (avg) (percentage) 0.190 -
462.libquantum (max) 14.000 -6.67%
462.libquantum (max) (percentage) 0.120 -7.69%
462.libquantum (unexplained diff vs previous) 8.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.070 -
464.h264ref 54.000 -
464.h264ref (avg) 54.000 -
464.h264ref (avg) (percentage) 0.560 -
464.h264ref (max) 56.000 -
464.h264ref (max) (percentage) 0.580 -
470.lbm 11.000 -
470.lbm (avg) 11.000 -
470.lbm (avg) (percentage) 0.070 -
470.lbm (max) 11.000 -
470.lbm (max) (percentage) 0.070 -
470.lbm (unexplained diff vs previous) 7.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.040 -
471.omnetpp 50.000 -
471.omnetpp (avg) 50.000 -
471.omnetpp (avg) (percentage) 0.690 -
471.omnetpp (max) 60.000 -1.64%
471.omnetpp (max) (percentage) 0.840 -
473.astar 9.000 -
473.astar (avg) 9.000 -
473.astar (avg) (percentage) 0.100 -
473.astar (max) 7.000 -
473.astar (max) (percentage) 0.080 -
482.sphinx3 79.000 -
482.sphinx3 (avg) 79.000 -
482.sphinx3 (avg) (percentage) 0.430 -
482.sphinx3 (max) 64.000 -
482.sphinx3 (max) (percentage) 0.350 -
483.xalancbmk 47.000 2.17%
483.xalancbmk (avg) 47.000 2.17%
483.xalancbmk (avg) (percentage) 0.840 -
483.xalancbmk (max) 53.000 3.92%
483.xalancbmk (max) (percentage) 0.940 -
Geometric Mean 4.780 -11.23%