Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/ea2ed80e6dac21bf19d87de8bef1ec01b00dbb8d (+ 984 commits)
test_url https://ci.linaro.org/job/tcwg_bmk_ci_llvm-build-tcwg_bmk_apm-llvm-master-aarch64-spec2k6-Oz_LTO/287/
View Options
Report
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0287 (llvmorg-16-init-06813-gea2ed80e6dac) 2022-09-24T09:23:44 0:00:00
Previous 0286 (llvmorg-20-init-17973-gf023da12d126) 2025-01-13T22:35:17 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 75 48
Performance Improvements 21 21
Added Tests 4 12
Unchanged Tests 980 999
Total Tests 1080

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
444.namd 6.25% 135372.000 143829.000 - 0.28% -
464.h264ref 3.84% 385311.000 400126.000 - 0.04% -
447.dealII 3.18% 320047.000 330217.000 - 1.54% -
453.povray 3.08% 671559.000 692210.000 - 0.10% -
403.gcc 3.04% 2416336.000 2489749.000 - 0.11% -
462.libquantum 2.86% 16512.000 16985.000 - -0.02% -
482.sphinx3 2.39% 117637.000 120454.000 - 0.17% -
456.hmmer 1.97% 111488.000 113689.000 - 0.07% -
400.perlbench 1.97% 883727.000 901163.000 - 0.10% -
401.bzip2 1.73% 43644.000 44397.000 - 0.09% -
473.astar 1.71% 23664.000 24068.000 - 1.17% -
471.omnetpp 1.71% 487258.000 495571.000 - 1.11% -
433.milc 1.62% 62150.000 63155.000 - 0.46% -
445.gobmk 1.59% 1821938.000 1850859.000 - 0.03% -
458.sjeng 1.54% 105415.000 107040.000 - 0.16% -
450.soplex 1.49% 228474.000 231883.000 - 3.33% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 14050.00% 72.000 10188.000 - 0.37% -
462.libquantum 11234.95% 103.000 11675.000 - 20.87% -
482.sphinx3 3201.96% 817.000 26977.000 - 13.45% -
450.soplex 2460.96% 543.000 13906.000 - -0.59% -
444.namd 1297.30% 666.000 9306.000 - -0.76% -
453.povray 1273.57% 526.000 7225.000 - -1.04% -
433.milc 1067.04% 1244.000 14518.000 - -0.49% -
429.mcf 1058.92% 1502.000 17407.000 - -7.92% -
400.perlbench 526.65% 1996.000 12508.000 - 12.84% -
447.dealII 403.43% 1922.000 9676.000 - 6.31% -
401.bzip2 360.40% 2727.000 12555.000 - -10.08% -
471.omnetpp 334.95% 3119.000 13566.000 - 13.84% -
470.lbm 298.66% 3215.000 12817.000 - 16.03% -
456.hmmer 275.23% 2915.000 10938.000 - 19.91% -
464.h264ref 206.71% 4499.000 13799.000 - 13.65% -
473.astar 190.52% 5126.000 14892.000 - 11.96% -
445.gobmk 133.67% 5444.000 12721.000 - 9.02% -
458.sjeng 93.11% 7523.000 14528.000 - -4.93% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
444.namd (unexplained diff vs previous) 32550.00% 2.000 653.000 - 0.00% -
444.namd (max) 30400.00% 2.000 610.000 - 25.77% -
444.namd 12100.00% 5.000 610.000 - 186.38% -
444.namd (avg) 12100.00% 5.000 610.000 - 186.38% -
429.mcf (unexplained diff vs previous) 3933.33% 3.000 121.000 - 0.00% -
433.milc (unexplained diff vs previous) 3687.50% 8.000 303.000 - 10000.00% -
433.milc 2860.00% 10.000 296.000 - 144.63% -
433.milc (avg) 2860.00% 10.000 296.000 - 144.63% -
482.sphinx3 (unexplained diff vs previous) 2850.00% 2.000 59.000 - -63.58% -
444.namd (unexplained diff vs previous) (percentage) 2240.00% 0.300 7.020 - 0.00% -
433.milc (max) 1457.89% 19.000 296.000 - 10.04% -
450.soplex (unexplained diff vs previous) 1391.67% 12.000 179.000 - 0.00% -
444.namd (max) (percentage) 1357.78% 0.450 6.560 - 26.64% -
470.lbm (max) 1271.23% 73.000 1001.000 - 16583.33% -
450.soplex 1157.14% 14.000 176.000 - 134.67% -
450.soplex (avg) 1157.14% 14.000 176.000 - 134.67% -
462.libquantum 800.00% 2.000 18.000 - 80.00% -
462.libquantum (avg) 800.00% 2.000 18.000 - 80.00% -
444.namd (avg) (percentage) 763.16% 0.760 6.560 - 187.72% -
482.sphinx3 (max) 555.56% 9.000 59.000 - -64.02% -
450.soplex (max) 486.67% 30.000 176.000 - 15.03% -
462.libquantum (max) 350.00% 4.000 18.000 - 500.00% -
482.sphinx3 321.43% 14.000 59.000 - -72.69% -
482.sphinx3 (avg) 321.43% 14.000 59.000 - -72.69% -
429.mcf (unexplained diff vs previous) (percentage) 250.00% 0.200 0.700 - 0.00% -
470.lbm (max) (percentage) 241.05% 2.290 7.810 - 12916.67% -
433.milc (unexplained diff vs previous) (percentage) 226.56% 0.640 2.090 - 10350.00% -
470.lbm (unexplained diff vs previous) 211.76% 17.000 53.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) 142.86% 7.000 17.000 - -85.83% -
433.milc (avg) (percentage) 131.82% 0.880 2.040 - 145.78% -
445.gobmk 100.00% 38.000 76.000 - 105.41% -
445.gobmk (avg) 100.00% 38.000 76.000 - 105.41% -
470.lbm 65.62% 32.000 53.000 - -17.19% -
470.lbm (avg) 65.62% 32.000 53.000 - -17.19% -
429.mcf 53.85% 78.000 120.000 - 73.91% -
429.mcf (avg) 53.85% 78.000 120.000 - 73.91% -
429.mcf (max) 37.05% 251.000 344.000 - 405.88% -
433.milc (max) (percentage) 32.47% 1.540 2.040 - 10.27% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests -38.71% 31.000 19.000 - 0.00% -
nb_compile_successful -29.63% 27.000 19.000 - 0.00% -
nb_execution_successful -29.63% 27.000 19.000 - 0.00% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (max) (percentage) -96.86% 4.140 0.130 - -87.00% -
462.libquantum (max) (percentage) -96.44% 4.490 0.160 - 300.00% -
462.libquantum (avg) (percentage) -93.10% 2.320 0.160 - 45.45% -
471.omnetpp (avg) (percentage) -92.82% 1.810 0.130 - -78.33% -
429.mcf (max) (percentage) -88.19% 16.770 1.980 - 450.00% -
482.sphinx3 (avg) (percentage) -87.78% 1.800 0.220 - -75.82% -
471.omnetpp (max) -86.82% 129.000 17.000 - -85.71% -
429.mcf (avg) (percentage) -86.78% 5.220 0.690 - 86.49% -
482.sphinx3 (max) (percentage) -81.51% 1.190 0.220 - -68.12% -
450.soplex (max) (percentage) -77.08% 5.540 1.270 - 15.45% -
445.gobmk (max) (percentage) -69.85% 1.990 0.600 - 0.00% -
471.omnetpp -69.64% 56.000 17.000 - -76.06% -
471.omnetpp (avg) -69.64% 56.000 17.000 - -76.06% -
470.lbm (avg) (percentage) -58.42% 1.010 0.420 - -27.59% -
450.soplex (avg) (percentage) -52.96% 2.700 1.270 - 135.19% -
450.soplex (unexplained diff vs previous) (percentage) -41.63% 2.210 1.290 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) -40.91% 0.220 0.130 - -87.13% -
445.gobmk (max) -29.63% 108.000 76.000 - 8.57% -
470.lbm (unexplained diff vs previous) (percentage) -22.64% 0.530 0.410 - 0.00% -
445.gobmk (avg) (percentage) -14.29% 0.700 0.600 - 87.50% -
482.sphinx3 (unexplained diff vs previous) (percentage) -8.33% 0.240 0.220 - -67.65% -

Added Tests - Code Size
483.xalancbmk

Added Tests - Execution
483.xalancbmk

Added Tests - sample variability
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 3.33% 224407.000 231883.000 - 1.49% -
447.dealII 1.54% 325219.000 330217.000 - 3.18% -
473.astar 1.17% 23789.000 24068.000 - 1.71% -
471.omnetpp 1.11% 490147.000 495571.000 - 1.71% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk 23.21% 9633.000 11869.000 - 0.00% -
462.libquantum 20.87% 9659.000 11675.000 - 11234.95% -
456.hmmer 19.91% 9122.000 10938.000 - 275.23% -
470.lbm 16.03% 11046.000 12817.000 - 298.66% -
471.omnetpp 13.84% 11917.000 13566.000 - 334.95% -
464.h264ref 13.65% 12142.000 13799.000 - 206.71% -
482.sphinx3 13.45% 23779.000 26977.000 - 3201.96% -
400.perlbench 12.84% 11085.000 12508.000 - 526.65% -
473.astar 11.96% 13301.000 14892.000 - 190.52% -
445.gobmk 9.02% 11668.000 12721.000 - 133.67% -
447.dealII 6.31% 9102.000 9676.000 - 403.43% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
470.lbm (max) 16583.33% 6.000 1001.000 - 1271.23% -
470.lbm (max) (percentage) 12916.67% 0.060 7.810 - 241.05% -
433.milc (unexplained diff vs previous) (percentage) 10350.00% 0.020 2.090 - 226.56% -
433.milc (unexplained diff vs previous) 10000.00% 3.000 303.000 - 3687.50% -
462.libquantum (max) 500.00% 3.000 18.000 - 350.00% -
429.mcf (max) (percentage) 450.00% 0.360 1.980 - -88.19% -
429.mcf (max) 405.88% 68.000 344.000 - 37.05% -
462.libquantum (max) (percentage) 300.00% 0.040 0.160 - -96.44% -
444.namd (avg) (percentage) 187.72% 2.280 6.560 - 763.16% -
444.namd 186.38% 213.000 610.000 - 12100.00% -
444.namd (avg) 186.38% 213.000 610.000 - 12100.00% -
433.milc (avg) (percentage) 145.78% 0.830 2.040 - 131.82% -
433.milc 144.63% 121.000 296.000 - 2860.00% -
433.milc (avg) 144.63% 121.000 296.000 - 2860.00% -
450.soplex (avg) (percentage) 135.19% 0.540 1.270 - -52.96% -
450.soplex 134.67% 75.000 176.000 - 1157.14% -
450.soplex (avg) 134.67% 75.000 176.000 - 1157.14% -
445.gobmk 105.41% 37.000 76.000 - 100.00% -
445.gobmk (avg) 105.41% 37.000 76.000 - 100.00% -
445.gobmk (avg) (percentage) 87.50% 0.320 0.600 - -14.29% -
429.mcf (avg) (percentage) 86.49% 0.370 0.690 - -86.78% -
462.libquantum 80.00% 10.000 18.000 - 800.00% -
462.libquantum (avg) 80.00% 10.000 18.000 - 800.00% -
429.mcf 73.91% 69.000 120.000 - 53.85% -
429.mcf (avg) 73.91% 69.000 120.000 - 53.85% -
462.libquantum (avg) (percentage) 45.45% 0.110 0.160 - -93.10% -
444.namd (max) (percentage) 26.64% 5.180 6.560 - 1357.78% -
444.namd (max) 25.77% 485.000 610.000 - 30400.00% -
450.soplex (max) (percentage) 15.45% 1.100 1.270 - -77.08% -
450.soplex (max) 15.03% 153.000 176.000 - 486.67% -
433.milc (max) (percentage) 10.27% 1.850 2.040 - 32.47% -
433.milc (max) 10.04% 269.000 296.000 - 1457.89% -
445.gobmk (max) 8.57% 70.000 76.000 - -29.63% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 -10.08% 13962.000 12555.000 - 360.40% -
429.mcf -7.92% 18905.000 17407.000 - 1058.92% -
458.sjeng -4.93% 15282.000 14528.000 - 93.11% -
453.povray -1.04% 7301.000 7225.000 - 1273.57% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
471.omnetpp (unexplained diff vs previous) (percentage) -87.13% 1.010 0.130 - -40.91% -
471.omnetpp (max) (percentage) -87.00% 1.000 0.130 - -96.86% -
471.omnetpp (unexplained diff vs previous) -85.83% 120.000 17.000 - 142.86% -
471.omnetpp (max) -85.71% 119.000 17.000 - -86.82% -
471.omnetpp (avg) (percentage) -78.33% 0.600 0.130 - -92.82% -
471.omnetpp -76.06% 71.000 17.000 - -69.64% -
471.omnetpp (avg) -76.06% 71.000 17.000 - -69.64% -
482.sphinx3 (avg) (percentage) -75.82% 0.910 0.220 - -87.78% -
482.sphinx3 -72.69% 216.000 59.000 - 321.43% -
482.sphinx3 (avg) -72.69% 216.000 59.000 - 321.43% -
482.sphinx3 (max) (percentage) -68.12% 0.690 0.220 - -81.51% -
482.sphinx3 (unexplained diff vs previous) (percentage) -67.65% 0.680 0.220 - -8.33% -
482.sphinx3 (max) -64.02% 164.000 59.000 - 555.56% -
482.sphinx3 (unexplained diff vs previous) -63.58% 162.000 59.000 - 2850.00% -
470.lbm (avg) (percentage) -27.59% 0.580 0.420 - -58.42% -
470.lbm -17.19% 64.000 53.000 - 65.62% -
470.lbm (avg) -17.19% 64.000 53.000 - 65.62% -

Added Tests - sample variability
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Report Time: 0.37s
Execution (seconds)
Name Current %
400.perlbench 12508.000 526.65%
401.bzip2 12555.000 360.40%
403.gcc 10188.000 14050.00%
429.mcf 17407.000 1058.92%
433.milc 14518.000 1067.04%
444.namd 9306.000 1297.30%
445.gobmk 12721.000 133.67%
447.dealII 9676.000 403.43%
450.soplex 13906.000 2460.96%
453.povray 7225.000 1273.57%
456.hmmer 10938.000 275.23%
458.sjeng 14528.000 93.11%
462.libquantum 11675.000 11234.95%
464.h264ref 13799.000 206.71%
470.lbm 12817.000 298.66%
471.omnetpp 13566.000 334.95%
473.astar 14892.000 190.52%
482.sphinx3 26977.000 3201.96%
483.xalancbmk 11869.000 -
Geometric Mean 12734.401 790.78%
Code Size (bytes)
Name Current %
400.perlbench 901163.000 1.97%
401.bzip2 44397.000 1.73%
403.gcc 2489749.000 3.04%
429.mcf 9249.000 -
433.milc 63155.000 1.62%
444.namd 143829.000 6.25%
445.gobmk 1850859.000 1.59%
447.dealII 330217.000 3.18%
450.soplex 231883.000 1.49%
453.povray 692210.000 3.08%
456.hmmer 113689.000 1.97%
458.sjeng 107040.000 1.54%
462.libquantum 16985.000 2.86%
464.h264ref 400126.000 3.84%
470.lbm 10273.000 -
471.omnetpp 495571.000 1.71%
473.astar 24068.000 1.71%
482.sphinx3 120454.000 2.39%
483.xalancbmk 2541511.000 -
Geometric Mean 170567.396 18.87%
Score
Name Current %
nb_compile_successful 19.000 -29.63%
nb_execution_successful 19.000 -29.63%
nb_total_tests 19.000 -38.71%
Geometric Mean 19.000 -32.80%
sample variability
Name Current %
429.mcf 120.000 53.85%
429.mcf (avg) 120.000 53.85%
429.mcf (avg) (percentage) 0.690 -86.78%
429.mcf (max) 344.000 37.05%
429.mcf (max) (percentage) 1.980 -88.19%
429.mcf (unexplained diff vs previous) 121.000 3933.33%
429.mcf (unexplained diff vs previous) (percentage) 0.700 250.00%
433.milc 296.000 2860.00%
433.milc (avg) 296.000 2860.00%
433.milc (avg) (percentage) 2.040 131.82%
433.milc (max) 296.000 1457.89%
433.milc (max) (percentage) 2.040 32.47%
433.milc (unexplained diff vs previous) 303.000 3687.50%
433.milc (unexplained diff vs previous) (percentage) 2.090 226.56%
444.namd 610.000 12100.00%
444.namd (avg) 610.000 12100.00%
444.namd (avg) (percentage) 6.560 763.16%
444.namd (max) 610.000 30400.00%
444.namd (max) (percentage) 6.560 1357.78%
444.namd (unexplained diff vs previous) 653.000 32550.00%
444.namd (unexplained diff vs previous) (percentage) 7.020 2240.00%
445.gobmk 76.000 100.00%
445.gobmk (avg) 76.000 100.00%
445.gobmk (avg) (percentage) 0.600 -14.29%
445.gobmk (max) 76.000 -29.63%
445.gobmk (max) (percentage) 0.600 -69.85%
445.gobmk (unexplained diff vs previous) 77.000 -
445.gobmk (unexplained diff vs previous) (percentage) 0.610 -
450.soplex 176.000 1157.14%
450.soplex (avg) 176.000 1157.14%
450.soplex (avg) (percentage) 1.270 -52.96%
450.soplex (max) 176.000 486.67%
450.soplex (max) (percentage) 1.270 -77.08%
450.soplex (unexplained diff vs previous) 179.000 1391.67%
450.soplex (unexplained diff vs previous) (percentage) 1.290 -41.63%
462.libquantum 18.000 800.00%
462.libquantum (avg) 18.000 800.00%
462.libquantum (avg) (percentage) 0.160 -93.10%
462.libquantum (max) 18.000 350.00%
462.libquantum (max) (percentage) 0.160 -96.44%
462.libquantum (unexplained diff vs previous) 19.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.160 -
470.lbm 53.000 65.62%
470.lbm (avg) 53.000 65.62%
470.lbm (avg) (percentage) 0.420 -58.42%
470.lbm (max) 1001.000 1271.23%
470.lbm (max) (percentage) 7.810 241.05%
470.lbm (unexplained diff vs previous) 53.000 211.76%
470.lbm (unexplained diff vs previous) (percentage) 0.410 -22.64%
471.omnetpp 17.000 -69.64%
471.omnetpp (avg) 17.000 -69.64%
471.omnetpp (avg) (percentage) 0.130 -92.82%
471.omnetpp (max) 17.000 -86.82%
471.omnetpp (max) (percentage) 0.130 -96.86%
471.omnetpp (unexplained diff vs previous) 17.000 142.86%
471.omnetpp (unexplained diff vs previous) (percentage) 0.130 -40.91%
482.sphinx3 59.000 321.43%
482.sphinx3 (avg) 59.000 321.43%
482.sphinx3 (avg) (percentage) 0.220 -87.78%
482.sphinx3 (max) 59.000 555.56%
482.sphinx3 (max) (percentage) 0.220 -81.51%
482.sphinx3 (unexplained diff vs previous) 59.000 2850.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.220 -8.33%
Geometric Mean 11.900 216.52%