Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/bce910242e7269446683cfcaffd3ebf5281a9125 (+ 519 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6-llvm-aarch64-master-Oz_LTO-build/5/
View Options
Report
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0005 (llvmorg-17-init-00152-gbce910242e72) 2023-01-21T08:05:02 0:00:00
Previous 0004 (llvmorg-17-init-03503-g5c943e368818) 2023-02-23T08:27:23 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 61 23
Performance Improvements 48 59
Added Tests 5 2
Unchanged Tests 642 672
Total Tests 756

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 2.95% 476109.000 490175.000 - 0.01% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 12136.90% 84.000 10279.000 - 1.27% -
462.libquantum 6468.03% 147.000 9655.000 - -0.04% -
482.sphinx3 2089.66% 1093.000 23933.000 - 0.65% -
450.soplex 1989.51% 667.000 13937.000 - -0.36% -
453.povray 1303.80% 527.000 7398.000 - 1.33% -
444.namd 1208.68% 714.000 9344.000 - -0.35% -
433.milc 765.40% 1685.000 14582.000 - -0.05% -
429.mcf 680.87% 2420.000 18897.000 - -0.04% -
400.perlbench 478.45% 1926.000 11141.000 - 0.51% -
447.dealII 371.47% 1914.000 9024.000 - -0.86% -
401.bzip2 345.78% 3115.000 13886.000 - -0.54% -
471.omnetpp 293.18% 3050.000 11992.000 - 0.63% -
456.hmmer 218.91% 2851.000 9092.000 - -0.33% -
470.lbm 174.48% 4040.000 11089.000 - 0.39% -
464.h264ref 157.22% 4663.000 11994.000 - -1.22% -
473.astar 139.99% 5554.000 13329.000 - 0.21% -
445.gobmk 114.54% 5453.000 11699.000 - 0.27% -
458.sjeng 103.26% 7537.000 15320.000 - 0.25% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum 1800.00% 1.000 19.000 - 90.00% -
462.libquantum (avg) 1800.00% 1.000 19.000 - 90.00% -
453.povray 1250.00% 4.000 54.000 - 1.89% -
453.povray (avg) 1250.00% 4.000 54.000 - 1.89% -
444.namd 1083.33% 12.000 142.000 - -33.33% -
444.namd (avg) 1083.33% 12.000 142.000 - -33.33% -
433.milc 975.00% 8.000 86.000 - -28.93% -
433.milc (avg) 975.00% 8.000 86.000 - -28.93% -
453.povray (max) 655.56% 9.000 68.000 - -11.69% -
400.perlbench (max) 545.45% 11.000 71.000 - -7.79% -
400.perlbench 522.22% 9.000 56.000 - 0.00% -
400.perlbench (avg) 522.22% 9.000 56.000 - 0.00% -
433.milc (max) 400.00% 2.000 10.000 - -96.28% -
471.omnetpp 321.43% 14.000 59.000 - -16.90% -
471.omnetpp (avg) 321.43% 14.000 59.000 - -16.90% -
471.omnetpp (max) 321.05% 19.000 80.000 - -32.77% -
482.sphinx3 314.29% 49.000 203.000 - -6.02% -
482.sphinx3 (avg) 314.29% 49.000 203.000 - -6.02% -
447.dealII 261.90% 21.000 76.000 - -7.32% -
447.dealII (avg) 261.90% 21.000 76.000 - -7.32% -
464.h264ref (max) 250.00% 2.000 7.000 - -78.12% -
462.libquantum (max) 238.46% 13.000 44.000 - 1366.67% -
456.hmmer 224.14% 29.000 94.000 - 17.50% -
456.hmmer (avg) 224.14% 29.000 94.000 - 17.50% -
450.soplex 213.64% 22.000 69.000 - -8.00% -
450.soplex (avg) 213.64% 22.000 69.000 - -8.00% -
464.h264ref 171.43% 14.000 38.000 - -13.64% -
464.h264ref (avg) 171.43% 14.000 38.000 - -13.64% -
445.gobmk 164.29% 14.000 37.000 - 0.00% -
445.gobmk (avg) 164.29% 14.000 37.000 - 0.00% -
445.gobmk (max) 130.77% 26.000 60.000 - -14.29% -
456.hmmer (max) 96.72% 61.000 120.000 - 76.47% -
445.gobmk (avg) (percentage) 23.08% 0.260 0.320 - 0.00% -
433.milc (avg) (percentage) 20.41% 0.490 0.590 - -28.92% -
447.dealII (max) 17.02% 47.000 55.000 - -56.35% -
400.perlbench (max) (percentage) 8.47% 0.590 0.640 - -8.57% -
445.gobmk (max) (percentage) 8.33% 0.480 0.520 - -13.33% -
483.xalancbmk (max) (percentage) 6.98% 0.430 0.460 - -9.80% -
471.omnetpp (max) (percentage) 6.35% 0.630 0.670 - -33.00% -
403.gcc (max) (percentage) 5.71% 0.350 0.370 - -7.50% -
471.omnetpp (avg) (percentage) 4.17% 0.480 0.500 - -16.67% -
456.hmmer (avg) (percentage) 1.96% 1.020 1.040 - 18.18% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
450.soplex -3.22% 231906.000 224443.000 - 0.02% -
447.dealII -1.14% 328959.000 325203.000 - -0.00% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) (percentage) -99.38% 20.990 0.130 - 0.00% -
401.bzip2 (max) (percentage) -98.97% 11.690 0.120 - 140.00% -
473.astar (max) (percentage) -98.94% 10.370 0.110 - 37.50% -
429.mcf (max) (percentage) -98.83% 26.570 0.310 - -13.89% -
444.namd (max) (percentage) -98.58% 4.230 0.060 - -98.84% -
482.sphinx3 (max) (percentage) -98.54% 33.620 0.490 - -28.99% -
450.soplex (max) (percentage) -98.32% 26.810 0.450 - -59.09% -
473.astar (max) -97.57% 575.000 14.000 - 40.00% -
470.lbm (max) (percentage) -96.98% 26.450 0.800 - 1233.33% -
401.bzip2 (max) -95.60% 364.000 16.000 - 166.67% -
429.mcf (unexplained diff vs previous) -95.08% 508.000 25.000 - 0.00% -
462.libquantum (max) (percentage) -94.83% 8.890 0.460 - 1050.00% -
473.astar (avg) (percentage) -94.12% 1.190 0.070 - 0.00% -
401.bzip2 (avg) (percentage) -93.94% 1.650 0.100 - 0.00% -
470.lbm (max) -91.76% 1068.000 88.000 - 1366.67% -
429.mcf (max) -90.97% 642.000 58.000 - -14.71% -
429.mcf (avg) (percentage) -90.81% 2.830 0.260 - -29.73% -
458.sjeng (max) (percentage) -90.76% 2.380 0.220 - -4.35% -
473.astar -86.36% 66.000 9.000 - 0.00% -
473.astar (avg) -86.36% 66.000 9.000 - 0.00% -
450.soplex (avg) (percentage) -84.94% 3.320 0.500 - -7.41% -
444.namd (max) -83.33% 30.000 5.000 - -98.97% -
462.libquantum (avg) (percentage) -82.91% 1.170 0.200 - 81.82% -
458.sjeng (max) -81.56% 179.000 33.000 - -5.71% -
482.sphinx3 (avg) (percentage) -81.40% 4.570 0.850 - -6.59% -
470.lbm (avg) (percentage) -81.23% 3.570 0.670 - 15.52% -
447.dealII (max) (percentage) -74.80% 2.460 0.620 - -55.40% -
401.bzip2 -74.51% 51.000 13.000 - 0.00% -
401.bzip2 (avg) -74.51% 51.000 13.000 - 0.00% -
482.sphinx3 (max) -68.12% 367.000 117.000 - -28.66% -
450.soplex (max) -65.17% 178.000 62.000 - -59.48% -
458.sjeng (avg) (percentage) -59.09% 0.440 0.180 - -21.74% -
433.milc (max) (percentage) -53.33% 0.150 0.070 - -96.22% -
453.povray (max) (percentage) -50.54% 1.860 0.920 - -13.21% -
470.lbm -48.61% 144.000 74.000 - 15.62% -
470.lbm (avg) -48.61% 144.000 74.000 - 15.62% -
456.hmmer (max) (percentage) -38.60% 2.150 1.320 - 76.00% -
429.mcf -27.94% 68.000 49.000 - -28.99% -
429.mcf (avg) -27.94% 68.000 49.000 - -28.99% -
447.dealII (avg) (percentage) -23.42% 1.110 0.850 - -6.59% -
458.sjeng -18.18% 33.000 27.000 - -22.86% -
458.sjeng (avg) -18.18% 33.000 27.000 - -22.86% -
453.povray (avg) (percentage) -15.12% 0.860 0.730 - 0.00% -
444.namd (avg) (percentage) -9.47% 1.690 1.530 - -32.89% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful 5.56% 18.000 19.000 - 0.00% -
nb_execution_successful 5.56% 18.000 19.000 - 0.00% -

Added Tests - Code Size
483.xalancbmk

Added Tests - Execution
483.xalancbmk

Added Tests - sample variability
483.xalancbmk
483.xalancbmk (avg)
483.xalancbmk (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
453.povray 1.33% 7301.000 7398.000 - 1303.80% -
403.gcc 1.27% 10150.000 10279.000 - 12136.90% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) 1366.67% 3.000 44.000 - 238.46% -
470.lbm (max) 1366.67% 6.000 88.000 - -91.76% -
470.lbm (max) (percentage) 1233.33% 0.060 0.800 - -96.98% -
462.libquantum (max) (percentage) 1050.00% 0.040 0.460 - -94.83% -
401.bzip2 (max) 166.67% 6.000 16.000 - -95.60% -
401.bzip2 (max) (percentage) 140.00% 0.050 0.120 - -98.97% -
462.libquantum 90.00% 10.000 19.000 - 1800.00% -
462.libquantum (avg) 90.00% 10.000 19.000 - 1800.00% -
462.libquantum (avg) (percentage) 81.82% 0.110 0.200 - -82.91% -
456.hmmer (max) 76.47% 68.000 120.000 - 96.72% -
456.hmmer (max) (percentage) 76.00% 0.750 1.320 - -38.60% -
473.astar (max) 40.00% 10.000 14.000 - -97.57% -
473.astar (max) (percentage) 37.50% 0.080 0.110 - -98.94% -
456.hmmer (avg) (percentage) 18.18% 0.880 1.040 - 1.96% -
456.hmmer 17.50% 80.000 94.000 - 224.14% -
456.hmmer (avg) 17.50% 80.000 94.000 - 224.14% -
470.lbm 15.62% 64.000 74.000 - -48.61% -
470.lbm (avg) 15.62% 64.000 74.000 - -48.61% -
470.lbm (avg) (percentage) 15.52% 0.580 0.670 - -81.23% -
453.povray 1.89% 53.000 54.000 - 1250.00% -
453.povray (avg) 1.89% 53.000 54.000 - 1250.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk -1.76% 9633.000 9463.000 - 0.00% -
464.h264ref -1.22% 12142.000 11994.000 - 157.22% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -98.97% 485.000 5.000 - -83.33% -
444.namd (max) (percentage) -98.84% 5.180 0.060 - -98.58% -
433.milc (max) -96.28% 269.000 10.000 - 400.00% -
433.milc (max) (percentage) -96.22% 1.850 0.070 - -53.33% -
464.h264ref (max) -78.12% 32.000 7.000 - 250.00% -
464.h264ref (max) (percentage) -77.78% 0.270 0.060 - 20.00% -
450.soplex (max) -59.48% 153.000 62.000 - -65.17% -
450.soplex (max) (percentage) -59.09% 1.100 0.450 - -98.32% -
447.dealII (max) -56.35% 126.000 55.000 - 17.02% -
447.dealII (max) (percentage) -55.40% 1.390 0.620 - -74.80% -
444.namd -33.33% 213.000 142.000 - 1083.33% -
444.namd (avg) -33.33% 213.000 142.000 - 1083.33% -
471.omnetpp (max) (percentage) -33.00% 1.000 0.670 - 6.35% -
444.namd (avg) (percentage) -32.89% 2.280 1.530 - -9.47% -
471.omnetpp (max) -32.77% 119.000 80.000 - 321.05% -
429.mcf (avg) (percentage) -29.73% 0.370 0.260 - -90.81% -
429.mcf -28.99% 69.000 49.000 - -27.94% -
429.mcf (avg) -28.99% 69.000 49.000 - -27.94% -
482.sphinx3 (max) (percentage) -28.99% 0.690 0.490 - -98.54% -
433.milc -28.93% 121.000 86.000 - 975.00% -
433.milc (avg) -28.93% 121.000 86.000 - 975.00% -
433.milc (avg) (percentage) -28.92% 0.830 0.590 - 20.41% -
482.sphinx3 (max) -28.66% 164.000 117.000 - -68.12% -
458.sjeng -22.86% 35.000 27.000 - -18.18% -
458.sjeng (avg) -22.86% 35.000 27.000 - -18.18% -
458.sjeng (avg) (percentage) -21.74% 0.230 0.180 - -59.09% -
471.omnetpp -16.90% 71.000 59.000 - 321.43% -
471.omnetpp (avg) -16.90% 71.000 59.000 - 321.43% -
471.omnetpp (avg) (percentage) -16.67% 0.600 0.500 - 4.17% -
429.mcf (max) -14.71% 68.000 58.000 - -90.97% -
445.gobmk (max) -14.29% 70.000 60.000 - 130.77% -
429.mcf (max) (percentage) -13.89% 0.360 0.310 - -98.83% -
464.h264ref -13.64% 44.000 38.000 - 171.43% -
464.h264ref (avg) -13.64% 44.000 38.000 - 171.43% -
464.h264ref (avg) (percentage) -13.51% 0.370 0.320 - 0.00% -
445.gobmk (max) (percentage) -13.33% 0.600 0.520 - 8.33% -
453.povray (max) (percentage) -13.21% 1.060 0.920 - -50.54% -
483.xalancbmk (max) -12.24% 49.000 43.000 - 0.00% -
453.povray (max) -11.69% 77.000 68.000 - 655.56% -
483.xalancbmk (max) (percentage) -9.80% 0.510 0.460 - 6.98% -
400.perlbench (max) (percentage) -8.57% 0.700 0.640 - 8.47% -
450.soplex -8.00% 75.000 69.000 - 213.64% -
450.soplex (avg) -8.00% 75.000 69.000 - 213.64% -
400.perlbench (max) -7.79% 77.000 71.000 - 545.45% -
403.gcc (max) (percentage) -7.50% 0.400 0.370 - 5.71% -
450.soplex (avg) (percentage) -7.41% 0.540 0.500 - -84.94% -
447.dealII -7.32% 82.000 76.000 - 261.90% -
447.dealII (avg) -7.32% 82.000 76.000 - 261.90% -
447.dealII (avg) (percentage) -6.59% 0.910 0.850 - -23.42% -
482.sphinx3 (avg) (percentage) -6.59% 0.910 0.850 - -81.40% -
482.sphinx3 -6.02% 216.000 203.000 - 314.29% -
482.sphinx3 (avg) -6.02% 216.000 203.000 - 314.29% -
458.sjeng (max) -5.71% 35.000 33.000 - -81.56% -
403.gcc (max) -5.00% 40.000 38.000 - 0.00% -
458.sjeng (max) (percentage) -4.35% 0.230 0.220 - -90.76% -
483.xalancbmk -2.44% 41.000 40.000 - 0.00% -
483.xalancbmk (avg) -2.44% 41.000 40.000 - 0.00% -

Added Tests - sample variability
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)

Report Time: 0.41s
Execution (seconds)
Name Current %
400.perlbench 11141.000 478.45%
401.bzip2 13886.000 345.78%
403.gcc 10279.000 12136.90%
429.mcf 18897.000 680.87%
433.milc 14582.000 765.40%
444.namd 9344.000 1208.68%
445.gobmk 11699.000 114.54%
447.dealII 9024.000 371.47%
450.soplex 13937.000 1989.51%
453.povray 7398.000 1303.80%
456.hmmer 9092.000 218.91%
458.sjeng 15320.000 103.26%
462.libquantum 9655.000 6468.03%
464.h264ref 11994.000 157.22%
470.lbm 11089.000 174.48%
471.omnetpp 11992.000 293.18%
473.astar 13329.000 139.99%
482.sphinx3 23933.000 2089.66%
483.xalancbmk 9463.000 -
Geometric Mean 11935.929 636.61%
Code Size (bytes)
Name Current %
400.perlbench 900599.000 -
401.bzip2 44317.000 -
403.gcc 2487781.000 -
429.mcf 9249.000 -
433.milc 62747.000 -
444.namd 143241.000 -
445.gobmk 1851043.000 -
447.dealII 325203.000 -1.14%
450.soplex 224443.000 -3.22%
453.povray 691350.000 -
456.hmmer 113629.000 -
458.sjeng 106896.000 -
462.libquantum 16965.000 -
464.h264ref 400034.000 -
470.lbm 10353.000 -
471.omnetpp 490175.000 2.95%
473.astar 23721.000 -
482.sphinx3 120138.000 -
483.xalancbmk 2538731.000 -
Geometric Mean 169783.065 16.17%
Score
Name Current %
nb_compile_successful 19.000 5.56%
nb_execution_successful 19.000 5.56%
nb_total_tests 19.000 -
Geometric Mean 19.000 3.67%
sample variability
Name Current %
400.perlbench 56.000 522.22%
400.perlbench (avg) 56.000 522.22%
400.perlbench (avg) (percentage) 0.510 -
400.perlbench (max) 71.000 545.45%
400.perlbench (max) (percentage) 0.640 8.47%
401.bzip2 13.000 -74.51%
401.bzip2 (avg) 13.000 -74.51%
401.bzip2 (avg) (percentage) 0.100 -93.94%
401.bzip2 (max) 16.000 -95.60%
401.bzip2 (max) (percentage) 0.120 -98.97%
403.gcc 31.000 -
403.gcc (avg) 31.000 -
403.gcc (avg) (percentage) 0.310 -
403.gcc (max) 38.000 -
403.gcc (max) (percentage) 0.370 5.71%
429.mcf 49.000 -27.94%
429.mcf (avg) 49.000 -27.94%
429.mcf (avg) (percentage) 0.260 -90.81%
429.mcf (max) 58.000 -90.97%
429.mcf (max) (percentage) 0.310 -98.83%
429.mcf (unexplained diff vs previous) 25.000 -95.08%
429.mcf (unexplained diff vs previous) (percentage) 0.130 -99.38%
433.milc 86.000 975.00%
433.milc (avg) 86.000 975.00%
433.milc (avg) (percentage) 0.590 20.41%
433.milc (max) 10.000 400.00%
433.milc (max) (percentage) 0.070 -53.33%
444.namd 142.000 1083.33%
444.namd (avg) 142.000 1083.33%
444.namd (avg) (percentage) 1.530 -9.47%
444.namd (max) 5.000 -83.33%
444.namd (max) (percentage) 0.060 -98.58%
445.gobmk 37.000 164.29%
445.gobmk (avg) 37.000 164.29%
445.gobmk (avg) (percentage) 0.320 23.08%
445.gobmk (max) 60.000 130.77%
445.gobmk (max) (percentage) 0.520 8.33%
447.dealII 76.000 261.90%
447.dealII (avg) 76.000 261.90%
447.dealII (avg) (percentage) 0.850 -23.42%
447.dealII (max) 55.000 17.02%
447.dealII (max) (percentage) 0.620 -74.80%
450.soplex 69.000 213.64%
450.soplex (avg) 69.000 213.64%
450.soplex (avg) (percentage) 0.500 -84.94%
450.soplex (max) 62.000 -65.17%
450.soplex (max) (percentage) 0.450 -98.32%
453.povray 54.000 1250.00%
453.povray (avg) 54.000 1250.00%
453.povray (avg) (percentage) 0.730 -15.12%
453.povray (max) 68.000 655.56%
453.povray (max) (percentage) 0.920 -50.54%
456.hmmer 94.000 224.14%
456.hmmer (avg) 94.000 224.14%
456.hmmer (avg) (percentage) 1.040 1.96%
456.hmmer (max) 120.000 96.72%
456.hmmer (max) (percentage) 1.320 -38.60%
458.sjeng 27.000 -18.18%
458.sjeng (avg) 27.000 -18.18%
458.sjeng (avg) (percentage) 0.180 -59.09%
458.sjeng (max) 33.000 -81.56%
458.sjeng (max) (percentage) 0.220 -90.76%
462.libquantum 19.000 1800.00%
462.libquantum (avg) 19.000 1800.00%
462.libquantum (avg) (percentage) 0.200 -82.91%
462.libquantum (max) 44.000 238.46%
462.libquantum (max) (percentage) 0.460 -94.83%
464.h264ref 38.000 171.43%
464.h264ref (avg) 38.000 171.43%
464.h264ref (avg) (percentage) 0.320 -
464.h264ref (max) 7.000 250.00%
464.h264ref (max) (percentage) 0.060 -
470.lbm 74.000 -48.61%
470.lbm (avg) 74.000 -48.61%
470.lbm (avg) (percentage) 0.670 -81.23%
470.lbm (max) 88.000 -91.76%
470.lbm (max) (percentage) 0.800 -96.98%
471.omnetpp 59.000 321.43%
471.omnetpp (avg) 59.000 321.43%
471.omnetpp (avg) (percentage) 0.500 4.17%
471.omnetpp (max) 80.000 321.05%
471.omnetpp (max) (percentage) 0.670 6.35%
473.astar 9.000 -86.36%
473.astar (avg) 9.000 -86.36%
473.astar (avg) (percentage) 0.070 -94.12%
473.astar (max) 14.000 -97.57%
473.astar (max) (percentage) 0.110 -98.94%
482.sphinx3 203.000 314.29%
482.sphinx3 (avg) 203.000 314.29%
482.sphinx3 (avg) (percentage) 0.850 -81.40%
482.sphinx3 (max) 117.000 -68.12%
482.sphinx3 (max) (percentage) 0.490 -98.54%
483.xalancbmk 40.000 -
483.xalancbmk (avg) 40.000 -
483.xalancbmk (avg) (percentage) 0.430 -
483.xalancbmk (max) 43.000 -
483.xalancbmk (max) (percentage) 0.460 6.98%
Geometric Mean 6.321 8.56%