Fields
hardware | None |
os | None |
Parameters
Fields
Name | Value |
---|
Parameters
Name | Value |
---|---|
git_llvm | https://github.com/llvm/llvm-project/commit/2869d31a7ac1e4748d171711614e9a89dfc0f75a (+ 197 commits) |
test_url | https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-aarch64-master-Oz_LTO-build/8/ |
Filter
llvm-aarch64-master-Oz_LTO test results
Run | Order | Start Time | Duration |
---|---|---|---|
Current | 0008 (llvmorg-17-init-04625-g2869d31a7ac1) | 2023-03-10T03:14:23 | 0:00:00 |
Previous | 0007 (llvmorg-17-init-04428-gcdd914a95952) | 2023-03-06T22:41:47 | 0:00:00 |
Baseline | 0001 (llvmorg-16-init-17110-gb1256f2345f6) | 2022-12-17T20:51:03 | 0:00:00 |
Tests Summary
Status Group | # | # (B) |
---|---|---|
Performance Regressions | 20 | 52 |
Performance Improvements | 66 | 63 |
Added Tests | 19 | 23 |
Unchanged Tests | 711 | 678 |
Total Tests | 816 |
Performance Regressions - Execution | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
433.milc | 3.32% | 1174.000 | 1213.000 | - | -91.69% | - |
403.gcc | 2.90% | 69.000 | 71.000 | - | -99.30% | - |
429.mcf | 1.99% | 1610.000 | 1642.000 | - | -91.31% | - |
400.perlbench | 1.78% | 1796.000 | 1828.000 | - | -83.51% | - |
450.soplex | 1.46% | 481.000 | 488.000 | - | -96.51% | - |
453.povray | 1.15% | 521.000 | 527.000 | - | -92.78% | - |
Performance Regressions - sample variability | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
473.astar (unexplained diff vs previous) (percentage) | 623.08% | 0.260 | 1.880 | - | 0.00% | - |
473.astar (unexplained diff vs previous) | 615.38% | 13.000 | 93.000 | - | 0.00% | - |
433.milc (max) | 185.71% | 14.000 | 40.000 | - | -85.13% | - |
433.milc (max) (percentage) | 172.13% | 1.220 | 3.320 | - | 79.46% | - |
433.milc | 66.67% | 6.000 | 10.000 | - | -91.74% | - |
433.milc (avg) | 66.67% | 6.000 | 10.000 | - | -91.74% | - |
433.milc (avg) (percentage) | 50.88% | 0.570 | 0.860 | - | 3.61% | - |
453.povray | 25.00% | 4.000 | 5.000 | - | -90.57% | - |
453.povray (avg) | 25.00% | 4.000 | 5.000 | - | -90.57% | - |
453.povray (avg) (percentage) | 3.23% | 0.930 | 0.960 | - | 31.51% | - |
470.lbm (avg) (percentage) | 1.44% | 4.860 | 4.930 | - | 750.00% | - |
429.mcf | 1.20% | 83.000 | 84.000 | - | 21.74% | - |
429.mcf (avg) | 1.20% | 83.000 | 84.000 | - | 21.74% | - |
473.astar (avg) (percentage) | 1.18% | 1.690 | 1.710 | - | 2342.86% | - |
Performance Improvements - Execution | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
470.lbm | -5.55% | 3300.000 | 3117.000 | - | -71.78% | - |
482.sphinx3 | -2.95% | 814.000 | 790.000 | - | -96.68% | - |
473.astar | -1.84% | 5050.000 | 4957.000 | - | -62.73% | - |
Performance Improvements - sample variability | Δ | Previous | Current | σ | Δ (B) | σ (B) |
---|---|---|---|---|---|---|
401.bzip2 (unexplained diff vs previous) (percentage) | -96.91% | 11.340 | 0.350 | - | 0.00% | - |
401.bzip2 (unexplained diff vs previous) | -96.86% | 287.000 | 9.000 | - | 0.00% | - |
471.omnetpp (unexplained diff vs previous) (percentage) | -94.74% | 0.570 | 0.030 | - | -97.03% | - |
471.omnetpp (unexplained diff vs previous) | -94.12% | 17.000 | 1.000 | - | -99.17% | - |
450.soplex (unexplained diff vs previous) (percentage) | -88.53% | 12.470 | 1.430 | - | 0.00% | - |
450.soplex (unexplained diff vs previous) | -88.33% | 60.000 | 7.000 | - | 0.00% | - |
444.namd (unexplained diff vs previous) | -66.67% | 3.000 | 1.000 | - | 0.00% | - |
444.namd (unexplained diff vs previous) (percentage) | -65.91% | 0.440 | 0.150 | - | 0.00% | - |
450.soplex (max) (percentage) | -56.13% | 21.060 | 9.240 | - | 740.00% | - |
462.libquantum (max) | -55.56% | 9.000 | 4.000 | - | 33.33% | - |
450.soplex (max) | -55.45% | 101.000 | 45.000 | - | -70.59% | - |
462.libquantum (max) (percentage) | -51.15% | 7.410 | 3.620 | - | 8950.00% | - |
444.namd (max) | -43.48% | 23.000 | 13.000 | - | -97.32% | - |
444.namd (max) (percentage) | -43.02% | 3.440 | 1.960 | - | -62.16% | - |
482.sphinx3 (max) | -23.71% | 232.000 | 177.000 | - | 7.93% | - |
482.sphinx3 (max) (percentage) | -21.59% | 28.580 | 22.410 | - | 3147.83% | - |
401.bzip2 (max) (percentage) | -16.68% | 10.190 | 8.490 | - | 16880.00% | - |
401.bzip2 (max) | -16.34% | 257.000 | 215.000 | - | 3483.33% | - |
456.hmmer | -10.71% | 28.000 | 25.000 | - | -68.75% | - |
456.hmmer (avg) | -10.71% | 28.000 | 25.000 | - | -68.75% | - |
471.omnetpp | -10.53% | 19.000 | 17.000 | - | -76.06% | - |
471.omnetpp (avg) | -10.53% | 19.000 | 17.000 | - | -76.06% | - |
447.dealII | -9.52% | 21.000 | 19.000 | - | -76.83% | - |
447.dealII (avg) | -9.52% | 21.000 | 19.000 | - | -76.83% | - |
470.lbm (max) | -9.35% | 727.000 | 659.000 | - | 10883.33% | - |
401.bzip2 | -9.09% | 66.000 | 60.000 | - | 361.54% | - |
401.bzip2 (avg) | -9.09% | 66.000 | 60.000 | - | 361.54% | - |
444.namd | -9.09% | 11.000 | 10.000 | - | -95.31% | - |
444.namd (avg) | -9.09% | 11.000 | 10.000 | - | -95.31% | - |
456.hmmer (avg) (percentage) | -8.91% | 1.010 | 0.920 | - | 4.55% | - |
447.dealII (avg) (percentage) | -8.77% | 1.140 | 1.040 | - | 14.29% | - |
401.bzip2 (avg) (percentage) | -8.75% | 2.630 | 2.400 | - | 2300.00% | - |
462.libquantum (avg) (percentage) | -8.47% | 1.770 | 1.620 | - | 1372.73% | - |
482.sphinx3 | -8.11% | 37.000 | 34.000 | - | -84.26% | - |
482.sphinx3 (avg) | -8.11% | 37.000 | 34.000 | - | -84.26% | - |
444.namd (avg) (percentage) | -7.93% | 1.640 | 1.510 | - | -33.77% | - |
471.omnetpp (avg) (percentage) | -7.81% | 0.640 | 0.590 | - | -1.67% | - |
473.astar (max) | -6.33% | 411.000 | 385.000 | - | 3750.00% | - |
450.soplex (avg) (percentage) | -6.05% | 5.120 | 4.810 | - | 790.74% | - |
471.omnetpp (max) (percentage) | -5.73% | 1.570 | 1.480 | - | 48.00% | - |
447.dealII (max) | -5.41% | 37.000 | 35.000 | - | -72.22% | - |
464.h264ref (max) (percentage) | -5.26% | 0.570 | 0.540 | - | 100.00% | - |
429.mcf (max) (percentage) | -4.77% | 19.930 | 18.980 | - | 5172.22% | - |
456.hmmer (max) (percentage) | -4.57% | 1.750 | 1.670 | - | 122.67% | - |
473.astar (max) (percentage) | -4.54% | 8.150 | 7.780 | - | 9625.00% | - |
447.dealII (max) (percentage) | -4.50% | 2.000 | 1.910 | - | 37.41% | - |
470.lbm | -4.38% | 160.000 | 153.000 | - | 139.06% | - |
470.lbm (avg) | -4.38% | 160.000 | 153.000 | - | 139.06% | - |
471.omnetpp (max) | -4.35% | 46.000 | 44.000 | - | -63.03% | - |
450.soplex | -4.17% | 24.000 | 23.000 | - | -69.33% | - |
450.soplex (avg) | -4.17% | 24.000 | 23.000 | - | -69.33% | - |
456.hmmer (max) | -4.17% | 48.000 | 46.000 | - | -32.35% | - |
482.sphinx3 (avg) (percentage) | -4.16% | 4.570 | 4.380 | - | 381.32% | - |
464.h264ref (max) | -4.00% | 25.000 | 24.000 | - | -25.00% | - |
470.lbm (max) (percentage) | -3.99% | 22.040 | 21.160 | - | 35166.67% | - |
458.sjeng (max) | -3.85% | 156.000 | 150.000 | - | 328.57% | - |
453.povray (max) (percentage) | -3.64% | 1.650 | 1.590 | - | 50.00% | - |
458.sjeng (max) (percentage) | -3.29% | 2.130 | 2.060 | - | 795.65% | - |
429.mcf (max) | -2.81% | 320.000 | 311.000 | - | 357.35% | - |
445.gobmk (max) (percentage) | -2.20% | 0.910 | 0.890 | - | 48.33% | - |
445.gobmk (max) | -2.13% | 47.000 | 46.000 | - | -34.29% | - |
473.astar | -1.18% | 85.000 | 84.000 | - | 833.33% | - |
473.astar (avg) | -1.18% | 85.000 | 84.000 | - | 833.33% | - |
Added Tests - sample variability |
---|
403.gcc |
403.gcc (avg) |
403.gcc (max) |
433.milc (unexplained diff vs previous) |
433.milc (unexplained diff vs previous) (percentage) |
447.dealII (unexplained diff vs previous) |
447.dealII (unexplained diff vs previous) (percentage) |
453.povray (unexplained diff vs previous) |
453.povray (unexplained diff vs previous) (percentage) |
456.hmmer (unexplained diff vs previous) |
456.hmmer (unexplained diff vs previous) (percentage) |
462.libquantum (unexplained diff vs previous) |
462.libquantum (unexplained diff vs previous) (percentage) |
464.h264ref (unexplained diff vs previous) |
464.h264ref (unexplained diff vs previous) (percentage) |
470.lbm (unexplained diff vs previous) |
470.lbm (unexplained diff vs previous) (percentage) |
482.sphinx3 (unexplained diff vs previous) |
482.sphinx3 (unexplained diff vs previous) (percentage) |
Performance Regressions - Code Size | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
450.soplex | 3.34% | 224407.000 | 231906.000 | - | 0.00% | - |
447.dealII | 1.15% | 325219.000 | 328955.000 | - | 0.00% | - |
Performance Regressions - sample variability | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
470.lbm (max) (percentage) | 35166.67% | 0.060 | 21.160 | - | -3.99% | - |
401.bzip2 (max) (percentage) | 16880.00% | 0.050 | 8.490 | - | -16.68% | - |
433.milc (unexplained diff vs previous) (percentage) | 16000.00% | 0.020 | 3.220 | - | 0.00% | - |
470.lbm (max) | 10883.33% | 6.000 | 659.000 | - | -9.35% | - |
473.astar (max) (percentage) | 9625.00% | 0.080 | 7.780 | - | -4.54% | - |
462.libquantum (max) (percentage) | 8950.00% | 0.040 | 3.620 | - | -51.15% | - |
429.mcf (max) (percentage) | 5172.22% | 0.360 | 18.980 | - | -4.77% | - |
473.astar (max) | 3750.00% | 10.000 | 385.000 | - | -6.33% | - |
401.bzip2 (max) | 3483.33% | 6.000 | 215.000 | - | -16.34% | - |
482.sphinx3 (max) (percentage) | 3147.83% | 0.690 | 22.410 | - | -21.59% | - |
473.astar (avg) (percentage) | 2342.86% | 0.070 | 1.710 | - | 1.18% | - |
401.bzip2 (avg) (percentage) | 2300.00% | 0.100 | 2.400 | - | -8.75% | - |
462.libquantum (avg) (percentage) | 1372.73% | 0.110 | 1.620 | - | -8.47% | - |
429.mcf (avg) (percentage) | 1294.59% | 0.370 | 5.160 | - | 0.00% | - |
433.milc (unexplained diff vs previous) | 1200.00% | 3.000 | 39.000 | - | 0.00% | - |
473.astar | 833.33% | 9.000 | 84.000 | - | -1.18% | - |
473.astar (avg) | 833.33% | 9.000 | 84.000 | - | -1.18% | - |
458.sjeng (max) (percentage) | 795.65% | 0.230 | 2.060 | - | -3.29% | - |
450.soplex (avg) (percentage) | 790.74% | 0.540 | 4.810 | - | -6.05% | - |
470.lbm (avg) (percentage) | 750.00% | 0.580 | 4.930 | - | 1.44% | - |
450.soplex (max) (percentage) | 740.00% | 1.100 | 9.240 | - | -56.13% | - |
482.sphinx3 (avg) (percentage) | 381.32% | 0.910 | 4.380 | - | -4.16% | - |
401.bzip2 | 361.54% | 13.000 | 60.000 | - | -9.09% | - |
401.bzip2 (avg) | 361.54% | 13.000 | 60.000 | - | -9.09% | - |
429.mcf (max) | 357.35% | 68.000 | 311.000 | - | -2.81% | - |
482.sphinx3 (unexplained diff vs previous) (percentage) | 347.06% | 0.680 | 3.040 | - | 0.00% | - |
458.sjeng (max) | 328.57% | 35.000 | 150.000 | - | -3.85% | - |
458.sjeng (avg) (percentage) | 147.83% | 0.230 | 0.570 | - | 0.00% | - |
470.lbm | 139.06% | 64.000 | 153.000 | - | -4.38% | - |
470.lbm (avg) | 139.06% | 64.000 | 153.000 | - | -4.38% | - |
456.hmmer (max) (percentage) | 122.67% | 0.750 | 1.670 | - | -4.57% | - |
464.h264ref (max) (percentage) | 100.00% | 0.270 | 0.540 | - | -5.26% | - |
433.milc (max) (percentage) | 79.46% | 1.850 | 3.320 | - | 172.13% | - |
453.povray (max) (percentage) | 50.00% | 1.060 | 1.590 | - | -3.64% | - |
445.gobmk (max) (percentage) | 48.33% | 0.600 | 0.890 | - | -2.20% | - |
471.omnetpp (max) (percentage) | 48.00% | 1.000 | 1.480 | - | -5.73% | - |
447.dealII (max) (percentage) | 37.41% | 1.390 | 1.910 | - | -4.50% | - |
445.gobmk (avg) (percentage) | 34.37% | 0.320 | 0.430 | - | 0.00% | - |
462.libquantum (max) | 33.33% | 3.000 | 4.000 | - | -55.56% | - |
453.povray (avg) (percentage) | 31.51% | 0.730 | 0.960 | - | 3.23% | - |
429.mcf | 21.74% | 69.000 | 84.000 | - | 1.20% | - |
429.mcf (avg) | 21.74% | 69.000 | 84.000 | - | 1.20% | - |
458.sjeng | 17.14% | 35.000 | 41.000 | - | 0.00% | - |
458.sjeng (avg) | 17.14% | 35.000 | 41.000 | - | 0.00% | - |
447.dealII (avg) (percentage) | 14.29% | 0.910 | 1.040 | - | -8.77% | - |
482.sphinx3 (max) | 7.93% | 164.000 | 177.000 | - | -23.71% | - |
456.hmmer (avg) (percentage) | 4.55% | 0.880 | 0.920 | - | -8.91% | - |
433.milc (avg) (percentage) | 3.61% | 0.830 | 0.860 | - | 50.88% | - |
Performance Regressions - Score | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
nb_compile_successful | -5.26% | 19.000 | 18.000 | - | 0.00% | - |
nb_execution_successful | -5.26% | 19.000 | 18.000 | - | 0.00% | - |
Performance Improvements - Code Size | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
471.omnetpp | -2.86% | 490147.000 | 476109.000 | - | 0.00% | - |
Performance Improvements - Execution | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
403.gcc | -99.30% | 10150.000 | 71.000 | - | 2.90% | - |
462.libquantum | -98.63% | 9659.000 | 132.000 | - | 0.00% | - |
482.sphinx3 | -96.68% | 23779.000 | 790.000 | - | -2.95% | - |
450.soplex | -96.51% | 13988.000 | 488.000 | - | 1.46% | - |
453.povray | -92.78% | 7301.000 | 527.000 | - | 1.15% | - |
444.namd | -92.75% | 9377.000 | 680.000 | - | -0.15% | - |
433.milc | -91.69% | 14590.000 | 1213.000 | - | 3.32% | - |
429.mcf | -91.31% | 18905.000 | 1642.000 | - | 1.99% | - |
400.perlbench | -83.51% | 11085.000 | 1828.000 | - | 1.78% | - |
401.bzip2 | -81.81% | 13962.000 | 2539.000 | - | 0.36% | - |
447.dealII | -79.51% | 9102.000 | 1865.000 | - | 0.38% | - |
471.omnetpp | -74.95% | 11917.000 | 2985.000 | - | 0.03% | - |
470.lbm | -71.78% | 11046.000 | 3117.000 | - | -5.55% | - |
456.hmmer | -69.45% | 9122.000 | 2787.000 | - | 0.07% | - |
464.h264ref | -62.99% | 12142.000 | 4494.000 | - | -0.42% | - |
473.astar | -62.73% | 13301.000 | 4957.000 | - | -1.84% | - |
445.gobmk | -54.96% | 11668.000 | 5255.000 | - | -0.11% | - |
458.sjeng | -52.24% | 15282.000 | 7298.000 | - | -0.83% | - |
Performance Improvements - sample variability | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
471.omnetpp (unexplained diff vs previous) | -99.17% | 120.000 | 1.000 | - | -94.12% | - |
444.namd (max) | -97.32% | 485.000 | 13.000 | - | -43.48% | - |
471.omnetpp (unexplained diff vs previous) (percentage) | -97.03% | 1.010 | 0.030 | - | -94.74% | - |
444.namd | -95.31% | 213.000 | 10.000 | - | -9.09% | - |
444.namd (avg) | -95.31% | 213.000 | 10.000 | - | -9.09% | - |
433.milc | -91.74% | 121.000 | 10.000 | - | 66.67% | - |
433.milc (avg) | -91.74% | 121.000 | 10.000 | - | 66.67% | - |
453.povray | -90.57% | 53.000 | 5.000 | - | 25.00% | - |
453.povray (avg) | -90.57% | 53.000 | 5.000 | - | 25.00% | - |
453.povray (max) | -89.61% | 77.000 | 8.000 | - | 0.00% | - |
400.perlbench (max) | -87.01% | 77.000 | 10.000 | - | 0.00% | - |
482.sphinx3 (unexplained diff vs previous) | -85.19% | 162.000 | 24.000 | - | 0.00% | - |
433.milc (max) | -85.13% | 269.000 | 40.000 | - | 185.71% | - |
482.sphinx3 | -84.26% | 216.000 | 34.000 | - | -8.11% | - |
482.sphinx3 (avg) | -84.26% | 216.000 | 34.000 | - | -8.11% | - |
400.perlbench | -83.93% | 56.000 | 9.000 | - | 0.00% | - |
400.perlbench (avg) | -83.93% | 56.000 | 9.000 | - | 0.00% | - |
462.libquantum | -80.00% | 10.000 | 2.000 | - | 0.00% | - |
462.libquantum (avg) | -80.00% | 10.000 | 2.000 | - | 0.00% | - |
447.dealII | -76.83% | 82.000 | 19.000 | - | -9.52% | - |
447.dealII (avg) | -76.83% | 82.000 | 19.000 | - | -9.52% | - |
471.omnetpp | -76.06% | 71.000 | 17.000 | - | -10.53% | - |
471.omnetpp (avg) | -76.06% | 71.000 | 17.000 | - | -10.53% | - |
447.dealII (max) | -72.22% | 126.000 | 35.000 | - | -5.41% | - |
450.soplex (max) | -70.59% | 153.000 | 45.000 | - | -55.45% | - |
450.soplex | -69.33% | 75.000 | 23.000 | - | -4.17% | - |
450.soplex (avg) | -69.33% | 75.000 | 23.000 | - | -4.17% | - |
456.hmmer | -68.75% | 80.000 | 25.000 | - | -10.71% | - |
456.hmmer (avg) | -68.75% | 80.000 | 25.000 | - | -10.71% | - |
464.h264ref | -63.64% | 44.000 | 16.000 | - | 0.00% | - |
464.h264ref (avg) | -63.64% | 44.000 | 16.000 | - | 0.00% | - |
471.omnetpp (max) | -63.03% | 119.000 | 44.000 | - | -4.35% | - |
444.namd (max) (percentage) | -62.16% | 5.180 | 1.960 | - | -43.02% | - |
445.gobmk | -40.54% | 37.000 | 22.000 | - | 0.00% | - |
445.gobmk (avg) | -40.54% | 37.000 | 22.000 | - | 0.00% | - |
445.gobmk (max) | -34.29% | 70.000 | 46.000 | - | -2.13% | - |
444.namd (avg) (percentage) | -33.77% | 2.280 | 1.510 | - | -7.93% | - |
456.hmmer (max) | -32.35% | 68.000 | 46.000 | - | -4.17% | - |
464.h264ref (max) | -25.00% | 32.000 | 24.000 | - | -4.00% | - |
400.perlbench (max) (percentage) | -20.00% | 0.700 | 0.560 | - | -1.75% | - |
483.xalancbmk (max) (percentage) | -17.65% | 0.510 | 0.420 | - | 0.00% | - |
403.gcc (max) (percentage) | -15.00% | 0.400 | 0.340 | - | 0.00% | - |
471.omnetpp (avg) (percentage) | -1.67% | 0.600 | 0.590 | - | -7.81% | - |
Performance Improvements - Score | Δ (B) | Baseline | Current | σ (B) | Δ | σ |
---|---|---|---|---|---|---|
nb_total_tests | 52.63% | 19.000 | 29.000 | - | 0.00% | - |
Added Tests - sample variability |
---|
401.bzip2 (unexplained diff vs previous) |
401.bzip2 (unexplained diff vs previous) (percentage) |
403.gcc |
403.gcc (avg) |
403.gcc (max) |
444.namd (unexplained diff vs previous) |
444.namd (unexplained diff vs previous) (percentage) |
447.dealII (unexplained diff vs previous) |
447.dealII (unexplained diff vs previous) (percentage) |
450.soplex (unexplained diff vs previous) |
450.soplex (unexplained diff vs previous) (percentage) |
453.povray (unexplained diff vs previous) |
453.povray (unexplained diff vs previous) (percentage) |
456.hmmer (unexplained diff vs previous) |
456.hmmer (unexplained diff vs previous) (percentage) |
462.libquantum (unexplained diff vs previous) |
462.libquantum (unexplained diff vs previous) (percentage) |
464.h264ref (unexplained diff vs previous) |
464.h264ref (unexplained diff vs previous) (percentage) |
470.lbm (unexplained diff vs previous) |
470.lbm (unexplained diff vs previous) (percentage) |
473.astar (unexplained diff vs previous) |
473.astar (unexplained diff vs previous) (percentage) |
Report Time: 0.36s