Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=56756c571eb6d33eb8035cf0deb37e472597af6b (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/2c3db94d9fd07681f8806dae93d555779ff4dfb1 (+ 1 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=42e48e720c78ab75eb8def9e866da52b0ac278df (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=f13102b78850c48082dca7202b8402d9dd6745b5 (unchanged)
regression GNU-1054
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/411/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0411 (basepoints/gcc-14-06032-g2c3db94d9fd) 2024-04-06T17:27:04 0:00:00
Previous 0410 (basepoints/gcc-14-06031-g37043268825) 2024-04-03T22:27:06 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 84 86
Performance Improvements 52 131
Added Tests 26 0
Unchanged Tests 1074 1019
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf 11.02% 1879.000 2086.000 - -87.84% -
403.gcc 8.93% 56.000 61.000 - -99.32% -
437.leslie3d 5.03% 11000.000 11553.000 - -23.01% -
473.astar 4.18% 5669.000 5906.000 - -60.45% -
401.bzip2 3.72% 2902.000 3010.000 - -80.18% -
450.soplex 3.64% 549.000 569.000 - -96.32% -
400.perlbench 3.34% 1529.000 1580.000 - -85.16% -
465.tonto 3.32% 11188.000 11560.000 - 40.89% -
482.sphinx3 2.86% 908.000 934.000 - -96.88% -
471.omnetpp 2.46% 2849.000 2919.000 - -72.82% -
459.GemsFDTD 2.23% 5954.000 6087.000 - -73.94% -
435.gromacs 1.92% 6933.000 7066.000 - -54.17% -
445.gobmk 1.78% 5157.000 5249.000 - -56.24% -
410.bwaves 1.74% 5905.000 6008.000 - -59.28% -
447.dealII 1.38% 1952.000 1979.000 - -79.70% -
470.lbm 1.22% 3694.000 3739.000 - -70.83% -
433.milc 1.13% 1147.000 1160.000 - -91.06% -
434.zeusmp 1.11% 2523.000 2551.000 - -80.69% -
458.sjeng 1.07% 7593.000 7674.000 - -46.24% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
436.cactusADM (max) 350.00% 6.000 27.000 - -95.11% -
436.cactusADM (max) (percentage) 312.50% 0.320 1.320 - -58.10% -
465.tonto (max) 261.32% 106.000 383.000 - 30.72% -
459.GemsFDTD (max) 255.26% 38.000 135.000 - -3.57% -
465.tonto (max) (percentage) 249.47% 0.950 3.320 - -7.26% -
459.GemsFDTD (max) (percentage) 248.44% 0.640 2.230 - 271.67% -
445.gobmk (max) 173.53% 34.000 93.000 - -49.18% -
445.gobmk (max) (percentage) 169.70% 0.660 1.780 - 16.34% -
464.h264ref (max) 130.43% 23.000 53.000 - -14.52% -
450.soplex (max) 122.22% 9.000 20.000 - -93.42% -
464.h264ref (max) (percentage) 121.95% 0.410 0.910 - 111.63% -
450.soplex (max) (percentage) 108.00% 1.750 3.640 - 84.77% -
434.zeusmp (max) 75.00% 16.000 28.000 - -90.70% -
434.zeusmp (max) (percentage) 73.44% 0.640 1.110 - -51.32% -
437.leslie3d (max) 38.66% 419.000 581.000 - 65.53% -
437.leslie3d (max) (percentage) 32.02% 3.810 5.030 - 114.96% -
400.perlbench (max) 18.18% 44.000 52.000 - -84.93% -
400.perlbench (max) (percentage) 15.57% 2.890 3.340 - 3.09% -
429.mcf 12.84% 109.000 123.000 - 36.67% -
429.mcf (avg) 12.84% 109.000 123.000 - 36.67% -
465.tonto 8.70% 138.000 150.000 - 9.49% -
465.tonto (avg) 8.70% 138.000 150.000 - 9.49% -
473.astar 7.37% 95.000 102.000 - 264.29% -
473.astar (avg) 7.37% 95.000 102.000 - 264.29% -
401.bzip2 6.56% 61.000 65.000 - -1.52% -
401.bzip2 (avg) 6.56% 61.000 65.000 - -1.52% -
400.perlbench 6.45% 31.000 33.000 - -78.85% -
400.perlbench (avg) 6.45% 31.000 33.000 - -78.85% -
482.sphinx3 5.88% 17.000 18.000 - -95.87% -
482.sphinx3 (avg) 5.88% 17.000 18.000 - -95.87% -
437.leslie3d 5.47% 475.000 501.000 - 40.34% -
437.leslie3d (avg) 5.47% 475.000 501.000 - 40.34% -
465.tonto (avg) (percentage) 4.84% 1.240 1.300 - -22.16% -
473.astar (max) 4.24% 236.000 246.000 - 1130.00% -
447.dealII 3.57% 28.000 29.000 - -72.12% -
447.dealII (avg) 3.57% 28.000 29.000 - -72.12% -
459.GemsFDTD 3.53% 85.000 88.000 - -58.10% -
459.GemsFDTD (avg) 3.53% 85.000 88.000 - -58.10% -
458.sjeng 3.33% 60.000 62.000 - 31.91% -
458.sjeng (avg) 3.33% 60.000 62.000 - 31.91% -
471.omnetpp 2.94% 68.000 70.000 - -58.82% -
471.omnetpp (avg) 2.94% 68.000 70.000 - -58.82% -
410.bwaves 2.86% 70.000 72.000 - -55.00% -
410.bwaves (avg) 2.86% 70.000 72.000 - -55.00% -
434.zeusmp 2.70% 37.000 38.000 - -80.51% -
434.zeusmp (avg) 2.70% 37.000 38.000 - -80.51% -
473.astar (avg) (percentage) 2.37% 1.690 1.730 - 810.53% -
470.lbm 2.33% 43.000 44.000 - -85.48% -
470.lbm (avg) 2.33% 43.000 44.000 - -85.48% -
435.gromacs 2.24% 134.000 137.000 - -48.50% -
435.gromacs (avg) 2.24% 134.000 137.000 - -48.50% -
403.gcc (avg) (percentage) 1.94% 6.180 6.300 - 932.79% -
464.h264ref 1.89% 53.000 54.000 - -30.77% -
464.h264ref (avg) 1.89% 53.000 54.000 - -30.77% -
445.gobmk 1.75% 57.000 58.000 - -36.96% -
445.gobmk (avg) 1.75% 57.000 58.000 - -36.96% -
400.perlbench (avg) (percentage) 1.44% 2.090 2.120 - 44.22% -
401.bzip2 (avg) (percentage) 1.41% 2.130 2.160 - 390.91% -
459.GemsFDTD (avg) (percentage) 1.40% 1.430 1.450 - 61.11% -
462.libquantum (max) (percentage) 1.37% 1.460 1.480 - 131.25% -
462.libquantum (unexplained diff vs previous) (percentage) 1.35% 1.480 1.500 0.000 14900.00% 0.000
458.sjeng (avg) (percentage) 1.25% 0.800 0.810 - 145.45% -
429.mcf (avg) (percentage) 1.20% 5.830 5.900 - 1013.21% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -3.85% 52.000 50.000 - 72.41% -
nb_execution_successful -3.85% 52.000 50.000 - 72.41% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -1.48% 135.000 133.000 - -98.85% -
436.cactusADM -1.32% 2128.000 2100.000 - -88.03% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (unexplained diff vs previous) (percentage) -55.73% 2.530 1.120 0.000 -65.85% 0.000
433.milc (unexplained diff vs previous) -55.17% 29.000 13.000 0.000 -96.94% 0.000
429.mcf (unexplained diff vs previous) (percentage) -50.30% 19.960 9.920 0.000 1737.04% 0.000
444.namd (unexplained diff vs previous) -50.00% 8.000 4.000 0.000 -99.43% 0.000
444.namd (unexplained diff vs previous) (percentage) -50.00% 1.100 0.550 0.000 -92.88% 0.000
429.mcf (unexplained diff vs previous) -44.80% 375.000 207.000 0.000 122.58% 0.000
410.bwaves (unexplained diff vs previous) (percentage) -37.59% 2.740 1.710 0.000 1610.00% 0.000
410.bwaves (unexplained diff vs previous) -36.42% 162.000 103.000 0.000 586.67% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) -31.70% 4.070 2.780 0.000 348.39% 0.000
482.sphinx3 (unexplained diff vs previous) -29.73% 37.000 26.000 0.000 -86.02% 0.000
458.sjeng (unexplained diff vs previous) (percentage) -29.33% 1.500 1.060 0.000 1666.67% 0.000
458.sjeng (unexplained diff vs previous) -28.95% 114.000 81.000 0.000 800.00% 0.000
458.sjeng (max) (percentage) -25.00% 1.480 1.110 - 404.55% -
458.sjeng (max) -24.11% 112.000 85.000 - 174.19% -
471.omnetpp (unexplained diff vs previous) (percentage) -21.31% 3.050 2.400 0.000 900.00% 0.000
470.lbm (unexplained diff vs previous) (percentage) -21.05% 1.520 1.200 0.000 990.91% 0.000
470.lbm (unexplained diff vs previous) -19.64% 56.000 45.000 0.000 221.43% 0.000
471.omnetpp (unexplained diff vs previous) -19.54% 87.000 70.000 0.000 169.23% 0.000
471.omnetpp (max) (percentage) -16.89% 2.960 2.460 - -60.13% -
482.sphinx3 (max) (percentage) -16.58% 3.920 3.270 - 122.45% -
471.omnetpp (max) -15.48% 84.000 71.000 - -89.27% -
444.namd (max) -14.29% 7.000 6.000 - -99.08% -
482.sphinx3 (max) -14.29% 35.000 30.000 - -93.17% -
444.namd (max) (percentage) -12.84% 1.090 0.950 - -86.77% -
470.lbm (max) (percentage) -12.75% 1.490 1.300 - -74.51% -
470.lbm (max) -12.73% 55.000 48.000 - -92.65% -
401.bzip2 (unexplained diff vs previous) (percentage) -12.44% 4.100 3.590 0.000 312.64% 0.000
410.bwaves (max) (percentage) -12.36% 2.670 2.340 - 1014.29% -
410.bwaves (max) -10.83% 157.000 140.000 - 366.67% -
433.milc (max) -10.71% 28.000 25.000 - -93.92% -
433.milc (max) (percentage) -10.12% 2.470 2.220 - -29.97% -
429.mcf (max) (percentage) -9.99% 24.930 22.440 - 4055.56% -
401.bzip2 (unexplained diff vs previous) -9.24% 119.000 108.000 0.000 -18.18% 0.000
473.astar (unexplained diff vs previous) (percentage) -8.03% 4.360 4.010 0.000 19950.00% 0.000
401.bzip2 (max) (percentage) -5.58% 3.940 3.720 - 322.73% -
435.gromacs (max) (percentage) -5.42% 2.030 1.920 - -4.95% -
453.povray (max) (percentage) -4.92% 1.830 1.740 - -31.23% -
447.dealII (max) (percentage) -4.79% 1.670 1.590 - 44.55% -
456.hmmer -4.17% 24.000 23.000 - -69.33% -
456.hmmer (avg) -4.17% 24.000 23.000 - -69.33% -
473.astar (unexplained diff vs previous) -4.05% 247.000 237.000 0.000 7800.00% 0.000
454.calculix (max) (percentage) -3.80% 1.580 1.520 - -67.66% -
435.gromacs (max) -3.57% 140.000 135.000 - -56.59% -
447.dealII (max) -3.12% 32.000 31.000 - -71.03% -
401.bzip2 (max) -2.63% 114.000 111.000 - -16.54% -
456.hmmer (avg) (percentage) -2.38% 0.840 0.820 - 20.59% -
403.gcc (max) (percentage) -2.11% 9.940 9.730 - 1251.39% -
481.wrf (max) (percentage) -2.08% 1.920 1.880 - -40.32% -
453.povray (avg) (percentage) -1.59% 1.890 1.860 - 48.80% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 3.12% 64.000 66.000 - 127.59% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)
403.gcc (unexplained diff vs previous)
403.gcc (unexplained diff vs previous) (percentage)
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
436.cactusADM (unexplained diff vs previous)
436.cactusADM (unexplained diff vs previous) (percentage)
437.leslie3d (unexplained diff vs previous)
437.leslie3d (unexplained diff vs previous) (percentage)
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
433.milc 4.00% 95873.000 99705.000 - 0.00% -
456.hmmer 1.41% 231209.000 234468.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 40.89% 8205.000 11560.000 - 3.32% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
473.astar (unexplained diff vs previous) (percentage) 19950.00% 0.020 4.010 0.000 -8.03% 0.000
462.libquantum (unexplained diff vs previous) (percentage) 14900.00% 0.010 1.500 0.000 1.35% 0.000
473.astar (unexplained diff vs previous) 7800.00% 3.000 237.000 0.000 -4.05% 0.000
429.mcf (max) (percentage) 4055.56% 0.540 22.440 - -9.99% -
473.astar (max) (percentage) 2885.71% 0.140 4.180 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) 1737.04% 0.540 9.920 0.000 -50.30% 0.000
458.sjeng (unexplained diff vs previous) (percentage) 1666.67% 0.060 1.060 0.000 -29.33% 0.000
410.bwaves (unexplained diff vs previous) (percentage) 1610.00% 0.100 1.710 0.000 -37.59% 0.000
483.xalancbmk (max) (percentage) 1300.00% 0.410 5.740 - 0.00% -
403.gcc (max) (percentage) 1251.39% 0.720 9.730 - -2.11% -
473.astar (max) 1130.00% 20.000 246.000 - 4.24% -
403.gcc (unexplained diff vs previous) (percentage) 1038.89% 0.720 8.200 0.000 0.00% 0.000
410.bwaves (max) (percentage) 1014.29% 0.210 2.340 - -12.36% -
429.mcf (avg) (percentage) 1013.21% 0.530 5.900 - 1.20% -
470.lbm (unexplained diff vs previous) (percentage) 990.91% 0.110 1.200 0.000 -21.05% 0.000
453.povray (unexplained diff vs previous) (percentage) 950.00% 0.020 0.210 0.000 0.00% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) 938.10% 0.210 2.180 0.000 0.00% 0.000
403.gcc (avg) (percentage) 932.79% 0.610 6.300 - 1.94% -
471.omnetpp (unexplained diff vs previous) (percentage) 900.00% 0.240 2.400 0.000 -21.31% 0.000
473.astar (avg) (percentage) 810.53% 0.190 1.730 - 2.37% -
458.sjeng (unexplained diff vs previous) 800.00% 9.000 81.000 0.000 -28.95% 0.000
464.h264ref (unexplained diff vs previous) (percentage) 800.00% 0.100 0.900 0.000 0.00% 0.000
410.bwaves (unexplained diff vs previous) 586.67% 15.000 103.000 0.000 -36.42% 0.000
465.tonto (unexplained diff vs previous) 490.48% 63.000 372.000 0.000 0.00% 0.000
429.mcf (max) 408.70% 92.000 468.000 - 0.00% -
458.sjeng (max) (percentage) 404.55% 0.220 1.110 - -25.00% -
401.bzip2 (avg) (percentage) 390.91% 0.440 2.160 - 1.41% -
410.bwaves (max) 366.67% 30.000 140.000 - -10.83% -
482.sphinx3 (unexplained diff vs previous) (percentage) 348.39% 0.620 2.780 0.000 -31.70% 0.000
401.bzip2 (max) (percentage) 322.73% 0.880 3.720 - -5.58% -
465.tonto (unexplained diff vs previous) (percentage) 318.18% 0.770 3.220 0.000 0.00% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) 312.64% 0.870 3.590 0.000 -12.44% 0.000
459.GemsFDTD (max) (percentage) 271.67% 0.600 2.230 - 248.44% -
473.astar 264.29% 28.000 102.000 - 7.37% -
473.astar (avg) 264.29% 28.000 102.000 - 7.37% -
464.h264ref (unexplained diff vs previous) 253.33% 15.000 53.000 0.000 0.00% 0.000
470.lbm (unexplained diff vs previous) 221.43% 14.000 45.000 0.000 -19.64% 0.000
459.GemsFDTD (unexplained diff vs previous) 177.08% 48.000 133.000 0.000 0.00% 0.000
458.sjeng (max) 174.19% 31.000 85.000 - -24.11% -
462.libquantum (avg) (percentage) 172.31% 0.650 1.770 - 0.00% -
471.omnetpp (unexplained diff vs previous) 169.23% 26.000 70.000 0.000 -19.54% 0.000
458.sjeng (avg) (percentage) 145.45% 0.330 0.810 - 1.25% -
450.soplex (avg) (percentage) 139.52% 1.240 2.970 - 0.34% -
462.libquantum (max) (percentage) 131.25% 0.640 1.480 - 1.37% -
429.mcf (unexplained diff vs previous) 122.58% 93.000 207.000 0.000 -44.80% 0.000
482.sphinx3 (max) (percentage) 122.45% 1.470 3.270 - -16.58% -
437.leslie3d (max) (percentage) 114.96% 2.340 5.030 - 32.02% -
464.h264ref (max) (percentage) 111.63% 0.430 0.910 - 121.95% -
437.leslie3d (unexplained diff vs previous) (percentage) 109.17% 2.290 4.790 0.000 0.00% 0.000
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 0.000 0.00% 0.000
450.soplex (max) (percentage) 84.77% 1.970 3.640 - 108.00% -
437.leslie3d (avg) (percentage) 82.35% 2.380 4.340 - 0.46% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 74.63% 2.010 3.510 0.000 0.00% 0.000
447.dealII (unexplained diff vs previous) (percentage) 74.36% 0.780 1.360 0.000 0.00% 0.000
464.h264ref (avg) (percentage) 70.37% 0.540 0.920 - 0.00% -
437.leslie3d (max) 65.53% 351.000 581.000 - 38.66% -
437.leslie3d (unexplained diff vs previous) 61.22% 343.000 553.000 0.000 0.00% 0.000
459.GemsFDTD (avg) (percentage) 61.11% 0.900 1.450 - 1.40% -
454.calculix (avg) (percentage) 52.24% 2.010 3.060 - 0.00% -
471.omnetpp (avg) (percentage) 52.20% 1.590 2.420 - 0.00% -
453.povray (avg) (percentage) 48.80% 1.250 1.860 - -1.59% -
445.gobmk (avg) (percentage) 45.45% 0.770 1.120 - 0.90% -
447.dealII (max) (percentage) 44.55% 1.100 1.590 - -4.79% -
400.perlbench (avg) (percentage) 44.22% 1.470 2.120 - 1.44% -
437.leslie3d 40.34% 357.000 501.000 - 5.47% -
437.leslie3d (avg) 40.34% 357.000 501.000 - 5.47% -
447.dealII (avg) (percentage) 37.38% 1.070 1.470 - 0.00% -
429.mcf 36.67% 90.000 123.000 - 12.84% -
429.mcf (avg) 36.67% 90.000 123.000 - 12.84% -
482.sphinx3 (avg) (percentage) 32.88% 1.460 1.940 - 0.52% -
458.sjeng 31.91% 47.000 62.000 - 3.33% -
458.sjeng (avg) 31.91% 47.000 62.000 - 3.33% -
465.tonto (max) 30.72% 293.000 383.000 - 261.32% -
456.hmmer (avg) (percentage) 20.59% 0.680 0.820 - -2.38% -
445.gobmk (max) (percentage) 16.34% 1.530 1.780 - 169.70% -
445.gobmk (unexplained diff vs previous) (percentage) 12.90% 1.550 1.750 0.000 0.00% 0.000
435.gromacs (avg) (percentage) 12.14% 1.730 1.940 - 0.00% -
410.bwaves (avg) (percentage) 10.09% 1.090 1.200 - 0.00% -
465.tonto 9.49% 137.000 150.000 - 8.70% -
465.tonto (avg) 9.49% 137.000 150.000 - 8.70% -
400.perlbench (max) (percentage) 3.09% 3.240 3.340 - 15.57% -
434.zeusmp (avg) (percentage) 1.35% 1.480 1.500 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
434.zeusmp -1.72% 165699.000 162853.000 - 0.00% -
459.GemsFDTD -1.50% 253716.000 249917.000 - 0.00% -
464.h264ref -1.46% 428364.000 422128.000 - 0.00% -
410.bwaves -1.45% 17880.000 17620.000 - 0.00% -
400.perlbench -1.03% 930652.000 921026.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.32% 9010.000 61.000 - 8.93% -
462.libquantum -98.85% 11614.000 133.000 - -1.48% -
482.sphinx3 -96.88% 29928.000 934.000 - 2.86% -
450.soplex -96.32% 15466.000 569.000 - 3.64% -
453.povray -92.56% 6544.000 487.000 - 0.21% -
444.namd -91.93% 9050.000 730.000 - 0.55% -
433.milc -91.06% 12976.000 1160.000 - 1.13% -
436.cactusADM -88.03% 17544.000 2100.000 - -1.32% -
429.mcf -87.84% 17158.000 2086.000 - 11.02% -
400.perlbench -85.16% 10650.000 1580.000 - 3.34% -
434.zeusmp -80.69% 13211.000 2551.000 - 1.11% -
401.bzip2 -80.18% 15185.000 3010.000 - 3.72% -
447.dealII -79.70% 9751.000 1979.000 - 1.38% -
459.GemsFDTD -73.94% 23356.000 6087.000 - 2.23% -
456.hmmer -73.52% 11046.000 2925.000 - 0.03% -
471.omnetpp -72.82% 10739.000 2919.000 - 2.46% -
470.lbm -70.83% 12819.000 3739.000 - 1.22% -
473.astar -60.45% 14932.000 5906.000 - 4.18% -
464.h264ref -59.53% 14556.000 5891.000 - 0.91% -
410.bwaves -59.28% 14755.000 6008.000 - 1.74% -
445.gobmk -56.24% 11995.000 5249.000 - 1.78% -
435.gromacs -54.17% 15419.000 7066.000 - 1.92% -
458.sjeng -46.24% 14275.000 7674.000 - 1.07% -
437.leslie3d -23.01% 15005.000 11553.000 - 5.03% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (unexplained diff vs previous) -99.43% 700.000 4.000 0.000 -50.00% 0.000
444.namd (max) -99.08% 649.000 6.000 - -14.29% -
456.hmmer (unexplained diff vs previous) -98.80% 83.000 1.000 0.000 0.00% 0.000
462.libquantum (max) -98.65% 74.000 1.000 - 0.00% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
433.milc (unexplained diff vs previous) -96.94% 425.000 13.000 0.000 -55.17% 0.000
444.namd -96.62% 355.000 12.000 - 0.00% -
444.namd (avg) -96.62% 355.000 12.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -96.00% 0.750 0.030 0.000 0.00% 0.000
482.sphinx3 -95.87% 436.000 18.000 - 5.88% -
482.sphinx3 (avg) -95.87% 436.000 18.000 - 5.88% -
453.povray (max) -95.15% 165.000 8.000 - 0.00% -
436.cactusADM (max) -95.11% 552.000 27.000 - 350.00% -
436.cactusADM (unexplained diff vs previous) -95.09% 570.000 28.000 0.000 0.00% 0.000
416.gamess (max) (percentage) -94.81% 2.890 0.150 - 0.00% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
433.milc (max) -93.92% 411.000 25.000 - -10.71% -
456.hmmer (max) -93.57% 140.000 9.000 - 0.00% -
450.soplex (unexplained diff vs previous) -93.57% 311.000 20.000 0.000 0.00% 0.000
450.soplex (max) -93.42% 304.000 20.000 - 122.22% -
482.sphinx3 (max) -93.17% 439.000 30.000 - -14.29% -
444.namd (unexplained diff vs previous) (percentage) -92.88% 7.730 0.550 0.000 -50.00% 0.000
470.lbm (max) -92.65% 653.000 48.000 - -12.73% -
403.gcc (unexplained diff vs previous) -92.31% 65.000 5.000 0.000 0.00% 0.000
403.gcc (max) -92.19% 64.000 5.000 - 0.00% -
433.milc -91.82% 220.000 18.000 - 0.00% -
433.milc (avg) -91.82% 220.000 18.000 - 0.00% -
450.soplex -91.62% 191.000 16.000 - 0.00% -
450.soplex (avg) -91.62% 191.000 16.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) -90.91% 308.000 28.000 0.000 0.00% 0.000
434.zeusmp (max) -90.70% 301.000 28.000 - 75.00% -
436.cactusADM -90.03% 291.000 29.000 - 0.00% -
436.cactusADM (avg) -90.03% 291.000 29.000 - 0.00% -
471.omnetpp (max) -89.27% 662.000 71.000 - -15.48% -
453.povray -88.89% 81.000 9.000 - 0.00% -
453.povray (avg) -88.89% 81.000 9.000 - 0.00% -
444.namd (max) (percentage) -86.77% 7.180 0.950 - -12.84% -
482.sphinx3 (unexplained diff vs previous) -86.02% 186.000 26.000 0.000 -29.73% 0.000
400.perlbench (unexplained diff vs previous) -85.71% 357.000 51.000 0.000 0.00% 0.000
470.lbm -85.48% 303.000 44.000 - 2.33% -
470.lbm (avg) -85.48% 303.000 44.000 - 2.33% -
400.perlbench (max) -84.93% 345.000 52.000 - 18.18% -
434.zeusmp -80.51% 195.000 38.000 - 2.70% -
434.zeusmp (avg) -80.51% 195.000 38.000 - 2.70% -
400.perlbench -78.85% 156.000 33.000 - 6.45% -
400.perlbench (avg) -78.85% 156.000 33.000 - 6.45% -
456.hmmer (max) (percentage) -75.59% 1.270 0.310 - 0.00% -
470.lbm (max) (percentage) -74.51% 5.100 1.300 - -12.75% -
447.dealII -72.12% 104.000 29.000 - 3.57% -
447.dealII (avg) -72.12% 104.000 29.000 - 3.57% -
447.dealII (max) -71.03% 107.000 31.000 - -3.12% -
456.hmmer -69.33% 75.000 23.000 - -4.17% -
456.hmmer (avg) -69.33% 75.000 23.000 - -4.17% -
454.calculix (max) (percentage) -67.66% 4.700 1.520 - -3.80% -
433.milc (unexplained diff vs previous) (percentage) -65.85% 3.280 1.120 0.000 -55.73% 0.000
447.dealII (unexplained diff vs previous) -64.47% 76.000 27.000 0.000 0.00% 0.000
416.gamess (avg) (percentage) -61.29% 2.170 0.840 - 0.00% -
471.omnetpp (max) (percentage) -60.13% 6.170 2.460 - -16.89% -
436.cactusADM (unexplained diff vs previous) (percentage) -59.08% 3.250 1.330 0.000 0.00% 0.000
471.omnetpp -58.82% 170.000 70.000 - 2.94% -
471.omnetpp (avg) -58.82% 170.000 70.000 - 2.94% -
435.gromacs (unexplained diff vs previous) -58.18% 318.000 133.000 0.000 0.00% 0.000
459.GemsFDTD -58.10% 210.000 88.000 - 3.53% -
459.GemsFDTD (avg) -58.10% 210.000 88.000 - 3.53% -
436.cactusADM (max) (percentage) -58.10% 3.150 1.320 - 312.50% -
444.namd (avg) (percentage) -57.00% 3.930 1.690 - -0.59% -
435.gromacs (max) -56.59% 311.000 135.000 - -3.57% -
410.bwaves -55.00% 160.000 72.000 - 2.86% -
410.bwaves (avg) -55.00% 160.000 72.000 - 2.86% -
434.zeusmp (unexplained diff vs previous) (percentage) -52.79% 2.330 1.100 0.000 0.00% 0.000
434.zeusmp (max) (percentage) -51.32% 2.280 1.110 - 73.44% -
445.gobmk (unexplained diff vs previous) -50.54% 186.000 92.000 0.000 0.00% 0.000
470.lbm (avg) (percentage) -49.79% 2.370 1.190 - 0.00% -
445.gobmk (max) -49.18% 183.000 93.000 - 173.53% -
435.gromacs -48.50% 266.000 137.000 - 2.24% -
435.gromacs (avg) -48.50% 266.000 137.000 - 2.24% -
481.wrf (max) (percentage) -40.32% 3.150 1.880 - -2.08% -
445.gobmk -36.96% 92.000 58.000 - 1.75% -
445.gobmk (avg) -36.96% 92.000 58.000 - 1.75% -
453.povray (max) (percentage) -31.23% 2.530 1.740 - -4.92% -
464.h264ref -30.77% 78.000 54.000 - 1.89% -
464.h264ref (avg) -30.77% 78.000 54.000 - 1.89% -
433.milc (max) (percentage) -29.97% 3.170 2.220 - -10.12% -
465.tonto (avg) (percentage) -22.16% 1.670 1.300 - 4.84% -
401.bzip2 (unexplained diff vs previous) -18.18% 132.000 108.000 0.000 -9.24% 0.000
401.bzip2 (max) -16.54% 133.000 111.000 - -2.63% -
436.cactusADM (avg) (percentage) -16.27% 1.660 1.390 - 0.00% -
464.h264ref (max) -14.52% 62.000 53.000 - 130.43% -
481.wrf (avg) (percentage) -12.17% 1.890 1.660 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) -8.74% 2.060 1.880 0.000 0.00% 0.000
465.tonto (max) (percentage) -7.26% 3.580 3.320 - 249.47% -
433.milc (avg) (percentage) -6.47% 1.700 1.590 - 0.00% -
435.gromacs (max) (percentage) -4.95% 2.020 1.920 - -5.42% -
400.perlbench (unexplained diff vs previous) (percentage) -3.58% 3.350 3.230 0.000 0.00% 0.000
459.GemsFDTD (max) -3.57% 140.000 135.000 - 255.26% -
401.bzip2 -1.52% 66.000 65.000 - 6.56% -
401.bzip2 (avg) -1.52% 66.000 65.000 - 6.56% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 127.59% 29.000 66.000 - 3.12% -
nb_compile_successful 72.41% 29.000 50.000 - -3.85% -
nb_execution_successful 72.41% 29.000 50.000 - -3.85% -

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1580.000 3.34%
401.bzip2 3010.000 3.72%
403.gcc 61.000 8.93%
410.bwaves 6008.000 1.74%
429.mcf 2086.000 11.02%
433.milc 1160.000 1.13%
434.zeusmp 2551.000 1.11%
435.gromacs 7066.000 1.92%
436.cactusADM 2100.000 -1.32%
437.leslie3d 11553.000 5.03%
444.namd 730.000 -
445.gobmk 5249.000 1.78%
447.dealII 1979.000 1.38%
450.soplex 569.000 3.64%
453.povray 487.000 -
456.hmmer 2925.000 -
458.sjeng 7674.000 1.07%
459.GemsFDTD 6087.000 2.23%
462.libquantum 133.000 -1.48%
464.h264ref 5891.000 -
465.tonto 11560.000 3.32%
470.lbm 3739.000 1.22%
471.omnetpp 2919.000 2.46%
473.astar 5906.000 4.18%
482.sphinx3 934.000 2.86%
Geometric Mean 2179.821 2.40%
Code Size (bytes)
Name Current %
400.perlbench 921026.000 -
401.bzip2 49690.000 -
403.gcc 2683585.000 -
410.bwaves 17620.000 -
429.mcf 10979.000 -
433.milc 99705.000 -
434.zeusmp 162853.000 -
435.gromacs 677825.000 -
436.cactusADM 551472.000 -
437.leslie3d 64764.000 -
444.namd 190766.000 -
445.gobmk 1399857.000 -
447.dealII 2118483.000 -
450.soplex 259893.000 -
453.povray 710554.000 -
456.hmmer 234468.000 -
458.sjeng 120025.000 -
459.GemsFDTD 249917.000 -
462.libquantum 31353.000 -
464.h264ref 422128.000 -
465.tonto 2874533.000 -
470.lbm 10916.000 -
471.omnetpp 566332.000 -
473.astar 32220.000 -
482.sphinx3 152612.000 -
Geometric Mean 207006.588 -
Score
Name Current %
nb_compile_successful 50.000 -3.85%
nb_execution_successful 50.000 -3.85%
nb_total_tests 66.000 3.12%
Geometric Mean 54.848 -1.58%
sample variability
Name Current %
400.perlbench 33.000 6.45%
400.perlbench (avg) 33.000 6.45%
400.perlbench (avg) (percentage) 2.120 1.44%
400.perlbench (max) 52.000 18.18%
400.perlbench (max) (percentage) 3.340 15.57%
400.perlbench (unexplained diff vs previous) 51.000 -
400.perlbench (unexplained diff vs previous) (percentage) 3.230 -
401.bzip2 65.000 6.56%
401.bzip2 (avg) 65.000 6.56%
401.bzip2 (avg) (percentage) 2.160 1.41%
401.bzip2 (max) 111.000 -2.63%
401.bzip2 (max) (percentage) 3.720 -5.58%
401.bzip2 (unexplained diff vs previous) 108.000 -9.24%
401.bzip2 (unexplained diff vs previous) (percentage) 3.590 -12.44%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 6.300 1.94%
403.gcc (max) 5.000 -
403.gcc (max) (percentage) 9.730 -2.11%
403.gcc (unexplained diff vs previous) 5.000 -
403.gcc (unexplained diff vs previous) (percentage) 8.200 -
410.bwaves 72.000 2.86%
410.bwaves (avg) 72.000 2.86%
410.bwaves (avg) (percentage) 1.200 -
410.bwaves (max) 140.000 -10.83%
410.bwaves (max) (percentage) 2.340 -12.36%
410.bwaves (unexplained diff vs previous) 103.000 -36.42%
410.bwaves (unexplained diff vs previous) (percentage) 1.710 -37.59%
416.gamess (avg) (percentage) 0.840 -
416.gamess (max) (percentage) 0.150 -
429.mcf 123.000 12.84%
429.mcf (avg) 123.000 12.84%
429.mcf (avg) (percentage) 5.900 1.20%
429.mcf (max) 468.000 -
429.mcf (max) (percentage) 22.440 -9.99%
429.mcf (unexplained diff vs previous) 207.000 -44.80%
429.mcf (unexplained diff vs previous) (percentage) 9.920 -50.30%
433.milc 18.000 -
433.milc (avg) 18.000 -
433.milc (avg) (percentage) 1.590 -
433.milc (max) 25.000 -10.71%
433.milc (max) (percentage) 2.220 -10.12%
433.milc (unexplained diff vs previous) 13.000 -55.17%
433.milc (unexplained diff vs previous) (percentage) 1.120 -55.73%
434.zeusmp 38.000 2.70%
434.zeusmp (avg) 38.000 2.70%
434.zeusmp (avg) (percentage) 1.500 -
434.zeusmp (max) 28.000 75.00%
434.zeusmp (max) (percentage) 1.110 73.44%
434.zeusmp (unexplained diff vs previous) 28.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 1.100 -
435.gromacs 137.000 2.24%
435.gromacs (avg) 137.000 2.24%
435.gromacs (avg) (percentage) 1.940 -
435.gromacs (max) 135.000 -3.57%
435.gromacs (max) (percentage) 1.920 -5.42%
435.gromacs (unexplained diff vs previous) 133.000 -
435.gromacs (unexplained diff vs previous) (percentage) 1.880 -
436.cactusADM 29.000 -
436.cactusADM (avg) 29.000 -
436.cactusADM (avg) (percentage) 1.390 -
436.cactusADM (max) 27.000 350.00%
436.cactusADM (max) (percentage) 1.320 312.50%
436.cactusADM (unexplained diff vs previous) 28.000 -
436.cactusADM (unexplained diff vs previous) (percentage) 1.330 -
437.leslie3d 501.000 5.47%
437.leslie3d (avg) 501.000 5.47%
437.leslie3d (avg) (percentage) 4.340 -
437.leslie3d (max) 581.000 38.66%
437.leslie3d (max) (percentage) 5.030 32.02%
437.leslie3d (unexplained diff vs previous) 553.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 4.790 -
444.namd 12.000 -
444.namd (avg) 12.000 -
444.namd (avg) (percentage) 1.690 -
444.namd (max) 6.000 -14.29%
444.namd (max) (percentage) 0.950 -12.84%
444.namd (unexplained diff vs previous) 4.000 -50.00%
444.namd (unexplained diff vs previous) (percentage) 0.550 -50.00%
445.gobmk 58.000 1.75%
445.gobmk (avg) 58.000 1.75%
445.gobmk (avg) (percentage) 1.120 -
445.gobmk (max) 93.000 173.53%
445.gobmk (max) (percentage) 1.780 169.70%
445.gobmk (unexplained diff vs previous) 92.000 -
445.gobmk (unexplained diff vs previous) (percentage) 1.750 -
447.dealII 29.000 3.57%
447.dealII (avg) 29.000 3.57%
447.dealII (avg) (percentage) 1.470 -
447.dealII (max) 31.000 -3.12%
447.dealII (max) (percentage) 1.590 -4.79%
447.dealII (unexplained diff vs previous) 27.000 -
447.dealII (unexplained diff vs previous) (percentage) 1.360 -
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 2.970 -
450.soplex (max) 20.000 122.22%
450.soplex (max) (percentage) 3.640 108.00%
450.soplex (unexplained diff vs previous) 20.000 -
450.soplex (unexplained diff vs previous) (percentage) 3.510 -
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.860 -1.59%
453.povray (max) 8.000 -
453.povray (max) (percentage) 1.740 -4.92%
453.povray (unexplained diff vs previous) 1.000 -
453.povray (unexplained diff vs previous) (percentage) 0.210 -
454.calculix (avg) (percentage) 3.060 -
454.calculix (max) (percentage) 1.520 -3.80%
456.hmmer 23.000 -4.17%
456.hmmer (avg) 23.000 -4.17%
456.hmmer (avg) (percentage) 0.820 -2.38%
456.hmmer (max) 9.000 -
456.hmmer (max) (percentage) 0.310 -
456.hmmer (unexplained diff vs previous) 1.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.030 -
458.sjeng 62.000 3.33%
458.sjeng (avg) 62.000 3.33%
458.sjeng (avg) (percentage) 0.810 1.25%
458.sjeng (max) 85.000 -24.11%
458.sjeng (max) (percentage) 1.110 -25.00%
458.sjeng (unexplained diff vs previous) 81.000 -28.95%
458.sjeng (unexplained diff vs previous) (percentage) 1.060 -29.33%
459.GemsFDTD 88.000 3.53%
459.GemsFDTD (avg) 88.000 3.53%
459.GemsFDTD (avg) (percentage) 1.450 1.40%
459.GemsFDTD (max) 135.000 255.26%
459.GemsFDTD (max) (percentage) 2.230 248.44%
459.GemsFDTD (unexplained diff vs previous) 133.000 -
459.GemsFDTD (unexplained diff vs previous) (percentage) 2.180 -
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.770 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.480 1.37%
462.libquantum (unexplained diff vs previous) 2.000 -
462.libquantum (unexplained diff vs previous) (percentage) 1.500 1.35%
464.h264ref 54.000 1.89%
464.h264ref (avg) 54.000 1.89%
464.h264ref (avg) (percentage) 0.920 -
464.h264ref (max) 53.000 130.43%
464.h264ref (max) (percentage) 0.910 121.95%
464.h264ref (unexplained diff vs previous) 53.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.900 -
465.tonto 150.000 8.70%
465.tonto (avg) 150.000 8.70%
465.tonto (avg) (percentage) 1.300 4.84%
465.tonto (max) 383.000 261.32%
465.tonto (max) (percentage) 3.320 249.47%
465.tonto (unexplained diff vs previous) 372.000 -
465.tonto (unexplained diff vs previous) (percentage) 3.220 -
470.lbm 44.000 2.33%
470.lbm (avg) 44.000 2.33%
470.lbm (avg) (percentage) 1.190 -
470.lbm (max) 48.000 -12.73%
470.lbm (max) (percentage) 1.300 -12.75%
470.lbm (unexplained diff vs previous) 45.000 -19.64%
470.lbm (unexplained diff vs previous) (percentage) 1.200 -21.05%
471.omnetpp 70.000 2.94%
471.omnetpp (avg) 70.000 2.94%
471.omnetpp (avg) (percentage) 2.420 -
471.omnetpp (max) 71.000 -15.48%
471.omnetpp (max) (percentage) 2.460 -16.89%
471.omnetpp (unexplained diff vs previous) 70.000 -19.54%
471.omnetpp (unexplained diff vs previous) (percentage) 2.400 -21.31%
473.astar 102.000 7.37%
473.astar (avg) 102.000 7.37%
473.astar (avg) (percentage) 1.730 2.37%
473.astar (max) 246.000 4.24%
473.astar (max) (percentage) 4.180 -
473.astar (unexplained diff vs previous) 237.000 -4.05%
473.astar (unexplained diff vs previous) (percentage) 4.010 -8.03%
481.wrf (avg) (percentage) 1.660 -
481.wrf (max) (percentage) 1.880 -2.08%
482.sphinx3 18.000 5.88%
482.sphinx3 (avg) 18.000 5.88%
482.sphinx3 (avg) (percentage) 1.940 -
482.sphinx3 (max) 30.000 -14.29%
482.sphinx3 (max) (percentage) 3.270 -16.58%
482.sphinx3 (unexplained diff vs previous) 26.000 -29.73%
482.sphinx3 (unexplained diff vs previous) (percentage) 2.780 -31.70%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.740 -
Geometric Mean 9.713 13.02%