Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=56756c571eb6d33eb8035cf0deb37e472597af6b (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/4ecfa6c4a9df19b364cbdf6d9de5f0b98c22bd8f (+ 4 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=42e48e720c78ab75eb8def9e866da52b0ac278df (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=f13102b78850c48082dca7202b8402d9dd6745b5 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/415/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0415 (basepoints/gcc-14-06036-g4ecfa6c4a9df) 2024-04-08T19:42:03 0:00:00
Previous 0413 (no-change) 2024-04-08T02:57:03 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 79 85
Performance Improvements 72 128
Unchanged Tests 1085 1023
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 7.14% 56.000 60.000 - -99.33% -
436.cactusADM 1.72% 2094.000 2130.000 - -87.86% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (unexplained diff vs previous) 3400.00% 2.000 70.000 0.000 677.78% 0.000
458.sjeng (unexplained diff vs previous) (percentage) 2966.67% 0.030 0.920 0.000 1433.33% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 1100.00% 0.240 2.880 0.000 1100.00% 0.000
471.omnetpp (unexplained diff vs previous) 1071.43% 7.000 82.000 0.000 215.38% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) 1046.43% 0.280 3.210 0.000 1428.57% 0.000
459.GemsFDTD (unexplained diff vs previous) 1017.65% 17.000 190.000 0.000 295.83% 0.000
445.gobmk (unexplained diff vs previous) (percentage) 938.10% 0.210 2.180 0.000 40.65% 0.000
445.gobmk (unexplained diff vs previous) 918.18% 11.000 112.000 0.000 -39.78% 0.000
473.astar (unexplained diff vs previous) (percentage) 848.78% 0.410 3.890 0.000 19350.00% 0.000
473.astar (unexplained diff vs previous) 816.67% 24.000 220.000 0.000 7233.33% 0.000
450.soplex (unexplained diff vs previous) (percentage) 661.43% 0.700 5.330 0.000 165.17% 0.000
433.milc (unexplained diff vs previous) (percentage) 642.31% 0.260 1.930 0.000 -41.16% 0.000
433.milc (unexplained diff vs previous) 633.33% 3.000 22.000 0.000 -94.82% 0.000
450.soplex (unexplained diff vs previous) 625.00% 4.000 29.000 0.000 -90.68% 0.000
436.cactusADM (unexplained diff vs previous) 500.00% 6.000 36.000 0.000 -93.68% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) 482.76% 0.290 1.690 0.000 -48.00% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) 472.88% 0.590 3.380 0.000 288.51% 0.000
401.bzip2 (unexplained diff vs previous) 450.00% 18.000 99.000 0.000 -25.00% 0.000
429.mcf (unexplained diff vs previous) (percentage) 449.12% 1.710 9.390 0.000 1638.89% 0.000
429.mcf (unexplained diff vs previous) 402.86% 35.000 176.000 0.000 89.25% 0.000
464.h264ref (unexplained diff vs previous) (percentage) 340.00% 0.200 0.880 0.000 780.00% 0.000
464.h264ref (unexplained diff vs previous) 325.00% 12.000 51.000 0.000 240.00% 0.000
470.lbm (unexplained diff vs previous) 260.00% 10.000 36.000 0.000 157.14% 0.000
470.lbm (unexplained diff vs previous) (percentage) 259.26% 0.270 0.970 0.000 781.82% 0.000
410.bwaves (unexplained diff vs previous) (percentage) 255.81% 0.430 1.530 0.000 1430.00% 0.000
410.bwaves (unexplained diff vs previous) 250.00% 26.000 91.000 0.000 506.67% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) 246.32% 0.950 3.290 0.000 430.65% 0.000
482.sphinx3 (unexplained diff vs previous) 233.33% 9.000 30.000 0.000 -83.87% 0.000
465.tonto (unexplained diff vs previous) (percentage) 216.05% 0.810 2.560 0.000 232.47% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) 207.58% 0.660 2.030 0.000 -12.88% 0.000
465.tonto (unexplained diff vs previous) 207.53% 93.000 286.000 0.000 353.97% 0.000
444.namd (unexplained diff vs previous) (percentage) 207.41% 0.270 0.830 0.000 -89.26% 0.000
434.zeusmp (unexplained diff vs previous) 200.00% 17.000 51.000 0.000 -83.44% 0.000
444.namd (unexplained diff vs previous) 200.00% 2.000 6.000 0.000 -99.14% 0.000
456.hmmer (max) 155.56% 9.000 23.000 - -83.57% -
456.hmmer (max) (percentage) 132.35% 0.340 0.790 - -37.80% -
456.hmmer (unexplained diff vs previous) 130.00% 10.000 23.000 0.000 -72.29% 0.000
456.hmmer (unexplained diff vs previous) (percentage) 129.41% 0.340 0.780 0.000 4.00% 0.000
400.perlbench (unexplained diff vs previous) (percentage) 110.37% 1.350 2.840 0.000 -15.22% 0.000
400.perlbench (unexplained diff vs previous) 104.76% 21.000 43.000 0.000 -87.96% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) 96.62% 3.250 6.390 0.000 179.04% 0.000
434.zeusmp (max) (percentage) 95.10% 1.020 1.990 - -12.72% -
434.zeusmp (max) 92.31% 26.000 50.000 - -83.39% -
437.leslie3d (unexplained diff vs previous) 84.62% 364.000 672.000 0.000 95.92% 0.000
459.GemsFDTD (max) (percentage) 67.20% 1.860 3.110 - 418.33% -
450.soplex (max) (percentage) 67.00% 3.030 5.060 - 156.85% -
435.gromacs (unexplained diff vs previous) (percentage) 65.83% 2.400 3.980 0.000 93.20% 0.000
459.GemsFDTD (max) 61.95% 113.000 183.000 - 30.71% -
435.gromacs (unexplained diff vs previous) 59.20% 174.000 277.000 0.000 -12.89% 0.000
450.soplex (max) 58.82% 17.000 27.000 - -91.12% -
436.cactusADM (max) 56.52% 23.000 36.000 - -93.48% -
436.cactusADM (max) (percentage) 56.36% 1.100 1.720 - -45.40% -
435.gromacs (max) (percentage) 55.69% 2.460 3.830 - 89.60% -
435.gromacs (max) 49.44% 178.000 266.000 - -14.47% -
437.leslie3d (max) (percentage) 43.44% 4.190 6.010 - 156.84% -
471.omnetpp (max) (percentage) 36.59% 2.050 2.800 - -54.62% -
437.leslie3d (max) 35.04% 468.000 632.000 - 80.06% -
471.omnetpp (max) 33.90% 59.000 79.000 - -88.07% -
445.gobmk (max) (percentage) 30.67% 1.630 2.130 - 39.22% -
445.gobmk (max) 28.24% 85.000 109.000 - -40.44% -
482.sphinx3 (max) (percentage) 8.16% 2.940 3.180 - 116.33% -
482.sphinx3 (max) 7.41% 27.000 29.000 - -93.39% -
401.bzip2 (max) (percentage) 5.48% 3.100 3.270 - 271.59% -
464.h264ref (max) (percentage) 4.82% 0.830 0.870 - 102.33% -
464.h264ref (max) 4.17% 48.000 50.000 - -19.35% -
436.cactusADM 3.57% 28.000 29.000 - -90.03% -
436.cactusADM (avg) 3.57% 28.000 29.000 - -90.03% -
459.GemsFDTD (avg) (percentage) 2.82% 1.420 1.460 - 62.22% -
465.tonto (avg) (percentage) 2.34% 1.280 1.310 - -21.56% -
473.astar (max) (percentage) 2.19% 3.660 3.740 - 2571.43% -
401.bzip2 (max) 2.15% 93.000 95.000 - -28.57% -
435.gromacs (avg) (percentage) 2.05% 1.950 1.990 - 15.03% -
445.gobmk (avg) (percentage) 1.80% 1.110 1.130 - 46.75% -
473.astar (avg) (percentage) 1.75% 1.710 1.740 - 815.79% -
450.soplex (avg) (percentage) 1.37% 2.930 2.970 - 139.52% -
458.sjeng (avg) (percentage) 1.27% 0.790 0.800 - 142.42% -
482.sphinx3 (avg) (percentage) 1.04% 1.930 1.950 - 33.56% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf -8.58% 2051.000 1875.000 - -89.07% -
437.leslie3d -6.01% 11189.000 10517.000 - -29.91% -
450.soplex -5.06% 573.000 544.000 - -96.48% -
435.gromacs -3.83% 7240.000 6963.000 - -54.84% -
473.astar -3.74% 5882.000 5662.000 - -62.08% -
401.bzip2 -3.27% 3028.000 2929.000 - -80.71% -
482.sphinx3 -3.18% 943.000 913.000 - -96.95% -
459.GemsFDTD -3.11% 6104.000 5914.000 - -74.68% -
471.omnetpp -2.80% 2926.000 2844.000 - -73.52% -
400.perlbench -2.76% 1559.000 1516.000 - -85.77% -
465.tonto -2.49% 11467.000 11181.000 - 36.27% -
445.gobmk -2.13% 5260.000 5148.000 - -57.08% -
434.zeusmp -1.99% 2568.000 2517.000 - -80.95% -
433.milc -1.89% 1163.000 1141.000 - -91.21% -
410.bwaves -1.51% 6034.000 5943.000 - -59.72% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
447.dealII (unexplained diff vs previous) -68.42% 19.000 6.000 0.000 -92.11% 0.000
447.dealII (unexplained diff vs previous) (percentage) -68.04% 0.970 0.310 0.000 -60.26% 0.000
453.povray (max) -37.50% 8.000 5.000 - -96.97% -
453.povray (unexplained diff vs previous) (percentage) -34.43% 1.220 0.800 0.000 3900.00% 0.000
453.povray (unexplained diff vs previous) -33.33% 6.000 4.000 0.000 300.00% 0.000
453.povray (max) (percentage) -32.74% 1.680 1.130 - -55.34% -
444.namd (max) -16.67% 6.000 5.000 - -99.23% -
429.mcf (max) -13.65% 425.000 367.000 - 298.91% -
447.dealII (max) (percentage) -9.09% 1.210 1.100 - 0.00% -
447.dealII (max) -8.70% 23.000 21.000 - -80.37% -
465.tonto (max) -8.62% 348.000 318.000 - 8.53% -
410.bwaves (max) -8.53% 129.000 118.000 - 293.33% -
444.namd -8.33% 12.000 11.000 - -96.90% -
444.namd (avg) -8.33% 12.000 11.000 - -96.90% -
458.sjeng (max) -8.00% 75.000 69.000 - 122.58% -
429.mcf -7.56% 119.000 110.000 - 22.22% -
429.mcf (avg) -7.56% 119.000 110.000 - 22.22% -
458.sjeng (max) (percentage) -7.07% 0.990 0.920 - 318.18% -
470.lbm (max) -6.82% 44.000 41.000 - -93.72% -
410.bwaves (max) (percentage) -6.54% 2.140 2.000 - 852.38% -
465.tonto (max) (percentage) -6.25% 3.040 2.850 - -20.39% -
470.lbm (max) (percentage) -5.88% 1.190 1.120 - -78.04% -
433.milc (max) (percentage) -5.83% 2.060 1.940 - -38.80% -
444.namd (max) (percentage) -5.75% 0.870 0.820 - -88.58% -
429.mcf (max) (percentage) -5.68% 20.770 19.590 - 3527.78% -
433.milc -5.56% 18.000 17.000 - -92.27% -
433.milc (avg) -5.56% 18.000 17.000 - -92.27% -
482.sphinx3 -5.56% 18.000 17.000 - -96.10% -
482.sphinx3 (avg) -5.56% 18.000 17.000 - -96.10% -
437.leslie3d -5.38% 483.000 457.000 - 28.01% -
437.leslie3d (avg) -5.38% 483.000 457.000 - 28.01% -
403.gcc (max) (percentage) -5.14% 8.370 7.940 - 1002.78% -
462.libquantum (max) (percentage) -5.11% 1.370 1.300 - 103.12% -
400.perlbench (max) -4.65% 43.000 41.000 - -88.12% -
433.milc (max) -4.35% 23.000 22.000 - -94.65% -
447.dealII -3.57% 28.000 27.000 - -74.04% -
447.dealII (avg) -3.57% 28.000 27.000 - -74.04% -
400.perlbench -3.12% 32.000 31.000 - -80.13% -
400.perlbench (avg) -3.12% 32.000 31.000 - -80.13% -
401.bzip2 -3.12% 64.000 62.000 - -6.06% -
401.bzip2 (avg) -3.12% 64.000 62.000 - -6.06% -
471.omnetpp -2.90% 69.000 67.000 - -60.59% -
471.omnetpp (avg) -2.90% 69.000 67.000 - -60.59% -
454.calculix (max) (percentage) -2.70% 1.480 1.440 - -69.36% -
434.zeusmp -2.63% 38.000 37.000 - -81.03% -
434.zeusmp (avg) -2.63% 38.000 37.000 - -81.03% -
470.lbm -2.27% 44.000 43.000 - -85.81% -
470.lbm (avg) -2.27% 44.000 43.000 - -85.81% -
435.gromacs -2.13% 141.000 138.000 - -48.12% -
435.gromacs (avg) -2.13% 141.000 138.000 - -48.12% -
447.dealII (avg) (percentage) -2.05% 1.460 1.430 - 33.64% -
473.astar -2.00% 100.000 98.000 - 250.00% -
473.astar (avg) -2.00% 100.000 98.000 - 250.00% -
473.astar (max) -1.86% 215.000 211.000 - 955.00% -
481.wrf (max) (percentage) -1.62% 1.850 1.820 - -42.22% -
462.libquantum (unexplained diff vs previous) (percentage) -1.33% 0.750 0.740 0.000 7300.00% 0.000
453.povray (avg) (percentage) -1.08% 1.850 1.830 - 46.40% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
433.milc 4.00% 95873.000 99705.000 - 0.00% -
456.hmmer 1.41% 231209.000 234468.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 36.27% 8205.000 11181.000 - -2.49% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
473.astar (unexplained diff vs previous) (percentage) 19350.00% 0.020 3.890 0.000 848.78% 0.000
462.libquantum (unexplained diff vs previous) (percentage) 7300.00% 0.010 0.740 0.000 -1.33% 0.000
473.astar (unexplained diff vs previous) 7233.33% 3.000 220.000 0.000 816.67% 0.000
453.povray (unexplained diff vs previous) (percentage) 3900.00% 0.020 0.800 0.000 -34.43% 0.000
429.mcf (max) (percentage) 3527.78% 0.540 19.590 - -5.68% -
473.astar (max) (percentage) 2571.43% 0.140 3.740 - 2.19% -
429.mcf (unexplained diff vs previous) (percentage) 1638.89% 0.540 9.390 0.000 449.12% 0.000
458.sjeng (unexplained diff vs previous) (percentage) 1433.33% 0.060 0.920 0.000 2966.67% 0.000
410.bwaves (unexplained diff vs previous) (percentage) 1430.00% 0.100 1.530 0.000 255.81% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) 1428.57% 0.210 3.210 0.000 1046.43% 0.000
483.xalancbmk (max) (percentage) 1297.56% 0.410 5.730 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 1100.00% 0.240 2.880 0.000 1100.00% 0.000
429.mcf (avg) (percentage) 1009.43% 0.530 5.880 - 0.68% -
403.gcc (max) (percentage) 1002.78% 0.720 7.940 - -5.14% -
473.astar (max) 955.00% 20.000 211.000 - -1.86% -
403.gcc (avg) (percentage) 945.90% 0.610 6.380 - 0.00% -
410.bwaves (max) (percentage) 852.38% 0.210 2.000 - -6.54% -
473.astar (avg) (percentage) 815.79% 0.190 1.740 - 1.75% -
470.lbm (unexplained diff vs previous) (percentage) 781.82% 0.110 0.970 0.000 259.26% 0.000
464.h264ref (unexplained diff vs previous) (percentage) 780.00% 0.100 0.880 0.000 340.00% 0.000
458.sjeng (unexplained diff vs previous) 677.78% 9.000 70.000 0.000 3400.00% 0.000
410.bwaves (unexplained diff vs previous) 506.67% 15.000 91.000 0.000 250.00% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) 430.65% 0.620 3.290 0.000 246.32% 0.000
459.GemsFDTD (max) (percentage) 418.33% 0.600 3.110 - 67.20% -
401.bzip2 (avg) (percentage) 388.64% 0.440 2.150 - 0.94% -
465.tonto (unexplained diff vs previous) 353.97% 63.000 286.000 0.000 207.53% 0.000
458.sjeng (max) (percentage) 318.18% 0.220 0.920 - -7.07% -
453.povray (unexplained diff vs previous) 300.00% 1.000 4.000 0.000 -33.33% 0.000
429.mcf (max) 298.91% 92.000 367.000 - -13.65% -
459.GemsFDTD (unexplained diff vs previous) 295.83% 48.000 190.000 0.000 1017.65% 0.000
410.bwaves (max) 293.33% 30.000 118.000 - -8.53% -
401.bzip2 (unexplained diff vs previous) (percentage) 288.51% 0.870 3.380 0.000 472.88% 0.000
401.bzip2 (max) (percentage) 271.59% 0.880 3.270 - 5.48% -
473.astar 250.00% 28.000 98.000 - -2.00% -
473.astar (avg) 250.00% 28.000 98.000 - -2.00% -
464.h264ref (unexplained diff vs previous) 240.00% 15.000 51.000 0.000 325.00% 0.000
465.tonto (unexplained diff vs previous) (percentage) 232.47% 0.770 2.560 0.000 216.05% 0.000
471.omnetpp (unexplained diff vs previous) 215.38% 26.000 82.000 0.000 1071.43% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) 179.04% 2.290 6.390 0.000 96.62% 0.000
462.libquantum (avg) (percentage) 167.69% 0.650 1.740 - -0.57% -
450.soplex (unexplained diff vs previous) (percentage) 165.17% 2.010 5.330 0.000 661.43% 0.000
470.lbm (unexplained diff vs previous) 157.14% 14.000 36.000 0.000 260.00% 0.000
450.soplex (max) (percentage) 156.85% 1.970 5.060 - 67.00% -
437.leslie3d (max) (percentage) 156.84% 2.340 6.010 - 43.44% -
458.sjeng (avg) (percentage) 142.42% 0.330 0.800 - 1.27% -
450.soplex (avg) (percentage) 139.52% 1.240 2.970 - 1.37% -
458.sjeng (max) 122.58% 31.000 69.000 - -8.00% -
482.sphinx3 (max) (percentage) 116.33% 1.470 3.180 - 8.16% -
462.libquantum (max) (percentage) 103.12% 0.640 1.300 - -5.11% -
464.h264ref (max) (percentage) 102.33% 0.430 0.870 - 4.82% -
437.leslie3d (unexplained diff vs previous) 95.92% 343.000 672.000 0.000 84.62% 0.000
435.gromacs (unexplained diff vs previous) (percentage) 93.20% 2.060 3.980 0.000 65.83% 0.000
435.gromacs (max) (percentage) 89.60% 2.020 3.830 - 55.69% -
429.mcf (unexplained diff vs previous) 89.25% 93.000 176.000 0.000 402.86% 0.000
437.leslie3d (avg) (percentage) 82.77% 2.380 4.350 - 0.69% -
437.leslie3d (max) 80.06% 351.000 632.000 - 35.04% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
464.h264ref (avg) (percentage) 68.52% 0.540 0.910 - 0.00% -
459.GemsFDTD (avg) (percentage) 62.22% 0.900 1.460 - 2.82% -
454.calculix (avg) (percentage) 52.24% 2.010 3.060 - 0.00% -
471.omnetpp (avg) (percentage) 50.31% 1.590 2.390 - 0.00% -
445.gobmk (avg) (percentage) 46.75% 0.770 1.130 - 1.80% -
453.povray (avg) (percentage) 46.40% 1.250 1.830 - -1.08% -
400.perlbench (avg) (percentage) 43.54% 1.470 2.110 - 0.48% -
445.gobmk (unexplained diff vs previous) (percentage) 40.65% 1.550 2.180 0.000 938.10% 0.000
445.gobmk (max) (percentage) 39.22% 1.530 2.130 - 30.67% -
447.dealII (avg) (percentage) 33.64% 1.070 1.430 - -2.05% -
482.sphinx3 (avg) (percentage) 33.56% 1.460 1.950 - 1.04% -
459.GemsFDTD (max) 30.71% 140.000 183.000 - 61.95% -
437.leslie3d 28.01% 357.000 457.000 - -5.38% -
437.leslie3d (avg) 28.01% 357.000 457.000 - -5.38% -
458.sjeng 27.66% 47.000 60.000 - 0.00% -
458.sjeng (avg) 27.66% 47.000 60.000 - 0.00% -
429.mcf 22.22% 90.000 110.000 - -7.56% -
429.mcf (avg) 22.22% 90.000 110.000 - -7.56% -
456.hmmer (avg) (percentage) 19.12% 0.680 0.810 - 0.00% -
435.gromacs (avg) (percentage) 15.03% 1.730 1.990 - 2.05% -
410.bwaves (avg) (percentage) 10.09% 1.090 1.200 - 0.84% -
465.tonto (max) 8.53% 293.000 318.000 - -8.62% -
465.tonto 6.57% 137.000 146.000 - 0.00% -
465.tonto (avg) 6.57% 137.000 146.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) 4.00% 0.750 0.780 0.000 129.41% 0.000

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
434.zeusmp -1.72% 165699.000 162853.000 - 0.00% -
459.GemsFDTD -1.50% 253716.000 249917.000 - 0.00% -
464.h264ref -1.46% 428364.000 422128.000 - 0.00% -
410.bwaves -1.45% 17880.000 17620.000 - 0.00% -
400.perlbench -1.03% 930652.000 921026.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.33% 9010.000 60.000 - 7.14% -
462.libquantum -98.84% 11614.000 135.000 - 0.75% -
482.sphinx3 -96.95% 29928.000 913.000 - -3.18% -
450.soplex -96.48% 15466.000 544.000 - -5.06% -
453.povray -92.41% 6544.000 497.000 - 0.81% -
444.namd -91.98% 9050.000 726.000 - -0.82% -
433.milc -91.21% 12976.000 1141.000 - -1.89% -
429.mcf -89.07% 17158.000 1875.000 - -8.58% -
436.cactusADM -87.86% 17544.000 2130.000 - 1.72% -
400.perlbench -85.77% 10650.000 1516.000 - -2.76% -
434.zeusmp -80.95% 13211.000 2517.000 - -1.99% -
401.bzip2 -80.71% 15185.000 2929.000 - -3.27% -
447.dealII -79.96% 9751.000 1954.000 - -0.31% -
459.GemsFDTD -74.68% 23356.000 5914.000 - -3.11% -
471.omnetpp -73.52% 10739.000 2844.000 - -2.80% -
456.hmmer -73.40% 11046.000 2938.000 - 0.79% -
470.lbm -71.19% 12819.000 3693.000 - -0.97% -
473.astar -62.08% 14932.000 5662.000 - -3.74% -
464.h264ref -59.96% 14556.000 5828.000 - -0.87% -
410.bwaves -59.72% 14755.000 5943.000 - -1.51% -
445.gobmk -57.08% 11995.000 5148.000 - -2.13% -
435.gromacs -54.84% 15419.000 6963.000 - -3.83% -
458.sjeng -46.72% 14275.000 7606.000 - -0.91% -
437.leslie3d -29.91% 15005.000 10517.000 - -6.01% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.23% 649.000 5.000 - -16.67% -
444.namd (unexplained diff vs previous) -99.14% 700.000 6.000 0.000 200.00% 0.000
462.libquantum (max) -98.65% 74.000 1.000 - 0.00% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
453.povray (max) -96.97% 165.000 5.000 - -37.50% -
444.namd -96.90% 355.000 11.000 - -8.33% -
444.namd (avg) -96.90% 355.000 11.000 - -8.33% -
482.sphinx3 -96.10% 436.000 17.000 - -5.56% -
482.sphinx3 (avg) -96.10% 436.000 17.000 - -5.56% -
416.gamess (max) (percentage) -95.16% 2.890 0.140 - 0.00% -
433.milc (unexplained diff vs previous) -94.82% 425.000 22.000 0.000 633.33% 0.000
433.milc (max) -94.65% 411.000 22.000 - -4.35% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
403.gcc (max) -93.75% 64.000 4.000 - 0.00% -
470.lbm (max) -93.72% 653.000 41.000 - -6.82% -
436.cactusADM (unexplained diff vs previous) -93.68% 570.000 36.000 0.000 500.00% 0.000
436.cactusADM (max) -93.48% 552.000 36.000 - 56.52% -
482.sphinx3 (max) -93.39% 439.000 29.000 - 7.41% -
433.milc -92.27% 220.000 17.000 - -5.56% -
433.milc (avg) -92.27% 220.000 17.000 - -5.56% -
447.dealII (unexplained diff vs previous) -92.11% 76.000 6.000 0.000 -68.42% 0.000
450.soplex -91.62% 191.000 16.000 - 0.00% -
450.soplex (avg) -91.62% 191.000 16.000 - 0.00% -
450.soplex (max) -91.12% 304.000 27.000 - 58.82% -
450.soplex (unexplained diff vs previous) -90.68% 311.000 29.000 0.000 625.00% 0.000
436.cactusADM -90.03% 291.000 29.000 - 3.57% -
436.cactusADM (avg) -90.03% 291.000 29.000 - 3.57% -
444.namd (unexplained diff vs previous) (percentage) -89.26% 7.730 0.830 0.000 207.41% 0.000
453.povray -88.89% 81.000 9.000 - 0.00% -
453.povray (avg) -88.89% 81.000 9.000 - 0.00% -
444.namd (max) (percentage) -88.58% 7.180 0.820 - -5.75% -
400.perlbench (max) -88.12% 345.000 41.000 - -4.65% -
471.omnetpp (max) -88.07% 662.000 79.000 - 33.90% -
400.perlbench (unexplained diff vs previous) -87.96% 357.000 43.000 0.000 104.76% 0.000
470.lbm -85.81% 303.000 43.000 - -2.27% -
470.lbm (avg) -85.81% 303.000 43.000 - -2.27% -
482.sphinx3 (unexplained diff vs previous) -83.87% 186.000 30.000 0.000 233.33% 0.000
456.hmmer (max) -83.57% 140.000 23.000 - 155.56% -
434.zeusmp (unexplained diff vs previous) -83.44% 308.000 51.000 0.000 200.00% 0.000
434.zeusmp (max) -83.39% 301.000 50.000 - 92.31% -
434.zeusmp -81.03% 195.000 37.000 - -2.63% -
434.zeusmp (avg) -81.03% 195.000 37.000 - -2.63% -
447.dealII (max) -80.37% 107.000 21.000 - -8.70% -
400.perlbench -80.13% 156.000 31.000 - -3.12% -
400.perlbench (avg) -80.13% 156.000 31.000 - -3.12% -
470.lbm (max) (percentage) -78.04% 5.100 1.120 - -5.88% -
447.dealII -74.04% 104.000 27.000 - -3.57% -
447.dealII (avg) -74.04% 104.000 27.000 - -3.57% -
456.hmmer (unexplained diff vs previous) -72.29% 83.000 23.000 0.000 130.00% 0.000
454.calculix (max) (percentage) -69.36% 4.700 1.440 - -2.70% -
456.hmmer -69.33% 75.000 23.000 - 0.00% -
456.hmmer (avg) -69.33% 75.000 23.000 - 0.00% -
416.gamess (avg) (percentage) -61.29% 2.170 0.840 - 0.00% -
471.omnetpp -60.59% 170.000 67.000 - -2.90% -
471.omnetpp (avg) -60.59% 170.000 67.000 - -2.90% -
447.dealII (unexplained diff vs previous) (percentage) -60.26% 0.780 0.310 0.000 -68.04% 0.000
459.GemsFDTD -59.05% 210.000 86.000 - 0.00% -
459.GemsFDTD (avg) -59.05% 210.000 86.000 - 0.00% -
444.namd (avg) (percentage) -58.02% 3.930 1.650 - -0.60% -
410.bwaves -55.62% 160.000 71.000 - 0.00% -
410.bwaves (avg) -55.62% 160.000 71.000 - 0.00% -
453.povray (max) (percentage) -55.34% 2.530 1.130 - -32.74% -
471.omnetpp (max) (percentage) -54.62% 6.170 2.800 - 36.59% -
470.lbm (avg) (percentage) -50.21% 2.370 1.180 - 0.00% -
435.gromacs -48.12% 266.000 138.000 - -2.13% -
435.gromacs (avg) -48.12% 266.000 138.000 - -2.13% -
436.cactusADM (unexplained diff vs previous) (percentage) -48.00% 3.250 1.690 0.000 482.76% 0.000
436.cactusADM (max) (percentage) -45.40% 3.150 1.720 - 56.36% -
481.wrf (max) (percentage) -42.22% 3.150 1.820 - -1.62% -
433.milc (unexplained diff vs previous) (percentage) -41.16% 3.280 1.930 0.000 642.31% 0.000
445.gobmk (max) -40.44% 183.000 109.000 - 28.24% -
445.gobmk (unexplained diff vs previous) -39.78% 186.000 112.000 0.000 918.18% 0.000
433.milc (max) (percentage) -38.80% 3.170 1.940 - -5.83% -
456.hmmer (max) (percentage) -37.80% 1.270 0.790 - 132.35% -
445.gobmk -36.96% 92.000 58.000 - 0.00% -
445.gobmk (avg) -36.96% 92.000 58.000 - 0.00% -
464.h264ref -32.05% 78.000 53.000 - 0.00% -
464.h264ref (avg) -32.05% 78.000 53.000 - 0.00% -
401.bzip2 (max) -28.57% 133.000 95.000 - 2.15% -
401.bzip2 (unexplained diff vs previous) -25.00% 132.000 99.000 0.000 450.00% 0.000
465.tonto (avg) (percentage) -21.56% 1.670 1.310 - 2.34% -
465.tonto (max) (percentage) -20.39% 3.580 2.850 - -6.25% -
464.h264ref (max) -19.35% 62.000 50.000 - 4.17% -
436.cactusADM (avg) (percentage) -17.47% 1.660 1.370 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) -15.22% 3.350 2.840 0.000 110.37% 0.000
400.perlbench (max) (percentage) -14.81% 3.240 2.760 - -0.72% -
435.gromacs (max) -14.47% 311.000 266.000 - 49.44% -
435.gromacs (unexplained diff vs previous) -12.89% 318.000 277.000 0.000 59.20% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) -12.88% 2.330 2.030 0.000 207.58% 0.000
434.zeusmp (max) (percentage) -12.72% 2.280 1.990 - 95.10% -
481.wrf (avg) (percentage) -12.17% 1.890 1.660 - 0.00% -
433.milc (avg) (percentage) -7.65% 1.700 1.570 - 0.00% -
401.bzip2 -6.06% 66.000 62.000 - -3.12% -
401.bzip2 (avg) -6.06% 66.000 62.000 - -3.12% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 127.59% 29.000 66.000 - 0.00% -
nb_compile_successful 72.41% 29.000 50.000 - 0.00% -
nb_execution_successful 72.41% 29.000 50.000 - 0.00% -

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1516.000 -2.76%
401.bzip2 2929.000 -3.27%
403.gcc 60.000 7.14%
410.bwaves 5943.000 -1.51%
429.mcf 1875.000 -8.58%
433.milc 1141.000 -1.89%
434.zeusmp 2517.000 -1.99%
435.gromacs 6963.000 -3.83%
436.cactusADM 2130.000 1.72%
437.leslie3d 10517.000 -6.01%
444.namd 726.000 -
445.gobmk 5148.000 -2.13%
447.dealII 1954.000 -
450.soplex 544.000 -5.06%
453.povray 497.000 -
456.hmmer 2938.000 -
458.sjeng 7606.000 -
459.GemsFDTD 5914.000 -3.11%
462.libquantum 135.000 -
464.h264ref 5828.000 -
465.tonto 11181.000 -2.49%
470.lbm 3693.000 -
471.omnetpp 2844.000 -2.80%
473.astar 5662.000 -3.74%
482.sphinx3 913.000 -3.18%
Geometric Mean 2131.953 -1.84%
Code Size (bytes)
Name Current %
400.perlbench 921026.000 -
401.bzip2 49690.000 -
403.gcc 2683585.000 -
410.bwaves 17620.000 -
429.mcf 10979.000 -
433.milc 99705.000 -
434.zeusmp 162853.000 -
435.gromacs 677825.000 -
436.cactusADM 551472.000 -
437.leslie3d 64764.000 -
444.namd 190766.000 -
445.gobmk 1399857.000 -
447.dealII 2118483.000 -
450.soplex 259893.000 -
453.povray 710554.000 -
456.hmmer 234468.000 -
458.sjeng 120025.000 -
459.GemsFDTD 249917.000 -
462.libquantum 31353.000 -
464.h264ref 422128.000 -
465.tonto 2874533.000 -
470.lbm 10916.000 -
471.omnetpp 566332.000 -
473.astar 32220.000 -
482.sphinx3 152612.000 -
Geometric Mean 207006.588 -
sample variability
Name Current %
400.perlbench 31.000 -3.12%
400.perlbench (avg) 31.000 -3.12%
400.perlbench (avg) (percentage) 2.110 -
400.perlbench (max) 41.000 -4.65%
400.perlbench (max) (percentage) 2.760 -
400.perlbench (unexplained diff vs previous) 43.000 104.76%
400.perlbench (unexplained diff vs previous) (percentage) 2.840 110.37%
401.bzip2 62.000 -3.12%
401.bzip2 (avg) 62.000 -3.12%
401.bzip2 (avg) (percentage) 2.150 -
401.bzip2 (max) 95.000 2.15%
401.bzip2 (max) (percentage) 3.270 5.48%
401.bzip2 (unexplained diff vs previous) 99.000 450.00%
401.bzip2 (unexplained diff vs previous) (percentage) 3.380 472.88%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 6.380 -
403.gcc (max) 4.000 -
403.gcc (max) (percentage) 7.940 -5.14%
410.bwaves 71.000 -
410.bwaves (avg) 71.000 -
410.bwaves (avg) (percentage) 1.200 -
410.bwaves (max) 118.000 -8.53%
410.bwaves (max) (percentage) 2.000 -6.54%
410.bwaves (unexplained diff vs previous) 91.000 250.00%
410.bwaves (unexplained diff vs previous) (percentage) 1.530 255.81%
416.gamess (avg) (percentage) 0.840 -
416.gamess (max) (percentage) 0.140 -
429.mcf 110.000 -7.56%
429.mcf (avg) 110.000 -7.56%
429.mcf (avg) (percentage) 5.880 -
429.mcf (max) 367.000 -13.65%
429.mcf (max) (percentage) 19.590 -5.68%
429.mcf (unexplained diff vs previous) 176.000 402.86%
429.mcf (unexplained diff vs previous) (percentage) 9.390 449.12%
433.milc 17.000 -5.56%
433.milc (avg) 17.000 -5.56%
433.milc (avg) (percentage) 1.570 -
433.milc (max) 22.000 -4.35%
433.milc (max) (percentage) 1.940 -5.83%
433.milc (unexplained diff vs previous) 22.000 633.33%
433.milc (unexplained diff vs previous) (percentage) 1.930 642.31%
434.zeusmp 37.000 -2.63%
434.zeusmp (avg) 37.000 -2.63%
434.zeusmp (avg) (percentage) 1.490 -
434.zeusmp (max) 50.000 92.31%
434.zeusmp (max) (percentage) 1.990 95.10%
434.zeusmp (unexplained diff vs previous) 51.000 200.00%
434.zeusmp (unexplained diff vs previous) (percentage) 2.030 207.58%
435.gromacs 138.000 -2.13%
435.gromacs (avg) 138.000 -2.13%
435.gromacs (avg) (percentage) 1.990 2.05%
435.gromacs (max) 266.000 49.44%
435.gromacs (max) (percentage) 3.830 55.69%
435.gromacs (unexplained diff vs previous) 277.000 59.20%
435.gromacs (unexplained diff vs previous) (percentage) 3.980 65.83%
436.cactusADM 29.000 3.57%
436.cactusADM (avg) 29.000 3.57%
436.cactusADM (avg) (percentage) 1.370 -
436.cactusADM (max) 36.000 56.52%
436.cactusADM (max) (percentage) 1.720 56.36%
436.cactusADM (unexplained diff vs previous) 36.000 500.00%
436.cactusADM (unexplained diff vs previous) (percentage) 1.690 482.76%
437.leslie3d 457.000 -5.38%
437.leslie3d (avg) 457.000 -5.38%
437.leslie3d (avg) (percentage) 4.350 -
437.leslie3d (max) 632.000 35.04%
437.leslie3d (max) (percentage) 6.010 43.44%
437.leslie3d (unexplained diff vs previous) 672.000 84.62%
437.leslie3d (unexplained diff vs previous) (percentage) 6.390 96.62%
444.namd 11.000 -8.33%
444.namd (avg) 11.000 -8.33%
444.namd (avg) (percentage) 1.650 -
444.namd (max) 5.000 -16.67%
444.namd (max) (percentage) 0.820 -5.75%
444.namd (unexplained diff vs previous) 6.000 200.00%
444.namd (unexplained diff vs previous) (percentage) 0.830 207.41%
445.gobmk 58.000 -
445.gobmk (avg) 58.000 -
445.gobmk (avg) (percentage) 1.130 1.80%
445.gobmk (max) 109.000 28.24%
445.gobmk (max) (percentage) 2.130 30.67%
445.gobmk (unexplained diff vs previous) 112.000 918.18%
445.gobmk (unexplained diff vs previous) (percentage) 2.180 938.10%
447.dealII 27.000 -3.57%
447.dealII (avg) 27.000 -3.57%
447.dealII (avg) (percentage) 1.430 -2.05%
447.dealII (max) 21.000 -8.70%
447.dealII (max) (percentage) 1.100 -9.09%
447.dealII (unexplained diff vs previous) 6.000 -68.42%
447.dealII (unexplained diff vs previous) (percentage) 0.310 -68.04%
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 2.970 1.37%
450.soplex (max) 27.000 58.82%
450.soplex (max) (percentage) 5.060 67.00%
450.soplex (unexplained diff vs previous) 29.000 625.00%
450.soplex (unexplained diff vs previous) (percentage) 5.330 661.43%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.830 -1.08%
453.povray (max) 5.000 -37.50%
453.povray (max) (percentage) 1.130 -32.74%
453.povray (unexplained diff vs previous) 4.000 -33.33%
453.povray (unexplained diff vs previous) (percentage) 0.800 -34.43%
454.calculix (avg) (percentage) 3.060 -
454.calculix (max) (percentage) 1.440 -2.70%
456.hmmer 23.000 -
456.hmmer (avg) 23.000 -
456.hmmer (avg) (percentage) 0.810 -
456.hmmer (max) 23.000 155.56%
456.hmmer (max) (percentage) 0.790 132.35%
456.hmmer (unexplained diff vs previous) 23.000 130.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.780 129.41%
458.sjeng 60.000 -
458.sjeng (avg) 60.000 -
458.sjeng (avg) (percentage) 0.800 1.27%
458.sjeng (max) 69.000 -8.00%
458.sjeng (max) (percentage) 0.920 -7.07%
458.sjeng (unexplained diff vs previous) 70.000 3400.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.920 2966.67%
459.GemsFDTD 86.000 -
459.GemsFDTD (avg) 86.000 -
459.GemsFDTD (avg) (percentage) 1.460 2.82%
459.GemsFDTD (max) 183.000 61.95%
459.GemsFDTD (max) (percentage) 3.110 67.20%
459.GemsFDTD (unexplained diff vs previous) 190.000 1017.65%
459.GemsFDTD (unexplained diff vs previous) (percentage) 3.210 1046.43%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.740 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.300 -5.11%
462.libquantum (unexplained diff vs previous) 1.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.740 -1.33%
464.h264ref 53.000 -
464.h264ref (avg) 53.000 -
464.h264ref (avg) (percentage) 0.910 -
464.h264ref (max) 50.000 4.17%
464.h264ref (max) (percentage) 0.870 4.82%
464.h264ref (unexplained diff vs previous) 51.000 325.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.880 340.00%
465.tonto 146.000 -
465.tonto (avg) 146.000 -
465.tonto (avg) (percentage) 1.310 2.34%
465.tonto (max) 318.000 -8.62%
465.tonto (max) (percentage) 2.850 -6.25%
465.tonto (unexplained diff vs previous) 286.000 207.53%
465.tonto (unexplained diff vs previous) (percentage) 2.560 216.05%
470.lbm 43.000 -2.27%
470.lbm (avg) 43.000 -2.27%
470.lbm (avg) (percentage) 1.180 -
470.lbm (max) 41.000 -6.82%
470.lbm (max) (percentage) 1.120 -5.88%
470.lbm (unexplained diff vs previous) 36.000 260.00%
470.lbm (unexplained diff vs previous) (percentage) 0.970 259.26%
471.omnetpp 67.000 -2.90%
471.omnetpp (avg) 67.000 -2.90%
471.omnetpp (avg) (percentage) 2.390 -
471.omnetpp (max) 79.000 33.90%
471.omnetpp (max) (percentage) 2.800 36.59%
471.omnetpp (unexplained diff vs previous) 82.000 1071.43%
471.omnetpp (unexplained diff vs previous) (percentage) 2.880 1100.00%
473.astar 98.000 -2.00%
473.astar (avg) 98.000 -2.00%
473.astar (avg) (percentage) 1.740 1.75%
473.astar (max) 211.000 -1.86%
473.astar (max) (percentage) 3.740 2.19%
473.astar (unexplained diff vs previous) 220.000 816.67%
473.astar (unexplained diff vs previous) (percentage) 3.890 848.78%
481.wrf (avg) (percentage) 1.660 -
481.wrf (max) (percentage) 1.820 -1.62%
482.sphinx3 17.000 -5.56%
482.sphinx3 (avg) 17.000 -5.56%
482.sphinx3 (avg) (percentage) 1.950 1.04%
482.sphinx3 (max) 29.000 7.41%
482.sphinx3 (max) (percentage) 3.180 8.16%
482.sphinx3 (unexplained diff vs previous) 30.000 233.33%
482.sphinx3 (unexplained diff vs previous) (percentage) 3.290 246.32%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.730 -
Geometric Mean 10.315 46.61%