Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c2f60ac565f1d369fde98146a16f1d3ef79e1000 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/e722a1f42b28092c9f709a3f758fc4fe57db32b0 (+ 34 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=cb9cae962c315139941a6eb0d07c6a71b658f5bb (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=911893ac7b96a82cd5251f3bb864b892a8fe91cf (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/34/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0034 (basepoints/gcc-14-00018-ge722a1f42b2) 2023-04-16T13:46:45 0:00:00
Previous 0033 (3-changed-components) 2023-04-14T21:05:43 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 44 74
Performance Improvements 131 141
Added Tests 6 6
Unchanged Tests 1055 1015
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 13.81% 11248.000 12801.000 - -29.16% -
403.gcc 5.26% 57.000 60.000 - -99.33% -
400.perlbench 2.17% 1570.000 1604.000 - -85.70% -
434.zeusmp 1.43% 2727.000 2766.000 - -86.79% -
453.povray 1.08% 554.000 560.000 - -92.43% -
450.soplex 1.01% 596.000 602.000 - -96.17% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (unexplained diff vs previous) 1033.33% 3.000 34.000 - -2.86% -
400.perlbench (unexplained diff vs previous) (percentage) 1015.79% 0.190 2.120 - 583.87% -
445.gobmk (unexplained diff vs previous) 800.00% 1.000 9.000 - -50.00% -
445.gobmk (unexplained diff vs previous) (percentage) 800.00% 0.020 0.180 - 20.00% -
410.bwaves (unexplained diff vs previous) (percentage) 500.00% 0.030 0.180 - -89.66% -
465.tonto (unexplained diff vs previous) (percentage) 500.00% 0.120 0.720 - -36.28% -
465.tonto (unexplained diff vs previous) 487.50% 16.000 94.000 - -36.49% -
410.bwaves (unexplained diff vs previous) 450.00% 2.000 11.000 - -95.72% -
437.leslie3d (unexplained diff vs previous) 341.19% 352.000 1553.000 - 730.48% -
437.leslie3d (unexplained diff vs previous) (percentage) 287.54% 3.130 12.130 - 1077.67% -
437.leslie3d (max) 220.11% 552.000 1767.000 - 215.54% -
437.leslie3d (max) (percentage) 181.26% 4.910 13.810 - 345.48% -
470.lbm (unexplained diff vs previous) 175.00% 4.000 11.000 - -98.66% -
470.lbm (unexplained diff vs previous) (percentage) 163.64% 0.110 0.290 - -94.25% -
400.perlbench (max) 112.50% 16.000 34.000 - -61.80% -
400.perlbench (max) (percentage) 106.67% 1.050 2.170 - 171.25% -
434.zeusmp (unexplained diff vs previous) 105.26% 19.000 39.000 - -91.70% -
434.zeusmp (unexplained diff vs previous) (percentage) 101.43% 0.700 1.410 - -37.33% -
434.zeusmp (max) 95.00% 20.000 39.000 - -91.89% -
456.hmmer (unexplained diff vs previous) 91.67% 12.000 23.000 - -76.53% -
456.hmmer (unexplained diff vs previous) (percentage) 88.89% 0.270 0.510 - -1.92% -
434.zeusmp (max) (percentage) 88.16% 0.760 1.430 - -37.83% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 73.33% 0.150 0.260 - -88.70% -
459.GemsFDTD (unexplained diff vs previous) 72.73% 11.000 19.000 - -96.75% -
465.tonto (max) 32.39% 71.000 94.000 - -57.85% -
465.tonto (max) (percentage) 30.91% 0.550 0.720 - -57.89% -
437.leslie3d 25.57% 395.000 496.000 - 64.78% -
437.leslie3d (avg) 25.57% 395.000 496.000 - 64.78% -
456.hmmer (max) 21.05% 19.000 23.000 - -76.53% -
456.hmmer (max) (percentage) 20.93% 0.430 0.520 - 0.00% -
453.povray (max) 20.00% 5.000 6.000 - -96.86% -
464.h264ref (unexplained diff vs previous) 20.00% 10.000 12.000 - -79.31% -
464.h264ref (unexplained diff vs previous) (percentage) 18.75% 0.160 0.190 - -50.00% -
437.leslie3d (avg) (percentage) 10.23% 3.520 3.880 - 132.34% -
435.gromacs (unexplained diff vs previous) 9.09% 55.000 60.000 - -79.45% -
459.GemsFDTD (max) (percentage) 8.33% 0.240 0.260 - -88.60% -
435.gromacs (unexplained diff vs previous) (percentage) 8.22% 0.730 0.790 - -54.34% -
459.GemsFDTD (max) 5.88% 17.000 18.000 - -96.89% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
473.astar -1.24% 4935.000 4874.000 - -62.46% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) -93.48% 92.000 6.000 - -83.78% -
429.mcf (max) (percentage) -92.65% 9.800 0.720 - 125.00% -
471.omnetpp (unexplained diff vs previous) (percentage) -89.31% 1.590 0.170 - -74.63% -
471.omnetpp (unexplained diff vs previous) -89.13% 46.000 5.000 - -91.94% -
462.libquantum (unexplained diff vs previous) (percentage) -80.06% 3.110 0.620 - 675.00% -
462.libquantum (unexplained diff vs previous) -80.00% 5.000 1.000 - -92.86% -
450.soplex (max) -77.78% 27.000 6.000 - -91.30% -
450.soplex (max) (percentage) -77.01% 4.610 1.060 - 140.91% -
416.gamess (unexplained diff vs previous) (percentage) -61.90% 0.210 0.080 - -97.38% -
416.gamess (unexplained diff vs previous) -61.29% 31.000 12.000 - -98.66% -
458.sjeng (unexplained diff vs previous) -58.33% 12.000 5.000 - -89.58% -
447.dealII (unexplained diff vs previous) (percentage) -57.53% 0.730 0.310 - -58.67% -
447.dealII (unexplained diff vs previous) -57.14% 14.000 6.000 - -91.18% -
458.sjeng (unexplained diff vs previous) (percentage) -56.25% 0.160 0.070 - -76.67% -
433.milc (max) -33.33% 3.000 2.000 - -97.94% -
444.namd (max) -20.00% 10.000 8.000 - -99.09% -
462.libquantum (max) -20.00% 5.000 4.000 - -69.23% -
471.omnetpp (max) -17.39% 46.000 38.000 - -77.91% -
482.sphinx3 (unexplained diff vs previous) (percentage) -16.92% 0.650 0.540 - -82.35% -
462.libquantum (max) (percentage) -16.82% 3.210 2.670 - 3237.50% -
471.omnetpp (max) (percentage) -16.77% 1.610 1.340 - -27.96% -
482.sphinx3 (unexplained diff vs previous) -16.67% 6.000 5.000 - -99.55% -
481.wrf (unexplained diff vs previous) (percentage) -15.62% 0.320 0.270 - -93.37% -
481.wrf (unexplained diff vs previous) -15.00% 20.000 17.000 - -98.10% -
450.soplex (unexplained diff vs previous) (percentage) -14.53% 1.170 1.000 - 127.27% -
450.soplex (unexplained diff vs previous) -14.29% 7.000 6.000 - -91.30% -
401.bzip2 (max) -12.50% 24.000 21.000 - -68.66% -
416.gamess (max) -12.50% 40.000 35.000 - -96.21% -
447.dealII (max) (percentage) -12.50% 0.720 0.630 - -61.82% -
416.gamess (max) (percentage) -11.11% 0.270 0.240 - -92.36% -
458.sjeng (max) -11.11% 18.000 16.000 - -66.67% -
444.namd (max) (percentage) -10.81% 1.110 0.990 - -82.03% -
473.astar (max) -10.45% 67.000 60.000 - -1.64% -
454.calculix (max) (percentage) -10.00% 2.600 2.340 - 178.57% -
444.namd (unexplained diff vs previous) -10.00% 10.000 9.000 - -98.92% -
401.bzip2 (max) (percentage) -9.88% 0.810 0.730 - 82.50% -
444.namd (unexplained diff vs previous) (percentage) -9.09% 1.100 1.000 - -80.88% -
473.astar (max) (percentage) -8.82% 1.360 1.240 - 163.83% -
458.sjeng (max) (percentage) -8.33% 0.240 0.220 - -26.67% -
447.dealII (max) -7.69% 13.000 12.000 - -92.00% -
470.lbm (max) -7.69% 13.000 12.000 - -98.66% -
473.astar (unexplained diff vs previous) -7.58% 66.000 61.000 - 0.00% -
433.milc (max) (percentage) -7.14% 0.280 0.260 - -63.38% -
473.astar (unexplained diff vs previous) (percentage) -6.72% 1.340 1.250 - 165.96% -
445.gobmk (max) -6.45% 31.000 29.000 - -12.12% -
482.sphinx3 (max) -6.25% 16.000 15.000 - -98.70% -
482.sphinx3 (max) (percentage) -6.04% 1.820 1.710 - -45.71% -
435.gromacs (max) (percentage) -6.04% 1.490 1.400 - -20.45% -
481.wrf (max) -5.56% 54.000 51.000 - -94.42% -
470.lbm (max) (percentage) -5.56% 0.360 0.340 - -93.80% -
435.gromacs (max) -5.36% 112.000 106.000 - -64.19% -
436.cactusADM (max) (percentage) -5.26% 0.380 0.360 - -92.87% -
464.h264ref (max) -5.13% 39.000 37.000 - -37.29% -
445.gobmk (max) (percentage) -4.92% 0.610 0.580 - 114.81% -
482.sphinx3 -4.88% 41.000 39.000 - -95.46% -
482.sphinx3 (avg) -4.88% 41.000 39.000 - -95.46% -
464.h264ref (max) (percentage) -4.76% 0.630 0.600 - 53.85% -
481.wrf (max) (percentage) -4.65% 0.860 0.820 - -80.29% -
401.bzip2 (unexplained diff vs previous) -4.55% 22.000 21.000 - -69.57% -
429.mcf -4.41% 68.000 65.000 - 16.07% -
429.mcf (avg) -4.41% 68.000 65.000 - 16.07% -
481.wrf -4.10% 244.000 234.000 - -58.58% -
481.wrf (avg) -4.10% 244.000 234.000 - -58.58% -
401.bzip2 (unexplained diff vs previous) (percentage) -4.05% 0.740 0.710 - 73.17% -
458.sjeng -3.97% 151.000 145.000 - 302.78% -
458.sjeng (avg) -3.97% 151.000 145.000 - 302.78% -
454.calculix (avg) (percentage) -3.90% 5.130 4.930 - 142.86% -
481.wrf (avg) (percentage) -3.87% 3.880 3.730 - 45.14% -
464.h264ref -3.80% 158.000 152.000 - 157.63% -
464.h264ref (avg) -3.80% 158.000 152.000 - 157.63% -
459.GemsFDTD (avg) (percentage) -3.59% 3.620 3.490 - 151.08% -
445.gobmk (avg) (percentage) -3.59% 2.230 2.150 - 877.27% -
458.sjeng (avg) (percentage) -3.55% 1.970 1.900 - 726.09% -
445.gobmk -3.54% 113.000 109.000 - 303.70% -
445.gobmk (avg) -3.54% 113.000 109.000 - 303.70% -
464.h264ref (avg) (percentage) -3.53% 2.550 2.460 - 530.77% -
429.mcf (avg) (percentage) -3.47% 7.200 6.950 - 1347.92% -
459.GemsFDTD -3.46% 260.000 251.000 - -28.69% -
459.GemsFDTD (avg) -3.46% 260.000 251.000 - -28.69% -
471.omnetpp (avg) (percentage) -3.38% 4.440 4.290 - 263.56% -
447.dealII (avg) (percentage) -3.37% 3.260 3.150 - 262.07% -
433.milc (avg) (percentage) -3.35% 3.880 3.750 - 443.48% -
450.soplex -3.33% 30.000 29.000 - -21.62% -
450.soplex (avg) -3.33% 30.000 29.000 - -21.62% -
403.gcc (max) (percentage) -3.25% 5.840 5.650 - 857.63% -
447.dealII -3.23% 62.000 60.000 - -24.05% -
447.dealII (avg) -3.23% 62.000 60.000 - -24.05% -
410.bwaves -3.17% 189.000 183.000 - -36.68% -
410.bwaves (avg) -3.17% 189.000 183.000 - -36.68% -
436.cactusADM (avg) (percentage) -3.17% 3.790 3.670 - 52.92% -
470.lbm (avg) (percentage) -3.16% 4.430 4.290 - 15.01% -
410.bwaves (avg) (percentage) -3.14% 3.180 3.080 - 57.14% -
416.gamess (avg) (percentage) -3.14% 2.230 2.160 - 30.91% -
471.omnetpp -3.12% 128.000 124.000 - 13.76% -
471.omnetpp (avg) -3.12% 128.000 124.000 - 13.76% -
436.cactusADM -3.06% 98.000 95.000 - -83.76% -
436.cactusADM (avg) -3.06% 98.000 95.000 - -83.76% -
416.gamess -3.03% 330.000 320.000 - -34.02% -
416.gamess (avg) -3.03% 330.000 320.000 - -34.02% -
470.lbm -3.03% 165.000 160.000 - -73.68% -
470.lbm (avg) -3.03% 165.000 160.000 - -73.68% -
450.soplex (avg) (percentage) -2.95% 5.080 4.930 - 1954.17% -
482.sphinx3 (avg) (percentage) -2.93% 4.440 4.310 - 83.40% -
462.libquantum (avg) (percentage) -2.88% 3.470 3.370 - 1673.68% -
473.astar -2.88% 139.000 135.000 - 164.71% -
473.astar (avg) -2.88% 139.000 135.000 - 164.71% -
444.namd -2.86% 35.000 34.000 - -91.79% -
444.namd (avg) -2.86% 35.000 34.000 - -91.79% -
435.gromacs (avg) (percentage) -2.76% 3.260 3.170 - 233.68% -
456.hmmer (avg) (percentage) -2.73% 2.200 2.140 - 613.33% -
401.bzip2 -2.70% 111.000 108.000 - 140.00% -
401.bzip2 (avg) -2.70% 111.000 108.000 - 140.00% -
401.bzip2 (avg) (percentage) -2.66% 3.760 3.660 - 1255.56% -
444.namd (avg) (percentage) -2.53% 3.960 3.860 - 48.46% -
453.povray (avg) (percentage) -2.49% 3.610 3.520 - 130.07% -
433.milc -2.33% 43.000 42.000 - -55.32% -
433.milc (avg) -2.33% 43.000 42.000 - -55.32% -
434.zeusmp (avg) (percentage) -2.26% 3.540 3.460 - 183.61% -
456.hmmer -2.06% 97.000 95.000 - 69.64% -
456.hmmer (avg) -2.06% 97.000 95.000 - 69.64% -
435.gromacs -2.03% 246.000 241.000 - 50.62% -
435.gromacs (avg) -2.03% 246.000 241.000 - 50.62% -
410.bwaves (max) (percentage) -1.82% 0.550 0.540 - -69.49% -
473.astar (avg) (percentage) -1.77% 2.820 2.770 - 592.50% -
465.tonto (avg) (percentage) -1.72% 1.160 1.140 - 0.00% -
465.tonto -1.32% 151.000 149.000 - 0.68% -
465.tonto (avg) -1.32% 151.000 149.000 - 0.68% -
400.perlbench (avg) (percentage) -1.25% 3.200 3.160 - 710.26% -
434.zeusmp -1.04% 96.000 95.000 - -62.75% -
434.zeusmp (avg) -1.04% 96.000 95.000 - -62.75% -

Added Tests - sample variability
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
436.cactusADM (unexplained diff vs previous)
436.cactusADM (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 3237.50% 0.080 2.670 - -16.82% -
450.soplex (avg) (percentage) 1954.17% 0.240 4.930 - -2.95% -
462.libquantum (avg) (percentage) 1673.68% 0.190 3.370 - -2.88% -
429.mcf (avg) (percentage) 1347.92% 0.480 6.950 - -3.47% -
401.bzip2 (avg) (percentage) 1255.56% 0.270 3.660 - -2.66% -
437.leslie3d (unexplained diff vs previous) (percentage) 1077.67% 1.030 12.130 - 287.54% -
445.gobmk (avg) (percentage) 877.27% 0.220 2.150 - -3.59% -
403.gcc (max) (percentage) 857.63% 0.590 5.650 - -3.25% -
437.leslie3d (unexplained diff vs previous) 730.48% 187.000 1553.000 - 341.19% -
458.sjeng (avg) (percentage) 726.09% 0.230 1.900 - -3.55% -
400.perlbench (avg) (percentage) 710.26% 0.390 3.160 - -1.25% -
462.libquantum (unexplained diff vs previous) (percentage) 675.00% 0.080 0.620 - -80.06% -
456.hmmer (avg) (percentage) 613.33% 0.300 2.140 - -2.73% -
473.astar (avg) (percentage) 592.50% 0.400 2.770 - -1.77% -
400.perlbench (unexplained diff vs previous) (percentage) 583.87% 0.310 2.120 - 1015.79% -
464.h264ref (avg) (percentage) 530.77% 0.390 2.460 - -3.53% -
433.milc (avg) (percentage) 443.48% 0.690 3.750 - -3.35% -
437.leslie3d (max) (percentage) 345.48% 3.100 13.810 - 181.26% -
445.gobmk 303.70% 27.000 109.000 - -3.54% -
445.gobmk (avg) 303.70% 27.000 109.000 - -3.54% -
458.sjeng 302.78% 36.000 145.000 - -3.97% -
458.sjeng (avg) 302.78% 36.000 145.000 - -3.97% -
403.gcc (avg) (percentage) 295.35% 0.430 1.700 - 0.00% -
471.omnetpp (avg) (percentage) 263.56% 1.180 4.290 - -3.38% -
447.dealII (avg) (percentage) 262.07% 0.870 3.150 - -3.37% -
435.gromacs (avg) (percentage) 233.68% 0.950 3.170 - -2.76% -
437.leslie3d (max) 215.54% 560.000 1767.000 - 220.11% -
434.zeusmp (avg) (percentage) 183.61% 1.220 3.460 - -2.26% -
454.calculix (max) (percentage) 178.57% 0.840 2.340 - -10.00% -
400.perlbench (max) (percentage) 171.25% 0.800 2.170 - 106.67% -
473.astar (unexplained diff vs previous) (percentage) 165.96% 0.470 1.250 - -6.72% -
473.astar 164.71% 51.000 135.000 - -2.88% -
473.astar (avg) 164.71% 51.000 135.000 - -2.88% -
473.astar (max) (percentage) 163.83% 0.470 1.240 - -8.82% -
464.h264ref 157.63% 59.000 152.000 - -3.80% -
464.h264ref (avg) 157.63% 59.000 152.000 - -3.80% -
459.GemsFDTD (avg) (percentage) 151.08% 1.390 3.490 - -3.59% -
454.calculix (avg) (percentage) 142.86% 2.030 4.930 - -3.90% -
450.soplex (max) (percentage) 140.91% 0.440 1.060 - -77.01% -
401.bzip2 140.00% 45.000 108.000 - -2.70% -
401.bzip2 (avg) 140.00% 45.000 108.000 - -2.70% -
437.leslie3d (avg) (percentage) 132.34% 1.670 3.880 - 10.23% -
453.povray (avg) (percentage) 130.07% 1.530 3.520 - -2.49% -
450.soplex (unexplained diff vs previous) (percentage) 127.27% 0.440 1.000 - -14.53% -
429.mcf (max) (percentage) 125.00% 0.320 0.720 - -92.65% -
445.gobmk (max) (percentage) 114.81% 0.270 0.580 - -4.92% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
483.xalancbmk (max) (percentage) 93.48% 0.920 1.780 - -0.56% -
482.sphinx3 (avg) (percentage) 83.40% 2.350 4.310 - -2.93% -
401.bzip2 (max) (percentage) 82.50% 0.400 0.730 - -9.88% -
401.bzip2 (unexplained diff vs previous) (percentage) 73.17% 0.410 0.710 - -4.05% -
456.hmmer 69.64% 56.000 95.000 - -2.06% -
456.hmmer (avg) 69.64% 56.000 95.000 - -2.06% -
437.leslie3d 64.78% 301.000 496.000 - 25.57% -
437.leslie3d (avg) 64.78% 301.000 496.000 - 25.57% -
410.bwaves (avg) (percentage) 57.14% 1.960 3.080 - -3.14% -
464.h264ref (max) (percentage) 53.85% 0.390 0.600 - -4.76% -
436.cactusADM (avg) (percentage) 52.92% 2.400 3.670 - -3.17% -
435.gromacs 50.62% 160.000 241.000 - -2.03% -
435.gromacs (avg) 50.62% 160.000 241.000 - -2.03% -
444.namd (avg) (percentage) 48.46% 2.600 3.860 - -2.53% -
481.wrf (avg) (percentage) 45.14% 2.570 3.730 - -3.87% -
416.gamess (avg) (percentage) 30.91% 1.650 2.160 - -3.14% -
445.gobmk (unexplained diff vs previous) (percentage) 20.00% 0.150 0.180 - 800.00% -
400.perlbench 16.28% 43.000 50.000 - 0.00% -
400.perlbench (avg) 16.28% 43.000 50.000 - 0.00% -
429.mcf 16.07% 56.000 65.000 - -4.41% -
429.mcf (avg) 16.07% 56.000 65.000 - -4.41% -
470.lbm (avg) (percentage) 15.01% 3.730 4.290 - -3.16% -
471.omnetpp 13.76% 109.000 124.000 - -3.12% -
471.omnetpp (avg) 13.76% 109.000 124.000 - -3.12% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -
nb_total_tests -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.81% 32491.000 61.000 - 0.00% -
403.gcc -99.33% 8890.000 60.000 - 5.26% -
462.libquantum -99.05% 17005.000 162.000 - 0.62% -
482.sphinx3 -97.47% 36586.000 925.000 - -0.54% -
450.soplex -96.17% 15709.000 602.000 - 1.01% -
444.namd -94.37% 15957.000 899.000 - -0.99% -
453.povray -92.43% 7395.000 560.000 - 1.08% -
429.mcf -91.94% 11727.000 945.000 - 0.00% -
433.milc -91.77% 13668.000 1125.000 - -0.09% -
436.cactusADM -89.31% 24396.000 2607.000 - 0.00% -
434.zeusmp -86.79% 20932.000 2766.000 - 1.43% -
400.perlbench -85.70% 11215.000 1604.000 - 2.17% -
401.bzip2 -82.58% 16971.000 2957.000 - -0.71% -
447.dealII -78.78% 9101.000 1931.000 - 0.31% -
470.lbm -77.10% 16312.000 3736.000 - -0.29% -
456.hmmer -76.47% 18995.000 4469.000 - 0.52% -
459.GemsFDTD -71.58% 25364.000 7209.000 - 0.26% -
481.wrf -71.39% 21991.000 6291.000 - -0.27% -
471.omnetpp -68.65% 9265.000 2905.000 - 0.17% -
473.astar -62.46% 12983.000 4874.000 - -1.24% -
410.bwaves -59.62% 14745.000 5954.000 - -0.18% -
464.h264ref -59.23% 15208.000 6201.000 - -0.19% -
445.gobmk -59.04% 12396.000 5077.000 - -0.18% -
435.gromacs -54.71% 16853.000 7632.000 - 0.79% -
458.sjeng -52.15% 16018.000 7665.000 - -0.07% -
416.gamess -49.62% 29409.000 14815.000 - -0.08% -
437.leslie3d -29.16% 18070.000 12801.000 - 13.81% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.63% 272.000 1.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -99.55% 1118.000 5.000 - -16.67% -
454.calculix -99.54% 659.000 3.000 - 0.00% -
454.calculix (avg) -99.54% 659.000 3.000 - 0.00% -
436.cactusADM (max) -99.27% 1231.000 9.000 - 0.00% -
444.namd (max) -99.09% 879.000 8.000 - -20.00% -
433.milc (unexplained diff vs previous) -98.96% 96.000 1.000 - 0.00% -
444.namd (unexplained diff vs previous) -98.92% 834.000 9.000 - -10.00% -
482.sphinx3 (max) -98.70% 1152.000 15.000 - -6.25% -
470.lbm (unexplained diff vs previous) -98.66% 822.000 11.000 - 175.00% -
416.gamess (unexplained diff vs previous) -98.66% 896.000 12.000 - -61.29% -
470.lbm (max) -98.66% 893.000 12.000 - -7.69% -
481.wrf (unexplained diff vs previous) -98.10% 894.000 17.000 - -15.00% -
433.milc (max) -97.94% 97.000 2.000 - -33.33% -
416.gamess (unexplained diff vs previous) (percentage) -97.38% 3.050 0.080 - -61.90% -
403.gcc -97.37% 38.000 1.000 - 0.00% -
403.gcc (avg) -97.37% 38.000 1.000 - 0.00% -
459.GemsFDTD (max) -96.89% 578.000 18.000 - 5.88% -
453.povray (max) -96.86% 191.000 6.000 - 20.00% -
459.GemsFDTD (unexplained diff vs previous) -96.75% 584.000 19.000 - 72.73% -
453.povray (unexplained diff vs previous) -96.45% 169.000 6.000 - 0.00% -
416.gamess (max) -96.21% 923.000 35.000 - -12.50% -
410.bwaves (unexplained diff vs previous) -95.72% 257.000 11.000 - 450.00% -
482.sphinx3 -95.46% 859.000 39.000 - -4.88% -
482.sphinx3 (avg) -95.46% 859.000 39.000 - -4.88% -
481.wrf (max) -94.42% 914.000 51.000 - -5.56% -
470.lbm (unexplained diff vs previous) (percentage) -94.25% 5.040 0.290 - 163.64% -
403.gcc (max) -94.23% 52.000 3.000 - 0.00% -
470.lbm (max) (percentage) -93.80% 5.480 0.340 - -5.56% -
481.wrf (unexplained diff vs previous) (percentage) -93.37% 4.070 0.270 - -15.62% -
436.cactusADM (max) (percentage) -92.87% 5.050 0.360 - -5.26% -
462.libquantum (unexplained diff vs previous) -92.86% 14.000 1.000 - -80.00% -
416.gamess (max) (percentage) -92.36% 3.140 0.240 - -11.11% -
447.dealII (max) -92.00% 150.000 12.000 - -7.69% -
471.omnetpp (unexplained diff vs previous) -91.94% 62.000 5.000 - -89.13% -
434.zeusmp (max) -91.89% 481.000 39.000 - 95.00% -
444.namd -91.79% 414.000 34.000 - -2.86% -
444.namd (avg) -91.79% 414.000 34.000 - -2.86% -
434.zeusmp (unexplained diff vs previous) -91.70% 470.000 39.000 - 105.26% -
450.soplex (max) -91.30% 69.000 6.000 - -77.78% -
450.soplex (unexplained diff vs previous) -91.30% 69.000 6.000 - -14.29% -
447.dealII (unexplained diff vs previous) -91.18% 68.000 6.000 - -57.14% -
410.bwaves (unexplained diff vs previous) (percentage) -89.66% 1.740 0.180 - 500.00% -
458.sjeng (unexplained diff vs previous) -89.58% 48.000 5.000 - -58.33% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -88.70% 2.300 0.260 - 73.33% -
459.GemsFDTD (max) (percentage) -88.60% 2.280 0.260 - 8.33% -
410.bwaves (max) -87.69% 260.000 32.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -87.14% 0.700 0.090 - 0.00% -
462.libquantum -84.38% 32.000 5.000 - 0.00% -
462.libquantum (avg) -84.38% 32.000 5.000 - 0.00% -
429.mcf (max) -83.78% 37.000 6.000 - -93.48% -
436.cactusADM -83.76% 585.000 95.000 - -3.06% -
436.cactusADM (avg) -83.76% 585.000 95.000 - -3.06% -
453.povray -83.19% 113.000 19.000 - 0.00% -
453.povray (avg) -83.19% 113.000 19.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -82.35% 3.060 0.540 - -16.92% -
444.namd (max) (percentage) -82.03% 5.510 0.990 - -10.81% -
444.namd (unexplained diff vs previous) (percentage) -80.88% 5.230 1.000 - -9.09% -
481.wrf (max) (percentage) -80.29% 4.160 0.820 - -4.65% -
435.gromacs (unexplained diff vs previous) -79.45% 292.000 60.000 - 9.09% -
464.h264ref (unexplained diff vs previous) -79.31% 58.000 12.000 - 20.00% -
471.omnetpp (max) -77.91% 172.000 38.000 - -17.39% -
458.sjeng (unexplained diff vs previous) (percentage) -76.67% 0.300 0.070 - -56.25% -
456.hmmer (max) -76.53% 98.000 23.000 - 21.05% -
456.hmmer (unexplained diff vs previous) -76.53% 98.000 23.000 - 91.67% -
471.omnetpp (unexplained diff vs previous) (percentage) -74.63% 0.670 0.170 - -89.31% -
470.lbm -73.68% 608.000 160.000 - -3.03% -
470.lbm (avg) -73.68% 608.000 160.000 - -3.03% -
401.bzip2 (unexplained diff vs previous) -69.57% 69.000 21.000 - -4.55% -
410.bwaves (max) (percentage) -69.49% 1.770 0.540 - -1.82% -
462.libquantum (max) -69.23% 13.000 4.000 - -20.00% -
401.bzip2 (max) -68.66% 67.000 21.000 - -12.50% -
458.sjeng (max) -66.67% 48.000 16.000 - -11.11% -
435.gromacs (max) -64.19% 296.000 106.000 - -5.36% -
433.milc (max) (percentage) -63.38% 0.710 0.260 - -7.14% -
434.zeusmp -62.75% 255.000 95.000 - -1.04% -
434.zeusmp (avg) -62.75% 255.000 95.000 - -1.04% -
447.dealII (max) (percentage) -61.82% 1.650 0.630 - -12.50% -
400.perlbench (max) -61.80% 89.000 34.000 - 112.50% -
447.dealII (unexplained diff vs previous) (percentage) -58.67% 0.750 0.310 - -57.53% -
481.wrf -58.58% 565.000 234.000 - -4.10% -
481.wrf (avg) -58.58% 565.000 234.000 - -4.10% -
453.povray (max) (percentage) -58.30% 2.590 1.080 - 0.93% -
465.tonto (max) (percentage) -57.89% 1.710 0.720 - 30.91% -
465.tonto (max) -57.85% 223.000 94.000 - 32.39% -
433.milc -55.32% 94.000 42.000 - -2.33% -
433.milc (avg) -55.32% 94.000 42.000 - -2.33% -
435.gromacs (unexplained diff vs previous) (percentage) -54.34% 1.730 0.790 - 8.22% -
453.povray (unexplained diff vs previous) (percentage) -53.28% 2.290 1.070 - -0.93% -
445.gobmk (unexplained diff vs previous) -50.00% 18.000 9.000 - 800.00% -
464.h264ref (unexplained diff vs previous) (percentage) -50.00% 0.380 0.190 - 18.75% -
482.sphinx3 (max) (percentage) -45.71% 3.150 1.710 - -6.04% -
434.zeusmp (max) (percentage) -37.83% 2.300 1.430 - 88.16% -
434.zeusmp (unexplained diff vs previous) (percentage) -37.33% 2.250 1.410 - 101.43% -
464.h264ref (max) -37.29% 59.000 37.000 - -5.13% -
410.bwaves -36.68% 289.000 183.000 - -3.17% -
410.bwaves (avg) -36.68% 289.000 183.000 - -3.17% -
465.tonto (unexplained diff vs previous) -36.49% 148.000 94.000 - 487.50% -
465.tonto (unexplained diff vs previous) (percentage) -36.28% 1.130 0.720 - 500.00% -
416.gamess -34.02% 485.000 320.000 - -3.03% -
416.gamess (avg) -34.02% 485.000 320.000 - -3.03% -
459.GemsFDTD -28.69% 352.000 251.000 - -3.46% -
459.GemsFDTD (avg) -28.69% 352.000 251.000 - -3.46% -
471.omnetpp (max) (percentage) -27.96% 1.860 1.340 - -16.77% -
458.sjeng (max) (percentage) -26.67% 0.300 0.220 - -8.33% -
447.dealII -24.05% 79.000 60.000 - -3.23% -
447.dealII (avg) -24.05% 79.000 60.000 - -3.23% -
450.soplex -21.62% 37.000 29.000 - -3.33% -
450.soplex (avg) -21.62% 37.000 29.000 - -3.33% -
435.gromacs (max) (percentage) -20.45% 1.760 1.400 - -6.04% -
445.gobmk (max) -12.12% 33.000 29.000 - -6.45% -
400.perlbench (unexplained diff vs previous) -2.86% 35.000 34.000 - 1033.33% -
456.hmmer (unexplained diff vs previous) (percentage) -1.92% 0.520 0.510 - 88.89% -
473.astar (max) -1.64% 61.000 60.000 - -10.45% -

Added Tests - sample variability
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
436.cactusADM (unexplained diff vs previous)
436.cactusADM (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1604.000 2.17%
401.bzip2 2957.000 -
403.gcc 60.000 5.26%
410.bwaves 5954.000 -
416.gamess 14815.000 -
429.mcf 945.000 -
433.milc 1125.000 -
434.zeusmp 2766.000 1.43%
435.gromacs 7632.000 -
436.cactusADM 2607.000 -
437.leslie3d 12801.000 13.81%
444.namd 899.000 -
445.gobmk 5077.000 -
447.dealII 1931.000 -
450.soplex 602.000 1.01%
453.povray 560.000 1.08%
454.calculix 61.000 -
456.hmmer 4469.000 -
458.sjeng 7665.000 -
459.GemsFDTD 7209.000 -
462.libquantum 162.000 -
464.h264ref 6201.000 -
465.tonto 13135.000 -
470.lbm 3736.000 -
471.omnetpp 2905.000 -
473.astar 4874.000 -1.24%
481.wrf 6291.000 -
482.sphinx3 925.000 -
Geometric Mean 2190.236 -
Code Size (bytes)
Name Current %
400.perlbench 563490.000 -
401.bzip2 33660.000 -
403.gcc 1589886.000 -
410.bwaves 14722.000 -
416.gamess 4554168.000 -
429.mcf 7709.000 -
433.milc 47493.000 -
434.zeusmp 140952.000 -
435.gromacs 394002.000 -
436.cactusADM 257008.000 -
437.leslie3d 48001.000 -
444.namd 162770.000 -
445.gobmk 1088240.000 -
447.dealII 191588.000 -
450.soplex 135600.000 -
453.povray 488845.000 -
454.calculix 682283.000 -
456.hmmer 74969.000 -
458.sjeng 81189.000 -
459.GemsFDTD 195934.000 -
462.libquantum 13074.000 -
464.h264ref 301398.000 -
465.tonto 1435824.000 -
470.lbm 10845.000 -
471.omnetpp 295673.000 -
473.astar 15549.000 -
481.wrf 1539900.000 -
482.sphinx3 80058.000 -
Geometric Mean 157307.434 -
sample variability
Name Current %
400.perlbench 50.000 -
400.perlbench (avg) 50.000 -
400.perlbench (avg) (percentage) 3.160 -1.25%
400.perlbench (max) 34.000 112.50%
400.perlbench (max) (percentage) 2.170 106.67%
400.perlbench (unexplained diff vs previous) 34.000 1033.33%
400.perlbench (unexplained diff vs previous) (percentage) 2.120 1015.79%
401.bzip2 108.000 -2.70%
401.bzip2 (avg) 108.000 -2.70%
401.bzip2 (avg) (percentage) 3.660 -2.66%
401.bzip2 (max) 21.000 -12.50%
401.bzip2 (max) (percentage) 0.730 -9.88%
401.bzip2 (unexplained diff vs previous) 21.000 -4.55%
401.bzip2 (unexplained diff vs previous) (percentage) 0.710 -4.05%
403.gcc 1.000 -
403.gcc (avg) 1.000 -
403.gcc (avg) (percentage) 1.700 -
403.gcc (max) 3.000 -
403.gcc (max) (percentage) 5.650 -3.25%
410.bwaves 183.000 -3.17%
410.bwaves (avg) 183.000 -3.17%
410.bwaves (avg) (percentage) 3.080 -3.14%
410.bwaves (max) 32.000 -
410.bwaves (max) (percentage) 0.540 -1.82%
410.bwaves (unexplained diff vs previous) 11.000 450.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.180 500.00%
416.gamess 320.000 -3.03%
416.gamess (avg) 320.000 -3.03%
416.gamess (avg) (percentage) 2.160 -3.14%
416.gamess (max) 35.000 -12.50%
416.gamess (max) (percentage) 0.240 -11.11%
416.gamess (unexplained diff vs previous) 12.000 -61.29%
416.gamess (unexplained diff vs previous) (percentage) 0.080 -61.90%
429.mcf 65.000 -4.41%
429.mcf (avg) 65.000 -4.41%
429.mcf (avg) (percentage) 6.950 -3.47%
429.mcf (max) 6.000 -93.48%
429.mcf (max) (percentage) 0.720 -92.65%
429.mcf (unexplained diff vs previous) 0.000 -
429.mcf (unexplained diff vs previous) (percentage) 0.000 -
433.milc 42.000 -2.33%
433.milc (avg) 42.000 -2.33%
433.milc (avg) (percentage) 3.750 -3.35%
433.milc (max) 2.000 -33.33%
433.milc (max) (percentage) 0.260 -7.14%
433.milc (unexplained diff vs previous) 1.000 -
433.milc (unexplained diff vs previous) (percentage) 0.090 -
434.zeusmp 95.000 -1.04%
434.zeusmp (avg) 95.000 -1.04%
434.zeusmp (avg) (percentage) 3.460 -2.26%
434.zeusmp (max) 39.000 95.00%
434.zeusmp (max) (percentage) 1.430 88.16%
434.zeusmp (unexplained diff vs previous) 39.000 105.26%
434.zeusmp (unexplained diff vs previous) (percentage) 1.410 101.43%
435.gromacs 241.000 -2.03%
435.gromacs (avg) 241.000 -2.03%
435.gromacs (avg) (percentage) 3.170 -2.76%
435.gromacs (max) 106.000 -5.36%
435.gromacs (max) (percentage) 1.400 -6.04%
435.gromacs (unexplained diff vs previous) 60.000 9.09%
435.gromacs (unexplained diff vs previous) (percentage) 0.790 8.22%
436.cactusADM 95.000 -3.06%
436.cactusADM (avg) 95.000 -3.06%
436.cactusADM (avg) (percentage) 3.670 -3.17%
436.cactusADM (max) 9.000 -
436.cactusADM (max) (percentage) 0.360 -5.26%
436.cactusADM (unexplained diff vs previous) 0.000 -
436.cactusADM (unexplained diff vs previous) (percentage) 0.000 -
437.leslie3d 496.000 25.57%
437.leslie3d (avg) 496.000 25.57%
437.leslie3d (avg) (percentage) 3.880 10.23%
437.leslie3d (max) 1767.000 220.11%
437.leslie3d (max) (percentage) 13.810 181.26%
437.leslie3d (unexplained diff vs previous) 1553.000 341.19%
437.leslie3d (unexplained diff vs previous) (percentage) 12.130 287.54%
444.namd 34.000 -2.86%
444.namd (avg) 34.000 -2.86%
444.namd (avg) (percentage) 3.860 -2.53%
444.namd (max) 8.000 -20.00%
444.namd (max) (percentage) 0.990 -10.81%
444.namd (unexplained diff vs previous) 9.000 -10.00%
444.namd (unexplained diff vs previous) (percentage) 1.000 -9.09%
445.gobmk 109.000 -3.54%
445.gobmk (avg) 109.000 -3.54%
445.gobmk (avg) (percentage) 2.150 -3.59%
445.gobmk (max) 29.000 -6.45%
445.gobmk (max) (percentage) 0.580 -4.92%
445.gobmk (unexplained diff vs previous) 9.000 800.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.180 800.00%
447.dealII 60.000 -3.23%
447.dealII (avg) 60.000 -3.23%
447.dealII (avg) (percentage) 3.150 -3.37%
447.dealII (max) 12.000 -7.69%
447.dealII (max) (percentage) 0.630 -12.50%
447.dealII (unexplained diff vs previous) 6.000 -57.14%
447.dealII (unexplained diff vs previous) (percentage) 0.310 -57.53%
450.soplex 29.000 -3.33%
450.soplex (avg) 29.000 -3.33%
450.soplex (avg) (percentage) 4.930 -2.95%
450.soplex (max) 6.000 -77.78%
450.soplex (max) (percentage) 1.060 -77.01%
450.soplex (unexplained diff vs previous) 6.000 -14.29%
450.soplex (unexplained diff vs previous) (percentage) 1.000 -14.53%
453.povray 19.000 -
453.povray (avg) 19.000 -
453.povray (avg) (percentage) 3.520 -2.49%
453.povray (max) 6.000 20.00%
453.povray (max) (percentage) 1.080 -
453.povray (unexplained diff vs previous) 6.000 -
453.povray (unexplained diff vs previous) (percentage) 1.070 -
454.calculix 3.000 -
454.calculix (avg) 3.000 -
454.calculix (avg) (percentage) 4.930 -3.90%
454.calculix (max) 1.000 -
454.calculix (max) (percentage) 2.340 -10.00%
454.calculix (unexplained diff vs previous) 0.000 -
454.calculix (unexplained diff vs previous) (percentage) 0.000 -
456.hmmer 95.000 -2.06%
456.hmmer (avg) 95.000 -2.06%
456.hmmer (avg) (percentage) 2.140 -2.73%
456.hmmer (max) 23.000 21.05%
456.hmmer (max) (percentage) 0.520 20.93%
456.hmmer (unexplained diff vs previous) 23.000 91.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.510 88.89%
458.sjeng 145.000 -3.97%
458.sjeng (avg) 145.000 -3.97%
458.sjeng (avg) (percentage) 1.900 -3.55%
458.sjeng (max) 16.000 -11.11%
458.sjeng (max) (percentage) 0.220 -8.33%
458.sjeng (unexplained diff vs previous) 5.000 -58.33%
458.sjeng (unexplained diff vs previous) (percentage) 0.070 -56.25%
459.GemsFDTD 251.000 -3.46%
459.GemsFDTD (avg) 251.000 -3.46%
459.GemsFDTD (avg) (percentage) 3.490 -3.59%
459.GemsFDTD (max) 18.000 5.88%
459.GemsFDTD (max) (percentage) 0.260 8.33%
459.GemsFDTD (unexplained diff vs previous) 19.000 72.73%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.260 73.33%
462.libquantum 5.000 -
462.libquantum (avg) 5.000 -
462.libquantum (avg) (percentage) 3.370 -2.88%
462.libquantum (max) 4.000 -20.00%
462.libquantum (max) (percentage) 2.670 -16.82%
462.libquantum (unexplained diff vs previous) 1.000 -80.00%
462.libquantum (unexplained diff vs previous) (percentage) 0.620 -80.06%
464.h264ref 152.000 -3.80%
464.h264ref (avg) 152.000 -3.80%
464.h264ref (avg) (percentage) 2.460 -3.53%
464.h264ref (max) 37.000 -5.13%
464.h264ref (max) (percentage) 0.600 -4.76%
464.h264ref (unexplained diff vs previous) 12.000 20.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.190 18.75%
465.tonto 149.000 -1.32%
465.tonto (avg) 149.000 -1.32%
465.tonto (avg) (percentage) 1.140 -1.72%
465.tonto (max) 94.000 32.39%
465.tonto (max) (percentage) 0.720 30.91%
465.tonto (unexplained diff vs previous) 94.000 487.50%
465.tonto (unexplained diff vs previous) (percentage) 0.720 500.00%
470.lbm 160.000 -3.03%
470.lbm (avg) 160.000 -3.03%
470.lbm (avg) (percentage) 4.290 -3.16%
470.lbm (max) 12.000 -7.69%
470.lbm (max) (percentage) 0.340 -5.56%
470.lbm (unexplained diff vs previous) 11.000 175.00%
470.lbm (unexplained diff vs previous) (percentage) 0.290 163.64%
471.omnetpp 124.000 -3.12%
471.omnetpp (avg) 124.000 -3.12%
471.omnetpp (avg) (percentage) 4.290 -3.38%
471.omnetpp (max) 38.000 -17.39%
471.omnetpp (max) (percentage) 1.340 -16.77%
471.omnetpp (unexplained diff vs previous) 5.000 -89.13%
471.omnetpp (unexplained diff vs previous) (percentage) 0.170 -89.31%
473.astar 135.000 -2.88%
473.astar (avg) 135.000 -2.88%
473.astar (avg) (percentage) 2.770 -1.77%
473.astar (max) 60.000 -10.45%
473.astar (max) (percentage) 1.240 -8.82%
473.astar (unexplained diff vs previous) 61.000 -7.58%
473.astar (unexplained diff vs previous) (percentage) 1.250 -6.72%
481.wrf 234.000 -4.10%
481.wrf (avg) 234.000 -4.10%
481.wrf (avg) (percentage) 3.730 -3.87%
481.wrf (max) 51.000 -5.56%
481.wrf (max) (percentage) 0.820 -4.65%
481.wrf (unexplained diff vs previous) 17.000 -15.00%
481.wrf (unexplained diff vs previous) (percentage) 0.270 -15.62%
482.sphinx3 39.000 -4.88%
482.sphinx3 (avg) 39.000 -4.88%
482.sphinx3 (avg) (percentage) 4.310 -2.93%
482.sphinx3 (max) 15.000 -6.25%
482.sphinx3 (max) (percentage) 1.710 -6.04%
482.sphinx3 (unexplained diff vs previous) 5.000 -16.67%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.540 -16.92%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.780 -
Geometric Mean 5.653 -20.49%