Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=de2efa143e3652d69c278dd1eb10a856593917c0 (+ 44 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/5d2a360f0a541646abb11efdbabc33c6a04de7ee (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=f957f47df75b9fab995754011491edebc6feb147 (+ 26 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=288a2f83ed42ce91cf92c2b08dad298a3da0cd9a (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/134/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0134 (2-changed-components) 2023-10-28T22:26:29 0:00:00
Previous 0131 (basepoints/gcc-14-04988-g5d2a360f0a5) 2023-10-22T20:52:49 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 33 68
Performance Improvements 119 160
Added Tests 2 2
Unchanged Tests 1082 1006
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
433.milc 1.14% 1136.000 1149.000 - -91.15% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
401.bzip2 (unexplained diff vs previous) (percentage) 357.14% 0.070 0.320 - -63.22% -
401.bzip2 (unexplained diff vs previous) 350.00% 2.000 9.000 - -93.18% -
434.zeusmp (unexplained diff vs previous) 220.00% 5.000 16.000 - -94.81% -
434.zeusmp (unexplained diff vs previous) (percentage) 215.00% 0.200 0.630 - -72.96% -
434.zeusmp (max) 166.67% 6.000 16.000 - -94.68% -
434.zeusmp (max) (percentage) 166.67% 0.240 0.640 - -71.93% -
444.namd (unexplained diff vs previous) 150.00% 2.000 5.000 - -99.29% -
444.namd (unexplained diff vs previous) (percentage) 150.00% 0.280 0.700 - -90.94% -
470.lbm (unexplained diff vs previous) 100.00% 3.000 6.000 - -57.14% -
470.lbm (unexplained diff vs previous) (percentage) 100.00% 0.080 0.160 - 45.45% -
459.GemsFDTD (max) 85.00% 20.000 37.000 - -73.57% -
459.GemsFDTD (unexplained diff vs previous) 85.00% 20.000 37.000 - -22.92% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 84.85% 0.330 0.610 - 190.48% -
459.GemsFDTD (max) (percentage) 82.35% 0.340 0.620 - 3.33% -
465.tonto (unexplained diff vs previous) 47.83% 23.000 34.000 - -46.03% -
465.tonto (unexplained diff vs previous) (percentage) 45.45% 0.220 0.320 - -58.44% -
462.libquantum (max) (percentage) 35.78% 1.090 1.480 - 131.25% -
471.omnetpp (unexplained diff vs previous) (percentage) 34.23% 1.490 2.000 - 733.33% -
471.omnetpp (unexplained diff vs previous) 31.82% 44.000 58.000 - 123.08% -
433.milc (unexplained diff vs previous) 30.00% 10.000 13.000 - -96.94% -
433.milc (unexplained diff vs previous) (percentage) 28.41% 0.880 1.130 - -65.55% -
482.sphinx3 (unexplained diff vs previous) (percentage) 25.00% 0.440 0.550 - -11.29% -
482.sphinx3 (unexplained diff vs previous) 25.00% 4.000 5.000 - -97.31% -
416.gamess (unexplained diff vs previous) (percentage) 14.29% 0.140 0.160 - -94.63% -
447.dealII (unexplained diff vs previous) (percentage) 12.09% 0.910 1.020 - 30.77% -
416.gamess (unexplained diff vs previous) 11.11% 18.000 20.000 - -97.04% -
447.dealII (unexplained diff vs previous) 11.11% 18.000 20.000 - -73.68% -
447.dealII (max) (percentage) 10.99% 0.910 1.010 - -8.18% -
444.namd (max) (percentage) 9.37% 0.640 0.700 - -90.25% -
454.calculix (unexplained diff vs previous) (percentage) 6.72% 6.250 6.670 - 33250.00% -
447.dealII (max) 5.56% 18.000 19.000 - -82.24% -
454.calculix (avg) (percentage) 1.96% 3.580 3.650 - 81.59% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
454.calculix -6.25% 64.000 60.000 - -99.74% -
403.gcc -5.00% 60.000 57.000 - -99.37% -
437.leslie3d -3.12% 10529.000 10200.000 - -32.02% -
471.omnetpp -1.96% 2953.000 2895.000 - -73.04% -
462.libquantum -1.48% 135.000 133.000 - -98.85% -
447.dealII -1.01% 1987.000 1967.000 - -79.83% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
445.gobmk (unexplained diff vs previous) -86.67% 15.000 2.000 - -98.92% -
445.gobmk (unexplained diff vs previous) (percentage) -86.21% 0.290 0.040 - -97.42% -
450.soplex (unexplained diff vs previous) (percentage) -80.22% 0.910 0.180 - -91.04% -
436.cactusADM (unexplained diff vs previous) -80.00% 5.000 1.000 - -99.82% -
450.soplex (unexplained diff vs previous) -80.00% 5.000 1.000 - -99.68% -
410.bwaves (unexplained diff vs previous) (percentage) -79.45% 0.730 0.150 - 50.00% -
410.bwaves (unexplained diff vs previous) -79.07% 43.000 9.000 - -40.00% -
458.sjeng (unexplained diff vs previous) -78.57% 14.000 3.000 - -66.67% -
436.cactusADM (unexplained diff vs previous) (percentage) -78.26% 0.230 0.050 - -98.46% -
458.sjeng (unexplained diff vs previous) (percentage) -77.78% 0.180 0.040 - -33.33% -
435.gromacs (unexplained diff vs previous) -66.67% 54.000 18.000 - -94.34% -
435.gromacs (unexplained diff vs previous) (percentage) -66.23% 0.770 0.260 - -87.38% -
453.povray (max) -62.50% 8.000 3.000 - -98.18% -
410.bwaves (max) -60.76% 79.000 31.000 - 3.33% -
410.bwaves (max) (percentage) -60.00% 1.350 0.540 - 157.14% -
453.povray (max) (percentage) -56.25% 1.760 0.770 - -69.57% -
429.mcf (unexplained diff vs previous) (percentage) -54.55% 0.110 0.050 - -90.74% -
429.mcf (unexplained diff vs previous) -50.00% 2.000 1.000 - -98.92% -
481.wrf (unexplained diff vs previous) -50.00% 38.000 19.000 - -83.04% -
481.wrf (unexplained diff vs previous) (percentage) -50.00% 0.700 0.350 - -47.76% -
450.soplex (max) (percentage) -34.65% 1.270 0.830 - -57.87% -
435.gromacs (max) -34.21% 76.000 50.000 - -83.92% -
435.gromacs (max) (percentage) -33.94% 1.090 0.720 - -64.36% -
450.soplex (max) -33.33% 6.000 4.000 - -98.68% -
453.povray (unexplained diff vs previous) -25.00% 4.000 3.000 - 200.00% -
454.calculix (max) -25.00% 4.000 3.000 - -99.72% -
453.povray (unexplained diff vs previous) (percentage) -24.39% 0.820 0.620 - 3000.00% -
473.astar (unexplained diff vs previous) (percentage) -18.75% 0.320 0.260 - 1200.00% -
473.astar (unexplained diff vs previous) -16.67% 18.000 15.000 - 400.00% -
471.omnetpp (max) -8.86% 79.000 72.000 - -89.12% -
481.wrf (max) -7.89% 38.000 35.000 - -93.40% -
473.astar (max) -7.14% 140.000 130.000 - 550.00% -
473.astar (max) (percentage) -6.88% 2.470 2.300 - 1542.86% -
470.lbm (max) (percentage) -6.82% 0.880 0.820 - -83.92% -
433.milc (max) -6.67% 15.000 14.000 - -96.59% -
433.milc (max) (percentage) -6.52% 1.380 1.290 - -59.31% -
471.omnetpp (max) (percentage) -6.34% 2.680 2.510 - -59.32% -
470.lbm (max) -6.25% 32.000 30.000 - -95.41% -
429.mcf (max) -5.76% 347.000 327.000 - 255.43% -
429.mcf (max) (percentage) -5.72% 18.530 17.470 - 3135.19% -
481.wrf (max) (percentage) -5.71% 0.700 0.660 - -79.05% -
437.leslie3d (max) -5.27% 891.000 844.000 - 140.46% -
482.sphinx3 (max) -5.26% 19.000 18.000 - -95.90% -
454.calculix (max) (percentage) -4.95% 6.670 6.340 - 34.89% -
458.sjeng (max) -4.76% 21.000 20.000 - -35.48% -
482.sphinx3 -4.35% 23.000 22.000 - -94.95% -
482.sphinx3 (avg) -4.35% 23.000 22.000 - -94.95% -
456.hmmer (max) (percentage) -4.00% 0.250 0.240 - -81.10% -
437.leslie3d -3.87% 465.000 447.000 - 25.21% -
437.leslie3d (avg) -3.87% 465.000 447.000 - 25.21% -
482.sphinx3 (max) (percentage) -3.70% 2.160 2.080 - 41.50% -
458.sjeng (max) (percentage) -3.57% 0.280 0.270 - 22.73% -
416.gamess (avg) (percentage) -3.33% 1.200 1.160 - -46.54% -
400.perlbench (max) (percentage) -3.30% 2.120 2.050 - -36.73% -
401.bzip2 (max) (percentage) -3.30% 0.910 0.880 - 0.00% -
400.perlbench (avg) (percentage) -3.18% 2.830 2.740 - 86.39% -
400.perlbench (max) -3.12% 32.000 31.000 - -91.01% -
456.hmmer -2.94% 34.000 33.000 - -56.00% -
456.hmmer (avg) -2.94% 34.000 33.000 - -56.00% -
447.dealII -2.78% 36.000 35.000 - -66.35% -
447.dealII (avg) -2.78% 36.000 35.000 - -66.35% -
481.wrf -2.73% 110.000 107.000 - -66.35% -
481.wrf (avg) -2.73% 110.000 107.000 - -66.35% -
445.gobmk (avg) (percentage) -2.72% 1.470 1.430 - 85.71% -
465.tonto (avg) (percentage) -2.68% 1.490 1.450 - -13.17% -
416.gamess -2.68% 149.000 145.000 - -70.53% -
416.gamess (avg) -2.68% 149.000 145.000 - -70.53% -
445.gobmk -2.67% 75.000 73.000 - -20.65% -
445.gobmk (avg) -2.67% 75.000 73.000 - -20.65% -
458.sjeng -2.60% 77.000 75.000 - 59.57% -
458.sjeng (avg) -2.60% 77.000 75.000 - 59.57% -
435.gromacs -2.58% 155.000 151.000 - -43.23% -
435.gromacs (avg) -2.58% 155.000 151.000 - -43.23% -
471.omnetpp -2.56% 78.000 76.000 - -55.29% -
471.omnetpp (avg) -2.56% 78.000 76.000 - -55.29% -
465.tonto -2.52% 159.000 155.000 - 13.14% -
465.tonto (avg) -2.52% 159.000 155.000 - 13.14% -
481.wrf (avg) (percentage) -2.46% 2.030 1.980 - 4.76% -
464.h264ref (avg) (percentage) -2.36% 1.270 1.240 - 129.63% -
400.perlbench -2.33% 43.000 42.000 - -73.08% -
400.perlbench (avg) -2.33% 43.000 42.000 - -73.08% -
435.gromacs (avg) (percentage) -2.26% 2.210 2.160 - 24.86% -
437.leslie3d (max) (percentage) -2.24% 8.470 8.280 - 253.85% -
465.tonto (max) (percentage) -2.17% 0.460 0.450 - -87.43% -
450.soplex (avg) (percentage) -2.06% 3.400 3.330 - 168.55% -
453.povray (avg) (percentage) -2.06% 2.430 2.380 - 90.40% -
465.tonto (max) -2.04% 49.000 48.000 - -83.62% -
458.sjeng (avg) (percentage) -2.00% 1.000 0.980 - 196.97% -
401.bzip2 (avg) (percentage) -1.98% 2.020 1.980 - 350.00% -
473.astar -1.92% 104.000 102.000 - 264.29% -
473.astar (avg) -1.92% 104.000 102.000 - 264.29% -
429.mcf (avg) (percentage) -1.91% 5.770 5.660 - 967.92% -
429.mcf -1.85% 108.000 106.000 - 17.78% -
429.mcf (avg) -1.85% 108.000 106.000 - 17.78% -
470.lbm (avg) (percentage) -1.79% 1.680 1.650 - -30.38% -
401.bzip2 -1.75% 57.000 56.000 - -15.15% -
401.bzip2 (avg) -1.75% 57.000 56.000 - -15.15% -
436.cactusADM (avg) (percentage) -1.74% 1.720 1.690 - 1.81% -
444.namd (avg) (percentage) -1.72% 2.330 2.290 - -41.73% -
456.hmmer (avg) (percentage) -1.71% 1.170 1.150 - 69.12% -
403.gcc (max) (percentage) -1.71% 6.450 6.340 - 780.56% -
473.astar (avg) (percentage) -1.63% 1.840 1.810 - 852.63% -
447.dealII (avg) (percentage) -1.61% 1.860 1.830 - 71.03% -
470.lbm -1.61% 62.000 61.000 - -79.87% -
470.lbm (avg) -1.61% 62.000 61.000 - -79.87% -
482.sphinx3 (avg) (percentage) -1.58% 2.530 2.490 - 70.55% -
410.bwaves (avg) (percentage) -1.37% 1.460 1.440 - 32.11% -
464.h264ref -1.35% 74.000 73.000 - -6.41% -
464.h264ref (avg) -1.35% 74.000 73.000 - -6.41% -
410.bwaves -1.16% 86.000 85.000 - -46.88% -
410.bwaves (avg) -1.16% 86.000 85.000 - -46.88% -
459.GemsFDTD (avg) (percentage) -1.10% 1.810 1.790 - 98.89% -
434.zeusmp (avg) (percentage) -1.07% 1.870 1.850 - 25.00% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 30.65% 8205.000 10720.000 - 0.32% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (unexplained diff vs previous) (percentage) 33250.00% 0.020 6.670 - 6.72% -
462.libquantum (unexplained diff vs previous) (percentage) 14900.00% 0.010 1.500 - 0.00% -
429.mcf (max) (percentage) 3135.19% 0.540 17.470 - -5.72% -
453.povray (unexplained diff vs previous) (percentage) 3000.00% 0.020 0.620 - -24.39% -
473.astar (max) (percentage) 1542.86% 0.140 2.300 - -6.88% -
483.xalancbmk (max) (percentage) 1336.59% 0.410 5.890 - -0.17% -
473.astar (unexplained diff vs previous) (percentage) 1200.00% 0.020 0.260 - -18.75% -
429.mcf (avg) (percentage) 967.92% 0.530 5.660 - -1.91% -
473.astar (avg) (percentage) 852.63% 0.190 1.810 - -1.63% -
403.gcc (avg) (percentage) 839.34% 0.610 5.730 - 0.00% -
403.gcc (max) (percentage) 780.56% 0.720 6.340 - -1.71% -
471.omnetpp (unexplained diff vs previous) (percentage) 733.33% 0.240 2.000 - 34.23% -
473.astar (max) 550.00% 20.000 130.000 - -7.14% -
473.astar (unexplained diff vs previous) 400.00% 3.000 15.000 - -16.67% -
401.bzip2 (avg) (percentage) 350.00% 0.440 1.980 - -1.98% -
473.astar 264.29% 28.000 102.000 - -1.92% -
473.astar (avg) 264.29% 28.000 102.000 - -1.92% -
429.mcf (max) 255.43% 92.000 327.000 - -5.76% -
437.leslie3d (max) (percentage) 253.85% 2.340 8.280 - -2.24% -
462.libquantum (avg) (percentage) 210.77% 0.650 2.020 - -0.49% -
453.povray (unexplained diff vs previous) 200.00% 1.000 3.000 - -25.00% -
458.sjeng (avg) (percentage) 196.97% 0.330 0.980 - -2.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 190.48% 0.210 0.610 - 84.85% -
450.soplex (avg) (percentage) 168.55% 1.240 3.330 - -2.06% -
410.bwaves (max) (percentage) 157.14% 0.210 0.540 - -60.00% -
437.leslie3d (max) 140.46% 351.000 844.000 - -5.27% -
462.libquantum (max) (percentage) 131.25% 0.640 1.480 - 35.78% -
464.h264ref (avg) (percentage) 129.63% 0.540 1.240 - -2.36% -
471.omnetpp (unexplained diff vs previous) 123.08% 26.000 58.000 - 31.82% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - 0.00% -
459.GemsFDTD (avg) (percentage) 98.89% 0.900 1.790 - -1.10% -
453.povray (avg) (percentage) 90.40% 1.250 2.380 - -2.06% -
400.perlbench (avg) (percentage) 86.39% 1.470 2.740 - -3.18% -
445.gobmk (avg) (percentage) 85.71% 0.770 1.430 - -2.72% -
437.leslie3d (avg) (percentage) 84.45% 2.380 4.390 - -0.68% -
454.calculix (avg) (percentage) 81.59% 2.010 3.650 - 1.96% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
447.dealII (avg) (percentage) 71.03% 1.070 1.830 - -1.61% -
482.sphinx3 (avg) (percentage) 70.55% 1.460 2.490 - -1.58% -
456.hmmer (avg) (percentage) 69.12% 0.680 1.150 - -1.71% -
471.omnetpp (avg) (percentage) 66.04% 1.590 2.640 - -0.38% -
458.sjeng 59.57% 47.000 75.000 - -2.60% -
458.sjeng (avg) 59.57% 47.000 75.000 - -2.60% -
410.bwaves (unexplained diff vs previous) (percentage) 50.00% 0.100 0.150 - -79.45% -
470.lbm (unexplained diff vs previous) (percentage) 45.45% 0.110 0.160 - 100.00% -
482.sphinx3 (max) (percentage) 41.50% 1.470 2.080 - -3.70% -
437.leslie3d (unexplained diff vs previous) (percentage) 41.05% 2.290 3.230 - 0.00% -
454.calculix (max) (percentage) 34.89% 4.700 6.340 - -4.95% -
410.bwaves (avg) (percentage) 32.11% 1.090 1.440 - -1.37% -
447.dealII (unexplained diff vs previous) (percentage) 30.77% 0.780 1.020 - 12.09% -
437.leslie3d 25.21% 357.000 447.000 - -3.87% -
437.leslie3d (avg) 25.21% 357.000 447.000 - -3.87% -
434.zeusmp (avg) (percentage) 25.00% 1.480 1.850 - -1.07% -
435.gromacs (avg) (percentage) 24.86% 1.730 2.160 - -2.26% -
458.sjeng (max) (percentage) 22.73% 0.220 0.270 - -3.57% -
433.milc (avg) (percentage) 19.41% 1.700 2.030 - -0.49% -
429.mcf 17.78% 90.000 106.000 - -1.85% -
429.mcf (avg) 17.78% 90.000 106.000 - -1.85% -
465.tonto 13.14% 137.000 155.000 - -2.52% -
465.tonto (avg) 13.14% 137.000 155.000 - -2.52% -
464.h264ref (max) (percentage) 6.98% 0.430 0.460 - -2.13% -
481.wrf (avg) (percentage) 4.76% 1.890 1.980 - -2.46% -
459.GemsFDTD (max) (percentage) 3.33% 0.600 0.620 - 82.35% -
410.bwaves (max) 3.33% 30.000 31.000 - -60.76% -
436.cactusADM (avg) (percentage) 1.81% 1.660 1.690 - -1.74% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -3.57% 120221.000 115929.000 - 0.00% -
464.h264ref -2.36% 428364.000 418236.000 - 0.00% -
434.zeusmp -1.64% 165699.000 162977.000 - 0.00% -
459.GemsFDTD -1.57% 253716.000 249721.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.74% 22860.000 60.000 - -6.25% -
403.gcc -99.37% 9010.000 57.000 - -5.00% -
462.libquantum -98.85% 11614.000 133.000 - -1.48% -
482.sphinx3 -96.98% 29928.000 905.000 - -0.55% -
450.soplex -96.46% 15466.000 547.000 - -0.18% -
453.povray -92.57% 6544.000 486.000 - -0.61% -
444.namd -92.14% 9050.000 711.000 - -0.70% -
433.milc -91.15% 12976.000 1149.000 - 1.14% -
429.mcf -89.07% 17158.000 1876.000 - 0.05% -
436.cactusADM -87.76% 17544.000 2147.000 - 0.05% -
400.perlbench -85.43% 10650.000 1552.000 - 0.00% -
401.bzip2 -81.21% 15185.000 2853.000 - 0.32% -
434.zeusmp -80.86% 13211.000 2528.000 - 0.64% -
447.dealII -79.83% 9751.000 1967.000 - -1.01% -
459.GemsFDTD -74.11% 23356.000 6046.000 - 0.62% -
456.hmmer -73.49% 11046.000 2928.000 - 0.10% -
471.omnetpp -73.04% 10739.000 2895.000 - -1.96% -
470.lbm -71.14% 12819.000 3700.000 - -0.16% -
481.wrf -67.61% 16834.000 5453.000 - 0.35% -
473.astar -61.94% 14932.000 5683.000 - -0.26% -
410.bwaves -59.92% 14755.000 5914.000 - -0.15% -
464.h264ref -59.54% 14556.000 5890.000 - 0.08% -
445.gobmk -56.94% 11995.000 5165.000 - -0.04% -
435.gromacs -54.62% 15419.000 6997.000 - -0.26% -
458.sjeng -45.81% 14275.000 7736.000 - 0.04% -
416.gamess -44.91% 22704.000 12508.000 - 0.16% -
437.leslie3d -32.02% 15005.000 10200.000 - -3.12% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (unexplained diff vs previous) -99.82% 570.000 1.000 - -80.00% -
454.calculix (max) -99.72% 1074.000 3.000 - -25.00% -
450.soplex (unexplained diff vs previous) -99.68% 311.000 1.000 - -80.00% -
454.calculix -99.56% 459.000 2.000 - 0.00% -
454.calculix (avg) -99.56% 459.000 2.000 - 0.00% -
444.namd (max) -99.38% 649.000 4.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.29% 700.000 5.000 - 150.00% -
429.mcf (unexplained diff vs previous) -98.92% 93.000 1.000 - -50.00% -
445.gobmk (unexplained diff vs previous) -98.92% 186.000 2.000 - -86.67% -
450.soplex (max) -98.68% 304.000 4.000 - -33.33% -
462.libquantum (max) -98.65% 74.000 1.000 - 0.00% -
436.cactusADM (max) -98.55% 552.000 8.000 - 0.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -98.46% 3.250 0.050 - -78.26% -
453.povray (max) -98.18% 165.000 3.000 - -62.50% -
445.gobmk (unexplained diff vs previous) (percentage) -97.42% 1.550 0.040 - -86.21% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -97.31% 186.000 5.000 - 25.00% -
416.gamess (unexplained diff vs previous) -97.04% 676.000 20.000 - 11.11% -
433.milc (unexplained diff vs previous) -96.94% 425.000 13.000 - 30.00% -
416.gamess (max) -96.80% 656.000 21.000 - 0.00% -
433.milc (max) -96.59% 411.000 14.000 - -6.67% -
456.hmmer (unexplained diff vs previous) -96.39% 83.000 3.000 - 0.00% -
482.sphinx3 (max) -95.90% 439.000 18.000 - -5.26% -
444.namd -95.49% 355.000 16.000 - 0.00% -
444.namd (avg) -95.49% 355.000 16.000 - 0.00% -
470.lbm (max) -95.41% 653.000 30.000 - -6.25% -
403.gcc (max) -95.31% 64.000 3.000 - 0.00% -
456.hmmer (max) -95.00% 140.000 7.000 - 0.00% -
482.sphinx3 -94.95% 436.000 22.000 - -4.35% -
482.sphinx3 (avg) -94.95% 436.000 22.000 - -4.35% -
434.zeusmp (unexplained diff vs previous) -94.81% 308.000 16.000 - 220.00% -
434.zeusmp (max) -94.68% 301.000 16.000 - 166.67% -
416.gamess (unexplained diff vs previous) (percentage) -94.63% 2.980 0.160 - 14.29% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
435.gromacs (unexplained diff vs previous) -94.34% 318.000 18.000 - -66.67% -
416.gamess (max) (percentage) -94.12% 2.890 0.170 - 0.00% -
481.wrf (max) -93.40% 530.000 35.000 - -7.89% -
401.bzip2 (unexplained diff vs previous) -93.18% 132.000 9.000 - 350.00% -
450.soplex (unexplained diff vs previous) (percentage) -91.04% 2.010 0.180 - -80.22% -
400.perlbench (max) -91.01% 345.000 31.000 - -3.12% -
444.namd (unexplained diff vs previous) (percentage) -90.94% 7.730 0.700 - 150.00% -
429.mcf (unexplained diff vs previous) (percentage) -90.74% 0.540 0.050 - -54.55% -
450.soplex -90.58% 191.000 18.000 - 0.00% -
450.soplex (avg) -90.58% 191.000 18.000 - 0.00% -
444.namd (max) (percentage) -90.25% 7.180 0.700 - 9.37% -
433.milc -89.55% 220.000 23.000 - 0.00% -
433.milc (avg) -89.55% 220.000 23.000 - 0.00% -
471.omnetpp (max) -89.12% 662.000 72.000 - -8.86% -
436.cactusADM -87.63% 291.000 36.000 - 0.00% -
436.cactusADM (avg) -87.63% 291.000 36.000 - 0.00% -
445.gobmk (max) -87.43% 183.000 23.000 - 0.00% -
465.tonto (max) (percentage) -87.43% 3.580 0.450 - -2.17% -
435.gromacs (unexplained diff vs previous) (percentage) -87.38% 2.060 0.260 - -66.23% -
436.cactusADM (max) (percentage) -87.30% 3.150 0.400 - -2.44% -
456.hmmer (unexplained diff vs previous) (percentage) -86.67% 0.750 0.100 - 0.00% -
453.povray -86.42% 81.000 11.000 - 0.00% -
453.povray (avg) -86.42% 81.000 11.000 - 0.00% -
435.gromacs (max) -83.92% 311.000 50.000 - -34.21% -
470.lbm (max) (percentage) -83.92% 5.100 0.820 - -6.82% -
465.tonto (max) -83.62% 293.000 48.000 - -2.04% -
481.wrf (unexplained diff vs previous) -83.04% 112.000 19.000 - -50.00% -
447.dealII (max) -82.24% 107.000 19.000 - 5.56% -
401.bzip2 (max) -81.20% 133.000 25.000 - 0.00% -
456.hmmer (max) (percentage) -81.10% 1.270 0.240 - -4.00% -
470.lbm -79.87% 303.000 61.000 - -1.61% -
470.lbm (avg) -79.87% 303.000 61.000 - -1.61% -
481.wrf (max) (percentage) -79.05% 3.150 0.660 - -5.71% -
434.zeusmp -76.41% 195.000 46.000 - 0.00% -
434.zeusmp (avg) -76.41% 195.000 46.000 - 0.00% -
447.dealII (unexplained diff vs previous) -73.68% 76.000 20.000 - 11.11% -
459.GemsFDTD (max) -73.57% 140.000 37.000 - 85.00% -
400.perlbench -73.08% 156.000 42.000 - -2.33% -
400.perlbench (avg) -73.08% 156.000 42.000 - -2.33% -
434.zeusmp (unexplained diff vs previous) (percentage) -72.96% 2.330 0.630 - 215.00% -
434.zeusmp (max) (percentage) -71.93% 2.280 0.640 - 166.67% -
416.gamess -70.53% 492.000 145.000 - -2.68% -
416.gamess (avg) -70.53% 492.000 145.000 - -2.68% -
445.gobmk (max) (percentage) -69.93% 1.530 0.460 - 0.00% -
453.povray (max) (percentage) -69.57% 2.530 0.770 - -56.25% -
458.sjeng (unexplained diff vs previous) -66.67% 9.000 3.000 - -78.57% -
464.h264ref (unexplained diff vs previous) -66.67% 15.000 5.000 - 0.00% -
481.wrf -66.35% 318.000 107.000 - -2.73% -
481.wrf (avg) -66.35% 318.000 107.000 - -2.73% -
447.dealII -66.35% 104.000 35.000 - -2.78% -
447.dealII (avg) -66.35% 104.000 35.000 - -2.78% -
433.milc (unexplained diff vs previous) (percentage) -65.55% 3.280 1.130 - 28.41% -
435.gromacs (max) (percentage) -64.36% 2.020 0.720 - -33.94% -
401.bzip2 (unexplained diff vs previous) (percentage) -63.22% 0.870 0.320 - 357.14% -
471.omnetpp (max) (percentage) -59.32% 6.170 2.510 - -6.34% -
433.milc (max) (percentage) -59.31% 3.170 1.290 - -6.52% -
465.tonto (unexplained diff vs previous) (percentage) -58.44% 0.770 0.320 - 45.45% -
450.soplex (max) (percentage) -57.87% 1.970 0.830 - -34.65% -
470.lbm (unexplained diff vs previous) -57.14% 14.000 6.000 - 100.00% -
464.h264ref (max) -56.45% 62.000 27.000 - 0.00% -
456.hmmer -56.00% 75.000 33.000 - -2.94% -
456.hmmer (avg) -56.00% 75.000 33.000 - -2.94% -
471.omnetpp -55.29% 170.000 76.000 - -2.56% -
471.omnetpp (avg) -55.29% 170.000 76.000 - -2.56% -
459.GemsFDTD -48.57% 210.000 108.000 - 0.00% -
459.GemsFDTD (avg) -48.57% 210.000 108.000 - 0.00% -
481.wrf (unexplained diff vs previous) (percentage) -47.76% 0.670 0.350 - -50.00% -
410.bwaves -46.88% 160.000 85.000 - -1.16% -
410.bwaves (avg) -46.88% 160.000 85.000 - -1.16% -
416.gamess (avg) (percentage) -46.54% 2.170 1.160 - -3.33% -
465.tonto (unexplained diff vs previous) -46.03% 63.000 34.000 - 47.83% -
435.gromacs -43.23% 266.000 151.000 - -2.58% -
435.gromacs (avg) -43.23% 266.000 151.000 - -2.58% -
444.namd (avg) (percentage) -41.73% 3.930 2.290 - -1.72% -
410.bwaves (unexplained diff vs previous) -40.00% 15.000 9.000 - -79.07% -
400.perlbench (max) (percentage) -36.73% 3.240 2.050 - -3.30% -
458.sjeng (max) -35.48% 31.000 20.000 - -4.76% -
458.sjeng (unexplained diff vs previous) (percentage) -33.33% 0.060 0.040 - -77.78% -
470.lbm (avg) (percentage) -30.38% 2.370 1.650 - -1.79% -
459.GemsFDTD (unexplained diff vs previous) -22.92% 48.000 37.000 - 85.00% -
445.gobmk -20.65% 92.000 73.000 - -2.67% -
445.gobmk (avg) -20.65% 92.000 73.000 - -2.67% -
464.h264ref (unexplained diff vs previous) (percentage) -20.00% 0.100 0.080 - 0.00% -
454.calculix (unexplained diff vs previous) -20.00% 5.000 4.000 - 0.00% -
401.bzip2 -15.15% 66.000 56.000 - -1.75% -
401.bzip2 (avg) -15.15% 66.000 56.000 - -1.75% -
465.tonto (avg) (percentage) -13.17% 1.670 1.450 - -2.68% -
482.sphinx3 (unexplained diff vs previous) (percentage) -11.29% 0.620 0.550 - 25.00% -
447.dealII (max) (percentage) -8.18% 1.100 1.010 - 10.99% -
464.h264ref -6.41% 78.000 73.000 - -1.35% -
464.h264ref (avg) -6.41% 78.000 73.000 - -1.35% -
437.leslie3d (unexplained diff vs previous) -4.08% 343.000 329.000 - 0.00% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1552.000 -
401.bzip2 2853.000 -
403.gcc 57.000 -5.00%
410.bwaves 5914.000 -
416.gamess 12508.000 -
429.mcf 1876.000 -
433.milc 1149.000 1.14%
434.zeusmp 2528.000 -
435.gromacs 6997.000 -
436.cactusADM 2147.000 -
437.leslie3d 10200.000 -3.12%
444.namd 711.000 -
445.gobmk 5165.000 -
447.dealII 1967.000 -1.01%
450.soplex 547.000 -
453.povray 486.000 -
454.calculix 60.000 -6.25%
456.hmmer 2928.000 -
458.sjeng 7736.000 -
459.GemsFDTD 6046.000 -
462.libquantum 133.000 -1.48%
464.h264ref 5890.000 -
465.tonto 10720.000 -
470.lbm 3700.000 -
471.omnetpp 2895.000 -1.96%
473.astar 5683.000 -
481.wrf 5453.000 -
482.sphinx3 905.000 -
Geometric Mean 2060.759 -
Code Size (bytes)
Name Current %
400.perlbench 925068.000 -
401.bzip2 49698.000 -
403.gcc 2675209.000 -
410.bwaves 17640.000 -
416.gamess 6229239.000 -
429.mcf 10979.000 -
433.milc 95589.000 -
434.zeusmp 162977.000 -
435.gromacs 678065.000 -
436.cactusADM 551192.000 -
437.leslie3d 64764.000 -
444.namd 190762.000 -
445.gobmk 1399537.000 -
447.dealII 2122038.000 -
450.soplex 259980.000 -
453.povray 702162.000 -
454.calculix 1268791.000 -
456.hmmer 230488.000 -
458.sjeng 115929.000 -
459.GemsFDTD 249721.000 -
462.libquantum 31321.000 -
464.h264ref 418236.000 -
465.tonto 2868553.000 -
470.lbm 10916.000 -
471.omnetpp 566294.000 -
473.astar 32056.000 -
481.wrf 2621400.000 -
482.sphinx3 152548.000 -
Geometric Mean 271911.577 -
sample variability
Name Current %
400.perlbench 42.000 -2.33%
400.perlbench (avg) 42.000 -2.33%
400.perlbench (avg) (percentage) 2.740 -3.18%
400.perlbench (max) 31.000 -3.12%
400.perlbench (max) (percentage) 2.050 -3.30%
400.perlbench (unexplained diff vs previous) 0.000 -
400.perlbench (unexplained diff vs previous) (percentage) 0.000 -
401.bzip2 56.000 -1.75%
401.bzip2 (avg) 56.000 -1.75%
401.bzip2 (avg) (percentage) 1.980 -1.98%
401.bzip2 (max) 25.000 -
401.bzip2 (max) (percentage) 0.880 -3.30%
401.bzip2 (unexplained diff vs previous) 9.000 350.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.320 357.14%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 5.730 -
403.gcc (max) 3.000 -
403.gcc (max) (percentage) 6.340 -1.71%
410.bwaves 85.000 -1.16%
410.bwaves (avg) 85.000 -1.16%
410.bwaves (avg) (percentage) 1.440 -1.37%
410.bwaves (max) 31.000 -60.76%
410.bwaves (max) (percentage) 0.540 -60.00%
410.bwaves (unexplained diff vs previous) 9.000 -79.07%
410.bwaves (unexplained diff vs previous) (percentage) 0.150 -79.45%
416.gamess 145.000 -2.68%
416.gamess (avg) 145.000 -2.68%
416.gamess (avg) (percentage) 1.160 -3.33%
416.gamess (max) 21.000 -
416.gamess (max) (percentage) 0.170 -
416.gamess (unexplained diff vs previous) 20.000 11.11%
416.gamess (unexplained diff vs previous) (percentage) 0.160 14.29%
429.mcf 106.000 -1.85%
429.mcf (avg) 106.000 -1.85%
429.mcf (avg) (percentage) 5.660 -1.91%
429.mcf (max) 327.000 -5.76%
429.mcf (max) (percentage) 17.470 -5.72%
429.mcf (unexplained diff vs previous) 1.000 -50.00%
429.mcf (unexplained diff vs previous) (percentage) 0.050 -54.55%
433.milc 23.000 -
433.milc (avg) 23.000 -
433.milc (avg) (percentage) 2.030 -
433.milc (max) 14.000 -6.67%
433.milc (max) (percentage) 1.290 -6.52%
433.milc (unexplained diff vs previous) 13.000 30.00%
433.milc (unexplained diff vs previous) (percentage) 1.130 28.41%
434.zeusmp 46.000 -
434.zeusmp (avg) 46.000 -
434.zeusmp (avg) (percentage) 1.850 -1.07%
434.zeusmp (max) 16.000 166.67%
434.zeusmp (max) (percentage) 0.640 166.67%
434.zeusmp (unexplained diff vs previous) 16.000 220.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.630 215.00%
435.gromacs 151.000 -2.58%
435.gromacs (avg) 151.000 -2.58%
435.gromacs (avg) (percentage) 2.160 -2.26%
435.gromacs (max) 50.000 -34.21%
435.gromacs (max) (percentage) 0.720 -33.94%
435.gromacs (unexplained diff vs previous) 18.000 -66.67%
435.gromacs (unexplained diff vs previous) (percentage) 0.260 -66.23%
436.cactusADM 36.000 -
436.cactusADM (avg) 36.000 -
436.cactusADM (avg) (percentage) 1.690 -1.74%
436.cactusADM (max) 8.000 -
436.cactusADM (max) (percentage) 0.400 -
436.cactusADM (unexplained diff vs previous) 1.000 -80.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.050 -78.26%
437.leslie3d 447.000 -3.87%
437.leslie3d (avg) 447.000 -3.87%
437.leslie3d (avg) (percentage) 4.390 -
437.leslie3d (max) 844.000 -5.27%
437.leslie3d (max) (percentage) 8.280 -2.24%
437.leslie3d (unexplained diff vs previous) 329.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 3.230 -
444.namd 16.000 -
444.namd (avg) 16.000 -
444.namd (avg) (percentage) 2.290 -1.72%
444.namd (max) 4.000 -
444.namd (max) (percentage) 0.700 9.37%
444.namd (unexplained diff vs previous) 5.000 150.00%
444.namd (unexplained diff vs previous) (percentage) 0.700 150.00%
445.gobmk 73.000 -2.67%
445.gobmk (avg) 73.000 -2.67%
445.gobmk (avg) (percentage) 1.430 -2.72%
445.gobmk (max) 23.000 -
445.gobmk (max) (percentage) 0.460 -
445.gobmk (unexplained diff vs previous) 2.000 -86.67%
445.gobmk (unexplained diff vs previous) (percentage) 0.040 -86.21%
447.dealII 35.000 -2.78%
447.dealII (avg) 35.000 -2.78%
447.dealII (avg) (percentage) 1.830 -1.61%
447.dealII (max) 19.000 5.56%
447.dealII (max) (percentage) 1.010 10.99%
447.dealII (unexplained diff vs previous) 20.000 11.11%
447.dealII (unexplained diff vs previous) (percentage) 1.020 12.09%
450.soplex 18.000 -
450.soplex (avg) 18.000 -
450.soplex (avg) (percentage) 3.330 -2.06%
450.soplex (max) 4.000 -33.33%
450.soplex (max) (percentage) 0.830 -34.65%
450.soplex (unexplained diff vs previous) 1.000 -80.00%
450.soplex (unexplained diff vs previous) (percentage) 0.180 -80.22%
453.povray 11.000 -
453.povray (avg) 11.000 -
453.povray (avg) (percentage) 2.380 -2.06%
453.povray (max) 3.000 -62.50%
453.povray (max) (percentage) 0.770 -56.25%
453.povray (unexplained diff vs previous) 3.000 -25.00%
453.povray (unexplained diff vs previous) (percentage) 0.620 -24.39%
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 3.650 1.96%
454.calculix (max) 3.000 -25.00%
454.calculix (max) (percentage) 6.340 -4.95%
454.calculix (unexplained diff vs previous) 4.000 -
454.calculix (unexplained diff vs previous) (percentage) 6.670 6.72%
456.hmmer 33.000 -2.94%
456.hmmer (avg) 33.000 -2.94%
456.hmmer (avg) (percentage) 1.150 -1.71%
456.hmmer (max) 7.000 -
456.hmmer (max) (percentage) 0.240 -4.00%
456.hmmer (unexplained diff vs previous) 3.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.100 -
458.sjeng 75.000 -2.60%
458.sjeng (avg) 75.000 -2.60%
458.sjeng (avg) (percentage) 0.980 -2.00%
458.sjeng (max) 20.000 -4.76%
458.sjeng (max) (percentage) 0.270 -3.57%
458.sjeng (unexplained diff vs previous) 3.000 -78.57%
458.sjeng (unexplained diff vs previous) (percentage) 0.040 -77.78%
459.GemsFDTD 108.000 -
459.GemsFDTD (avg) 108.000 -
459.GemsFDTD (avg) (percentage) 1.790 -1.10%
459.GemsFDTD (max) 37.000 85.00%
459.GemsFDTD (max) (percentage) 0.620 82.35%
459.GemsFDTD (unexplained diff vs previous) 37.000 85.00%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.610 84.85%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 2.020 -
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.480 35.78%
462.libquantum (unexplained diff vs previous) 2.000 -
462.libquantum (unexplained diff vs previous) (percentage) 1.500 -
464.h264ref 73.000 -1.35%
464.h264ref (avg) 73.000 -1.35%
464.h264ref (avg) (percentage) 1.240 -2.36%
464.h264ref (max) 27.000 -
464.h264ref (max) (percentage) 0.460 -
464.h264ref (unexplained diff vs previous) 5.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.080 -
465.tonto 155.000 -2.52%
465.tonto (avg) 155.000 -2.52%
465.tonto (avg) (percentage) 1.450 -2.68%
465.tonto (max) 48.000 -2.04%
465.tonto (max) (percentage) 0.450 -2.17%
465.tonto (unexplained diff vs previous) 34.000 47.83%
465.tonto (unexplained diff vs previous) (percentage) 0.320 45.45%
470.lbm 61.000 -1.61%
470.lbm (avg) 61.000 -1.61%
470.lbm (avg) (percentage) 1.650 -1.79%
470.lbm (max) 30.000 -6.25%
470.lbm (max) (percentage) 0.820 -6.82%
470.lbm (unexplained diff vs previous) 6.000 100.00%
470.lbm (unexplained diff vs previous) (percentage) 0.160 100.00%
471.omnetpp 76.000 -2.56%
471.omnetpp (avg) 76.000 -2.56%
471.omnetpp (avg) (percentage) 2.640 -
471.omnetpp (max) 72.000 -8.86%
471.omnetpp (max) (percentage) 2.510 -6.34%
471.omnetpp (unexplained diff vs previous) 58.000 31.82%
471.omnetpp (unexplained diff vs previous) (percentage) 2.000 34.23%
473.astar 102.000 -1.92%
473.astar (avg) 102.000 -1.92%
473.astar (avg) (percentage) 1.810 -1.63%
473.astar (max) 130.000 -7.14%
473.astar (max) (percentage) 2.300 -6.88%
473.astar (unexplained diff vs previous) 15.000 -16.67%
473.astar (unexplained diff vs previous) (percentage) 0.260 -18.75%
481.wrf 107.000 -2.73%
481.wrf (avg) 107.000 -2.73%
481.wrf (avg) (percentage) 1.980 -2.46%
481.wrf (max) 35.000 -7.89%
481.wrf (max) (percentage) 0.660 -5.71%
481.wrf (unexplained diff vs previous) 19.000 -50.00%
481.wrf (unexplained diff vs previous) (percentage) 0.350 -50.00%
482.sphinx3 22.000 -4.35%
482.sphinx3 (avg) 22.000 -4.35%
482.sphinx3 (avg) (percentage) 2.490 -1.58%
482.sphinx3 (max) 18.000 -5.26%
482.sphinx3 (max) (percentage) 2.080 -3.70%
482.sphinx3 (unexplained diff vs previous) 5.000 25.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.550 25.00%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.890 -
Geometric Mean 5.218 4.54%