Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=de2efa143e3652d69c278dd1eb10a856593917c0 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/7ab79a40b546a1470abaf76bec74c63e9990fe47 (+ 1 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=f957f47df75b9fab995754011491edebc6feb147 (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=288a2f83ed42ce91cf92c2b08dad298a3da0cd9a (unchanged)
regression GNU-995
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/137/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0137 (basepoints/gcc-14-05110-g7ab79a40b54) 2023-11-02T11:36:44 0:00:00
Previous 0136 (basepoints/gcc-14-05109-ga291237b628) 2023-10-31T23:13:19 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 30 61
Performance Improvements 110 158
Added Tests 16 6
Unchanged Tests 1080 1011
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
400.perlbench 1.77% 925068.000 941436.000 - 1.16% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (unexplained diff vs previous) 850.00% 2.000 19.000 - -26.92% -
471.omnetpp (unexplained diff vs previous) (percentage) 828.57% 0.070 0.650 - 170.83% -
459.GemsFDTD (unexplained diff vs previous) 675.00% 4.000 31.000 - -35.42% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 642.86% 0.070 0.520 - 147.62% -
410.bwaves (unexplained diff vs previous) 260.00% 10.000 36.000 - 140.00% -
410.bwaves (unexplained diff vs previous) (percentage) 258.82% 0.170 0.610 - 510.00% -
456.hmmer (unexplained diff vs previous) 166.67% 6.000 16.000 - -80.72% -
456.hmmer (unexplained diff vs previous) (percentage) 161.90% 0.210 0.550 - -26.67% -
433.milc (unexplained diff vs previous) (percentage) 158.82% 0.170 0.440 - -86.59% -
433.milc (unexplained diff vs previous) 150.00% 2.000 5.000 - -98.82% -
456.hmmer (max) (percentage) 129.17% 0.240 0.550 - -56.69% -
456.hmmer (max) 114.29% 7.000 15.000 - -89.29% -
401.bzip2 (unexplained diff vs previous) 100.00% 2.000 4.000 - -96.97% -
401.bzip2 (unexplained diff vs previous) (percentage) 100.00% 0.070 0.140 - -83.91% -
482.sphinx3 (unexplained diff vs previous) 100.00% 2.000 4.000 - -97.85% -
482.sphinx3 (unexplained diff vs previous) (percentage) 100.00% 0.220 0.440 - -29.03% -
435.gromacs (max) (percentage) 91.18% 0.680 1.300 - -35.64% -
435.gromacs (max) 89.36% 47.000 89.000 - -71.38% -
436.cactusADM (unexplained diff vs previous) (percentage) 89.19% 0.370 0.700 - -78.46% -
436.cactusADM (unexplained diff vs previous) 87.50% 8.000 15.000 - -97.37% -
436.cactusADM (max) (percentage) 84.21% 0.380 0.700 - -77.78% -
436.cactusADM (max) 75.00% 8.000 14.000 - -97.46% -
450.soplex (unexplained diff vs previous) 33.33% 3.000 4.000 - -98.71% -
450.soplex (unexplained diff vs previous) (percentage) 32.73% 0.550 0.730 - -63.68% -
410.bwaves (max) 28.57% 28.000 36.000 - 20.00% -
410.bwaves (max) (percentage) 27.08% 0.480 0.610 - 190.48% -
481.wrf (max) 18.18% 33.000 39.000 - -92.64% -
481.wrf (max) (percentage) 17.74% 0.620 0.730 - -76.83% -
437.leslie3d (avg) (percentage) 1.09% 4.590 4.640 - 94.96% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -7.13% 11521.000 10699.000 - -28.70% -
454.calculix -3.17% 63.000 61.000 - -99.73% -
403.gcc -1.75% 57.000 56.000 - -99.38% -
464.h264ref -1.56% 5885.000 5793.000 - -60.20% -
400.perlbench -1.48% 1559.000 1536.000 - -85.58% -
435.gromacs -1.30% 7000.000 6909.000 - -55.19% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) (percentage) -95.50% 16.680 0.750 - 38.89% -
429.mcf (max) -95.48% 310.000 14.000 - -84.78% -
473.astar (max) -88.71% 124.000 14.000 - -30.00% -
473.astar (max) (percentage) -88.13% 2.190 0.260 - 85.71% -
470.lbm (max) (percentage) -75.64% 0.780 0.190 - -96.27% -
470.lbm (max) -75.00% 28.000 7.000 - -98.93% -
473.astar (unexplained diff vs previous) -68.75% 16.000 5.000 - 66.67% -
473.astar (unexplained diff vs previous) (percentage) -67.86% 0.280 0.090 - 350.00% -
444.namd (unexplained diff vs previous) -66.67% 6.000 2.000 - -99.71% -
444.namd (unexplained diff vs previous) (percentage) -66.67% 0.840 0.280 - -96.38% -
465.tonto (unexplained diff vs previous) -66.67% 57.000 19.000 - -69.84% -
465.tonto (unexplained diff vs previous) (percentage) -66.04% 0.530 0.180 - -76.62% -
458.sjeng (max) -50.00% 20.000 10.000 - -67.74% -
445.gobmk (max) -47.83% 23.000 12.000 - -93.44% -
445.gobmk (max) (percentage) -46.67% 0.450 0.240 - -84.31% -
458.sjeng (max) (percentage) -46.15% 0.260 0.140 - -36.36% -
445.gobmk (unexplained diff vs previous) (percentage) -40.00% 0.100 0.060 - -96.13% -
445.gobmk (unexplained diff vs previous) -40.00% 5.000 3.000 - -98.39% -
437.leslie3d (unexplained diff vs previous) -37.77% 1321.000 822.000 - 139.65% -
453.povray (max) -33.33% 6.000 4.000 - -97.58% -
437.leslie3d (unexplained diff vs previous) (percentage) -33.04% 11.470 7.680 - 235.37% -
453.povray (max) (percentage) -25.20% 1.230 0.920 - -63.64% -
458.sjeng (unexplained diff vs previous) (percentage) -23.08% 0.130 0.100 - 66.67% -
458.sjeng (unexplained diff vs previous) -20.00% 10.000 8.000 - -11.11% -
416.gamess (max) -19.05% 21.000 17.000 - -97.41% -
416.gamess (max) (percentage) -17.65% 0.170 0.140 - -95.16% -
444.namd (max) -16.67% 6.000 5.000 - -99.23% -
434.zeusmp (max) -13.33% 15.000 13.000 - -95.68% -
429.mcf (unexplained diff vs previous) (percentage) -12.79% 0.860 0.750 - 38.89% -
401.bzip2 (max) -12.50% 8.000 7.000 - -94.74% -
429.mcf (unexplained diff vs previous) -12.50% 16.000 14.000 - -84.95% -
437.leslie3d (max) -12.27% 1491.000 1308.000 - 272.65% -
434.zeusmp (max) (percentage) -11.67% 0.600 0.530 - -76.75% -
471.omnetpp (max) -10.64% 47.000 42.000 - -93.66% -
433.milc (max) (percentage) -10.53% 0.950 0.850 - -73.19% -
433.milc (max) -10.00% 10.000 9.000 - -97.81% -
471.omnetpp (max) (percentage) -9.82% 1.630 1.470 - -76.18% -
459.GemsFDTD (max) -8.82% 34.000 31.000 - -77.86% -
465.tonto (max) -8.77% 57.000 52.000 - -82.25% -
482.sphinx3 (max) (percentage) -8.00% 0.500 0.460 - -68.71% -
465.tonto (max) (percentage) -7.55% 0.530 0.490 - -86.31% -
459.GemsFDTD (max) (percentage) -7.02% 0.570 0.530 - -11.67% -
401.bzip2 (max) (percentage) -6.90% 0.290 0.270 - -69.32% -
400.perlbench (max) -6.45% 31.000 29.000 - -91.59% -
447.dealII (max) (percentage) -6.38% 0.940 0.880 - -20.00% -
444.namd -6.25% 16.000 15.000 - -95.77% -
444.namd (avg) -6.25% 16.000 15.000 - -95.77% -
437.leslie3d -6.06% 528.000 496.000 - 38.94% -
437.leslie3d (avg) -6.06% 528.000 496.000 - 38.94% -
444.namd (max) (percentage) -5.95% 0.840 0.790 - -89.00% -
437.leslie3d (max) (percentage) -5.56% 12.950 12.230 - 422.65% -
447.dealII (max) -5.56% 18.000 17.000 - -84.11% -
462.libquantum (max) (percentage) -5.11% 1.370 1.300 - 103.12% -
482.sphinx3 -4.55% 22.000 21.000 - -95.18% -
482.sphinx3 (avg) -4.55% 22.000 21.000 - -95.18% -
450.soplex (max) (percentage) -3.90% 0.770 0.740 - -62.44% -
401.bzip2 -3.64% 55.000 53.000 - -19.70% -
401.bzip2 (avg) -3.64% 55.000 53.000 - -19.70% -
454.calculix (max) (percentage) -3.63% 6.060 5.840 - 24.26% -
470.lbm -3.33% 60.000 58.000 - -80.86% -
470.lbm (avg) -3.33% 60.000 58.000 - -80.86% -
465.tonto -3.25% 154.000 149.000 - 8.76% -
465.tonto (avg) -3.25% 154.000 149.000 - 8.76% -
456.hmmer -3.03% 33.000 32.000 - -57.33% -
456.hmmer (avg) -3.03% 33.000 32.000 - -57.33% -
465.tonto (avg) (percentage) -2.80% 1.430 1.390 - -16.77% -
445.gobmk -2.78% 72.000 70.000 - -23.91% -
445.gobmk (avg) -2.78% 72.000 70.000 - -23.91% -
481.wrf -2.78% 108.000 105.000 - -66.98% -
481.wrf (avg) -2.78% 108.000 105.000 - -66.98% -
416.gamess -2.76% 145.000 141.000 - -71.34% -
416.gamess (avg) -2.76% 145.000 141.000 - -71.34% -
458.sjeng -2.70% 74.000 72.000 - 53.19% -
458.sjeng (avg) -2.70% 74.000 72.000 - 53.19% -
435.gromacs -2.65% 151.000 147.000 - -44.74% -
435.gromacs (avg) -2.65% 151.000 147.000 - -44.74% -
416.gamess (avg) (percentage) -2.59% 1.160 1.130 - -47.93% -
453.povray (avg) (percentage) -2.55% 2.350 2.290 - 83.20% -
400.perlbench (max) (percentage) -2.50% 2.000 1.950 - -39.81% -
434.zeusmp -2.17% 46.000 45.000 - -76.92% -
434.zeusmp (avg) -2.17% 46.000 45.000 - -76.92% -
434.zeusmp (avg) (percentage) -2.16% 1.850 1.810 - 22.30% -
445.gobmk (avg) (percentage) -2.14% 1.400 1.370 - 77.92% -
458.sjeng (avg) (percentage) -2.08% 0.960 0.940 - 184.85% -
401.bzip2 (avg) (percentage) -2.07% 1.930 1.890 - 329.55% -
462.libquantum (avg) (percentage) -2.01% 1.990 1.950 - 200.00% -
473.astar -1.98% 101.000 99.000 - 253.57% -
473.astar (avg) -1.98% 101.000 99.000 - 253.57% -
459.GemsFDTD -1.90% 105.000 103.000 - -50.95% -
459.GemsFDTD (avg) -1.90% 105.000 103.000 - -50.95% -
470.lbm (avg) (percentage) -1.85% 1.620 1.590 - -32.91% -
444.namd (avg) (percentage) -1.77% 2.260 2.220 - -43.51% -
459.GemsFDTD (avg) (percentage) -1.71% 1.750 1.720 - 91.11% -
473.astar (avg) (percentage) -1.69% 1.780 1.750 - 821.05% -
482.sphinx3 (avg) (percentage) -1.64% 2.440 2.400 - 64.38% -
429.mcf (avg) (percentage) -1.62% 5.570 5.480 - 933.96% -
450.soplex (avg) (percentage) -1.53% 3.260 3.210 - 158.87% -
481.wrf (avg) (percentage) -1.52% 1.980 1.950 - 3.17% -
410.bwaves (avg) (percentage) -1.41% 1.420 1.400 - 28.44% -
464.h264ref -1.41% 71.000 70.000 - -10.26% -
464.h264ref (avg) -1.41% 71.000 70.000 - -10.26% -
403.gcc (max) (percentage) -1.28% 6.250 6.170 - 756.94% -
471.omnetpp (avg) (percentage) -1.16% 2.590 2.560 - 61.01% -
433.milc (avg) (percentage) -1.01% 1.990 1.970 - 15.88% -

Added Tests - sample variability
416.gamess (unexplained diff vs previous)
416.gamess (unexplained diff vs previous) (percentage)
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)
481.wrf (unexplained diff vs previous)
481.wrf (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
400.perlbench 1.16% 930652.000 941436.000 - 1.77% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 31.12% 8205.000 10758.000 - -0.18% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (unexplained diff vs previous) (percentage) 16300.00% 0.020 3.280 - 0.00% -
483.xalancbmk (max) (percentage) 1331.71% 0.410 5.870 - -0.17% -
429.mcf (avg) (percentage) 933.96% 0.530 5.480 - -1.62% -
403.gcc (avg) (percentage) 839.34% 0.610 5.730 - 0.00% -
473.astar (avg) (percentage) 821.05% 0.190 1.750 - -1.69% -
403.gcc (max) (percentage) 756.94% 0.720 6.170 - -1.28% -
410.bwaves (unexplained diff vs previous) (percentage) 510.00% 0.100 0.610 - 258.82% -
437.leslie3d (max) (percentage) 422.65% 2.340 12.230 - -5.56% -
473.astar (unexplained diff vs previous) (percentage) 350.00% 0.020 0.090 - -67.86% -
401.bzip2 (avg) (percentage) 329.55% 0.440 1.890 - -2.07% -
437.leslie3d (max) 272.65% 351.000 1308.000 - -12.27% -
473.astar 253.57% 28.000 99.000 - -1.98% -
473.astar (avg) 253.57% 28.000 99.000 - -1.98% -
437.leslie3d (unexplained diff vs previous) (percentage) 235.37% 2.290 7.680 - -33.04% -
462.libquantum (avg) (percentage) 200.00% 0.650 1.950 - -2.01% -
410.bwaves (max) (percentage) 190.48% 0.210 0.610 - 27.08% -
458.sjeng (avg) (percentage) 184.85% 0.330 0.940 - -2.08% -
471.omnetpp (unexplained diff vs previous) (percentage) 170.83% 0.240 0.650 - 828.57% -
450.soplex (avg) (percentage) 158.87% 1.240 3.210 - -1.53% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 147.62% 0.210 0.520 - 642.86% -
410.bwaves (unexplained diff vs previous) 140.00% 15.000 36.000 - 260.00% -
437.leslie3d (unexplained diff vs previous) 139.65% 343.000 822.000 - -37.77% -
464.h264ref (avg) (percentage) 124.07% 0.540 1.210 - 0.00% -
462.libquantum (max) (percentage) 103.12% 0.640 1.300 - -5.11% -
437.leslie3d (avg) (percentage) 94.96% 2.380 4.640 - 1.09% -
459.GemsFDTD (avg) (percentage) 91.11% 0.900 1.720 - -1.71% -
400.perlbench (avg) (percentage) 86.39% 1.470 2.740 - 0.00% -
473.astar (max) (percentage) 85.71% 0.140 0.260 - -88.13% -
453.povray (avg) (percentage) 83.20% 1.250 2.290 - -2.55% -
454.calculix (avg) (percentage) 81.09% 2.010 3.640 - -0.27% -
445.gobmk (avg) (percentage) 77.92% 0.770 1.370 - -2.14% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
447.dealII (avg) (percentage) 67.29% 1.070 1.790 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) 66.67% 0.060 0.100 - -23.08% -
473.astar (unexplained diff vs previous) 66.67% 3.000 5.000 - -68.75% -
456.hmmer (avg) (percentage) 64.71% 0.680 1.120 - -0.88% -
482.sphinx3 (avg) (percentage) 64.38% 1.460 2.400 - -1.64% -
471.omnetpp (avg) (percentage) 61.01% 1.590 2.560 - -1.16% -
458.sjeng 53.19% 47.000 72.000 - -2.70% -
458.sjeng (avg) 53.19% 47.000 72.000 - -2.70% -
437.leslie3d 38.94% 357.000 496.000 - -6.06% -
437.leslie3d (avg) 38.94% 357.000 496.000 - -6.06% -
429.mcf (max) (percentage) 38.89% 0.540 0.750 - -95.50% -
429.mcf (unexplained diff vs previous) (percentage) 38.89% 0.540 0.750 - -12.79% -
410.bwaves (avg) (percentage) 28.44% 1.090 1.400 - -1.41% -
454.calculix (max) (percentage) 24.26% 4.700 5.840 - -3.63% -
435.gromacs (avg) (percentage) 23.70% 1.730 2.140 - -0.93% -
434.zeusmp (avg) (percentage) 22.30% 1.480 1.810 - -2.16% -
410.bwaves (max) 20.00% 30.000 36.000 - 28.57% -
433.milc (avg) (percentage) 15.88% 1.700 1.970 - -1.01% -
429.mcf 13.33% 90.000 102.000 - -0.97% -
429.mcf (avg) 13.33% 90.000 102.000 - -0.97% -
481.wrf (unexplained diff vs previous) (percentage) 10.45% 0.670 0.740 - 0.00% -
465.tonto 8.76% 137.000 149.000 - -3.25% -
465.tonto (avg) 8.76% 137.000 149.000 - -3.25% -
464.h264ref (max) (percentage) 4.65% 0.430 0.450 - 0.00% -
481.wrf (avg) (percentage) 3.17% 1.890 1.950 - -1.52% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -3.57% 120221.000 115929.000 - 0.00% -
464.h264ref -2.37% 428364.000 418224.000 - -0.00% -
434.zeusmp -1.65% 165699.000 162969.000 - 0.00% -
459.GemsFDTD -1.57% 253716.000 249721.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.73% 22860.000 61.000 - -3.17% -
403.gcc -99.38% 9010.000 56.000 - -1.75% -
462.libquantum -98.85% 11614.000 134.000 - 0.00% -
482.sphinx3 -96.98% 29928.000 903.000 - -0.44% -
450.soplex -96.46% 15466.000 548.000 - 0.74% -
453.povray -92.48% 6544.000 492.000 - 0.00% -
444.namd -92.10% 9050.000 715.000 - -0.28% -
433.milc -91.20% 12976.000 1142.000 - -0.44% -
429.mcf -89.08% 17158.000 1874.000 - 0.75% -
436.cactusADM -87.80% 17544.000 2140.000 - -0.70% -
400.perlbench -85.58% 10650.000 1536.000 - -1.48% -
401.bzip2 -81.20% 15185.000 2855.000 - 0.14% -
434.zeusmp -80.87% 13211.000 2527.000 - 0.04% -
447.dealII -79.86% 9751.000 1964.000 - -0.30% -
459.GemsFDTD -74.26% 23356.000 6011.000 - -0.51% -
456.hmmer -73.69% 11046.000 2906.000 - -0.55% -
471.omnetpp -72.88% 10739.000 2912.000 - 0.66% -
470.lbm -71.07% 12819.000 3708.000 - 0.00% -
481.wrf -67.83% 16834.000 5416.000 - -0.73% -
473.astar -61.87% 14932.000 5694.000 - -0.09% -
464.h264ref -60.20% 14556.000 5793.000 - -1.56% -
410.bwaves -59.74% 14755.000 5940.000 - 0.61% -
445.gobmk -56.96% 11995.000 5163.000 - 0.06% -
435.gromacs -55.19% 15419.000 6909.000 - -1.30% -
458.sjeng -45.82% 14275.000 7734.000 - 0.10% -
416.gamess -44.87% 22704.000 12516.000 - -0.04% -
437.leslie3d -28.70% 15005.000 10699.000 - -7.13% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.72% 1074.000 3.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.71% 700.000 2.000 - -66.67% -
434.zeusmp (unexplained diff vs previous) -99.68% 308.000 1.000 - 0.00% -
454.calculix -99.56% 459.000 2.000 - 0.00% -
454.calculix (avg) -99.56% 459.000 2.000 - 0.00% -
416.gamess (unexplained diff vs previous) -99.26% 676.000 5.000 - 0.00% -
444.namd (max) -99.23% 649.000 5.000 - -16.67% -
482.sphinx3 (max) -99.09% 439.000 4.000 - 0.00% -
470.lbm (max) -98.93% 653.000 7.000 - -75.00% -
433.milc (unexplained diff vs previous) -98.82% 425.000 5.000 - 150.00% -
450.soplex (unexplained diff vs previous) -98.71% 311.000 4.000 - 33.33% -
450.soplex (max) -98.68% 304.000 4.000 - 0.00% -
416.gamess (unexplained diff vs previous) (percentage) -98.66% 2.980 0.040 - 0.00% -
462.libquantum (max) -98.65% 74.000 1.000 - 0.00% -
445.gobmk (unexplained diff vs previous) -98.39% 186.000 3.000 - -40.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -98.28% 2.330 0.040 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -97.85% 186.000 4.000 - 100.00% -
433.milc (max) -97.81% 411.000 9.000 - -10.00% -
453.povray (max) -97.58% 165.000 4.000 - -33.33% -
436.cactusADM (max) -97.46% 552.000 14.000 - 75.00% -
416.gamess (max) -97.41% 656.000 17.000 - -19.05% -
436.cactusADM (unexplained diff vs previous) -97.37% 570.000 15.000 - 87.50% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) -96.97% 132.000 4.000 - 100.00% -
444.namd (unexplained diff vs previous) (percentage) -96.38% 7.730 0.280 - -66.67% -
470.lbm (max) (percentage) -96.27% 5.100 0.190 - -75.64% -
445.gobmk (unexplained diff vs previous) (percentage) -96.13% 1.550 0.060 - -40.00% -
444.namd -95.77% 355.000 15.000 - -6.25% -
444.namd (avg) -95.77% 355.000 15.000 - -6.25% -
434.zeusmp (max) -95.68% 301.000 13.000 - -13.33% -
403.gcc (max) -95.31% 64.000 3.000 - 0.00% -
482.sphinx3 -95.18% 436.000 21.000 - -4.55% -
482.sphinx3 (avg) -95.18% 436.000 21.000 - -4.55% -
416.gamess (max) (percentage) -95.16% 2.890 0.140 - -17.65% -
401.bzip2 (max) -94.74% 133.000 7.000 - -12.50% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
471.omnetpp (max) -93.66% 662.000 42.000 - -10.64% -
445.gobmk (max) -93.44% 183.000 12.000 - -47.83% -
481.wrf (max) -92.64% 530.000 39.000 - 18.18% -
400.perlbench (max) -91.59% 345.000 29.000 - -6.45% -
450.soplex -91.10% 191.000 17.000 - 0.00% -
450.soplex (avg) -91.10% 191.000 17.000 - 0.00% -
433.milc -90.00% 220.000 22.000 - 0.00% -
433.milc (avg) -90.00% 220.000 22.000 - 0.00% -
456.hmmer (max) -89.29% 140.000 15.000 - 114.29% -
444.namd (max) (percentage) -89.00% 7.180 0.790 - -5.95% -
436.cactusADM -87.97% 291.000 35.000 - 0.00% -
436.cactusADM (avg) -87.97% 291.000 35.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -86.59% 3.280 0.440 - 158.82% -
453.povray -86.42% 81.000 11.000 - 0.00% -
453.povray (avg) -86.42% 81.000 11.000 - 0.00% -
465.tonto (max) (percentage) -86.31% 3.580 0.490 - -7.55% -
429.mcf (unexplained diff vs previous) -84.95% 93.000 14.000 - -12.50% -
429.mcf (max) -84.78% 92.000 14.000 - -95.48% -
445.gobmk (max) (percentage) -84.31% 1.530 0.240 - -46.67% -
447.dealII (max) -84.11% 107.000 17.000 - -5.56% -
401.bzip2 (unexplained diff vs previous) (percentage) -83.91% 0.870 0.140 - 100.00% -
465.tonto (max) -82.25% 293.000 52.000 - -8.77% -
470.lbm -80.86% 303.000 58.000 - -3.33% -
470.lbm (avg) -80.86% 303.000 58.000 - -3.33% -
456.hmmer (unexplained diff vs previous) -80.72% 83.000 16.000 - 166.67% -
436.cactusADM (unexplained diff vs previous) (percentage) -78.46% 3.250 0.700 - 89.19% -
459.GemsFDTD (max) -77.86% 140.000 31.000 - -8.82% -
436.cactusADM (max) (percentage) -77.78% 3.150 0.700 - 84.21% -
434.zeusmp -76.92% 195.000 45.000 - -2.17% -
434.zeusmp (avg) -76.92% 195.000 45.000 - -2.17% -
481.wrf (max) (percentage) -76.83% 3.150 0.730 - 17.74% -
434.zeusmp (max) (percentage) -76.75% 2.280 0.530 - -11.67% -
465.tonto (unexplained diff vs previous) (percentage) -76.62% 0.770 0.180 - -66.04% -
471.omnetpp (max) (percentage) -76.18% 6.170 1.470 - -9.82% -
433.milc (max) (percentage) -73.19% 3.170 0.850 - -10.53% -
400.perlbench -73.08% 156.000 42.000 - 0.00% -
400.perlbench (avg) -73.08% 156.000 42.000 - 0.00% -
435.gromacs (unexplained diff vs previous) -71.38% 318.000 91.000 - 0.00% -
435.gromacs (max) -71.38% 311.000 89.000 - 89.36% -
416.gamess -71.34% 492.000 141.000 - -2.76% -
416.gamess (avg) -71.34% 492.000 141.000 - -2.76% -
465.tonto (unexplained diff vs previous) -69.84% 63.000 19.000 - -66.67% -
401.bzip2 (max) (percentage) -69.32% 0.880 0.270 - -6.90% -
482.sphinx3 (max) (percentage) -68.71% 1.470 0.460 - -8.00% -
458.sjeng (max) -67.74% 31.000 10.000 - -50.00% -
481.wrf -66.98% 318.000 105.000 - -2.78% -
481.wrf (avg) -66.98% 318.000 105.000 - -2.78% -
447.dealII -66.35% 104.000 35.000 - 0.00% -
447.dealII (avg) -66.35% 104.000 35.000 - 0.00% -
481.wrf (unexplained diff vs previous) -64.29% 112.000 40.000 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) -63.68% 2.010 0.730 - 32.73% -
453.povray (max) (percentage) -63.64% 2.530 0.920 - -25.20% -
450.soplex (max) (percentage) -62.44% 1.970 0.740 - -3.90% -
454.calculix (unexplained diff vs previous) -60.00% 5.000 2.000 - 0.00% -
464.h264ref (max) -58.06% 62.000 26.000 - 0.00% -
456.hmmer -57.33% 75.000 32.000 - -3.03% -
456.hmmer (avg) -57.33% 75.000 32.000 - -3.03% -
456.hmmer (max) (percentage) -56.69% 1.270 0.550 - 129.17% -
471.omnetpp -56.47% 170.000 74.000 - 0.00% -
471.omnetpp (avg) -56.47% 170.000 74.000 - 0.00% -
459.GemsFDTD -50.95% 210.000 103.000 - -1.90% -
459.GemsFDTD (avg) -50.95% 210.000 103.000 - -1.90% -
410.bwaves -48.12% 160.000 83.000 - 0.00% -
410.bwaves (avg) -48.12% 160.000 83.000 - 0.00% -
416.gamess (avg) (percentage) -47.93% 2.170 1.130 - -2.59% -
435.gromacs -44.74% 266.000 147.000 - -2.65% -
435.gromacs (avg) -44.74% 266.000 147.000 - -2.65% -
444.namd (avg) (percentage) -43.51% 3.930 2.220 - -1.77% -
400.perlbench (max) (percentage) -39.81% 3.240 1.950 - -2.50% -
458.sjeng (max) (percentage) -36.36% 0.220 0.140 - -46.15% -
435.gromacs (unexplained diff vs previous) (percentage) -35.92% 2.060 1.320 - 0.00% -
435.gromacs (max) (percentage) -35.64% 2.020 1.300 - 91.18% -
459.GemsFDTD (unexplained diff vs previous) -35.42% 48.000 31.000 - 675.00% -
470.lbm (avg) (percentage) -32.91% 2.370 1.590 - -1.85% -
473.astar (max) -30.00% 20.000 14.000 - -88.71% -
482.sphinx3 (unexplained diff vs previous) (percentage) -29.03% 0.620 0.440 - 100.00% -
471.omnetpp (unexplained diff vs previous) -26.92% 26.000 19.000 - 850.00% -
456.hmmer (unexplained diff vs previous) (percentage) -26.67% 0.750 0.550 - 161.90% -
445.gobmk -23.91% 92.000 70.000 - -2.78% -
445.gobmk (avg) -23.91% 92.000 70.000 - -2.78% -
447.dealII (max) (percentage) -20.00% 1.100 0.880 - -6.38% -
401.bzip2 -19.70% 66.000 53.000 - -3.64% -
401.bzip2 (avg) -19.70% 66.000 53.000 - -3.64% -
465.tonto (avg) (percentage) -16.77% 1.670 1.390 - -2.80% -
459.GemsFDTD (max) (percentage) -11.67% 0.600 0.530 - -7.02% -
458.sjeng (unexplained diff vs previous) -11.11% 9.000 8.000 - -20.00% -
464.h264ref -10.26% 78.000 70.000 - -1.41% -
464.h264ref (avg) -10.26% 78.000 70.000 - -1.41% -

Added Tests - sample variability
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1536.000 -1.48%
401.bzip2 2855.000 -
403.gcc 56.000 -1.75%
410.bwaves 5940.000 -
416.gamess 12516.000 -
429.mcf 1874.000 -
433.milc 1142.000 -
434.zeusmp 2527.000 -
435.gromacs 6909.000 -1.30%
436.cactusADM 2140.000 -
437.leslie3d 10699.000 -7.13%
444.namd 715.000 -
445.gobmk 5163.000 -
447.dealII 1964.000 -
450.soplex 548.000 -
453.povray 492.000 -
454.calculix 61.000 -3.17%
456.hmmer 2906.000 -
458.sjeng 7734.000 -
459.GemsFDTD 6011.000 -
462.libquantum 134.000 -
464.h264ref 5793.000 -1.56%
465.tonto 10758.000 -
470.lbm 3708.000 -
471.omnetpp 2912.000 -
473.astar 5694.000 -
481.wrf 5416.000 -
482.sphinx3 903.000 -
Geometric Mean 2062.084 -
Code Size (bytes)
Name Current %
400.perlbench 941436.000 1.77%
401.bzip2 49698.000 -
403.gcc 2671145.000 -
410.bwaves 17640.000 -
416.gamess 6241643.000 -
429.mcf 10979.000 -
433.milc 95589.000 -
434.zeusmp 162969.000 -
435.gromacs 678045.000 -
436.cactusADM 551180.000 -
437.leslie3d 64764.000 -
444.namd 190762.000 -
445.gobmk 1399537.000 -
447.dealII 2122042.000 -
450.soplex 259980.000 -
453.povray 702162.000 -
454.calculix 1268791.000 -
456.hmmer 230496.000 -
458.sjeng 115929.000 -
459.GemsFDTD 249721.000 -
462.libquantum 31321.000 -
464.h264ref 418224.000 -
465.tonto 2868541.000 -
470.lbm 10916.000 -
471.omnetpp 566294.000 -
473.astar 32056.000 -
481.wrf 2617276.000 -
482.sphinx3 152548.000 -
Geometric Mean 272070.274 -
sample variability
Name Current %
400.perlbench 42.000 -
400.perlbench (avg) 42.000 -
400.perlbench (avg) (percentage) 2.740 -
400.perlbench (max) 29.000 -6.45%
400.perlbench (max) (percentage) 1.950 -2.50%
401.bzip2 53.000 -3.64%
401.bzip2 (avg) 53.000 -3.64%
401.bzip2 (avg) (percentage) 1.890 -2.07%
401.bzip2 (max) 7.000 -12.50%
401.bzip2 (max) (percentage) 0.270 -6.90%
401.bzip2 (unexplained diff vs previous) 4.000 100.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.140 100.00%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 5.730 -
403.gcc (max) 3.000 -
403.gcc (max) (percentage) 6.170 -1.28%
410.bwaves 83.000 -
410.bwaves (avg) 83.000 -
410.bwaves (avg) (percentage) 1.400 -1.41%
410.bwaves (max) 36.000 28.57%
410.bwaves (max) (percentage) 0.610 27.08%
410.bwaves (unexplained diff vs previous) 36.000 260.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.610 258.82%
416.gamess 141.000 -2.76%
416.gamess (avg) 141.000 -2.76%
416.gamess (avg) (percentage) 1.130 -2.59%
416.gamess (max) 17.000 -19.05%
416.gamess (max) (percentage) 0.140 -17.65%
416.gamess (unexplained diff vs previous) 5.000 -
416.gamess (unexplained diff vs previous) (percentage) 0.040 -
429.mcf 102.000 -
429.mcf (avg) 102.000 -
429.mcf (avg) (percentage) 5.480 -1.62%
429.mcf (max) 14.000 -95.48%
429.mcf (max) (percentage) 0.750 -95.50%
429.mcf (unexplained diff vs previous) 14.000 -12.50%
429.mcf (unexplained diff vs previous) (percentage) 0.750 -12.79%
433.milc 22.000 -
433.milc (avg) 22.000 -
433.milc (avg) (percentage) 1.970 -1.01%
433.milc (max) 9.000 -10.00%
433.milc (max) (percentage) 0.850 -10.53%
433.milc (unexplained diff vs previous) 5.000 150.00%
433.milc (unexplained diff vs previous) (percentage) 0.440 158.82%
434.zeusmp 45.000 -2.17%
434.zeusmp (avg) 45.000 -2.17%
434.zeusmp (avg) (percentage) 1.810 -2.16%
434.zeusmp (max) 13.000 -13.33%
434.zeusmp (max) (percentage) 0.530 -11.67%
434.zeusmp (unexplained diff vs previous) 1.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 0.040 -
435.gromacs 147.000 -2.65%
435.gromacs (avg) 147.000 -2.65%
435.gromacs (avg) (percentage) 2.140 -
435.gromacs (max) 89.000 89.36%
435.gromacs (max) (percentage) 1.300 91.18%
435.gromacs (unexplained diff vs previous) 91.000 -
435.gromacs (unexplained diff vs previous) (percentage) 1.320 -
436.cactusADM 35.000 -
436.cactusADM (avg) 35.000 -
436.cactusADM (avg) (percentage) 1.650 -
436.cactusADM (max) 14.000 75.00%
436.cactusADM (max) (percentage) 0.700 84.21%
436.cactusADM (unexplained diff vs previous) 15.000 87.50%
436.cactusADM (unexplained diff vs previous) (percentage) 0.700 89.19%
437.leslie3d 496.000 -6.06%
437.leslie3d (avg) 496.000 -6.06%
437.leslie3d (avg) (percentage) 4.640 1.09%
437.leslie3d (max) 1308.000 -12.27%
437.leslie3d (max) (percentage) 12.230 -5.56%
437.leslie3d (unexplained diff vs previous) 822.000 -37.77%
437.leslie3d (unexplained diff vs previous) (percentage) 7.680 -33.04%
444.namd 15.000 -6.25%
444.namd (avg) 15.000 -6.25%
444.namd (avg) (percentage) 2.220 -1.77%
444.namd (max) 5.000 -16.67%
444.namd (max) (percentage) 0.790 -5.95%
444.namd (unexplained diff vs previous) 2.000 -66.67%
444.namd (unexplained diff vs previous) (percentage) 0.280 -66.67%
445.gobmk 70.000 -2.78%
445.gobmk (avg) 70.000 -2.78%
445.gobmk (avg) (percentage) 1.370 -2.14%
445.gobmk (max) 12.000 -47.83%
445.gobmk (max) (percentage) 0.240 -46.67%
445.gobmk (unexplained diff vs previous) 3.000 -40.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.060 -40.00%
447.dealII 35.000 -
447.dealII (avg) 35.000 -
447.dealII (avg) (percentage) 1.790 -
447.dealII (max) 17.000 -5.56%
447.dealII (max) (percentage) 0.880 -6.38%
450.soplex 17.000 -
450.soplex (avg) 17.000 -
450.soplex (avg) (percentage) 3.210 -1.53%
450.soplex (max) 4.000 -
450.soplex (max) (percentage) 0.740 -3.90%
450.soplex (unexplained diff vs previous) 4.000 33.33%
450.soplex (unexplained diff vs previous) (percentage) 0.730 32.73%
453.povray 11.000 -
453.povray (avg) 11.000 -
453.povray (avg) (percentage) 2.290 -2.55%
453.povray (max) 4.000 -33.33%
453.povray (max) (percentage) 0.920 -25.20%
453.povray (unexplained diff vs previous) 0.000 -
453.povray (unexplained diff vs previous) (percentage) 0.000 -
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 3.640 -
454.calculix (max) 3.000 -
454.calculix (max) (percentage) 5.840 -3.63%
454.calculix (unexplained diff vs previous) 2.000 -
454.calculix (unexplained diff vs previous) (percentage) 3.280 -
456.hmmer 32.000 -3.03%
456.hmmer (avg) 32.000 -3.03%
456.hmmer (avg) (percentage) 1.120 -
456.hmmer (max) 15.000 114.29%
456.hmmer (max) (percentage) 0.550 129.17%
456.hmmer (unexplained diff vs previous) 16.000 166.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.550 161.90%
458.sjeng 72.000 -2.70%
458.sjeng (avg) 72.000 -2.70%
458.sjeng (avg) (percentage) 0.940 -2.08%
458.sjeng (max) 10.000 -50.00%
458.sjeng (max) (percentage) 0.140 -46.15%
458.sjeng (unexplained diff vs previous) 8.000 -20.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.100 -23.08%
459.GemsFDTD 103.000 -1.90%
459.GemsFDTD (avg) 103.000 -1.90%
459.GemsFDTD (avg) (percentage) 1.720 -1.71%
459.GemsFDTD (max) 31.000 -8.82%
459.GemsFDTD (max) (percentage) 0.530 -7.02%
459.GemsFDTD (unexplained diff vs previous) 31.000 675.00%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.520 642.86%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.950 -2.01%
462.libquantum (max) 1.000 -
462.libquantum (max) (percentage) 1.300 -5.11%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 70.000 -1.41%
464.h264ref (avg) 70.000 -1.41%
464.h264ref (avg) (percentage) 1.210 -
464.h264ref (max) 26.000 -
464.h264ref (max) (percentage) 0.450 -
465.tonto 149.000 -3.25%
465.tonto (avg) 149.000 -3.25%
465.tonto (avg) (percentage) 1.390 -2.80%
465.tonto (max) 52.000 -8.77%
465.tonto (max) (percentage) 0.490 -7.55%
465.tonto (unexplained diff vs previous) 19.000 -66.67%
465.tonto (unexplained diff vs previous) (percentage) 0.180 -66.04%
470.lbm 58.000 -3.33%
470.lbm (avg) 58.000 -3.33%
470.lbm (avg) (percentage) 1.590 -1.85%
470.lbm (max) 7.000 -75.00%
470.lbm (max) (percentage) 0.190 -75.64%
470.lbm (unexplained diff vs previous) 0.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.000 -
471.omnetpp 74.000 -
471.omnetpp (avg) 74.000 -
471.omnetpp (avg) (percentage) 2.560 -1.16%
471.omnetpp (max) 42.000 -10.64%
471.omnetpp (max) (percentage) 1.470 -9.82%
471.omnetpp (unexplained diff vs previous) 19.000 850.00%
471.omnetpp (unexplained diff vs previous) (percentage) 0.650 828.57%
473.astar 99.000 -1.98%
473.astar (avg) 99.000 -1.98%
473.astar (avg) (percentage) 1.750 -1.69%
473.astar (max) 14.000 -88.71%
473.astar (max) (percentage) 0.260 -88.13%
473.astar (unexplained diff vs previous) 5.000 -68.75%
473.astar (unexplained diff vs previous) (percentage) 0.090 -67.86%
481.wrf 105.000 -2.78%
481.wrf (avg) 105.000 -2.78%
481.wrf (avg) (percentage) 1.950 -1.52%
481.wrf (max) 39.000 18.18%
481.wrf (max) (percentage) 0.730 17.74%
481.wrf (unexplained diff vs previous) 40.000 -
481.wrf (unexplained diff vs previous) (percentage) 0.740 -
482.sphinx3 21.000 -4.55%
482.sphinx3 (avg) 21.000 -4.55%
482.sphinx3 (avg) (percentage) 2.400 -1.64%
482.sphinx3 (max) 4.000 -
482.sphinx3 (max) (percentage) 0.460 -8.00%
482.sphinx3 (unexplained diff vs previous) 4.000 100.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.440 100.00%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.870 -
Geometric Mean 4.123 -36.20%