Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=164cc86b81de7ab25682a234b978134d08fb3009 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/f7884f7673444b8a2c10ea0981d480f2e82dd16a (+ 1 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=b6e3898194bbae78910bbe9cd086937014961e45 (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=dd22391461d1c15b8296f13695365005f8be2e90 (unchanged)
regression GNU-1183
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/199/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0199 (basepoints/gcc-14-05879-gf7884f76734) 2024-02-29T15:25:05 0:00:00
Previous 0196 (3-changed-components) 2024-02-23T15:56:10 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 30 69
Performance Improvements 107 153
Added Tests 1 1
Unchanged Tests 1098 1013
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
433.milc 4.28% 95617.000 99705.000 - 4.00% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 1.67% 60.000 61.000 - -99.32% -
450.soplex 1.42% 565.000 573.000 - -96.30% -
429.mcf 1.40% 1500.000 1521.000 - -91.14% -
471.omnetpp 1.30% 2927.000 2965.000 - -72.39% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
453.povray (max) 300.00% 3.000 12.000 - -92.73% -
453.povray (max) (percentage) 289.55% 0.670 2.610 - 3.16% -
434.zeusmp (unexplained diff vs previous) 200.00% 1.000 3.000 - -99.03% -
434.zeusmp (unexplained diff vs previous) (percentage) 200.00% 0.040 0.120 - -94.85% -
400.perlbench (unexplained diff vs previous) 150.00% 4.000 10.000 - -97.20% -
400.perlbench (unexplained diff vs previous) (percentage) 146.15% 0.260 0.640 - -80.90% -
473.astar (unexplained diff vs previous) 86.67% 15.000 28.000 - 833.33% -
447.dealII (unexplained diff vs previous) (percentage) 82.84% 1.340 2.450 - 214.10% -
473.astar (unexplained diff vs previous) (percentage) 80.77% 0.260 0.470 - 2250.00% -
400.perlbench (max) 80.00% 5.000 9.000 - -97.39% -
447.dealII (unexplained diff vs previous) 77.78% 27.000 48.000 - -36.84% -
447.dealII (max) (percentage) 75.74% 1.360 2.390 - 117.27% -
447.dealII (max) 70.37% 27.000 46.000 - -57.01% -
400.perlbench (max) (percentage) 68.42% 0.380 0.640 - -80.25% -
459.GemsFDTD (max) 36.36% 22.000 30.000 - -78.57% -
459.GemsFDTD (max) (percentage) 32.43% 0.370 0.490 - -18.33% -
459.GemsFDTD (unexplained diff vs previous) 30.43% 23.000 30.000 - -37.50% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 28.95% 0.380 0.490 - 133.33% -
437.leslie3d (unexplained diff vs previous) (percentage) 28.03% 2.890 3.700 - 61.57% -
437.leslie3d (unexplained diff vs previous) 23.33% 330.000 407.000 - 18.66% -
471.omnetpp (unexplained diff vs previous) 15.15% 33.000 38.000 - 46.15% -
471.omnetpp (unexplained diff vs previous) (percentage) 13.27% 1.130 1.280 - 433.33% -
437.leslie3d (max) (percentage) 1.42% 3.510 3.560 - 52.14% -
447.dealII (avg) (percentage) 1.36% 1.470 1.490 - 39.25% -
429.mcf (max) 1.35% 148.000 150.000 - 63.04% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -3.56% 11419.000 11012.000 - -26.61% -
453.povray -2.61% 499.000 486.000 - -92.57% -
447.dealII -2.39% 2009.000 1961.000 - -79.89% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
445.gobmk (unexplained diff vs previous) -73.33% 15.000 4.000 - -97.85% -
445.gobmk (unexplained diff vs previous) (percentage) -72.41% 0.290 0.080 - -94.84% -
450.soplex (unexplained diff vs previous) (percentage) -71.77% 4.960 1.400 - -30.35% -
450.soplex (unexplained diff vs previous) -71.43% 28.000 8.000 - -97.43% -
433.milc (unexplained diff vs previous) -66.67% 6.000 2.000 - -99.53% -
433.milc (unexplained diff vs previous) (percentage) -66.67% 0.510 0.170 - -94.82% -
435.gromacs (unexplained diff vs previous) (percentage) -65.20% 2.040 0.710 - -65.53% -
435.gromacs (unexplained diff vs previous) -65.07% 146.000 51.000 - -83.96% -
436.cactusADM (unexplained diff vs previous) (percentage) -58.24% 0.910 0.380 - -88.31% -
436.cactusADM (unexplained diff vs previous) -57.89% 19.000 8.000 - -98.60% -
456.hmmer (unexplained diff vs previous) (percentage) -57.14% 0.070 0.030 - -96.00% -
458.sjeng (unexplained diff vs previous) -55.26% 38.000 17.000 - 88.89% -
458.sjeng (unexplained diff vs previous) (percentage) -55.10% 0.490 0.220 - 266.67% -
444.namd (max) -50.00% 8.000 4.000 - -99.38% -
444.namd (unexplained diff vs previous) -50.00% 6.000 3.000 - -99.57% -
444.namd (unexplained diff vs previous) (percentage) -50.00% 0.820 0.410 - -94.70% -
456.hmmer (unexplained diff vs previous) -50.00% 2.000 1.000 - -98.80% -
444.namd (max) (percentage) -44.04% 1.090 0.610 - -91.50% -
410.bwaves (unexplained diff vs previous) -43.75% 32.000 18.000 - 20.00% -
410.bwaves (unexplained diff vs previous) (percentage) -43.40% 0.530 0.300 - 200.00% -
401.bzip2 (unexplained diff vs previous) -28.57% 21.000 15.000 - -88.64% -
470.lbm (unexplained diff vs previous) -28.57% 14.000 10.000 - -28.57% -
401.bzip2 (unexplained diff vs previous) (percentage) -28.57% 0.700 0.500 - -42.53% -
470.lbm (unexplained diff vs previous) (percentage) -27.03% 0.370 0.270 - 145.45% -
410.bwaves (max) -18.75% 32.000 26.000 - -13.33% -
410.bwaves (max) (percentage) -16.98% 0.530 0.440 - 109.52% -
470.lbm (max) (percentage) -16.22% 0.370 0.310 - -93.92% -
470.lbm (max) -15.38% 13.000 11.000 - -98.32% -
434.zeusmp (max) -14.29% 21.000 18.000 - -94.02% -
435.gromacs (max) (percentage) -12.50% 2.000 1.750 - -13.37% -
450.soplex (max) (percentage) -12.50% 4.720 4.130 - 109.64% -
436.cactusADM (max) (percentage) -12.22% 0.900 0.790 - -74.92% -
435.gromacs (max) -11.97% 142.000 125.000 - -59.81% -
450.soplex (max) -11.54% 26.000 23.000 - -92.43% -
433.milc (max) -11.11% 9.000 8.000 - -98.05% -
436.cactusADM (max) -11.11% 18.000 16.000 - -97.10% -
464.h264ref (max) -11.11% 18.000 16.000 - -74.19% -
482.sphinx3 (max) -11.11% 9.000 8.000 - -98.18% -
434.zeusmp (max) (percentage) -10.98% 0.820 0.730 - -67.98% -
465.tonto (max) (percentage) -10.81% 0.740 0.660 - -81.56% -
465.tonto (max) -10.71% 84.000 75.000 - -74.40% -
433.milc (max) (percentage) -10.39% 0.770 0.690 - -78.23% -
471.omnetpp (max) (percentage) -10.00% 4.000 3.600 - -41.65% -
464.h264ref (max) (percentage) -9.68% 0.310 0.280 - -34.88% -
471.omnetpp (max) -9.40% 117.000 106.000 - -83.99% -
401.bzip2 (max) -7.50% 40.000 37.000 - -72.18% -
462.libquantum (max) (percentage) -7.19% 1.670 1.550 - 142.19% -
456.hmmer (max) -7.14% 14.000 13.000 - -90.71% -
401.bzip2 (max) (percentage) -6.67% 1.350 1.260 - 43.18% -
429.mcf (unexplained diff vs previous) (percentage) -6.12% 1.470 1.380 - 155.56% -
433.milc -5.26% 19.000 18.000 - -91.82% -
433.milc (avg) -5.26% 19.000 18.000 - -91.82% -
454.calculix (max) (percentage) -4.97% 1.610 1.530 - -67.45% -
416.gamess (unexplained diff vs previous) (percentage) -4.55% 0.220 0.210 - -92.95% -
429.mcf (unexplained diff vs previous) -4.55% 22.000 21.000 - -77.42% -
456.hmmer (max) (percentage) -4.17% 0.480 0.460 - -63.78% -
437.leslie3d -4.05% 494.000 474.000 - 32.77% -
437.leslie3d (avg) -4.05% 494.000 474.000 - 32.77% -
456.hmmer -3.85% 26.000 25.000 - -66.67% -
456.hmmer (avg) -3.85% 26.000 25.000 - -66.67% -
416.gamess (unexplained diff vs previous) -3.57% 28.000 27.000 - -96.01% -
403.gcc (max) (percentage) -3.32% 11.140 10.770 - 1395.83% -
401.bzip2 -3.08% 65.000 63.000 - -4.55% -
401.bzip2 (avg) -3.08% 65.000 63.000 - -4.55% -
481.wrf (max) (percentage) -2.84% 2.110 2.050 - -34.92% -
434.zeusmp -2.50% 40.000 39.000 - -80.00% -
434.zeusmp (avg) -2.50% 40.000 39.000 - -80.00% -
416.gamess (avg) (percentage) -2.22% 0.900 0.880 - -59.45% -
470.lbm -2.17% 46.000 45.000 - -85.15% -
470.lbm (avg) -2.17% 46.000 45.000 - -85.15% -
473.astar (max) (percentage) -2.15% 2.790 2.730 - 1850.00% -
482.sphinx3 (max) (percentage) -2.04% 0.980 0.960 - -34.69% -
437.leslie3d (max) -2.00% 400.000 392.000 - 11.68% -
400.perlbench (avg) (percentage) -1.90% 2.100 2.060 - 40.14% -
434.zeusmp (avg) (percentage) -1.90% 1.580 1.550 - 4.73% -
458.sjeng (max) -1.85% 54.000 53.000 - 70.97% -
473.astar (max) -1.83% 164.000 161.000 - 705.00% -
416.gamess -1.79% 112.000 110.000 - -77.64% -
416.gamess (avg) -1.79% 112.000 110.000 - -77.64% -
445.gobmk (avg) (percentage) -1.72% 1.160 1.140 - 48.05% -
481.wrf (max) -1.72% 116.000 114.000 - -78.49% -
444.namd (avg) (percentage) -1.67% 1.800 1.770 - -54.96% -
445.gobmk -1.67% 60.000 59.000 - -35.87% -
445.gobmk (avg) -1.67% 60.000 59.000 - -35.87% -
410.bwaves (avg) (percentage) -1.65% 1.210 1.190 - 9.17% -
470.lbm (avg) (percentage) -1.61% 1.240 1.220 - -48.52% -
458.sjeng -1.59% 63.000 62.000 - 31.91% -
458.sjeng (avg) -1.59% 63.000 62.000 - 31.91% -
435.gromacs (avg) (percentage) -1.52% 1.970 1.940 - 12.14% -
401.bzip2 (avg) (percentage) -1.38% 2.170 2.140 - 386.36% -
462.libquantum (unexplained diff vs previous) (percentage) -1.37% 0.730 0.720 - 7100.00% -
410.bwaves -1.37% 73.000 72.000 - -55.00% -
410.bwaves (avg) -1.37% 73.000 72.000 - -55.00% -
459.GemsFDTD (avg) (percentage) -1.35% 1.480 1.460 - 62.22% -
433.milc (avg) (percentage) -1.23% 1.630 1.610 - -5.29% -
473.astar (avg) (percentage) -1.17% 1.710 1.690 - 789.47% -
456.hmmer (avg) (percentage) -1.12% 0.890 0.880 - 29.41% -
459.GemsFDTD -1.11% 90.000 89.000 - -57.62% -
459.GemsFDTD (avg) -1.11% 90.000 89.000 - -57.62% -
462.libquantum (avg) (percentage) -1.10% 1.820 1.800 - 176.92% -
429.mcf (avg) (percentage) -1.04% 5.750 5.690 - 973.58% -
482.sphinx3 (avg) (percentage) -1.03% 1.950 1.930 - 32.19% -
473.astar -1.00% 100.000 99.000 - 253.57% -
473.astar (avg) -1.00% 100.000 99.000 - 253.57% -

Added Tests - sample variability
454.calculix (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
433.milc 4.00% 95873.000 99705.000 - 4.28% -
456.hmmer 1.42% 231209.000 234488.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 39.02% 8205.000 11407.000 - -0.51% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
453.povray (unexplained diff vs previous) (percentage) 13250.00% 0.020 2.670 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) 7100.00% 0.010 0.720 - -1.37% -
473.astar (unexplained diff vs previous) (percentage) 2250.00% 0.020 0.470 - 80.77% -
473.astar (max) (percentage) 1850.00% 0.140 2.730 - -2.15% -
429.mcf (max) (percentage) 1729.63% 0.540 9.880 - 0.00% -
403.gcc (max) (percentage) 1395.83% 0.720 10.770 - -3.32% -
483.xalancbmk (max) (percentage) 1302.44% 0.410 5.750 - -0.17% -
453.povray (unexplained diff vs previous) 1200.00% 1.000 13.000 - 0.00% -
429.mcf (avg) (percentage) 973.58% 0.530 5.690 - -1.04% -
403.gcc (avg) (percentage) 913.11% 0.610 6.180 - 0.00% -
473.astar (unexplained diff vs previous) 833.33% 3.000 28.000 - 86.67% -
473.astar (avg) (percentage) 789.47% 0.190 1.690 - -1.17% -
473.astar (max) 705.00% 20.000 161.000 - -1.83% -
471.omnetpp (unexplained diff vs previous) (percentage) 433.33% 0.240 1.280 - 13.27% -
401.bzip2 (avg) (percentage) 386.36% 0.440 2.140 - -1.38% -
458.sjeng (unexplained diff vs previous) (percentage) 266.67% 0.060 0.220 - -55.10% -
473.astar 253.57% 28.000 99.000 - -1.00% -
473.astar (avg) 253.57% 28.000 99.000 - -1.00% -
458.sjeng (max) (percentage) 218.18% 0.220 0.700 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) 214.10% 0.780 2.450 - 82.84% -
410.bwaves (unexplained diff vs previous) (percentage) 200.00% 0.100 0.300 - -43.40% -
462.libquantum (avg) (percentage) 176.92% 0.650 1.800 - -1.10% -
429.mcf (unexplained diff vs previous) (percentage) 155.56% 0.540 1.380 - -6.12% -
470.lbm (unexplained diff vs previous) (percentage) 145.45% 0.110 0.270 - -27.03% -
458.sjeng (avg) (percentage) 145.45% 0.330 0.810 - -1.22% -
462.libquantum (max) (percentage) 142.19% 0.640 1.550 - -7.19% -
450.soplex (avg) (percentage) 141.13% 1.240 2.990 - -0.99% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 133.33% 0.210 0.490 - 28.95% -
447.dealII (max) (percentage) 117.27% 1.100 2.390 - 75.74% -
450.soplex (max) (percentage) 109.64% 1.970 4.130 - -12.50% -
410.bwaves (max) (percentage) 109.52% 0.210 0.440 - -16.98% -
458.sjeng (unexplained diff vs previous) 88.89% 9.000 17.000 - -55.26% -
437.leslie3d (avg) (percentage) 81.09% 2.380 4.310 - -0.46% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
464.h264ref (avg) (percentage) 75.93% 0.540 0.950 - 0.00% -
458.sjeng (max) 70.97% 31.000 53.000 - -1.85% -
429.mcf (max) 63.04% 92.000 150.000 - 1.35% -
459.GemsFDTD (avg) (percentage) 62.22% 0.900 1.460 - -1.35% -
437.leslie3d (unexplained diff vs previous) (percentage) 61.57% 2.290 3.700 - 28.03% -
454.calculix (avg) (percentage) 58.21% 2.010 3.180 - 0.32% -
453.povray (avg) (percentage) 52.80% 1.250 1.910 - 0.53% -
471.omnetpp (avg) (percentage) 52.20% 1.590 2.420 - -0.82% -
437.leslie3d (max) (percentage) 52.14% 2.340 3.560 - 1.42% -
445.gobmk (avg) (percentage) 48.05% 0.770 1.140 - -1.72% -
471.omnetpp (unexplained diff vs previous) 46.15% 26.000 38.000 - 15.15% -
401.bzip2 (max) (percentage) 43.18% 0.880 1.260 - -6.67% -
400.perlbench (avg) (percentage) 40.14% 1.470 2.060 - -1.90% -
447.dealII (avg) (percentage) 39.25% 1.070 1.490 - 1.36% -
437.leslie3d 32.77% 357.000 474.000 - -4.05% -
437.leslie3d (avg) 32.77% 357.000 474.000 - -4.05% -
482.sphinx3 (avg) (percentage) 32.19% 1.460 1.930 - -1.03% -
458.sjeng 31.91% 47.000 62.000 - -1.59% -
458.sjeng (avg) 31.91% 47.000 62.000 - -1.59% -
456.hmmer (avg) (percentage) 29.41% 0.680 0.880 - -1.12% -
482.sphinx3 (unexplained diff vs previous) (percentage) 20.97% 0.620 0.750 - 0.00% -
410.bwaves (unexplained diff vs previous) 20.00% 15.000 18.000 - -43.75% -
437.leslie3d (unexplained diff vs previous) 18.66% 343.000 407.000 - 23.33% -
435.gromacs (avg) (percentage) 12.14% 1.730 1.940 - -1.52% -
437.leslie3d (max) 11.68% 351.000 392.000 - -2.00% -
410.bwaves (avg) (percentage) 9.17% 1.090 1.190 - -1.65% -
465.tonto 5.11% 137.000 144.000 - -0.69% -
465.tonto (avg) 5.11% 137.000 144.000 - -0.69% -
434.zeusmp (avg) (percentage) 4.73% 1.480 1.550 - -1.90% -
453.povray (max) (percentage) 3.16% 2.530 2.610 - 289.55% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
464.h264ref -2.43% 428364.000 417968.000 - 0.01% -
434.zeusmp -1.73% 165699.000 162829.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.73% 22860.000 61.000 - 0.00% -
403.gcc -99.32% 9010.000 61.000 - 1.67% -
462.libquantum -98.81% 11614.000 138.000 - 0.73% -
482.sphinx3 -96.88% 29928.000 933.000 - -0.74% -
450.soplex -96.30% 15466.000 573.000 - 1.42% -
453.povray -92.57% 6544.000 486.000 - -2.61% -
444.namd -91.86% 9050.000 737.000 - 0.41% -
429.mcf -91.14% 17158.000 1521.000 - 1.40% -
433.milc -90.98% 12976.000 1171.000 - -0.17% -
436.cactusADM -88.03% 17544.000 2100.000 - 0.38% -
400.perlbench -85.38% 10650.000 1557.000 - -0.64% -
434.zeusmp -80.51% 13211.000 2575.000 - 0.12% -
401.bzip2 -80.31% 15185.000 2990.000 - -0.50% -
447.dealII -79.89% 9751.000 1961.000 - -2.39% -
459.GemsFDTD -73.67% 23356.000 6149.000 - 0.49% -
456.hmmer -73.49% 11046.000 2928.000 - 0.03% -
471.omnetpp -72.39% 10739.000 2965.000 - 1.30% -
470.lbm -70.79% 12819.000 3744.000 - -0.27% -
481.wrf -66.82% 16834.000 5586.000 - 0.78% -
473.astar -60.42% 14932.000 5910.000 - 0.48% -
464.h264ref -59.41% 14556.000 5908.000 - -0.37% -
410.bwaves -58.96% 14755.000 6056.000 - 0.30% -
445.gobmk -56.24% 11995.000 5249.000 - 0.08% -
435.gromacs -53.34% 15419.000 7194.000 - 0.71% -
458.sjeng -45.99% 14275.000 7710.000 - -0.22% -
416.gamess -44.67% 22704.000 12561.000 - 0.22% -
437.leslie3d -26.61% 15005.000 11012.000 - -3.56% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.78% 459.000 1.000 - 0.00% -
454.calculix (avg) -99.78% 459.000 1.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.57% 700.000 3.000 - -50.00% -
433.milc (unexplained diff vs previous) -99.53% 425.000 2.000 - -66.67% -
444.namd (max) -99.38% 649.000 4.000 - -50.00% -
434.zeusmp (unexplained diff vs previous) -99.03% 308.000 3.000 - 200.00% -
456.hmmer (unexplained diff vs previous) -98.80% 83.000 1.000 - -50.00% -
436.cactusADM (unexplained diff vs previous) -98.60% 570.000 8.000 - -57.89% -
470.lbm (max) -98.32% 653.000 11.000 - -15.38% -
482.sphinx3 (max) -98.18% 439.000 8.000 - -11.11% -
433.milc (max) -98.05% 411.000 8.000 - -11.11% -
445.gobmk (unexplained diff vs previous) -97.85% 186.000 4.000 - -73.33% -
450.soplex (unexplained diff vs previous) -97.43% 311.000 8.000 - -71.43% -
400.perlbench (max) -97.39% 345.000 9.000 - 80.00% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
462.libquantum (max) -97.30% 74.000 2.000 - 0.00% -
400.perlbench (unexplained diff vs previous) -97.20% 357.000 10.000 - 150.00% -
436.cactusADM (max) -97.10% 552.000 16.000 - -11.11% -
444.namd -96.34% 355.000 13.000 - 0.00% -
444.namd (avg) -96.34% 355.000 13.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -96.24% 186.000 7.000 - 0.00% -
416.gamess (unexplained diff vs previous) -96.01% 676.000 27.000 - -3.57% -
456.hmmer (unexplained diff vs previous) (percentage) -96.00% 0.750 0.030 - -57.14% -
416.gamess (max) -95.88% 656.000 27.000 - 0.00% -
482.sphinx3 -95.87% 436.000 18.000 - 0.00% -
482.sphinx3 (avg) -95.87% 436.000 18.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -94.85% 2.330 0.120 - 200.00% -
445.gobmk (unexplained diff vs previous) (percentage) -94.84% 1.550 0.080 - -72.41% -
433.milc (unexplained diff vs previous) (percentage) -94.82% 3.280 0.170 - -66.67% -
444.namd (unexplained diff vs previous) (percentage) -94.70% 7.730 0.410 - -50.00% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
434.zeusmp (max) -94.02% 301.000 18.000 - -14.29% -
470.lbm (max) (percentage) -93.92% 5.100 0.310 - -16.22% -
416.gamess (unexplained diff vs previous) (percentage) -92.95% 2.980 0.210 - -4.55% -
453.povray (max) -92.73% 165.000 12.000 - 300.00% -
450.soplex (max) -92.43% 304.000 23.000 - -11.54% -
416.gamess (max) (percentage) -92.39% 2.890 0.220 - 0.00% -
433.milc -91.82% 220.000 18.000 - -5.26% -
433.milc (avg) -91.82% 220.000 18.000 - -5.26% -
444.namd (max) (percentage) -91.50% 7.180 0.610 - -44.04% -
450.soplex -91.10% 191.000 17.000 - 0.00% -
450.soplex (avg) -91.10% 191.000 17.000 - 0.00% -
456.hmmer (max) -90.71% 140.000 13.000 - -7.14% -
403.gcc (max) -90.62% 64.000 6.000 - 0.00% -
436.cactusADM -89.69% 291.000 30.000 - 0.00% -
436.cactusADM (avg) -89.69% 291.000 30.000 - 0.00% -
453.povray -88.89% 81.000 9.000 - 0.00% -
453.povray (avg) -88.89% 81.000 9.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) -88.64% 132.000 15.000 - -28.57% -
436.cactusADM (unexplained diff vs previous) (percentage) -88.31% 3.250 0.380 - -58.24% -
470.lbm -85.15% 303.000 45.000 - -2.17% -
470.lbm (avg) -85.15% 303.000 45.000 - -2.17% -
471.omnetpp (max) -83.99% 662.000 106.000 - -9.40% -
435.gromacs (unexplained diff vs previous) -83.96% 318.000 51.000 - -65.07% -
465.tonto (max) (percentage) -81.56% 3.580 0.660 - -10.81% -
445.gobmk (max) -81.42% 183.000 34.000 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) -80.90% 3.350 0.640 - 146.15% -
400.perlbench (max) (percentage) -80.25% 3.240 0.640 - 68.42% -
434.zeusmp -80.00% 195.000 39.000 - -2.50% -
434.zeusmp (avg) -80.00% 195.000 39.000 - -2.50% -
400.perlbench -79.49% 156.000 32.000 - 0.00% -
400.perlbench (avg) -79.49% 156.000 32.000 - 0.00% -
459.GemsFDTD (max) -78.57% 140.000 30.000 - 36.36% -
481.wrf (max) -78.49% 530.000 114.000 - -1.72% -
433.milc (max) (percentage) -78.23% 3.170 0.690 - -10.39% -
416.gamess -77.64% 492.000 110.000 - -1.79% -
416.gamess (avg) -77.64% 492.000 110.000 - -1.79% -
429.mcf (unexplained diff vs previous) -77.42% 93.000 21.000 - -4.55% -
436.cactusADM (max) (percentage) -74.92% 3.150 0.790 - -12.22% -
465.tonto (max) -74.40% 293.000 75.000 - -10.71% -
464.h264ref (max) -74.19% 62.000 16.000 - -11.11% -
401.bzip2 (max) -72.18% 133.000 37.000 - -7.50% -
447.dealII -72.12% 104.000 29.000 - 0.00% -
447.dealII (avg) -72.12% 104.000 29.000 - 0.00% -
481.wrf -70.44% 318.000 94.000 - 0.00% -
481.wrf (avg) -70.44% 318.000 94.000 - 0.00% -
434.zeusmp (max) (percentage) -67.98% 2.280 0.730 - -10.98% -
454.calculix (max) (percentage) -67.45% 4.700 1.530 - -4.97% -
456.hmmer -66.67% 75.000 25.000 - -3.85% -
456.hmmer (avg) -66.67% 75.000 25.000 - -3.85% -
435.gromacs (unexplained diff vs previous) (percentage) -65.53% 2.060 0.710 - -65.20% -
456.hmmer (max) (percentage) -63.78% 1.270 0.460 - -4.17% -
435.gromacs (max) -59.81% 311.000 125.000 - -11.97% -
416.gamess (avg) (percentage) -59.45% 2.170 0.880 - -2.22% -
471.omnetpp -58.24% 170.000 71.000 - 0.00% -
471.omnetpp (avg) -58.24% 170.000 71.000 - 0.00% -
459.GemsFDTD -57.62% 210.000 89.000 - -1.11% -
459.GemsFDTD (avg) -57.62% 210.000 89.000 - -1.11% -
447.dealII (max) -57.01% 107.000 46.000 - 70.37% -
445.gobmk (max) (percentage) -56.86% 1.530 0.660 - 0.00% -
410.bwaves -55.00% 160.000 72.000 - -1.37% -
410.bwaves (avg) -55.00% 160.000 72.000 - -1.37% -
444.namd (avg) (percentage) -54.96% 3.930 1.770 - -1.67% -
470.lbm (avg) (percentage) -48.52% 2.370 1.220 - -1.61% -
435.gromacs -47.74% 266.000 139.000 - -0.71% -
435.gromacs (avg) -47.74% 266.000 139.000 - -0.71% -
401.bzip2 (unexplained diff vs previous) (percentage) -42.53% 0.870 0.500 - -28.57% -
471.omnetpp (max) (percentage) -41.65% 6.170 3.600 - -10.00% -
459.GemsFDTD (unexplained diff vs previous) -37.50% 48.000 30.000 - 30.43% -
447.dealII (unexplained diff vs previous) -36.84% 76.000 48.000 - 77.78% -
445.gobmk -35.87% 92.000 59.000 - -1.67% -
445.gobmk (avg) -35.87% 92.000 59.000 - -1.67% -
481.wrf (max) (percentage) -34.92% 3.150 2.050 - -2.84% -
464.h264ref (max) (percentage) -34.88% 0.430 0.280 - -9.68% -
482.sphinx3 (max) (percentage) -34.69% 1.470 0.960 - -2.04% -
450.soplex (unexplained diff vs previous) (percentage) -30.35% 2.010 1.400 - -71.77% -
470.lbm (unexplained diff vs previous) -28.57% 14.000 10.000 - -28.57% -
464.h264ref -28.21% 78.000 56.000 - 0.00% -
464.h264ref (avg) -28.21% 78.000 56.000 - 0.00% -
465.tonto (avg) (percentage) -23.95% 1.670 1.270 - 0.00% -
459.GemsFDTD (max) (percentage) -18.33% 0.600 0.490 - 32.43% -
436.cactusADM (avg) (percentage) -13.86% 1.660 1.430 - -0.69% -
435.gromacs (max) (percentage) -13.37% 2.020 1.750 - -12.50% -
410.bwaves (max) -13.33% 30.000 26.000 - -18.75% -
481.wrf (avg) (percentage) -10.05% 1.890 1.700 - 0.00% -
433.milc (avg) (percentage) -5.29% 1.700 1.610 - -1.23% -
401.bzip2 -4.55% 66.000 63.000 - -3.08% -
401.bzip2 (avg) -4.55% 66.000 63.000 - -3.08% -
429.mcf -4.44% 90.000 86.000 - 0.00% -
429.mcf (avg) -4.44% 90.000 86.000 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 3.45% 29.000 30.000 - 0.00% -

Added Tests - sample variability
454.calculix (max)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1557.000 -
401.bzip2 2990.000 -
403.gcc 61.000 1.67%
410.bwaves 6056.000 -
416.gamess 12561.000 -
429.mcf 1521.000 1.40%
433.milc 1171.000 -
434.zeusmp 2575.000 -
435.gromacs 7194.000 -
436.cactusADM 2100.000 -
437.leslie3d 11012.000 -3.56%
444.namd 737.000 -
445.gobmk 5249.000 -
447.dealII 1961.000 -2.39%
450.soplex 573.000 1.42%
453.povray 486.000 -2.61%
454.calculix 61.000 -
456.hmmer 2928.000 -
458.sjeng 7710.000 -
459.GemsFDTD 6149.000 -
462.libquantum 138.000 -
464.h264ref 5908.000 -
465.tonto 11407.000 -
470.lbm 3744.000 -
471.omnetpp 2965.000 1.30%
473.astar 5910.000 -
481.wrf 5586.000 -
482.sphinx3 933.000 -
Geometric Mean 2091.361 -
Code Size (bytes)
Name Current %
400.perlbench 929250.000 -
401.bzip2 49690.000 -
403.gcc 2683613.000 -
410.bwaves 17640.000 -
416.gamess 6251075.000 -
429.mcf 10979.000 -
433.milc 99705.000 4.28%
434.zeusmp 162829.000 -
435.gromacs 677869.000 -
436.cactusADM 551516.000 -
437.leslie3d 64776.000 -
444.namd 190766.000 -
445.gobmk 1399885.000 -
447.dealII 2118491.000 -
450.soplex 259941.000 -
453.povray 706478.000 -
454.calculix 1270135.000 -
456.hmmer 234488.000 -
458.sjeng 120025.000 -
459.GemsFDTD 254373.000 -
462.libquantum 31353.000 -
464.h264ref 417968.000 -
465.tonto 2873397.000 -
470.lbm 10916.000 -
471.omnetpp 566460.000 -
473.astar 32228.000 -
481.wrf 2623276.000 -
482.sphinx3 152612.000 -
Geometric Mean 273250.959 -
sample variability
Name Current %
400.perlbench 32.000 -
400.perlbench (avg) 32.000 -
400.perlbench (avg) (percentage) 2.060 -1.90%
400.perlbench (max) 9.000 80.00%
400.perlbench (max) (percentage) 0.640 68.42%
400.perlbench (unexplained diff vs previous) 10.000 150.00%
400.perlbench (unexplained diff vs previous) (percentage) 0.640 146.15%
401.bzip2 63.000 -3.08%
401.bzip2 (avg) 63.000 -3.08%
401.bzip2 (avg) (percentage) 2.140 -1.38%
401.bzip2 (max) 37.000 -7.50%
401.bzip2 (max) (percentage) 1.260 -6.67%
401.bzip2 (unexplained diff vs previous) 15.000 -28.57%
401.bzip2 (unexplained diff vs previous) (percentage) 0.500 -28.57%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 6.180 -
403.gcc (max) 6.000 -
403.gcc (max) (percentage) 10.770 -3.32%
410.bwaves 72.000 -1.37%
410.bwaves (avg) 72.000 -1.37%
410.bwaves (avg) (percentage) 1.190 -1.65%
410.bwaves (max) 26.000 -18.75%
410.bwaves (max) (percentage) 0.440 -16.98%
410.bwaves (unexplained diff vs previous) 18.000 -43.75%
410.bwaves (unexplained diff vs previous) (percentage) 0.300 -43.40%
416.gamess 110.000 -1.79%
416.gamess (avg) 110.000 -1.79%
416.gamess (avg) (percentage) 0.880 -2.22%
416.gamess (max) 27.000 -
416.gamess (max) (percentage) 0.220 -
416.gamess (unexplained diff vs previous) 27.000 -3.57%
416.gamess (unexplained diff vs previous) (percentage) 0.210 -4.55%
429.mcf 86.000 -
429.mcf (avg) 86.000 -
429.mcf (avg) (percentage) 5.690 -1.04%
429.mcf (max) 150.000 1.35%
429.mcf (max) (percentage) 9.880 -
429.mcf (unexplained diff vs previous) 21.000 -4.55%
429.mcf (unexplained diff vs previous) (percentage) 1.380 -6.12%
433.milc 18.000 -5.26%
433.milc (avg) 18.000 -5.26%
433.milc (avg) (percentage) 1.610 -1.23%
433.milc (max) 8.000 -11.11%
433.milc (max) (percentage) 0.690 -10.39%
433.milc (unexplained diff vs previous) 2.000 -66.67%
433.milc (unexplained diff vs previous) (percentage) 0.170 -66.67%
434.zeusmp 39.000 -2.50%
434.zeusmp (avg) 39.000 -2.50%
434.zeusmp (avg) (percentage) 1.550 -1.90%
434.zeusmp (max) 18.000 -14.29%
434.zeusmp (max) (percentage) 0.730 -10.98%
434.zeusmp (unexplained diff vs previous) 3.000 200.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.120 200.00%
435.gromacs 139.000 -
435.gromacs (avg) 139.000 -
435.gromacs (avg) (percentage) 1.940 -1.52%
435.gromacs (max) 125.000 -11.97%
435.gromacs (max) (percentage) 1.750 -12.50%
435.gromacs (unexplained diff vs previous) 51.000 -65.07%
435.gromacs (unexplained diff vs previous) (percentage) 0.710 -65.20%
436.cactusADM 30.000 -
436.cactusADM (avg) 30.000 -
436.cactusADM (avg) (percentage) 1.430 -
436.cactusADM (max) 16.000 -11.11%
436.cactusADM (max) (percentage) 0.790 -12.22%
436.cactusADM (unexplained diff vs previous) 8.000 -57.89%
436.cactusADM (unexplained diff vs previous) (percentage) 0.380 -58.24%
437.leslie3d 474.000 -4.05%
437.leslie3d (avg) 474.000 -4.05%
437.leslie3d (avg) (percentage) 4.310 -
437.leslie3d (max) 392.000 -2.00%
437.leslie3d (max) (percentage) 3.560 1.42%
437.leslie3d (unexplained diff vs previous) 407.000 23.33%
437.leslie3d (unexplained diff vs previous) (percentage) 3.700 28.03%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.770 -1.67%
444.namd (max) 4.000 -50.00%
444.namd (max) (percentage) 0.610 -44.04%
444.namd (unexplained diff vs previous) 3.000 -50.00%
444.namd (unexplained diff vs previous) (percentage) 0.410 -50.00%
445.gobmk 59.000 -1.67%
445.gobmk (avg) 59.000 -1.67%
445.gobmk (avg) (percentage) 1.140 -1.72%
445.gobmk (max) 34.000 -
445.gobmk (max) (percentage) 0.660 -
445.gobmk (unexplained diff vs previous) 4.000 -73.33%
445.gobmk (unexplained diff vs previous) (percentage) 0.080 -72.41%
447.dealII 29.000 -
447.dealII (avg) 29.000 -
447.dealII (avg) (percentage) 1.490 1.36%
447.dealII (max) 46.000 70.37%
447.dealII (max) (percentage) 2.390 75.74%
447.dealII (unexplained diff vs previous) 48.000 77.78%
447.dealII (unexplained diff vs previous) (percentage) 2.450 82.84%
450.soplex 17.000 -
450.soplex (avg) 17.000 -
450.soplex (avg) (percentage) 2.990 -
450.soplex (max) 23.000 -11.54%
450.soplex (max) (percentage) 4.130 -12.50%
450.soplex (unexplained diff vs previous) 8.000 -71.43%
450.soplex (unexplained diff vs previous) (percentage) 1.400 -71.77%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.910 -
453.povray (max) 12.000 300.00%
453.povray (max) (percentage) 2.610 289.55%
453.povray (unexplained diff vs previous) 13.000 -
453.povray (unexplained diff vs previous) (percentage) 2.670 -
454.calculix 1.000 -
454.calculix (avg) 1.000 -
454.calculix (avg) (percentage) 3.180 -
454.calculix (max) 0.000 -
454.calculix (max) (percentage) 1.530 -4.97%
456.hmmer 25.000 -3.85%
456.hmmer (avg) 25.000 -3.85%
456.hmmer (avg) (percentage) 0.880 -1.12%
456.hmmer (max) 13.000 -7.14%
456.hmmer (max) (percentage) 0.460 -4.17%
456.hmmer (unexplained diff vs previous) 1.000 -50.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.030 -57.14%
458.sjeng 62.000 -1.59%
458.sjeng (avg) 62.000 -1.59%
458.sjeng (avg) (percentage) 0.810 -
458.sjeng (max) 53.000 -1.85%
458.sjeng (max) (percentage) 0.700 -
458.sjeng (unexplained diff vs previous) 17.000 -55.26%
458.sjeng (unexplained diff vs previous) (percentage) 0.220 -55.10%
459.GemsFDTD 89.000 -1.11%
459.GemsFDTD (avg) 89.000 -1.11%
459.GemsFDTD (avg) (percentage) 1.460 -1.35%
459.GemsFDTD (max) 30.000 36.36%
459.GemsFDTD (max) (percentage) 0.490 32.43%
459.GemsFDTD (unexplained diff vs previous) 30.000 30.43%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.490 28.95%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.800 -1.10%
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.550 -7.19%
462.libquantum (unexplained diff vs previous) 1.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.720 -1.37%
464.h264ref 56.000 -
464.h264ref (avg) 56.000 -
464.h264ref (avg) (percentage) 0.950 -
464.h264ref (max) 16.000 -11.11%
464.h264ref (max) (percentage) 0.280 -9.68%
465.tonto 144.000 -
465.tonto (avg) 144.000 -
465.tonto (avg) (percentage) 1.270 -
465.tonto (max) 75.000 -10.71%
465.tonto (max) (percentage) 0.660 -10.81%
470.lbm 45.000 -2.17%
470.lbm (avg) 45.000 -2.17%
470.lbm (avg) (percentage) 1.220 -1.61%
470.lbm (max) 11.000 -15.38%
470.lbm (max) (percentage) 0.310 -16.22%
470.lbm (unexplained diff vs previous) 10.000 -28.57%
470.lbm (unexplained diff vs previous) (percentage) 0.270 -27.03%
471.omnetpp 71.000 -
471.omnetpp (avg) 71.000 -
471.omnetpp (avg) (percentage) 2.420 -
471.omnetpp (max) 106.000 -9.40%
471.omnetpp (max) (percentage) 3.600 -10.00%
471.omnetpp (unexplained diff vs previous) 38.000 15.15%
471.omnetpp (unexplained diff vs previous) (percentage) 1.280 13.27%
473.astar 99.000 -1.00%
473.astar (avg) 99.000 -1.00%
473.astar (avg) (percentage) 1.690 -1.17%
473.astar (max) 161.000 -1.83%
473.astar (max) (percentage) 2.730 -2.15%
473.astar (unexplained diff vs previous) 28.000 86.67%
473.astar (unexplained diff vs previous) (percentage) 0.470 80.77%
481.wrf 94.000 -
481.wrf (avg) 94.000 -
481.wrf (avg) (percentage) 1.700 -
481.wrf (max) 114.000 -1.72%
481.wrf (max) (percentage) 2.050 -2.84%
482.sphinx3 18.000 -
482.sphinx3 (avg) 18.000 -
482.sphinx3 (avg) (percentage) 1.930 -1.03%
482.sphinx3 (max) 8.000 -11.11%
482.sphinx3 (max) (percentage) 0.960 -2.04%
482.sphinx3 (unexplained diff vs previous) 7.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.750 -
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.750 -
Geometric Mean 6.147 11.51%