Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=12d5d5bfd0201711ac3b14d8cd92589919a82b7a (+ 33 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/07e03761a7fc1626a6a74ed957e117f56981558c (+ 48 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=fd7ee2e6c5eb49e4a630a9978b4d668bff6354ee (+ 3 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=dd22391461d1c15b8296f13695365005f8be2e90 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/209/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0209 (3-changed-components) 2024-03-15T18:09:04 0:00:00
Previous 0208 (3-changed-components) 2024-03-11T20:10:10 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 38 14
Performance Improvements 82 187
Added Tests 2 2
Unchanged Tests 1114 1033
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 2.58% 2713.000 2783.000 - 2.17% -
401.bzip2 2.51% 2984.000 3059.000 - 4.01% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (unexplained diff vs previous) (percentage) 900.00% 0.030 0.300 - -99.89% -
456.hmmer (unexplained diff vs previous) 800.00% 1.000 9.000 - -99.89% -
458.sjeng (unexplained diff vs previous) (percentage) 333.33% 0.090 0.390 - -99.57% -
458.sjeng (unexplained diff vs previous) 314.29% 7.000 29.000 - -99.58% -
462.libquantum (unexplained diff vs previous) (percentage) 311.11% 0.720 2.960 - -99.97% -
462.libquantum (unexplained diff vs previous) 300.00% 1.000 4.000 - -99.97% -
401.bzip2 (unexplained diff vs previous) 257.14% 21.000 75.000 - -99.31% -
401.bzip2 (unexplained diff vs previous) (percentage) 250.00% 0.700 2.450 - -99.33% -
410.bwaves (unexplained diff vs previous) 200.00% 4.000 12.000 - -99.85% -
410.bwaves (unexplained diff vs previous) (percentage) 185.71% 0.070 0.200 - -99.85% -
473.astar (unexplained diff vs previous) 91.67% 12.000 23.000 - -99.77% -
473.astar (unexplained diff vs previous) (percentage) 91.30% 0.230 0.440 - -99.78% -
400.perlbench (unexplained diff vs previous) (percentage) 71.73% 1.910 3.280 - -99.38% -
400.perlbench (unexplained diff vs previous) 66.67% 30.000 50.000 - -99.43% -
400.perlbench (max) (percentage) 63.92% 1.940 3.180 - -96.22% -
400.perlbench (max) 60.00% 30.000 48.000 - -96.56% -
433.milc (unexplained diff vs previous) 57.14% 7.000 11.000 - -99.90% -
433.milc (unexplained diff vs previous) (percentage) 55.74% 0.610 0.950 - -99.90% -
462.libquantum (max) (percentage) 55.68% 1.850 2.880 - -97.09% -
462.libquantum (max) 50.00% 2.000 3.000 - -97.83% -
473.astar (max) (percentage) 22.22% 0.360 0.440 - -99.34% -
473.astar (max) 22.22% 18.000 22.000 - -99.35% -
456.hmmer (max) (percentage) 20.00% 0.250 0.300 - -99.59% -
401.bzip2 (max) 15.15% 66.000 76.000 - -96.71% -
456.hmmer (max) 14.29% 7.000 8.000 - -99.62% -
471.omnetpp (unexplained diff vs previous) 12.90% 62.000 70.000 - -99.34% -
401.bzip2 (max) (percentage) 12.56% 2.230 2.510 - -96.81% -
471.omnetpp (unexplained diff vs previous) (percentage) 10.04% 2.290 2.520 - -99.36% -
471.omnetpp (max) 7.58% 66.000 71.000 - -96.73% -
437.leslie3d (unexplained diff vs previous) (percentage) 6.62% 5.890 6.280 - -68.41% -
471.omnetpp (max) (percentage) 5.31% 2.450 2.580 - -96.76% -
401.bzip2 3.51% 57.000 59.000 - -63.35% -
401.bzip2 (avg) 3.51% 57.000 59.000 - -63.35% -
471.omnetpp 1.56% 64.000 65.000 - -70.85% -
471.omnetpp (avg) 1.56% 64.000 65.000 - -70.85% -
462.libquantum (avg) (percentage) 1.10% 1.810 1.830 - -76.11% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -5.91% 12177.000 11457.000 - -1.57% -
400.perlbench -3.18% 1573.000 1523.000 - -8.42% -
462.libquantum -2.88% 139.000 135.000 - -3.57% -
445.gobmk -1.59% 4975.000 4896.000 - -2.53% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
434.zeusmp (unexplained diff vs previous) -78.57% 14.000 3.000 - -99.97% -
434.zeusmp (unexplained diff vs previous) (percentage) -78.57% 0.560 0.120 - -99.97% -
453.povray (unexplained diff vs previous) -58.33% 12.000 5.000 - -99.91% -
453.povray (unexplained diff vs previous) (percentage) -57.92% 2.210 0.930 - -99.91% -
482.sphinx3 (unexplained diff vs previous) -50.00% 12.000 6.000 - -99.98% -
482.sphinx3 (unexplained diff vs previous) (percentage) -49.65% 1.430 0.720 - -99.97% -
434.zeusmp (max) -44.44% 18.000 10.000 - -99.51% -
434.zeusmp (max) (percentage) -43.06% 0.720 0.410 - -99.50% -
447.dealII (unexplained diff vs previous) -40.00% 15.000 9.000 - -99.87% -
450.soplex (max) -40.00% 5.000 3.000 - -99.45% -
450.soplex (unexplained diff vs previous) -40.00% 5.000 3.000 - -99.98% -
447.dealII (unexplained diff vs previous) (percentage) -40.00% 0.900 0.540 - -99.87% -
450.soplex (unexplained diff vs previous) (percentage) -40.00% 0.850 0.510 - -99.98% -
436.cactusADM (max) -37.50% 8.000 5.000 - -99.73% -
447.dealII (max) -31.58% 19.000 13.000 - -99.06% -
447.dealII (max) (percentage) -30.51% 1.180 0.820 - -98.98% -
436.cactusADM (max) (percentage) -26.32% 0.380 0.280 - -99.68% -
436.cactusADM (unexplained diff vs previous) (percentage) -26.32% 0.380 0.280 - -99.96% -
450.soplex (max) (percentage) -25.58% 0.860 0.640 - -99.33% -
436.cactusADM (unexplained diff vs previous) -25.00% 8.000 6.000 - -99.96% -
482.sphinx3 (max) -16.67% 12.000 10.000 - -98.90% -
429.mcf (unexplained diff vs previous) -12.50% 8.000 7.000 - -99.96% -
482.sphinx3 (max) (percentage) -12.41% 1.450 1.270 - -98.68% -
429.mcf (unexplained diff vs previous) (percentage) -12.28% 0.570 0.500 - -99.96% -
410.bwaves (max) (percentage) -12.00% 0.250 0.220 - -99.62% -
470.lbm (max) -11.76% 17.000 15.000 - -99.45% -
437.leslie3d (max) -11.15% 762.000 677.000 - -64.90% -
429.mcf (max) -10.00% 10.000 9.000 - -99.32% -
459.GemsFDTD (max) -10.00% 30.000 27.000 - -99.39% -
429.mcf (max) (percentage) -9.09% 0.770 0.700 - -99.25% -
453.povray (max) -9.09% 11.000 10.000 - -98.00% -
458.sjeng (max) -8.82% 34.000 31.000 - -99.14% -
458.sjeng (max) (percentage) -8.70% 0.460 0.420 - -99.12% -
459.GemsFDTD (max) (percentage) -8.62% 0.580 0.530 - -99.26% -
453.povray (max) (percentage) -8.29% 2.170 1.990 - -97.82% -
435.gromacs (max) -8.22% 73.000 67.000 - -98.12% -
435.gromacs (max) (percentage) -7.84% 1.020 0.940 - -98.09% -
464.h264ref (max) (percentage) -7.27% 0.550 0.510 - -99.06% -
410.bwaves (max) -7.14% 14.000 13.000 - -99.60% -
444.namd (max) (percentage) -6.73% 1.040 0.970 - -98.94% -
470.lbm (max) (percentage) -6.67% 0.450 0.420 - -99.42% -
464.h264ref (max) -6.45% 31.000 29.000 - -99.07% -
482.sphinx3 -5.88% 17.000 16.000 - -78.38% -
482.sphinx3 (avg) -5.88% 17.000 16.000 - -78.38% -
437.leslie3d (max) (percentage) -5.59% 6.260 5.910 - -64.35% -
437.leslie3d -5.32% 583.000 552.000 - 55.49% -
437.leslie3d (avg) -5.32% 583.000 552.000 - 55.49% -
433.milc (max) -4.35% 23.000 22.000 - -97.89% -
465.tonto (max) -3.49% 258.000 249.000 - -88.21% -
434.zeusmp -2.94% 34.000 33.000 - -77.55% -
434.zeusmp (avg) -2.94% 34.000 33.000 - -77.55% -
436.cactusADM -2.94% 34.000 33.000 - -73.39% -
436.cactusADM (avg) -2.94% 34.000 33.000 - -73.39% -
400.perlbench -2.78% 36.000 35.000 - -66.67% -
400.perlbench (avg) -2.78% 36.000 35.000 - -66.67% -
465.tonto (max) (percentage) -2.71% 2.210 2.150 - -88.87% -
433.milc (max) (percentage) -2.49% 2.010 1.960 - -97.84% -
470.lbm -2.08% 48.000 47.000 - -80.33% -
470.lbm (avg) -2.08% 48.000 47.000 - -80.33% -
447.dealII (avg) (percentage) -2.03% 1.480 1.450 - -74.87% -
454.calculix (max) (percentage) -1.98% 1.010 0.990 - -99.01% -
445.gobmk -1.89% 53.000 52.000 - -73.47% -
445.gobmk (avg) -1.89% 53.000 52.000 - -73.47% -
444.namd (avg) (percentage) -1.72% 1.740 1.710 - -81.45% -
403.gcc (max) (percentage) -1.61% 4.980 4.900 - -95.07% -
429.mcf (avg) (percentage) -1.56% 6.420 6.320 - -17.60% -
434.zeusmp (avg) (percentage) -1.46% 1.370 1.350 - -76.92% -
410.bwaves -1.43% 70.000 69.000 - -72.29% -
410.bwaves (avg) -1.43% 70.000 69.000 - -72.29% -
450.soplex (avg) (percentage) -1.42% 2.820 2.780 - -56.77% -
456.hmmer (avg) (percentage) -1.25% 0.800 0.790 - -82.09% -
436.cactusADM (avg) (percentage) -1.24% 1.610 1.590 - -73.41% -
459.GemsFDTD -1.18% 85.000 84.000 - -76.67% -
459.GemsFDTD (avg) -1.18% 85.000 84.000 - -76.67% -
429.mcf -1.12% 89.000 88.000 - -19.27% -
429.mcf (avg) -1.12% 89.000 88.000 - -19.27% -
453.povray (avg) (percentage) -1.12% 1.790 1.770 - -77.34% -
473.astar (avg) (percentage) -1.07% 1.870 1.850 - -55.53% -

Added Tests - sample variability
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 6.23% 10941.000 11623.000 - -0.74% -
410.bwaves 4.90% 5652.000 5929.000 - -0.20% -
401.bzip2 4.01% 2941.000 3059.000 - 2.51% -
450.soplex 3.17% 567.000 585.000 - -0.51% -
444.namd 2.68% 747.000 767.000 - 0.00% -
456.hmmer 2.46% 2882.000 2953.000 - -0.30% -
471.omnetpp 2.17% 2724.000 2783.000 - 2.58% -
473.astar 1.31% 5128.000 5195.000 - 0.44% -
436.cactusADM 1.30% 2083.000 2110.000 - -0.28% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 58.03% 3.050 4.820 - 0.63% -
437.leslie3d 55.49% 355.000 552.000 - -5.32% -
437.leslie3d (avg) 55.49% 355.000 552.000 - -5.32% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -13.79% 29.000 25.000 - 0.00% -
nb_execution_successful -13.79% 29.000 25.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -2.65% 244728.000 238232.000 - -0.28% -
434.zeusmp -2.56% 152037.000 148143.000 - 0.00% -
410.bwaves -1.67% 16760.000 16480.000 - 0.00% -
444.namd -1.61% 188385.000 185348.000 - 0.00% -
464.h264ref -1.46% 387748.000 382096.000 - -0.00% -
470.lbm -1.44% 9691.000 9551.000 - 0.00% -
435.gromacs -1.39% 499261.000 492297.000 - 0.00% -
437.leslie3d -1.33% 59479.000 58690.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -15.10% 6173.000 5241.000 - -0.59% -
482.sphinx3 -11.75% 945.000 834.000 - -0.71% -
400.perlbench -8.42% 1663.000 1523.000 - -3.18% -
462.libquantum -3.57% 140.000 135.000 - -2.88% -
447.dealII -3.19% 1724.000 1669.000 - 0.54% -
445.gobmk -2.53% 5023.000 4896.000 - -1.59% -
429.mcf -2.18% 1424.000 1393.000 - -0.50% -
416.gamess -1.97% 12465.000 12220.000 - 0.17% -
453.povray -1.83% 547.000 537.000 - -0.92% -
458.sjeng -1.72% 7567.000 7437.000 - -0.39% -
437.leslie3d -1.57% 11640.000 11457.000 - -5.91% -
435.gromacs -1.38% 7269.000 7169.000 - -0.17% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
450.soplex (unexplained diff vs previous) (percentage) -99.98% 2557.320 0.510 - -40.00% -
450.soplex (unexplained diff vs previous) -99.98% 14500.000 3.000 - -40.00% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 6.000 - -50.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.97% 2759.050 0.720 - -49.65% -
434.zeusmp (unexplained diff vs previous) -99.97% 11280.000 3.000 - -78.57% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.97% 448.690 0.120 - -78.57% -
462.libquantum (unexplained diff vs previous) -99.97% 13518.000 4.000 - 300.00% -
462.libquantum (unexplained diff vs previous) (percentage) -99.97% 9655.710 2.960 - 311.11% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.96% 787.130 0.280 - -26.32% -
436.cactusADM (unexplained diff vs previous) -99.96% 16396.000 6.000 - -25.00% -
429.mcf (unexplained diff vs previous) -99.96% 18593.000 7.000 - -12.50% -
429.mcf (unexplained diff vs previous) (percentage) -99.96% 1305.690 0.500 - -12.28% -
470.lbm (unexplained diff vs previous) (percentage) -99.95% 263.730 0.130 - 0.00% -
470.lbm (unexplained diff vs previous) -99.95% 9990.000 5.000 - 0.00% -
453.povray (unexplained diff vs previous) -99.91% 5707.000 5.000 - -58.33% -
453.povray (unexplained diff vs previous) (percentage) -99.91% 1043.330 0.930 - -57.92% -
433.milc (unexplained diff vs previous) (percentage) -99.90% 980.800 0.950 - 55.74% -
433.milc (unexplained diff vs previous) -99.90% 11289.000 11.000 - 57.14% -
456.hmmer (unexplained diff vs previous) (percentage) -99.89% 278.630 0.300 - 900.00% -
456.hmmer (unexplained diff vs previous) -99.89% 8030.000 9.000 - 800.00% -
447.dealII (unexplained diff vs previous) -99.87% 7017.000 9.000 - -40.00% -
447.dealII (unexplained diff vs previous) (percentage) -99.87% 407.020 0.540 - -40.00% -
410.bwaves (unexplained diff vs previous) (percentage) -99.85% 137.900 0.200 - 185.71% -
410.bwaves (unexplained diff vs previous) -99.85% 7794.000 12.000 - 200.00% -
473.astar (unexplained diff vs previous) (percentage) -99.78% 197.850 0.440 - 91.30% -
473.astar (unexplained diff vs previous) -99.77% 10146.000 23.000 - 91.67% -
416.gamess (max) -99.75% 4805.000 12.000 - 0.00% -
416.gamess (max) (percentage) -99.74% 38.550 0.100 - 0.00% -
436.cactusADM (max) -99.73% 1848.000 5.000 - -37.50% -
436.cactusADM (max) (percentage) -99.68% 88.730 0.280 - -26.32% -
456.hmmer (max) -99.62% 2120.000 8.000 - 14.29% -
410.bwaves (max) (percentage) -99.62% 57.970 0.220 - -12.00% -
410.bwaves (max) -99.60% 3276.000 13.000 - -7.14% -
456.hmmer (max) (percentage) -99.59% 73.590 0.300 - 20.00% -
458.sjeng (unexplained diff vs previous) -99.58% 6876.000 29.000 - 314.29% -
458.sjeng (unexplained diff vs previous) (percentage) -99.57% 90.870 0.390 - 333.33% -
434.zeusmp (max) -99.51% 2055.000 10.000 - -44.44% -
434.zeusmp (max) (percentage) -99.50% 81.770 0.410 - -43.06% -
470.lbm (max) -99.45% 2746.000 15.000 - -11.76% -
450.soplex (max) -99.45% 545.000 3.000 - -40.00% -
400.perlbench (unexplained diff vs previous) -99.43% 8772.000 50.000 - 66.67% -
470.lbm (max) (percentage) -99.42% 72.510 0.420 - -6.67% -
459.GemsFDTD (max) -99.39% 4411.000 27.000 - -10.00% -
400.perlbench (unexplained diff vs previous) (percentage) -99.38% 527.480 3.280 - 71.73% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.36% 391.300 2.520 - 10.04% -
473.astar (max) -99.35% 3406.000 22.000 - 22.22% -
471.omnetpp (unexplained diff vs previous) -99.34% 10659.000 70.000 - 12.90% -
473.astar (max) (percentage) -99.34% 66.430 0.440 - 22.22% -
450.soplex (max) (percentage) -99.33% 96.240 0.640 - -25.58% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.33% 368.340 2.450 - 250.00% -
429.mcf (max) -99.32% 1322.000 9.000 - -10.00% -
401.bzip2 (unexplained diff vs previous) -99.31% 10833.000 75.000 - 257.14% -
459.GemsFDTD (max) (percentage) -99.26% 71.460 0.530 - -8.62% -
429.mcf (max) (percentage) -99.25% 92.890 0.700 - -9.09% -
458.sjeng (max) -99.14% 3602.000 31.000 - -8.82% -
458.sjeng (max) (percentage) -99.12% 47.610 0.420 - -8.70% -
464.h264ref (max) -99.07% 3127.000 29.000 - -6.45% -
447.dealII (max) -99.06% 1384.000 13.000 - -31.58% -
464.h264ref (max) (percentage) -99.06% 54.290 0.510 - -7.27% -
454.calculix (max) (percentage) -99.01% 99.730 0.990 - -1.98% -
447.dealII (max) (percentage) -98.98% 80.280 0.820 - -30.51% -
444.namd (max) -98.98% 685.000 7.000 - 0.00% -
444.namd (max) (percentage) -98.94% 91.790 0.970 - -6.73% -
482.sphinx3 (max) -98.90% 911.000 10.000 - -16.67% -
482.sphinx3 (max) (percentage) -98.68% 96.500 1.270 - -12.41% -
445.gobmk (max) -98.49% 2643.000 40.000 - 0.00% -
445.gobmk (max) (percentage) -98.44% 52.630 0.820 - 0.00% -
481.wrf (max) (percentage) -98.41% 68.700 1.090 - -0.91% -
435.gromacs (max) -98.12% 3573.000 67.000 - -8.22% -
435.gromacs (max) (percentage) -98.09% 49.160 0.940 - -7.84% -
453.povray (max) -98.00% 499.000 10.000 - -9.09% -
433.milc (max) -97.89% 1044.000 22.000 - -4.35% -
433.milc (max) (percentage) -97.84% 90.750 1.960 - -2.49% -
462.libquantum (max) -97.83% 138.000 3.000 - 50.00% -
453.povray (max) (percentage) -97.82% 91.250 1.990 - -8.29% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
462.libquantum (max) (percentage) -97.09% 98.970 2.880 - 55.68% -
401.bzip2 (max) (percentage) -96.81% 78.650 2.510 - 12.56% -
471.omnetpp (max) (percentage) -96.76% 79.650 2.580 - 5.31% -
471.omnetpp (max) -96.73% 2169.000 71.000 - 7.58% -
401.bzip2 (max) -96.71% 2313.000 76.000 - 15.15% -
400.perlbench (max) -96.56% 1397.000 48.000 - 60.00% -
400.perlbench (max) (percentage) -96.22% 84.060 3.180 - 63.92% -
403.gcc (max) (percentage) -95.07% 99.360 4.900 - -1.61% -
465.tonto (max) (percentage) -88.87% 19.310 2.150 - -2.71% -
465.tonto (max) -88.21% 2112.000 249.000 - -3.49% -
416.gamess -82.36% 550.000 97.000 - 0.00% -
416.gamess (avg) -82.36% 550.000 97.000 - 0.00% -
456.hmmer (avg) (percentage) -82.09% 4.410 0.790 - -1.25% -
416.gamess (avg) (percentage) -81.90% 4.420 0.800 - 0.00% -
456.hmmer -81.89% 127.000 23.000 - 0.00% -
456.hmmer (avg) -81.89% 127.000 23.000 - 0.00% -
444.namd (avg) (percentage) -81.45% 9.220 1.710 - -1.72% -
444.namd -80.88% 68.000 13.000 - 0.00% -
444.namd (avg) -80.88% 68.000 13.000 - 0.00% -
470.lbm -80.33% 239.000 47.000 - -2.08% -
470.lbm (avg) -80.33% 239.000 47.000 - -2.08% -
470.lbm (avg) (percentage) -80.09% 6.330 1.260 - -0.79% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
464.h264ref -79.18% 293.000 61.000 - 0.00% -
464.h264ref (avg) -79.18% 293.000 61.000 - 0.00% -
464.h264ref (avg) (percentage) -78.98% 5.090 1.070 - 0.00% -
453.povray -78.57% 42.000 9.000 - 0.00% -
453.povray (avg) -78.57% 42.000 9.000 - 0.00% -
482.sphinx3 -78.38% 74.000 16.000 - -5.88% -
482.sphinx3 (avg) -78.38% 74.000 16.000 - -5.88% -
434.zeusmp -77.55% 147.000 33.000 - -2.94% -
434.zeusmp (avg) -77.55% 147.000 33.000 - -2.94% -
453.povray (avg) (percentage) -77.34% 7.810 1.770 - -1.12% -
434.zeusmp (avg) (percentage) -76.92% 5.850 1.350 - -1.46% -
459.GemsFDTD -76.67% 360.000 84.000 - -1.18% -
459.GemsFDTD (avg) -76.67% 360.000 84.000 - -1.18% -
462.libquantum (avg) (percentage) -76.11% 7.660 1.830 - 1.10% -
458.sjeng -75.93% 241.000 58.000 - 0.00% -
458.sjeng (avg) -75.93% 241.000 58.000 - 0.00% -
447.dealII -75.76% 99.000 24.000 - 0.00% -
447.dealII (avg) -75.76% 99.000 24.000 - 0.00% -
458.sjeng (avg) (percentage) -75.24% 3.190 0.790 - 0.00% -
447.dealII (avg) (percentage) -74.87% 5.770 1.450 - -2.03% -
482.sphinx3 (avg) (percentage) -74.40% 7.930 2.030 - -0.98% -
445.gobmk -73.47% 196.000 52.000 - -1.89% -
445.gobmk (avg) -73.47% 196.000 52.000 - -1.89% -
436.cactusADM (avg) (percentage) -73.41% 5.980 1.590 - -1.24% -
436.cactusADM -73.39% 124.000 33.000 - -2.94% -
436.cactusADM (avg) -73.39% 124.000 33.000 - -2.94% -
410.bwaves (avg) (percentage) -73.24% 4.410 1.180 - -0.84% -
433.milc (avg) (percentage) -72.73% 6.050 1.650 - -0.60% -
433.milc -72.46% 69.000 19.000 - 0.00% -
433.milc (avg) -72.46% 69.000 19.000 - 0.00% -
445.gobmk (avg) (percentage) -72.38% 3.910 1.080 - 0.00% -
410.bwaves -72.29% 249.000 69.000 - -1.43% -
410.bwaves (avg) -72.29% 249.000 69.000 - -1.43% -
459.GemsFDTD (avg) (percentage) -72.26% 5.840 1.620 - 0.00% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
471.omnetpp (avg) (percentage) -71.10% 8.200 2.370 - 0.00% -
471.omnetpp -70.85% 223.000 65.000 - 1.56% -
471.omnetpp (avg) -70.85% 223.000 65.000 - 1.56% -
437.leslie3d (unexplained diff vs previous) -68.89% 2314.000 720.000 - 0.42% -
437.leslie3d (unexplained diff vs previous) (percentage) -68.41% 19.880 6.280 - 6.62% -
400.perlbench -66.67% 105.000 35.000 - -2.78% -
400.perlbench (avg) -66.67% 105.000 35.000 - -2.78% -
437.leslie3d (max) -64.90% 1929.000 677.000 - -11.15% -
401.bzip2 (avg) (percentage) -64.42% 5.480 1.950 - 0.52% -
437.leslie3d (max) (percentage) -64.35% 16.580 5.910 - -5.59% -
401.bzip2 -63.35% 161.000 59.000 - 3.51% -
401.bzip2 (avg) -63.35% 161.000 59.000 - 3.51% -
400.perlbench (avg) (percentage) -63.15% 6.350 2.340 - 0.86% -
435.gromacs -61.04% 367.000 143.000 - 0.00% -
435.gromacs (avg) -61.04% 367.000 143.000 - 0.00% -
435.gromacs (avg) (percentage) -60.47% 5.060 2.000 - 0.00% -
450.soplex (avg) (percentage) -56.77% 6.430 2.780 - -1.42% -
450.soplex -55.56% 36.000 16.000 - 0.00% -
450.soplex (avg) -55.56% 36.000 16.000 - 0.00% -
473.astar (avg) (percentage) -55.53% 4.160 1.850 - -1.07% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
473.astar -54.93% 213.000 96.000 - 0.00% -
473.astar (avg) -54.93% 213.000 96.000 - 0.00% -
465.tonto (avg) (percentage) -54.49% 3.340 1.520 - 0.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
465.tonto -51.78% 365.000 176.000 - -0.56% -
465.tonto (avg) -51.78% 365.000 176.000 - -0.56% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf -19.27% 109.000 88.000 - -1.12% -
429.mcf (avg) -19.27% 109.000 88.000 - -1.12% -
429.mcf (avg) (percentage) -17.60% 7.670 6.320 - -1.56% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 13.79% 29.000 33.000 - 0.00% -

Added Tests - sample variability
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1523.000 -3.18%
401.bzip2 3059.000 2.51%
410.bwaves 5929.000 -
416.gamess 12220.000 -
429.mcf 1393.000 -
433.milc 1159.000 -
434.zeusmp 2511.000 -
435.gromacs 7169.000 -
436.cactusADM 2110.000 -
437.leslie3d 11457.000 -5.91%
444.namd 767.000 -
445.gobmk 4896.000 -1.59%
447.dealII 1669.000 -
450.soplex 585.000 -
453.povray 537.000 -
456.hmmer 2953.000 -
458.sjeng 7437.000 -
459.GemsFDTD 5241.000 -
462.libquantum 135.000 -2.88%
464.h264ref 5752.000 -
465.tonto 11623.000 -
470.lbm 3791.000 -
471.omnetpp 2783.000 2.58%
473.astar 5195.000 -
482.sphinx3 834.000 -
Geometric Mean 2593.179 -
Code Size (bytes)
Name Current %
400.perlbench 890587.000 -
401.bzip2 43919.000 -
410.bwaves 16480.000 -
416.gamess 5700921.000 -
429.mcf 9055.000 -
433.milc 60805.000 -
434.zeusmp 148143.000 -
435.gromacs 492297.000 -
436.cactusADM 350637.000 -
437.leslie3d 58690.000 -
444.namd 185348.000 -
445.gobmk 1383066.000 -
447.dealII 310444.000 -
450.soplex 205303.000 -
453.povray 656269.000 -
456.hmmer 108215.000 -
458.sjeng 103292.000 -
459.GemsFDTD 238232.000 -
462.libquantum 16085.000 -
464.h264ref 382096.000 -
465.tonto 1939926.000 -
470.lbm 9551.000 -
471.omnetpp 479794.000 -
473.astar 22880.000 -
482.sphinx3 112243.000 -
Geometric Mean 159736.912 -
sample variability
Name Current %
400.perlbench 35.000 -2.78%
400.perlbench (avg) 35.000 -2.78%
400.perlbench (avg) (percentage) 2.340 -
400.perlbench (max) 48.000 60.00%
400.perlbench (max) (percentage) 3.180 63.92%
400.perlbench (unexplained diff vs previous) 50.000 66.67%
400.perlbench (unexplained diff vs previous) (percentage) 3.280 71.73%
401.bzip2 59.000 3.51%
401.bzip2 (avg) 59.000 3.51%
401.bzip2 (avg) (percentage) 1.950 -
401.bzip2 (max) 76.000 15.15%
401.bzip2 (max) (percentage) 2.510 12.56%
401.bzip2 (unexplained diff vs previous) 75.000 257.14%
401.bzip2 (unexplained diff vs previous) (percentage) 2.450 250.00%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 4.900 -1.61%
410.bwaves 69.000 -1.43%
410.bwaves (avg) 69.000 -1.43%
410.bwaves (avg) (percentage) 1.180 -
410.bwaves (max) 13.000 -7.14%
410.bwaves (max) (percentage) 0.220 -12.00%
410.bwaves (unexplained diff vs previous) 12.000 200.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.200 185.71%
416.gamess 97.000 -
416.gamess (avg) 97.000 -
416.gamess (avg) (percentage) 0.800 -
416.gamess (max) 12.000 -
416.gamess (max) (percentage) 0.100 -
429.mcf 88.000 -1.12%
429.mcf (avg) 88.000 -1.12%
429.mcf (avg) (percentage) 6.320 -1.56%
429.mcf (max) 9.000 -10.00%
429.mcf (max) (percentage) 0.700 -9.09%
429.mcf (unexplained diff vs previous) 7.000 -12.50%
429.mcf (unexplained diff vs previous) (percentage) 0.500 -12.28%
433.milc 19.000 -
433.milc (avg) 19.000 -
433.milc (avg) (percentage) 1.650 -
433.milc (max) 22.000 -4.35%
433.milc (max) (percentage) 1.960 -2.49%
433.milc (unexplained diff vs previous) 11.000 57.14%
433.milc (unexplained diff vs previous) (percentage) 0.950 55.74%
434.zeusmp 33.000 -2.94%
434.zeusmp (avg) 33.000 -2.94%
434.zeusmp (avg) (percentage) 1.350 -1.46%
434.zeusmp (max) 10.000 -44.44%
434.zeusmp (max) (percentage) 0.410 -43.06%
434.zeusmp (unexplained diff vs previous) 3.000 -78.57%
434.zeusmp (unexplained diff vs previous) (percentage) 0.120 -78.57%
435.gromacs 143.000 -
435.gromacs (avg) 143.000 -
435.gromacs (avg) (percentage) 2.000 -
435.gromacs (max) 67.000 -8.22%
435.gromacs (max) (percentage) 0.940 -7.84%
436.cactusADM 33.000 -2.94%
436.cactusADM (avg) 33.000 -2.94%
436.cactusADM (avg) (percentage) 1.590 -1.24%
436.cactusADM (max) 5.000 -37.50%
436.cactusADM (max) (percentage) 0.280 -26.32%
436.cactusADM (unexplained diff vs previous) 6.000 -25.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.280 -26.32%
437.leslie3d 552.000 -5.32%
437.leslie3d (avg) 552.000 -5.32%
437.leslie3d (avg) (percentage) 4.820 -
437.leslie3d (max) 677.000 -11.15%
437.leslie3d (max) (percentage) 5.910 -5.59%
437.leslie3d (unexplained diff vs previous) 720.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 6.280 6.62%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.710 -1.72%
444.namd (max) 7.000 -
444.namd (max) (percentage) 0.970 -6.73%
444.namd (unexplained diff vs previous) 0.000 -
444.namd (unexplained diff vs previous) (percentage) 0.000 -
445.gobmk 52.000 -1.89%
445.gobmk (avg) 52.000 -1.89%
445.gobmk (avg) (percentage) 1.080 -
445.gobmk (max) 40.000 -
445.gobmk (max) (percentage) 0.820 -
447.dealII 24.000 -
447.dealII (avg) 24.000 -
447.dealII (avg) (percentage) 1.450 -2.03%
447.dealII (max) 13.000 -31.58%
447.dealII (max) (percentage) 0.820 -30.51%
447.dealII (unexplained diff vs previous) 9.000 -40.00%
447.dealII (unexplained diff vs previous) (percentage) 0.540 -40.00%
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 2.780 -1.42%
450.soplex (max) 3.000 -40.00%
450.soplex (max) (percentage) 0.640 -25.58%
450.soplex (unexplained diff vs previous) 3.000 -40.00%
450.soplex (unexplained diff vs previous) (percentage) 0.510 -40.00%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.770 -1.12%
453.povray (max) 10.000 -9.09%
453.povray (max) (percentage) 1.990 -8.29%
453.povray (unexplained diff vs previous) 5.000 -58.33%
453.povray (unexplained diff vs previous) (percentage) 0.930 -57.92%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 0.990 -1.98%
456.hmmer 23.000 -
456.hmmer (avg) 23.000 -
456.hmmer (avg) (percentage) 0.790 -1.25%
456.hmmer (max) 8.000 14.29%
456.hmmer (max) (percentage) 0.300 20.00%
456.hmmer (unexplained diff vs previous) 9.000 800.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.300 900.00%
458.sjeng 58.000 -
458.sjeng (avg) 58.000 -
458.sjeng (avg) (percentage) 0.790 -
458.sjeng (max) 31.000 -8.82%
458.sjeng (max) (percentage) 0.420 -8.70%
458.sjeng (unexplained diff vs previous) 29.000 314.29%
458.sjeng (unexplained diff vs previous) (percentage) 0.390 333.33%
459.GemsFDTD 84.000 -1.18%
459.GemsFDTD (avg) 84.000 -1.18%
459.GemsFDTD (avg) (percentage) 1.620 -
459.GemsFDTD (max) 27.000 -10.00%
459.GemsFDTD (max) (percentage) 0.530 -8.62%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.830 1.10%
462.libquantum (max) 3.000 50.00%
462.libquantum (max) (percentage) 2.880 55.68%
462.libquantum (unexplained diff vs previous) 4.000 300.00%
462.libquantum (unexplained diff vs previous) (percentage) 2.960 311.11%
464.h264ref 61.000 -
464.h264ref (avg) 61.000 -
464.h264ref (avg) (percentage) 1.070 -
464.h264ref (max) 29.000 -6.45%
464.h264ref (max) (percentage) 0.510 -7.27%
465.tonto 176.000 -
465.tonto (avg) 176.000 -
465.tonto (avg) (percentage) 1.520 -
465.tonto (max) 249.000 -3.49%
465.tonto (max) (percentage) 2.150 -2.71%
470.lbm 47.000 -2.08%
470.lbm (avg) 47.000 -2.08%
470.lbm (avg) (percentage) 1.260 -
470.lbm (max) 15.000 -11.76%
470.lbm (max) (percentage) 0.420 -6.67%
470.lbm (unexplained diff vs previous) 5.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.130 -
471.omnetpp 65.000 1.56%
471.omnetpp (avg) 65.000 1.56%
471.omnetpp (avg) (percentage) 2.370 -
471.omnetpp (max) 71.000 7.58%
471.omnetpp (max) (percentage) 2.580 5.31%
471.omnetpp (unexplained diff vs previous) 70.000 12.90%
471.omnetpp (unexplained diff vs previous) (percentage) 2.520 10.04%
473.astar 96.000 -
473.astar (avg) 96.000 -
473.astar (avg) (percentage) 1.850 -1.07%
473.astar (max) 22.000 22.22%
473.astar (max) (percentage) 0.440 22.22%
473.astar (unexplained diff vs previous) 23.000 91.67%
473.astar (unexplained diff vs previous) (percentage) 0.440 91.30%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 1.090 -
482.sphinx3 16.000 -5.88%
482.sphinx3 (avg) 16.000 -5.88%
482.sphinx3 (avg) (percentage) 2.030 -
482.sphinx3 (max) 10.000 -16.67%
482.sphinx3 (max) (percentage) 1.270 -12.41%
482.sphinx3 (unexplained diff vs previous) 6.000 -50.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.720 -49.65%
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 5.825 -9.05%