Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c33568d2c108478c25fe9959b5ce9d25b97efa6c (unchanged)
git_gcc https://gcc.gnu.org/git/gcc.git (6aba48a8cc128e54ee243d451ac9a843ff41c4f9) (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=0852c4aab7870adbd188f7d27985f1631c8596df (unchanged)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/cc2fbfb820fc476698d05a3b39f083124e82b2a3 (+ 337 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/301/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0301 (v6.12.7-77719-gcc2fbfb820fc) 2024-12-22T19:55:22 0:00:00
Previous 0300 (glibc-2.40.9000-00636-g0852c4aab78) 2024-12-17T23:09:43 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 34 70
Performance Improvements 106 137
Added Tests 2 2
Unchanged Tests 1094 1027
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (unexplained diff vs previous) (percentage) 1276.92% 0.130 1.790 - 477.42% -
400.perlbench (unexplained diff vs previous) 1250.00% 2.000 27.000 - -22.86% -
458.sjeng (unexplained diff vs previous) 157.14% 7.000 18.000 - -62.50% -
458.sjeng (unexplained diff vs previous) (percentage) 155.56% 0.090 0.230 - -23.33% -
429.mcf (unexplained diff vs previous) (percentage) 116.86% 1.720 3.730 - 4044.44% -
429.mcf (unexplained diff vs previous) 109.09% 22.000 46.000 - 360.00% -
429.mcf (max) (percentage) 105.71% 1.750 3.600 - 1025.00% -
473.astar (unexplained diff vs previous) (percentage) 101.54% 0.650 1.310 - 178.72% -
429.mcf (max) 100.00% 22.000 44.000 - 18.92% -
473.astar (unexplained diff vs previous) 100.00% 32.000 64.000 - 4.92% -
473.astar (max) (percentage) 78.08% 0.730 1.300 - 176.60% -
473.astar (max) 75.00% 36.000 63.000 - 3.28% -
482.sphinx3 (unexplained diff vs previous) (percentage) 48.93% 3.270 4.870 - 59.15% -
482.sphinx3 (unexplained diff vs previous) 41.94% 31.000 44.000 - -96.06% -
482.sphinx3 (max) (percentage) 37.28% 3.380 4.640 - 47.30% -
471.omnetpp (unexplained diff vs previous) 28.57% 21.000 27.000 - -56.45% -
482.sphinx3 (max) 28.12% 32.000 41.000 - -96.44% -
471.omnetpp (unexplained diff vs previous) (percentage) 27.63% 0.760 0.970 - 44.78% -
456.hmmer (unexplained diff vs previous) (percentage) 17.54% 0.570 0.670 - 28.85% -
456.hmmer (unexplained diff vs previous) 16.67% 24.000 28.000 - -71.43% -
456.hmmer (max) (percentage) 15.79% 0.570 0.660 - 26.92% -
456.hmmer (max) 12.50% 24.000 27.000 - -72.45% -
401.bzip2 (unexplained diff vs previous) (percentage) 11.23% 1.870 2.080 - 407.32% -
401.bzip2 (unexplained diff vs previous) 9.09% 55.000 60.000 - -13.04% -
401.bzip2 (max) (percentage) 6.84% 1.900 2.030 - 407.50% -
445.gobmk (unexplained diff vs previous) (percentage) 4.39% 1.140 1.190 - 693.33% -
401.bzip2 (max) 3.57% 56.000 58.000 - -13.43% -
445.gobmk (unexplained diff vs previous) 3.45% 58.000 60.000 - 233.33% -
445.gobmk (max) (percentage) 2.61% 1.150 1.180 - 337.04% -
482.sphinx3 (avg) (percentage) 2.53% 1.580 1.620 - -31.06% -
470.lbm (avg) (percentage) 1.98% 1.010 1.030 - -72.39% -
433.milc (avg) (percentage) 1.82% 1.100 1.120 - 62.32% -
445.gobmk (max) 1.72% 58.000 59.000 - 78.79% -
410.bwaves (avg) (percentage) 1.12% 0.890 0.900 - -54.08% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -5.30% 12814.000 12135.000 - -32.84% -
482.sphinx3 -4.64% 948.000 904.000 - -97.53% -
429.mcf -3.60% 1278.000 1232.000 - -89.49% -
433.milc -2.50% 1122.000 1094.000 - -92.00% -
470.lbm -2.41% 3779.000 3688.000 - -77.39% -
401.bzip2 -2.03% 2949.000 2889.000 - -82.98% -
410.bwaves -1.83% 6077.000 5966.000 - -59.54% -
400.perlbench -1.76% 1534.000 1507.000 - -86.56% -
459.GemsFDTD -1.75% 7251.000 7124.000 - -71.91% -
473.astar -1.30% 4934.000 4870.000 - -62.49% -
445.gobmk -1.18% 5104.000 5044.000 - -59.31% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
436.cactusADM (unexplained diff vs previous) -96.55% 29.000 1.000 - -99.91% -
436.cactusADM (unexplained diff vs previous) (percentage) -96.43% 1.120 0.040 - -99.17% -
434.zeusmp (unexplained diff vs previous) (percentage) -93.46% 1.070 0.070 - -96.89% -
434.zeusmp (unexplained diff vs previous) -93.33% 30.000 2.000 - -99.57% -
435.gromacs (unexplained diff vs previous) -89.14% 221.000 24.000 - -91.78% -
435.gromacs (unexplained diff vs previous) (percentage) -88.93% 2.890 0.320 - -81.50% -
450.soplex (unexplained diff vs previous) -76.92% 26.000 6.000 - -91.30% -
450.soplex (unexplained diff vs previous) (percentage) -76.69% 4.290 1.000 - 127.27% -
444.namd (unexplained diff vs previous) -75.00% 4.000 1.000 - -99.88% -
444.namd (unexplained diff vs previous) (percentage) -75.00% 0.440 0.110 - -97.90% -
465.tonto (max) -63.24% 185.000 68.000 - -69.51% -
465.tonto (max) (percentage) -62.94% 1.430 0.530 - -69.01% -
453.povray (unexplained diff vs previous) (percentage) -62.67% 1.500 0.560 - -75.55% -
453.povray (unexplained diff vs previous) -62.50% 8.000 3.000 - -98.22% -
465.tonto (unexplained diff vs previous) (percentage) -61.02% 0.590 0.230 - -79.65% -
464.h264ref (max) -60.98% 41.000 16.000 - -72.88% -
465.tonto (unexplained diff vs previous) -60.53% 76.000 30.000 - -79.73% -
464.h264ref (max) (percentage) -60.29% 0.680 0.270 - -30.77% -
447.dealII (unexplained diff vs previous) -50.00% 18.000 9.000 - -86.76% -
447.dealII (unexplained diff vs previous) (percentage) -49.46% 0.930 0.470 - -37.33% -
459.GemsFDTD (unexplained diff vs previous) -42.01% 219.000 127.000 - -78.25% -
464.h264ref (unexplained diff vs previous) (percentage) -41.67% 0.360 0.210 - -44.74% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -41.06% 3.020 1.780 - -22.61% -
464.h264ref (unexplained diff vs previous) -40.91% 22.000 13.000 - -77.59% -
434.zeusmp (max) -26.67% 30.000 22.000 - -95.43% -
435.gromacs (max) -25.11% 227.000 170.000 - -42.57% -
433.milc (max) -25.00% 36.000 27.000 - -72.16% -
434.zeusmp (max) (percentage) -25.00% 1.080 0.810 - -64.78% -
435.gromacs (max) (percentage) -24.83% 2.980 2.240 - 27.27% -
433.milc (max) (percentage) -22.36% 3.220 2.500 - 252.11% -
410.bwaves (max) -21.58% 139.000 109.000 - -58.08% -
410.bwaves (max) (percentage) -20.09% 2.290 1.830 - 3.39% -
433.milc (unexplained diff vs previous) -20.00% 35.000 28.000 - -70.83% -
410.bwaves (unexplained diff vs previous) -18.38% 136.000 111.000 - -56.81% -
459.GemsFDTD (max) -18.22% 225.000 184.000 - -68.17% -
433.milc (unexplained diff vs previous) (percentage) -17.95% 3.120 2.560 - 265.71% -
453.povray (max) (percentage) -17.01% 1.470 1.220 - -52.90% -
410.bwaves (unexplained diff vs previous) (percentage) -16.96% 2.240 1.860 - 6.90% -
459.GemsFDTD (max) (percentage) -16.72% 3.110 2.590 - 13.60% -
447.dealII (max) -16.67% 18.000 15.000 - -90.00% -
437.leslie3d (max) -14.89% 779.000 663.000 - 18.39% -
450.soplex (max) -14.81% 27.000 23.000 - -66.67% -
470.lbm (max) -14.56% 103.000 88.000 - -90.15% -
453.povray (max) -14.29% 7.000 6.000 - -96.86% -
447.dealII (max) (percentage) -12.77% 0.940 0.820 - -50.30% -
436.cactusADM (max) (percentage) -12.61% 1.110 0.970 - -80.79% -
450.soplex (max) (percentage) -12.50% 4.480 3.920 - 790.91% -
470.lbm (max) (percentage) -12.36% 2.750 2.410 - -56.02% -
436.cactusADM (max) -10.71% 28.000 25.000 - -97.97% -
400.perlbench (max) -10.34% 29.000 26.000 - -70.79% -
437.leslie3d (max) (percentage) -10.03% 6.080 5.470 - 76.45% -
470.lbm (unexplained diff vs previous) -9.90% 101.000 91.000 - -88.93% -
444.namd (max) -9.09% 11.000 10.000 - -98.86% -
437.leslie3d (unexplained diff vs previous) -7.62% 735.000 679.000 - 263.10% -
470.lbm (unexplained diff vs previous) (percentage) -7.49% 2.670 2.470 - -50.99% -
450.soplex -6.67% 15.000 14.000 - -62.16% -
450.soplex (avg) -6.67% 15.000 14.000 - -62.16% -
444.namd (max) (percentage) -6.56% 1.220 1.140 - -79.31% -
400.perlbench (max) (percentage) -6.28% 1.910 1.790 - 123.75% -
462.libquantum (max) (percentage) -5.96% 1.510 1.420 - 1675.00% -
437.leslie3d -5.07% 631.000 599.000 - 99.00% -
437.leslie3d (avg) -5.07% 631.000 599.000 - 99.00% -
471.omnetpp (max) (percentage) -4.95% 1.820 1.730 - -6.99% -
400.perlbench -4.35% 23.000 22.000 - -48.84% -
400.perlbench (avg) -4.35% 23.000 22.000 - -48.84% -
447.dealII -4.00% 25.000 24.000 - -69.62% -
447.dealII (avg) -4.00% 25.000 24.000 - -69.62% -
471.omnetpp (max) -4.00% 50.000 48.000 - -72.09% -
429.mcf -3.92% 51.000 49.000 - -12.50% -
429.mcf (avg) -3.92% 51.000 49.000 - -12.50% -
470.lbm -2.63% 38.000 37.000 - -93.91% -
470.lbm (avg) -2.63% 38.000 37.000 - -93.91% -
437.leslie3d (unexplained diff vs previous) (percentage) -2.44% 5.740 5.600 - 443.69% -
464.h264ref -2.33% 43.000 42.000 - -28.81% -
464.h264ref (avg) -2.33% 43.000 42.000 - -28.81% -
445.gobmk -2.27% 44.000 43.000 - 59.26% -
445.gobmk (avg) -2.27% 44.000 43.000 - 59.26% -
401.bzip2 -2.04% 49.000 48.000 - 6.67% -
401.bzip2 (avg) -2.04% 49.000 48.000 - 6.67% -
410.bwaves -1.85% 54.000 53.000 - -81.66% -
410.bwaves (avg) -1.85% 54.000 53.000 - -81.66% -
458.sjeng (avg) (percentage) -1.64% 0.610 0.600 - 160.87% -
447.dealII (avg) (percentage) -1.52% 1.320 1.300 - 49.43% -
435.gromacs -1.52% 132.000 130.000 - -18.75% -
435.gromacs (avg) -1.52% 132.000 130.000 - -18.75% -
462.libquantum (avg) (percentage) -1.47% 1.360 1.340 - 605.26% -
473.astar -1.47% 68.000 67.000 - 31.37% -
473.astar (avg) -1.47% 68.000 67.000 - 31.37% -
464.h264ref (avg) (percentage) -1.43% 0.700 0.690 - 76.92% -
465.tonto (avg) (percentage) -1.27% 0.790 0.780 - -31.58% -
459.GemsFDTD -1.25% 80.000 79.000 - -77.56% -
459.GemsFDTD (avg) -1.25% 80.000 79.000 - -77.56% -
436.cactusADM (avg) (percentage) -1.20% 0.830 0.820 - -65.83% -
435.gromacs (avg) (percentage) -1.16% 1.730 1.710 - 80.00% -
434.zeusmp (avg) (percentage) -1.05% 0.950 0.940 - -22.95% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.86% 1437761.000 1464510.000 - 0.00% -
473.astar 1.78% 15549.000 15825.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (unexplained diff vs previous) (percentage) 4044.44% 0.090 3.730 - 116.86% -
462.libquantum (max) (percentage) 1675.00% 0.080 1.420 - -5.96% -
403.gcc (max) (percentage) 1094.92% 0.590 7.050 - -0.28% -
429.mcf (max) (percentage) 1025.00% 0.320 3.600 - 105.71% -
450.soplex (avg) (percentage) 933.33% 0.240 2.480 - -0.80% -
450.soplex (max) (percentage) 790.91% 0.440 3.920 - -12.50% -
429.mcf (avg) (percentage) 733.33% 0.480 4.000 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) 693.33% 0.150 1.190 - 4.39% -
462.libquantum (avg) (percentage) 605.26% 0.190 1.340 - -1.47% -
401.bzip2 (avg) (percentage) 522.22% 0.270 1.680 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) 477.42% 0.310 1.790 - 1276.92% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
437.leslie3d (unexplained diff vs previous) (percentage) 443.69% 1.030 5.600 - -2.44% -
401.bzip2 (max) (percentage) 407.50% 0.400 2.030 - 6.84% -
401.bzip2 (unexplained diff vs previous) (percentage) 407.32% 0.410 2.080 - 11.23% -
429.mcf (unexplained diff vs previous) 360.00% 10.000 46.000 - 109.09% -
445.gobmk (max) (percentage) 337.04% 0.270 1.180 - 2.61% -
454.calculix (max) (percentage) 310.71% 0.840 3.450 - 0.00% -
445.gobmk (avg) (percentage) 295.45% 0.220 0.870 - 0.00% -
400.perlbench (avg) (percentage) 289.74% 0.390 1.520 - 0.66% -
433.milc (unexplained diff vs previous) (percentage) 265.71% 0.700 2.560 - -17.95% -
437.leslie3d (unexplained diff vs previous) 263.10% 187.000 679.000 - -7.62% -
433.milc (max) (percentage) 252.11% 0.710 2.500 - -22.36% -
473.astar (avg) (percentage) 245.00% 0.400 1.380 - -0.72% -
445.gobmk (unexplained diff vs previous) 233.33% 18.000 60.000 - 3.45% -
437.leslie3d (avg) (percentage) 195.81% 1.670 4.940 - 0.20% -
473.astar (unexplained diff vs previous) (percentage) 178.72% 0.470 1.310 - 101.54% -
473.astar (max) (percentage) 176.60% 0.470 1.300 - 78.08% -
458.sjeng (avg) (percentage) 160.87% 0.230 0.600 - -1.64% -
450.soplex (unexplained diff vs previous) (percentage) 127.27% 0.440 1.000 - -76.69% -
400.perlbench (max) (percentage) 123.75% 0.800 1.790 - -6.28% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
437.leslie3d 99.00% 301.000 599.000 - -5.07% -
437.leslie3d (avg) 99.00% 301.000 599.000 - -5.07% -
435.gromacs (avg) (percentage) 80.00% 0.950 1.710 - -1.16% -
445.gobmk (max) 78.79% 33.000 59.000 - 1.72% -
464.h264ref (avg) (percentage) 76.92% 0.390 0.690 - -1.43% -
437.leslie3d (max) (percentage) 76.45% 3.100 5.470 - -10.03% -
456.hmmer (avg) (percentage) 73.33% 0.300 0.520 - 0.00% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
483.xalancbmk (max) (percentage) 63.04% 0.920 1.500 - 0.00% -
433.milc (avg) (percentage) 62.32% 0.690 1.120 - 1.82% -
445.gobmk 59.26% 27.000 43.000 - -2.27% -
445.gobmk (avg) 59.26% 27.000 43.000 - -2.27% -
482.sphinx3 (unexplained diff vs previous) (percentage) 59.15% 3.060 4.870 - 48.93% -
471.omnetpp (avg) (percentage) 54.24% 1.180 1.820 - -0.55% -
447.dealII (avg) (percentage) 49.43% 0.870 1.300 - -1.52% -
482.sphinx3 (max) (percentage) 47.30% 3.150 4.640 - 37.28% -
471.omnetpp (unexplained diff vs previous) (percentage) 44.78% 0.670 0.970 - 27.63% -
473.astar 31.37% 51.000 67.000 - -1.47% -
473.astar (avg) 31.37% 51.000 67.000 - -1.47% -
456.hmmer (unexplained diff vs previous) (percentage) 28.85% 0.520 0.670 - 17.54% -
458.sjeng 27.78% 36.000 46.000 - 0.00% -
458.sjeng (avg) 27.78% 36.000 46.000 - 0.00% -
435.gromacs (max) (percentage) 27.27% 1.760 2.240 - -24.83% -
456.hmmer (max) (percentage) 26.92% 0.520 0.660 - 15.79% -
429.mcf (max) 18.92% 37.000 44.000 - 100.00% -
437.leslie3d (max) 18.39% 560.000 663.000 - -14.89% -
459.GemsFDTD (max) (percentage) 13.60% 2.280 2.590 - -16.72% -
410.bwaves (unexplained diff vs previous) (percentage) 6.90% 1.740 1.860 - -16.96% -
458.sjeng (max) (percentage) 6.67% 0.300 0.320 - 0.00% -
401.bzip2 6.67% 45.000 48.000 - -2.04% -
401.bzip2 (avg) 6.67% 45.000 48.000 - -2.04% -
473.astar (unexplained diff vs previous) 4.92% 61.000 64.000 - 100.00% -
410.bwaves (max) (percentage) 3.39% 1.770 1.830 - -20.09% -
473.astar (max) 3.28% 61.000 63.000 - 75.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - 0.00% -
nb_execution_successful -17.24% 29.000 24.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.83% 10845.000 10213.000 - 0.00% -
437.leslie3d -3.79% 48001.000 46184.000 - 0.00% -
434.zeusmp -1.66% 141060.000 138714.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.72% 17005.000 218.000 - 0.00% -
482.sphinx3 -97.53% 36586.000 904.000 - -4.64% -
450.soplex -96.18% 15709.000 600.000 - -0.99% -
444.namd -94.33% 15957.000 905.000 - -0.11% -
453.povray -92.72% 7395.000 538.000 - 0.56% -
433.milc -92.00% 13668.000 1094.000 - -2.50% -
429.mcf -89.49% 11727.000 1232.000 - -3.60% -
436.cactusADM -89.42% 24396.000 2582.000 - 0.04% -
434.zeusmp -86.63% 20932.000 2799.000 - -0.07% -
400.perlbench -86.56% 11215.000 1507.000 - -1.76% -
401.bzip2 -82.98% 16971.000 2889.000 - -2.03% -
447.dealII -78.93% 9101.000 1918.000 - -0.47% -
456.hmmer -77.93% 18995.000 4192.000 - -0.66% -
470.lbm -77.39% 16312.000 3688.000 - -2.41% -
459.GemsFDTD -71.91% 25364.000 7124.000 - -1.75% -
471.omnetpp -69.98% 9265.000 2781.000 - 0.98% -
473.astar -62.49% 12983.000 4870.000 - -1.30% -
464.h264ref -59.55% 15208.000 6152.000 - -0.21% -
410.bwaves -59.54% 14745.000 5966.000 - -1.83% -
445.gobmk -59.31% 12396.000 5044.000 - -1.18% -
435.gromacs -54.83% 16853.000 7612.000 - -0.31% -
458.sjeng -52.10% 16018.000 7673.000 - 0.24% -
437.leslie3d -32.84% 18070.000 12135.000 - -5.30% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (unexplained diff vs previous) -99.91% 1173.000 1.000 - -96.55% -
444.namd (unexplained diff vs previous) -99.88% 834.000 1.000 - -75.00% -
434.zeusmp (unexplained diff vs previous) -99.57% 470.000 2.000 - -93.33% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.17% 4.810 0.040 - -96.43% -
444.namd (max) -98.86% 879.000 10.000 - -9.09% -
482.sphinx3 -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (avg) -98.37% 859.000 14.000 - 0.00% -
453.povray (unexplained diff vs previous) -98.22% 169.000 3.000 - -62.50% -
436.cactusADM (max) -97.97% 1231.000 25.000 - -10.71% -
444.namd (unexplained diff vs previous) (percentage) -97.90% 5.230 0.110 - -75.00% -
444.namd -97.83% 414.000 9.000 - 0.00% -
444.namd (avg) -97.83% 414.000 9.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -96.89% 2.250 0.070 - -93.46% -
453.povray (max) -96.86% 191.000 6.000 - -14.29% -
482.sphinx3 (max) -96.44% 1152.000 41.000 - 28.12% -
436.cactusADM -96.41% 585.000 21.000 - 0.00% -
436.cactusADM (avg) -96.41% 585.000 21.000 - 0.00% -
416.gamess (max) (percentage) -96.18% 3.140 0.120 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -96.06% 1118.000 44.000 - 41.94% -
434.zeusmp (max) -95.43% 481.000 22.000 - -26.67% -
470.lbm -93.91% 608.000 37.000 - -2.63% -
470.lbm (avg) -93.91% 608.000 37.000 - -2.63% -
453.povray -93.81% 113.000 7.000 - 0.00% -
453.povray (avg) -93.81% 113.000 7.000 - 0.00% -
462.libquantum -93.75% 32.000 2.000 - 0.00% -
462.libquantum (avg) -93.75% 32.000 2.000 - 0.00% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
435.gromacs (unexplained diff vs previous) -91.78% 292.000 24.000 - -89.14% -
450.soplex (unexplained diff vs previous) -91.30% 69.000 6.000 - -76.92% -
470.lbm (max) -90.15% 893.000 88.000 - -14.56% -
447.dealII (max) -90.00% 150.000 15.000 - -16.67% -
434.zeusmp -89.80% 255.000 26.000 - 0.00% -
434.zeusmp (avg) -89.80% 255.000 26.000 - 0.00% -
470.lbm (unexplained diff vs previous) -88.93% 822.000 91.000 - -9.90% -
433.milc -87.23% 94.000 12.000 - 0.00% -
433.milc (avg) -87.23% 94.000 12.000 - 0.00% -
447.dealII (unexplained diff vs previous) -86.76% 68.000 9.000 - -50.00% -
410.bwaves -81.66% 289.000 53.000 - -1.85% -
410.bwaves (avg) -81.66% 289.000 53.000 - -1.85% -
435.gromacs (unexplained diff vs previous) (percentage) -81.50% 1.730 0.320 - -88.93% -
436.cactusADM (max) (percentage) -80.79% 5.050 0.970 - -12.61% -
465.tonto (unexplained diff vs previous) -79.73% 148.000 30.000 - -60.53% -
465.tonto (unexplained diff vs previous) (percentage) -79.65% 1.130 0.230 - -61.02% -
444.namd (max) (percentage) -79.31% 5.510 1.140 - -6.56% -
459.GemsFDTD (unexplained diff vs previous) -78.25% 584.000 127.000 - -42.01% -
464.h264ref (unexplained diff vs previous) -77.59% 58.000 13.000 - -40.91% -
459.GemsFDTD -77.56% 352.000 79.000 - -1.25% -
459.GemsFDTD (avg) -77.56% 352.000 79.000 - -1.25% -
462.libquantum (max) -76.92% 13.000 3.000 - 0.00% -
453.povray (unexplained diff vs previous) (percentage) -75.55% 2.290 0.560 - -62.67% -
464.h264ref (max) -72.88% 59.000 16.000 - -60.98% -
456.hmmer (max) -72.45% 98.000 27.000 - 12.50% -
470.lbm (avg) (percentage) -72.39% 3.730 1.030 - 1.98% -
433.milc (max) -72.16% 97.000 27.000 - -25.00% -
471.omnetpp (max) -72.09% 172.000 48.000 - -4.00% -
456.hmmer (unexplained diff vs previous) -71.43% 98.000 28.000 - 16.67% -
433.milc (unexplained diff vs previous) -70.83% 96.000 28.000 - -20.00% -
400.perlbench (max) -70.79% 89.000 26.000 - -10.34% -
447.dealII -69.62% 79.000 24.000 - -4.00% -
447.dealII (avg) -69.62% 79.000 24.000 - -4.00% -
465.tonto (max) -69.51% 223.000 68.000 - -63.24% -
465.tonto (max) (percentage) -69.01% 1.710 0.530 - -62.94% -
459.GemsFDTD (max) -68.17% 578.000 184.000 - -18.22% -
450.soplex (max) -66.67% 69.000 23.000 - -14.81% -
436.cactusADM (avg) (percentage) -65.83% 2.400 0.820 - -1.20% -
434.zeusmp (max) (percentage) -64.78% 2.300 0.810 - -25.00% -
456.hmmer -62.50% 56.000 21.000 - 0.00% -
456.hmmer (avg) -62.50% 56.000 21.000 - 0.00% -
458.sjeng (unexplained diff vs previous) -62.50% 48.000 18.000 - 157.14% -
450.soplex -62.16% 37.000 14.000 - -6.67% -
450.soplex (avg) -62.16% 37.000 14.000 - -6.67% -
444.namd (avg) (percentage) -59.23% 2.600 1.060 - -0.93% -
410.bwaves (max) -58.08% 260.000 109.000 - -21.58% -
410.bwaves (unexplained diff vs previous) -56.81% 257.000 111.000 - -18.38% -
471.omnetpp (unexplained diff vs previous) -56.45% 62.000 27.000 - 28.57% -
470.lbm (max) (percentage) -56.02% 5.480 2.410 - -12.36% -
471.omnetpp -54.13% 109.000 50.000 - 0.00% -
471.omnetpp (avg) -54.13% 109.000 50.000 - 0.00% -
410.bwaves (avg) (percentage) -54.08% 1.960 0.900 - 1.12% -
453.povray (max) (percentage) -52.90% 2.590 1.220 - -17.01% -
470.lbm (unexplained diff vs previous) (percentage) -50.99% 5.040 2.470 - -7.49% -
447.dealII (max) (percentage) -50.30% 1.650 0.820 - -12.77% -
458.sjeng (max) -50.00% 48.000 24.000 - 0.00% -
400.perlbench -48.84% 43.000 22.000 - -4.35% -
400.perlbench (avg) -48.84% 43.000 22.000 - -4.35% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) -44.74% 0.380 0.210 - -41.67% -
435.gromacs (max) -42.57% 296.000 170.000 - -25.11% -
447.dealII (unexplained diff vs previous) (percentage) -37.33% 0.750 0.470 - -49.46% -
465.tonto -31.76% 148.000 101.000 - -0.98% -
465.tonto (avg) -31.76% 148.000 101.000 - -0.98% -
465.tonto (avg) (percentage) -31.58% 1.140 0.780 - -1.27% -
482.sphinx3 (avg) (percentage) -31.06% 2.350 1.620 - 2.53% -
464.h264ref (max) (percentage) -30.77% 0.390 0.270 - -60.29% -
464.h264ref -28.81% 59.000 42.000 - -2.33% -
464.h264ref (avg) -28.81% 59.000 42.000 - -2.33% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) -23.33% 0.300 0.230 - 155.56% -
434.zeusmp (avg) (percentage) -22.95% 1.220 0.940 - -1.05% -
400.perlbench (unexplained diff vs previous) -22.86% 35.000 27.000 - 1250.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -22.61% 2.300 1.780 - -41.06% -
459.GemsFDTD (avg) (percentage) -19.42% 1.390 1.120 - 0.90% -
435.gromacs -18.75% 160.000 130.000 - -1.52% -
435.gromacs (avg) -18.75% 160.000 130.000 - -1.52% -
401.bzip2 (max) -13.43% 67.000 58.000 - 3.57% -
401.bzip2 (unexplained diff vs previous) -13.04% 69.000 60.000 - 9.09% -
429.mcf -12.50% 56.000 49.000 - -3.92% -
429.mcf (avg) -12.50% 56.000 49.000 - -3.92% -
453.povray (avg) (percentage) -10.46% 1.530 1.370 - -0.72% -
471.omnetpp (max) (percentage) -6.99% 1.860 1.730 - -4.95% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.36s
Execution (seconds)
Name Current %
400.perlbench 1507.000 -1.76%
401.bzip2 2889.000 -2.03%
410.bwaves 5966.000 -1.83%
429.mcf 1232.000 -3.60%
433.milc 1094.000 -2.50%
434.zeusmp 2799.000 -
435.gromacs 7612.000 -
436.cactusADM 2582.000 -
437.leslie3d 12135.000 -5.30%
444.namd 905.000 -
445.gobmk 5044.000 -1.18%
447.dealII 1918.000 -
450.soplex 600.000 -
453.povray 538.000 -
456.hmmer 4192.000 -
458.sjeng 7673.000 -
459.GemsFDTD 7124.000 -1.75%
462.libquantum 218.000 -
464.h264ref 6152.000 -
465.tonto 12972.000 -
470.lbm 3688.000 -2.41%
471.omnetpp 2781.000 -
473.astar 4870.000 -1.30%
482.sphinx3 904.000 -4.64%
Geometric Mean 2629.046 -1.22%
Code Size (bytes)
Name Current %
400.perlbench 562078.000 -
401.bzip2 33398.000 -
410.bwaves 14723.000 -
429.mcf 7717.000 -
433.milc 47733.000 -
434.zeusmp 138714.000 -
435.gromacs 393002.000 -
436.cactusADM 256626.000 -
437.leslie3d 46184.000 -
444.namd 163131.000 -
445.gobmk 1087552.000 -
447.dealII 189727.000 -
450.soplex 135852.000 -
453.povray 493733.000 -
456.hmmer 74389.000 -
458.sjeng 81349.000 -
459.GemsFDTD 194087.000 -
462.libquantum 13118.000 -
464.h264ref 301482.000 -
465.tonto 1464510.000 -
470.lbm 10213.000 -
471.omnetpp 296077.000 -
473.astar 15825.000 -
482.sphinx3 79598.000 -
Geometric Mean 105775.523 -
sample variability
Name Current %
400.perlbench 22.000 -4.35%
400.perlbench (avg) 22.000 -4.35%
400.perlbench (avg) (percentage) 1.520 -
400.perlbench (max) 26.000 -10.34%
400.perlbench (max) (percentage) 1.790 -6.28%
400.perlbench (unexplained diff vs previous) 27.000 1250.00%
400.perlbench (unexplained diff vs previous) (percentage) 1.790 1276.92%
401.bzip2 48.000 -2.04%
401.bzip2 (avg) 48.000 -2.04%
401.bzip2 (avg) (percentage) 1.680 -
401.bzip2 (max) 58.000 3.57%
401.bzip2 (max) (percentage) 2.030 6.84%
401.bzip2 (unexplained diff vs previous) 60.000 9.09%
401.bzip2 (unexplained diff vs previous) (percentage) 2.080 11.23%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 7.050 -
410.bwaves 53.000 -1.85%
410.bwaves (avg) 53.000 -1.85%
410.bwaves (avg) (percentage) 0.900 1.12%
410.bwaves (max) 109.000 -21.58%
410.bwaves (max) (percentage) 1.830 -20.09%
410.bwaves (unexplained diff vs previous) 111.000 -18.38%
410.bwaves (unexplained diff vs previous) (percentage) 1.860 -16.96%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.120 -
429.mcf 49.000 -3.92%
429.mcf (avg) 49.000 -3.92%
429.mcf (avg) (percentage) 4.000 -
429.mcf (max) 44.000 100.00%
429.mcf (max) (percentage) 3.600 105.71%
429.mcf (unexplained diff vs previous) 46.000 109.09%
429.mcf (unexplained diff vs previous) (percentage) 3.730 116.86%
433.milc 12.000 -
433.milc (avg) 12.000 -
433.milc (avg) (percentage) 1.120 1.82%
433.milc (max) 27.000 -25.00%
433.milc (max) (percentage) 2.500 -22.36%
433.milc (unexplained diff vs previous) 28.000 -20.00%
433.milc (unexplained diff vs previous) (percentage) 2.560 -17.95%
434.zeusmp 26.000 -
434.zeusmp (avg) 26.000 -
434.zeusmp (avg) (percentage) 0.940 -1.05%
434.zeusmp (max) 22.000 -26.67%
434.zeusmp (max) (percentage) 0.810 -25.00%
434.zeusmp (unexplained diff vs previous) 2.000 -93.33%
434.zeusmp (unexplained diff vs previous) (percentage) 0.070 -93.46%
435.gromacs 130.000 -1.52%
435.gromacs (avg) 130.000 -1.52%
435.gromacs (avg) (percentage) 1.710 -1.16%
435.gromacs (max) 170.000 -25.11%
435.gromacs (max) (percentage) 2.240 -24.83%
435.gromacs (unexplained diff vs previous) 24.000 -89.14%
435.gromacs (unexplained diff vs previous) (percentage) 0.320 -88.93%
436.cactusADM 21.000 -
436.cactusADM (avg) 21.000 -
436.cactusADM (avg) (percentage) 0.820 -1.20%
436.cactusADM (max) 25.000 -10.71%
436.cactusADM (max) (percentage) 0.970 -12.61%
436.cactusADM (unexplained diff vs previous) 1.000 -96.55%
436.cactusADM (unexplained diff vs previous) (percentage) 0.040 -96.43%
437.leslie3d 599.000 -5.07%
437.leslie3d (avg) 599.000 -5.07%
437.leslie3d (avg) (percentage) 4.940 -
437.leslie3d (max) 663.000 -14.89%
437.leslie3d (max) (percentage) 5.470 -10.03%
437.leslie3d (unexplained diff vs previous) 679.000 -7.62%
437.leslie3d (unexplained diff vs previous) (percentage) 5.600 -2.44%
444.namd 9.000 -
444.namd (avg) 9.000 -
444.namd (avg) (percentage) 1.060 -
444.namd (max) 10.000 -9.09%
444.namd (max) (percentage) 1.140 -6.56%
444.namd (unexplained diff vs previous) 1.000 -75.00%
444.namd (unexplained diff vs previous) (percentage) 0.110 -75.00%
445.gobmk 43.000 -2.27%
445.gobmk (avg) 43.000 -2.27%
445.gobmk (avg) (percentage) 0.870 -
445.gobmk (max) 59.000 1.72%
445.gobmk (max) (percentage) 1.180 2.61%
445.gobmk (unexplained diff vs previous) 60.000 3.45%
445.gobmk (unexplained diff vs previous) (percentage) 1.190 4.39%
447.dealII 24.000 -4.00%
447.dealII (avg) 24.000 -4.00%
447.dealII (avg) (percentage) 1.300 -1.52%
447.dealII (max) 15.000 -16.67%
447.dealII (max) (percentage) 0.820 -12.77%
447.dealII (unexplained diff vs previous) 9.000 -50.00%
447.dealII (unexplained diff vs previous) (percentage) 0.470 -49.46%
450.soplex 14.000 -6.67%
450.soplex (avg) 14.000 -6.67%
450.soplex (avg) (percentage) 2.480 -
450.soplex (max) 23.000 -14.81%
450.soplex (max) (percentage) 3.920 -12.50%
450.soplex (unexplained diff vs previous) 6.000 -76.92%
450.soplex (unexplained diff vs previous) (percentage) 1.000 -76.69%
453.povray 7.000 -
453.povray (avg) 7.000 -
453.povray (avg) (percentage) 1.370 -
453.povray (max) 6.000 -14.29%
453.povray (max) (percentage) 1.220 -17.01%
453.povray (unexplained diff vs previous) 3.000 -62.50%
453.povray (unexplained diff vs previous) (percentage) 0.560 -62.67%
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.450 -
456.hmmer 21.000 -
456.hmmer (avg) 21.000 -
456.hmmer (avg) (percentage) 0.520 -
456.hmmer (max) 27.000 12.50%
456.hmmer (max) (percentage) 0.660 15.79%
456.hmmer (unexplained diff vs previous) 28.000 16.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.670 17.54%
458.sjeng 46.000 -
458.sjeng (avg) 46.000 -
458.sjeng (avg) (percentage) 0.600 -1.64%
458.sjeng (max) 24.000 -
458.sjeng (max) (percentage) 0.320 -
458.sjeng (unexplained diff vs previous) 18.000 157.14%
458.sjeng (unexplained diff vs previous) (percentage) 0.230 155.56%
459.GemsFDTD 79.000 -1.25%
459.GemsFDTD (avg) 79.000 -1.25%
459.GemsFDTD (avg) (percentage) 1.120 -
459.GemsFDTD (max) 184.000 -18.22%
459.GemsFDTD (max) (percentage) 2.590 -16.72%
459.GemsFDTD (unexplained diff vs previous) 127.000 -42.01%
459.GemsFDTD (unexplained diff vs previous) (percentage) 1.780 -41.06%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.340 -1.47%
462.libquantum (max) 3.000 -
462.libquantum (max) (percentage) 1.420 -5.96%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 42.000 -2.33%
464.h264ref (avg) 42.000 -2.33%
464.h264ref (avg) (percentage) 0.690 -1.43%
464.h264ref (max) 16.000 -60.98%
464.h264ref (max) (percentage) 0.270 -60.29%
464.h264ref (unexplained diff vs previous) 13.000 -40.91%
464.h264ref (unexplained diff vs previous) (percentage) 0.210 -41.67%
465.tonto 101.000 -
465.tonto (avg) 101.000 -
465.tonto (avg) (percentage) 0.780 -1.27%
465.tonto (max) 68.000 -63.24%
465.tonto (max) (percentage) 0.530 -62.94%
465.tonto (unexplained diff vs previous) 30.000 -60.53%
465.tonto (unexplained diff vs previous) (percentage) 0.230 -61.02%
470.lbm 37.000 -2.63%
470.lbm (avg) 37.000 -2.63%
470.lbm (avg) (percentage) 1.030 1.98%
470.lbm (max) 88.000 -14.56%
470.lbm (max) (percentage) 2.410 -12.36%
470.lbm (unexplained diff vs previous) 91.000 -9.90%
470.lbm (unexplained diff vs previous) (percentage) 2.470 -7.49%
471.omnetpp 50.000 -
471.omnetpp (avg) 50.000 -
471.omnetpp (avg) (percentage) 1.820 -
471.omnetpp (max) 48.000 -4.00%
471.omnetpp (max) (percentage) 1.730 -4.95%
471.omnetpp (unexplained diff vs previous) 27.000 28.57%
471.omnetpp (unexplained diff vs previous) (percentage) 0.970 27.63%
473.astar 67.000 -1.47%
473.astar (avg) 67.000 -1.47%
473.astar (avg) (percentage) 1.380 -
473.astar (max) 63.000 75.00%
473.astar (max) (percentage) 1.300 78.08%
473.astar (unexplained diff vs previous) 64.000 100.00%
473.astar (unexplained diff vs previous) (percentage) 1.310 101.54%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.620 2.53%
482.sphinx3 (max) 41.000 28.12%
482.sphinx3 (max) (percentage) 4.640 37.28%
482.sphinx3 (unexplained diff vs previous) 44.000 41.94%
482.sphinx3 (unexplained diff vs previous) (percentage) 4.870 48.93%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.500 -
Geometric Mean 5.975 -23.38%