Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c33568d2c108478c25fe9959b5ce9d25b97efa6c (unchanged)
git_gcc https://gcc.gnu.org/git/gcc.git (6aba48a8cc128e54ee243d451ac9a843ff41c4f9) (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=77c7c4417410f4bf15ea246dd52603dc7be8de45 (+ 29 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/11747c52c04a32dd5e83ebffaf8030e0155c6ee7 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/280/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0280 (glibc-2.40.9000-00422-g77c7c441741) 2024-11-26T22:43:02 0:00:00
Previous 0275 (v6.11.10-74425-g11747c52c04a) 2024-11-17T11:43:59 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 22 56
Performance Improvements 115 151
Added Tests 2 2
Unchanged Tests 1097 1027
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
453.povray (unexplained diff vs previous) (percentage) 944.44% 0.180 1.880 - -17.90% -
453.povray (unexplained diff vs previous) 900.00% 1.000 10.000 - -94.08% -
459.GemsFDTD (unexplained diff vs previous) 550.00% 2.000 13.000 - -97.77% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 500.00% 0.030 0.180 - -92.17% -
444.namd (max) 200.00% 3.000 9.000 - -98.98% -
444.namd (max) (percentage) 125.00% 0.440 0.990 - -82.03% -
444.namd (unexplained diff vs previous) 125.00% 4.000 9.000 - -98.92% -
444.namd (unexplained diff vs previous) (percentage) 122.73% 0.440 0.980 - -81.26% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - -85.71% -
462.libquantum (unexplained diff vs previous) (percentage) 95.65% 0.460 0.900 - 1025.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 94.89% 1.370 2.670 - 298.51% -
471.omnetpp (unexplained diff vs previous) 89.74% 39.000 74.000 - 19.35% -
434.zeusmp (unexplained diff vs previous) 66.67% 12.000 20.000 - -95.74% -
434.zeusmp (unexplained diff vs previous) (percentage) 63.64% 0.440 0.720 - -68.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) 39.17% 1.200 1.670 - -45.42% -
482.sphinx3 (unexplained diff vs previous) 36.36% 11.000 15.000 - -98.66% -
453.povray (max) (percentage) 33.09% 1.390 1.850 - -28.57% -
482.sphinx3 (max) (percentage) 30.16% 1.260 1.640 - -47.94% -
453.povray (max) 28.57% 7.000 9.000 - -95.29% -
482.sphinx3 (max) 27.27% 11.000 14.000 - -98.78% -
434.zeusmp (max) 25.00% 16.000 20.000 - -95.84% -
434.zeusmp (max) (percentage) 23.73% 0.590 0.730 - -68.26% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp -2.60% 2844.000 2770.000 - -70.10% -
453.povray -1.85% 542.000 532.000 - -92.81% -
482.sphinx3 -1.64% 914.000 899.000 - -97.54% -
450.soplex -1.50% 602.000 593.000 - -96.23% -
435.gromacs -1.39% 7500.000 7396.000 - -56.11% -
473.astar -1.33% 4904.000 4839.000 - -62.73% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (unexplained diff vs previous) -94.44% 36.000 2.000 - -94.29% -
400.perlbench (unexplained diff vs previous) (percentage) -94.35% 2.300 0.130 - -58.06% -
410.bwaves (unexplained diff vs previous) -85.71% 28.000 4.000 - -98.44% -
410.bwaves (unexplained diff vs previous) (percentage) -85.11% 0.470 0.070 - -95.98% -
429.mcf (unexplained diff vs previous) -84.62% 13.000 2.000 - -80.00% -
429.mcf (unexplained diff vs previous) (percentage) -84.62% 1.040 0.160 - 77.78% -
470.lbm (unexplained diff vs previous) (percentage) -83.33% 0.300 0.050 - -99.01% -
458.sjeng (unexplained diff vs previous) -82.00% 50.000 9.000 - -81.25% -
470.lbm (unexplained diff vs previous) -81.82% 11.000 2.000 - -99.76% -
458.sjeng (unexplained diff vs previous) (percentage) -81.54% 0.650 0.120 - -60.00% -
437.leslie3d (unexplained diff vs previous) (percentage) -59.31% 2.040 0.830 - -19.42% -
437.leslie3d (unexplained diff vs previous) -59.04% 249.000 102.000 - -45.45% -
436.cactusADM (unexplained diff vs previous) (percentage) -51.61% 0.310 0.150 - -96.88% -
401.bzip2 (max) -50.00% 38.000 19.000 - -71.64% -
436.cactusADM (unexplained diff vs previous) -50.00% 8.000 4.000 - -99.66% -
450.soplex (max) -50.00% 20.000 10.000 - -85.51% -
450.soplex (max) (percentage) -49.85% 3.370 1.690 - 284.09% -
401.bzip2 (max) (percentage) -48.48% 1.320 0.680 - 70.00% -
445.gobmk (unexplained diff vs previous) -41.94% 31.000 18.000 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) -40.98% 0.610 0.360 - 140.00% -
433.milc (unexplained diff vs previous) (percentage) -35.71% 0.280 0.180 - -74.29% -
401.bzip2 (unexplained diff vs previous) -35.71% 14.000 9.000 - -86.96% -
401.bzip2 (unexplained diff vs previous) (percentage) -35.42% 0.480 0.310 - -24.39% -
433.milc (unexplained diff vs previous) -33.33% 3.000 2.000 - -97.92% -
465.tonto (unexplained diff vs previous) -32.29% 96.000 65.000 - -56.08% -
465.tonto (unexplained diff vs previous) (percentage) -31.08% 0.740 0.510 - -54.87% -
410.bwaves (max) -28.57% 28.000 20.000 - -92.31% -
436.cactusADM (max) (percentage) -25.81% 0.310 0.230 - -95.45% -
410.bwaves (max) (percentage) -25.53% 0.470 0.350 - -80.23% -
436.cactusADM (max) -25.00% 8.000 6.000 - -99.51% -
450.soplex (unexplained diff vs previous) -25.00% 12.000 9.000 - -86.96% -
450.soplex (unexplained diff vs previous) (percentage) -23.62% 1.990 1.520 - 245.45% -
429.mcf (max) -23.08% 13.000 10.000 - -72.97% -
473.astar (max) -20.00% 80.000 64.000 - 4.92% -
473.astar (max) (percentage) -18.90% 1.640 1.330 - 182.98% -
470.lbm (max) -18.18% 11.000 9.000 - -98.99% -
473.astar (unexplained diff vs previous) -17.72% 79.000 65.000 - 6.56% -
445.gobmk (max) -17.65% 34.000 28.000 - -15.15% -
429.mcf (max) (percentage) -17.14% 1.050 0.870 - 171.88% -
458.sjeng (max) (percentage) -16.92% 0.650 0.540 - 80.00% -
435.gromacs (unexplained diff vs previous) -16.80% 125.000 104.000 - -64.38% -
473.astar (unexplained diff vs previous) (percentage) -16.77% 1.610 1.340 - 185.11% -
458.sjeng (max) -16.33% 49.000 41.000 - -14.58% -
435.gromacs (unexplained diff vs previous) (percentage) -15.57% 1.670 1.410 - -18.50% -
445.gobmk (max) (percentage) -14.93% 0.670 0.570 - 111.11% -
447.dealII (unexplained diff vs previous) -14.29% 14.000 12.000 - -82.35% -
447.dealII (unexplained diff vs previous) (percentage) -13.89% 0.720 0.620 - -17.33% -
433.milc (max) (percentage) -11.11% 0.270 0.240 - -66.20% -
437.leslie3d (max) (percentage) -11.04% 7.610 6.770 - 118.39% -
456.hmmer (max) (percentage) -10.71% 0.280 0.250 - -51.92% -
437.leslie3d (max) -10.32% 930.000 834.000 - 48.93% -
470.lbm (max) (percentage) -10.00% 0.300 0.270 - -95.07% -
456.hmmer (max) -9.09% 11.000 10.000 - -89.80% -
471.omnetpp (max) -9.09% 99.000 90.000 - -47.67% -
400.perlbench (max) (percentage) -8.51% 2.350 2.150 - 168.75% -
400.perlbench (max) -8.33% 36.000 33.000 - -62.92% -
464.h264ref (max) -7.69% 13.000 12.000 - -79.66% -
435.gromacs (max) -7.14% 224.000 208.000 - -29.73% -
447.dealII (max) -7.14% 14.000 13.000 - -91.33% -
447.dealII (max) (percentage) -6.85% 0.730 0.680 - -58.79% -
450.soplex -6.67% 15.000 14.000 - -62.16% -
450.soplex (avg) -6.67% 15.000 14.000 - -62.16% -
471.omnetpp (max) (percentage) -6.30% 3.490 3.270 - 75.81% -
435.gromacs (max) (percentage) -5.69% 2.990 2.820 - 60.23% -
462.libquantum (max) (percentage) -5.13% 1.170 1.110 - 1287.50% -
465.tonto (max) -5.12% 215.000 204.000 - -8.52% -
465.tonto (max) (percentage) -4.79% 1.670 1.590 - -7.02% -
464.h264ref (max) (percentage) -4.55% 0.220 0.210 - -46.15% -
436.cactusADM -4.55% 22.000 21.000 - -96.41% -
436.cactusADM (avg) -4.55% 22.000 21.000 - -96.41% -
456.hmmer -4.55% 22.000 21.000 - -62.50% -
456.hmmer (avg) -4.55% 22.000 21.000 - -62.50% -
400.perlbench -4.17% 24.000 23.000 - -46.51% -
400.perlbench (avg) -4.17% 24.000 23.000 - -46.51% -
447.dealII -3.85% 26.000 25.000 - -68.35% -
447.dealII (avg) -3.85% 26.000 25.000 - -68.35% -
471.omnetpp -3.85% 52.000 50.000 - -54.13% -
471.omnetpp (avg) -3.85% 52.000 50.000 - -54.13% -
459.GemsFDTD (max) (percentage) -3.57% 0.280 0.270 - -88.16% -
459.GemsFDTD -2.53% 79.000 77.000 - -78.12% -
459.GemsFDTD (avg) -2.53% 79.000 77.000 - -78.12% -
435.gromacs -2.34% 128.000 125.000 - -21.88% -
435.gromacs (avg) -2.34% 128.000 125.000 - -21.88% -
445.gobmk -2.27% 44.000 43.000 - 59.26% -
445.gobmk (avg) -2.27% 44.000 43.000 - 59.26% -
464.h264ref -2.27% 44.000 43.000 - -27.12% -
464.h264ref (avg) -2.27% 44.000 43.000 - -27.12% -
458.sjeng -2.08% 48.000 47.000 - 30.56% -
458.sjeng (avg) -2.08% 48.000 47.000 - 30.56% -
470.lbm (avg) (percentage) -1.94% 1.030 1.010 - -72.92% -
429.mcf -1.92% 52.000 51.000 - -8.93% -
429.mcf (avg) -1.92% 52.000 51.000 - -8.93% -
456.hmmer (avg) (percentage) -1.89% 0.530 0.520 - 73.33% -
465.tonto -1.87% 107.000 105.000 - -29.05% -
465.tonto (avg) -1.87% 107.000 105.000 - -29.05% -
433.milc (avg) (percentage) -1.82% 1.100 1.080 - 56.52% -
459.GemsFDTD (avg) (percentage) -1.79% 1.120 1.100 - -20.86% -
458.sjeng (avg) (percentage) -1.59% 0.630 0.620 - 169.57% -
473.astar -1.45% 69.000 68.000 - 33.33% -
473.astar (avg) -1.45% 69.000 68.000 - 33.33% -
464.h264ref (avg) (percentage) -1.39% 0.720 0.710 - 82.05% -
400.perlbench (avg) (percentage) -1.29% 1.550 1.530 - 292.31% -
437.leslie3d (avg) (percentage) -1.22% 4.920 4.860 - 191.02% -
465.tonto (avg) (percentage) -1.20% 0.830 0.820 - -28.07% -
429.mcf (avg) (percentage) -1.20% 4.180 4.130 - 760.42% -
436.cactusADM (avg) (percentage) -1.18% 0.850 0.840 - -65.00% -
401.bzip2 (avg) (percentage) -1.15% 1.740 1.720 - 537.04% -
445.gobmk (avg) (percentage) -1.15% 0.870 0.860 - 290.91% -
410.bwaves (avg) (percentage) -1.11% 0.900 0.890 - -54.59% -

Added Tests - sample variability
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.86% 1437761.000 1464510.000 - 0.00% -
473.astar 1.78% 15549.000 15825.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 1287.50% 0.080 1.110 - -5.13% -
403.gcc (max) (percentage) 1108.47% 0.590 7.130 - -0.28% -
462.libquantum (unexplained diff vs previous) (percentage) 1025.00% 0.080 0.900 - 95.65% -
450.soplex (avg) (percentage) 950.00% 0.240 2.520 - -0.40% -
429.mcf (avg) (percentage) 760.42% 0.480 4.130 - -1.20% -
462.libquantum (avg) (percentage) 615.79% 0.190 1.360 - -0.73% -
401.bzip2 (avg) (percentage) 537.04% 0.270 1.720 - -1.15% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 0.00% -
454.calculix (max) (percentage) 310.71% 0.840 3.450 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 298.51% 0.670 2.670 - 94.89% -
400.perlbench (avg) (percentage) 292.31% 0.390 1.530 - -1.29% -
445.gobmk (avg) (percentage) 290.91% 0.220 0.860 - -1.15% -
450.soplex (max) (percentage) 284.09% 0.440 1.690 - -49.85% -
473.astar (avg) (percentage) 255.00% 0.400 1.420 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 245.45% 0.440 1.520 - -23.62% -
437.leslie3d (avg) (percentage) 191.02% 1.670 4.860 - -1.22% -
473.astar (unexplained diff vs previous) (percentage) 185.11% 0.470 1.340 - -16.77% -
473.astar (max) (percentage) 182.98% 0.470 1.330 - -18.90% -
429.mcf (max) (percentage) 171.88% 0.320 0.870 - -17.14% -
458.sjeng (avg) (percentage) 169.57% 0.230 0.620 - -1.59% -
400.perlbench (max) (percentage) 168.75% 0.800 2.150 - -8.51% -
445.gobmk (unexplained diff vs previous) (percentage) 140.00% 0.150 0.360 - -40.98% -
437.leslie3d (max) (percentage) 118.39% 3.100 6.770 - -11.04% -
445.gobmk (max) (percentage) 111.11% 0.270 0.570 - -14.93% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
437.leslie3d 99.00% 301.000 599.000 - -0.33% -
437.leslie3d (avg) 99.00% 301.000 599.000 - -0.33% -
464.h264ref (avg) (percentage) 82.05% 0.390 0.710 - -1.39% -
458.sjeng (max) (percentage) 80.00% 0.300 0.540 - -16.92% -
435.gromacs (avg) (percentage) 78.95% 0.950 1.700 - -0.58% -
429.mcf (unexplained diff vs previous) (percentage) 77.78% 0.090 0.160 - -84.62% -
471.omnetpp (max) (percentage) 75.81% 1.860 3.270 - -6.30% -
456.hmmer (avg) (percentage) 73.33% 0.300 0.520 - -1.89% -
401.bzip2 (max) (percentage) 70.00% 0.400 0.680 - -48.48% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 0.00% -
483.xalancbmk (max) (percentage) 63.04% 0.920 1.500 - 0.00% -
435.gromacs (max) (percentage) 60.23% 1.760 2.820 - -5.69% -
445.gobmk 59.26% 27.000 43.000 - -2.27% -
445.gobmk (avg) 59.26% 27.000 43.000 - -2.27% -
433.milc (avg) (percentage) 56.52% 0.690 1.080 - -1.82% -
471.omnetpp (avg) (percentage) 55.93% 1.180 1.840 - 0.55% -
447.dealII (avg) (percentage) 55.17% 0.870 1.350 - -0.74% -
437.leslie3d (max) 48.93% 560.000 834.000 - -10.32% -
473.astar 33.33% 51.000 68.000 - -1.45% -
473.astar (avg) 33.33% 51.000 68.000 - -1.45% -
458.sjeng 30.56% 36.000 47.000 - -2.08% -
458.sjeng (avg) 30.56% 36.000 47.000 - -2.08% -
471.omnetpp (unexplained diff vs previous) 19.35% 62.000 74.000 - 89.74% -
401.bzip2 11.11% 45.000 50.000 - 0.00% -
401.bzip2 (avg) 11.11% 45.000 50.000 - 0.00% -
473.astar (unexplained diff vs previous) 6.56% 61.000 65.000 - -17.72% -
473.astar (max) 4.92% 61.000 64.000 - -20.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - 0.00% -
nb_execution_successful -17.24% 29.000 24.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.83% 10845.000 10213.000 - 0.00% -
437.leslie3d -3.79% 48001.000 46184.000 - 0.00% -
434.zeusmp -1.66% 141060.000 138714.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.70% 17005.000 221.000 - 0.91% -
482.sphinx3 -97.54% 36586.000 899.000 - -1.64% -
450.soplex -96.23% 15709.000 593.000 - -1.50% -
444.namd -94.25% 15957.000 918.000 - 0.99% -
453.povray -92.81% 7395.000 532.000 - -1.85% -
433.milc -92.02% 13668.000 1091.000 - 0.18% -
429.mcf -89.34% 11727.000 1250.000 - 0.16% -
436.cactusADM -89.30% 24396.000 2610.000 - -0.15% -
434.zeusmp -86.73% 20932.000 2778.000 - 0.73% -
400.perlbench -86.02% 11215.000 1568.000 - 0.13% -
401.bzip2 -82.85% 16971.000 2911.000 - 0.31% -
447.dealII -78.88% 9101.000 1922.000 - -0.62% -
456.hmmer -77.96% 18995.000 4187.000 - 0.00% -
470.lbm -77.42% 16312.000 3683.000 - 0.05% -
459.GemsFDTD -72.11% 25364.000 7075.000 - -0.18% -
471.omnetpp -70.10% 9265.000 2770.000 - -2.60% -
473.astar -62.73% 12983.000 4839.000 - -1.33% -
410.bwaves -59.48% 14745.000 5974.000 - 0.07% -
464.h264ref -59.38% 15208.000 6177.000 - 0.21% -
445.gobmk -59.20% 12396.000 5057.000 - -0.35% -
435.gromacs -56.11% 16853.000 7396.000 - -1.39% -
458.sjeng -52.19% 16018.000 7658.000 - 0.12% -
437.leslie3d -31.78% 18070.000 12328.000 - 0.83% -
465.tonto -1.41% 13044.000 12860.000 - -0.50% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
470.lbm (unexplained diff vs previous) -99.76% 822.000 2.000 - -81.82% -
436.cactusADM (unexplained diff vs previous) -99.66% 1173.000 4.000 - -50.00% -
436.cactusADM (max) -99.51% 1231.000 6.000 - -25.00% -
470.lbm (unexplained diff vs previous) (percentage) -99.01% 5.040 0.050 - -83.33% -
470.lbm (max) -98.99% 893.000 9.000 - -18.18% -
444.namd (max) -98.98% 879.000 9.000 - 200.00% -
444.namd (unexplained diff vs previous) -98.92% 834.000 9.000 - 125.00% -
482.sphinx3 (max) -98.78% 1152.000 14.000 - 27.27% -
482.sphinx3 (unexplained diff vs previous) -98.66% 1118.000 15.000 - 36.36% -
410.bwaves (unexplained diff vs previous) -98.44% 257.000 4.000 - -85.71% -
482.sphinx3 -98.37% 859.000 14.000 - 0.00% -
482.sphinx3 (avg) -98.37% 859.000 14.000 - 0.00% -
433.milc (max) -97.94% 97.000 2.000 - 0.00% -
433.milc (unexplained diff vs previous) -97.92% 96.000 2.000 - -33.33% -
444.namd -97.83% 414.000 9.000 - 0.00% -
444.namd (avg) -97.83% 414.000 9.000 - 0.00% -
459.GemsFDTD (unexplained diff vs previous) -97.77% 584.000 13.000 - 550.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -96.88% 4.810 0.150 - -51.61% -
459.GemsFDTD (max) -96.71% 578.000 19.000 - 0.00% -
436.cactusADM -96.41% 585.000 21.000 - -4.55% -
436.cactusADM (avg) -96.41% 585.000 21.000 - -4.55% -
416.gamess (max) (percentage) -96.18% 3.140 0.120 - 0.00% -
410.bwaves (unexplained diff vs previous) (percentage) -95.98% 1.740 0.070 - -85.11% -
434.zeusmp (max) -95.84% 481.000 20.000 - 25.00% -
434.zeusmp (unexplained diff vs previous) -95.74% 470.000 20.000 - 66.67% -
436.cactusADM (max) (percentage) -95.45% 5.050 0.230 - -25.81% -
453.povray (max) -95.29% 191.000 9.000 - 28.57% -
470.lbm (max) (percentage) -95.07% 5.480 0.270 - -10.00% -
400.perlbench (unexplained diff vs previous) -94.29% 35.000 2.000 - -94.44% -
453.povray (unexplained diff vs previous) -94.08% 169.000 10.000 - 900.00% -
470.lbm -93.91% 608.000 37.000 - 0.00% -
470.lbm (avg) -93.91% 608.000 37.000 - 0.00% -
453.povray -93.81% 113.000 7.000 - 0.00% -
453.povray (avg) -93.81% 113.000 7.000 - 0.00% -
481.wrf (max) (percentage) -93.75% 4.160 0.260 - 0.00% -
410.bwaves (max) -92.31% 260.000 20.000 - -28.57% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -92.17% 2.300 0.180 - 500.00% -
447.dealII (max) -91.33% 150.000 13.000 - -7.14% -
462.libquantum -90.62% 32.000 3.000 - 0.00% -
462.libquantum (avg) -90.62% 32.000 3.000 - 0.00% -
434.zeusmp -89.80% 255.000 26.000 - 0.00% -
434.zeusmp (avg) -89.80% 255.000 26.000 - 0.00% -
456.hmmer (max) -89.80% 98.000 10.000 - -9.09% -
433.milc -88.30% 94.000 11.000 - 0.00% -
433.milc (avg) -88.30% 94.000 11.000 - 0.00% -
459.GemsFDTD (max) (percentage) -88.16% 2.280 0.270 - -3.57% -
401.bzip2 (unexplained diff vs previous) -86.96% 69.000 9.000 - -35.71% -
450.soplex (unexplained diff vs previous) -86.96% 69.000 9.000 - -25.00% -
462.libquantum (unexplained diff vs previous) -85.71% 14.000 2.000 - 100.00% -
450.soplex (max) -85.51% 69.000 10.000 - -50.00% -
462.libquantum (max) -84.62% 13.000 2.000 - 0.00% -
447.dealII (unexplained diff vs previous) -82.35% 68.000 12.000 - -14.29% -
444.namd (max) (percentage) -82.03% 5.510 0.990 - 125.00% -
410.bwaves -81.66% 289.000 53.000 - 0.00% -
410.bwaves (avg) -81.66% 289.000 53.000 - 0.00% -
444.namd (unexplained diff vs previous) (percentage) -81.26% 5.230 0.980 - 122.73% -
458.sjeng (unexplained diff vs previous) -81.25% 48.000 9.000 - -82.00% -
410.bwaves (max) (percentage) -80.23% 1.770 0.350 - -25.53% -
429.mcf (unexplained diff vs previous) -80.00% 10.000 2.000 - -84.62% -
464.h264ref (max) -79.66% 59.000 12.000 - -7.69% -
459.GemsFDTD -78.12% 352.000 77.000 - -2.53% -
459.GemsFDTD (avg) -78.12% 352.000 77.000 - -2.53% -
464.h264ref (unexplained diff vs previous) -77.59% 58.000 13.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -74.29% 0.700 0.180 - -35.71% -
429.mcf (max) -72.97% 37.000 10.000 - -23.08% -
470.lbm (avg) (percentage) -72.92% 3.730 1.010 - -1.94% -
401.bzip2 (max) -71.64% 67.000 19.000 - -50.00% -
447.dealII -68.35% 79.000 25.000 - -3.85% -
447.dealII (avg) -68.35% 79.000 25.000 - -3.85% -
434.zeusmp (max) (percentage) -68.26% 2.300 0.730 - 23.73% -
434.zeusmp (unexplained diff vs previous) (percentage) -68.00% 2.250 0.720 - 63.64% -
433.milc (max) (percentage) -66.20% 0.710 0.240 - -11.11% -
436.cactusADM (avg) (percentage) -65.00% 2.400 0.840 - -1.18% -
435.gromacs (unexplained diff vs previous) -64.38% 292.000 104.000 - -16.80% -
400.perlbench (max) -62.92% 89.000 33.000 - -8.33% -
456.hmmer -62.50% 56.000 21.000 - -4.55% -
456.hmmer (avg) -62.50% 56.000 21.000 - -4.55% -
450.soplex -62.16% 37.000 14.000 - -6.67% -
450.soplex (avg) -62.16% 37.000 14.000 - -6.67% -
458.sjeng (unexplained diff vs previous) (percentage) -60.00% 0.300 0.120 - -81.54% -
447.dealII (max) (percentage) -58.79% 1.650 0.680 - -6.85% -
444.namd (avg) (percentage) -58.46% 2.600 1.080 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) -58.06% 0.310 0.130 - -94.35% -
465.tonto (unexplained diff vs previous) -56.08% 148.000 65.000 - -32.29% -
465.tonto (unexplained diff vs previous) (percentage) -54.87% 1.130 0.510 - -31.08% -
410.bwaves (avg) (percentage) -54.59% 1.960 0.890 - -1.11% -
471.omnetpp -54.13% 109.000 50.000 - -3.85% -
471.omnetpp (avg) -54.13% 109.000 50.000 - -3.85% -
456.hmmer (max) (percentage) -51.92% 0.520 0.250 - -10.71% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - 0.00% -
482.sphinx3 (max) (percentage) -47.94% 3.150 1.640 - 30.16% -
471.omnetpp (max) -47.67% 172.000 90.000 - -9.09% -
400.perlbench -46.51% 43.000 23.000 - -4.17% -
400.perlbench (avg) -46.51% 43.000 23.000 - -4.17% -
464.h264ref (max) (percentage) -46.15% 0.390 0.210 - -4.55% -
437.leslie3d (unexplained diff vs previous) -45.45% 187.000 102.000 - -59.04% -
482.sphinx3 (unexplained diff vs previous) (percentage) -45.42% 3.060 1.670 - 39.17% -
464.h264ref (unexplained diff vs previous) (percentage) -44.74% 0.380 0.210 - 0.00% -
482.sphinx3 (avg) (percentage) -33.19% 2.350 1.570 - 0.00% -
435.gromacs (max) -29.73% 296.000 208.000 - -7.14% -
465.tonto -29.05% 148.000 105.000 - -1.87% -
465.tonto (avg) -29.05% 148.000 105.000 - -1.87% -
453.povray (max) (percentage) -28.57% 2.590 1.850 - 33.09% -
465.tonto (avg) (percentage) -28.07% 1.140 0.820 - -1.20% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - 0.00% -
464.h264ref -27.12% 59.000 43.000 - -2.27% -
464.h264ref (avg) -27.12% 59.000 43.000 - -2.27% -
401.bzip2 (unexplained diff vs previous) (percentage) -24.39% 0.410 0.310 - -35.42% -
434.zeusmp (avg) (percentage) -22.13% 1.220 0.950 - 0.00% -
435.gromacs -21.88% 160.000 125.000 - -2.34% -
435.gromacs (avg) -21.88% 160.000 125.000 - -2.34% -
459.GemsFDTD (avg) (percentage) -20.86% 1.390 1.100 - -1.79% -
437.leslie3d (unexplained diff vs previous) (percentage) -19.42% 1.030 0.830 - -59.31% -
435.gromacs (unexplained diff vs previous) (percentage) -18.50% 1.730 1.410 - -15.57% -
453.povray (unexplained diff vs previous) (percentage) -17.90% 2.290 1.880 - 944.44% -
447.dealII (unexplained diff vs previous) (percentage) -17.33% 0.750 0.620 - -13.89% -
445.gobmk (max) -15.15% 33.000 28.000 - -17.65% -
458.sjeng (max) -14.58% 48.000 41.000 - -16.33% -
453.povray (avg) (percentage) -9.15% 1.530 1.390 - 0.00% -
429.mcf -8.93% 56.000 51.000 - -1.92% -
429.mcf (avg) -8.93% 56.000 51.000 - -1.92% -
465.tonto (max) -8.52% 223.000 204.000 - -5.12% -
465.tonto (max) (percentage) -7.02% 1.710 1.590 - -4.79% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 0.00% -

Added Tests - sample variability
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1568.000 -
401.bzip2 2911.000 -
410.bwaves 5974.000 -
429.mcf 1250.000 -
433.milc 1091.000 -
434.zeusmp 2778.000 -
435.gromacs 7396.000 -1.39%
436.cactusADM 2610.000 -
437.leslie3d 12328.000 -
444.namd 918.000 -
445.gobmk 5057.000 -
447.dealII 1922.000 -
450.soplex 593.000 -1.50%
453.povray 532.000 -1.85%
456.hmmer 4187.000 -
458.sjeng 7658.000 -
459.GemsFDTD 7075.000 -
462.libquantum 221.000 -
464.h264ref 6177.000 -
465.tonto 12860.000 -
470.lbm 3683.000 -
471.omnetpp 2770.000 -2.60%
473.astar 4839.000 -1.33%
482.sphinx3 899.000 -1.64%
Geometric Mean 2632.151 -
Code Size (bytes)
Name Current %
400.perlbench 562078.000 -
401.bzip2 33398.000 -
410.bwaves 14723.000 -
429.mcf 7717.000 -
433.milc 47733.000 -
434.zeusmp 138714.000 -
435.gromacs 393002.000 -
436.cactusADM 256626.000 -
437.leslie3d 46184.000 -
444.namd 163131.000 -
445.gobmk 1087552.000 -
447.dealII 189727.000 -
450.soplex 135852.000 -
453.povray 493733.000 -
456.hmmer 74389.000 -
458.sjeng 81349.000 -
459.GemsFDTD 194087.000 -
462.libquantum 13118.000 -
464.h264ref 301482.000 -
465.tonto 1464510.000 -
470.lbm 10213.000 -
471.omnetpp 296077.000 -
473.astar 15825.000 -
482.sphinx3 79598.000 -
Geometric Mean 105775.523 -
sample variability
Name Current %
400.perlbench 23.000 -4.17%
400.perlbench (avg) 23.000 -4.17%
400.perlbench (avg) (percentage) 1.530 -1.29%
400.perlbench (max) 33.000 -8.33%
400.perlbench (max) (percentage) 2.150 -8.51%
400.perlbench (unexplained diff vs previous) 2.000 -94.44%
400.perlbench (unexplained diff vs previous) (percentage) 0.130 -94.35%
401.bzip2 50.000 -
401.bzip2 (avg) 50.000 -
401.bzip2 (avg) (percentage) 1.720 -1.15%
401.bzip2 (max) 19.000 -50.00%
401.bzip2 (max) (percentage) 0.680 -48.48%
401.bzip2 (unexplained diff vs previous) 9.000 -35.71%
401.bzip2 (unexplained diff vs previous) (percentage) 0.310 -35.42%
403.gcc (avg) (percentage) 2.370 -
403.gcc (max) (percentage) 7.130 -
410.bwaves 53.000 -
410.bwaves (avg) 53.000 -
410.bwaves (avg) (percentage) 0.890 -1.11%
410.bwaves (max) 20.000 -28.57%
410.bwaves (max) (percentage) 0.350 -25.53%
410.bwaves (unexplained diff vs previous) 4.000 -85.71%
410.bwaves (unexplained diff vs previous) (percentage) 0.070 -85.11%
416.gamess (avg) (percentage) 0.850 -
416.gamess (max) (percentage) 0.120 -
429.mcf 51.000 -1.92%
429.mcf (avg) 51.000 -1.92%
429.mcf (avg) (percentage) 4.130 -1.20%
429.mcf (max) 10.000 -23.08%
429.mcf (max) (percentage) 0.870 -17.14%
429.mcf (unexplained diff vs previous) 2.000 -84.62%
429.mcf (unexplained diff vs previous) (percentage) 0.160 -84.62%
433.milc 11.000 -
433.milc (avg) 11.000 -
433.milc (avg) (percentage) 1.080 -1.82%
433.milc (max) 2.000 -
433.milc (max) (percentage) 0.240 -11.11%
433.milc (unexplained diff vs previous) 2.000 -33.33%
433.milc (unexplained diff vs previous) (percentage) 0.180 -35.71%
434.zeusmp 26.000 -
434.zeusmp (avg) 26.000 -
434.zeusmp (avg) (percentage) 0.950 -
434.zeusmp (max) 20.000 25.00%
434.zeusmp (max) (percentage) 0.730 23.73%
434.zeusmp (unexplained diff vs previous) 20.000 66.67%
434.zeusmp (unexplained diff vs previous) (percentage) 0.720 63.64%
435.gromacs 125.000 -2.34%
435.gromacs (avg) 125.000 -2.34%
435.gromacs (avg) (percentage) 1.700 -
435.gromacs (max) 208.000 -7.14%
435.gromacs (max) (percentage) 2.820 -5.69%
435.gromacs (unexplained diff vs previous) 104.000 -16.80%
435.gromacs (unexplained diff vs previous) (percentage) 1.410 -15.57%
436.cactusADM 21.000 -4.55%
436.cactusADM (avg) 21.000 -4.55%
436.cactusADM (avg) (percentage) 0.840 -1.18%
436.cactusADM (max) 6.000 -25.00%
436.cactusADM (max) (percentage) 0.230 -25.81%
436.cactusADM (unexplained diff vs previous) 4.000 -50.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.150 -51.61%
437.leslie3d 599.000 -
437.leslie3d (avg) 599.000 -
437.leslie3d (avg) (percentage) 4.860 -1.22%
437.leslie3d (max) 834.000 -10.32%
437.leslie3d (max) (percentage) 6.770 -11.04%
437.leslie3d (unexplained diff vs previous) 102.000 -59.04%
437.leslie3d (unexplained diff vs previous) (percentage) 0.830 -59.31%
444.namd 9.000 -
444.namd (avg) 9.000 -
444.namd (avg) (percentage) 1.080 -
444.namd (max) 9.000 200.00%
444.namd (max) (percentage) 0.990 125.00%
444.namd (unexplained diff vs previous) 9.000 125.00%
444.namd (unexplained diff vs previous) (percentage) 0.980 122.73%
445.gobmk 43.000 -2.27%
445.gobmk (avg) 43.000 -2.27%
445.gobmk (avg) (percentage) 0.860 -1.15%
445.gobmk (max) 28.000 -17.65%
445.gobmk (max) (percentage) 0.570 -14.93%
445.gobmk (unexplained diff vs previous) 18.000 -41.94%
445.gobmk (unexplained diff vs previous) (percentage) 0.360 -40.98%
447.dealII 25.000 -3.85%
447.dealII (avg) 25.000 -3.85%
447.dealII (avg) (percentage) 1.350 -
447.dealII (max) 13.000 -7.14%
447.dealII (max) (percentage) 0.680 -6.85%
447.dealII (unexplained diff vs previous) 12.000 -14.29%
447.dealII (unexplained diff vs previous) (percentage) 0.620 -13.89%
450.soplex 14.000 -6.67%
450.soplex (avg) 14.000 -6.67%
450.soplex (avg) (percentage) 2.520 -
450.soplex (max) 10.000 -50.00%
450.soplex (max) (percentage) 1.690 -49.85%
450.soplex (unexplained diff vs previous) 9.000 -25.00%
450.soplex (unexplained diff vs previous) (percentage) 1.520 -23.62%
453.povray 7.000 -
453.povray (avg) 7.000 -
453.povray (avg) (percentage) 1.390 -
453.povray (max) 9.000 28.57%
453.povray (max) (percentage) 1.850 33.09%
453.povray (unexplained diff vs previous) 10.000 900.00%
453.povray (unexplained diff vs previous) (percentage) 1.880 944.44%
454.calculix (avg) (percentage) 3.440 -
454.calculix (max) (percentage) 3.450 -
456.hmmer 21.000 -4.55%
456.hmmer (avg) 21.000 -4.55%
456.hmmer (avg) (percentage) 0.520 -1.89%
456.hmmer (max) 10.000 -9.09%
456.hmmer (max) (percentage) 0.250 -10.71%
456.hmmer (unexplained diff vs previous) 0.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.000 -
458.sjeng 47.000 -2.08%
458.sjeng (avg) 47.000 -2.08%
458.sjeng (avg) (percentage) 0.620 -1.59%
458.sjeng (max) 41.000 -16.33%
458.sjeng (max) (percentage) 0.540 -16.92%
458.sjeng (unexplained diff vs previous) 9.000 -82.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.120 -81.54%
459.GemsFDTD 77.000 -2.53%
459.GemsFDTD (avg) 77.000 -2.53%
459.GemsFDTD (avg) (percentage) 1.100 -1.79%
459.GemsFDTD (max) 19.000 -
459.GemsFDTD (max) (percentage) 0.270 -3.57%
459.GemsFDTD (unexplained diff vs previous) 13.000 550.00%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.180 500.00%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 1.360 -
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.110 -5.13%
462.libquantum (unexplained diff vs previous) 2.000 100.00%
462.libquantum (unexplained diff vs previous) (percentage) 0.900 95.65%
464.h264ref 43.000 -2.27%
464.h264ref (avg) 43.000 -2.27%
464.h264ref (avg) (percentage) 0.710 -1.39%
464.h264ref (max) 12.000 -7.69%
464.h264ref (max) (percentage) 0.210 -4.55%
464.h264ref (unexplained diff vs previous) 13.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.210 -
465.tonto 105.000 -1.87%
465.tonto (avg) 105.000 -1.87%
465.tonto (avg) (percentage) 0.820 -1.20%
465.tonto (max) 204.000 -5.12%
465.tonto (max) (percentage) 1.590 -4.79%
465.tonto (unexplained diff vs previous) 65.000 -32.29%
465.tonto (unexplained diff vs previous) (percentage) 0.510 -31.08%
470.lbm 37.000 -
470.lbm (avg) 37.000 -
470.lbm (avg) (percentage) 1.010 -1.94%
470.lbm (max) 9.000 -18.18%
470.lbm (max) (percentage) 0.270 -10.00%
470.lbm (unexplained diff vs previous) 2.000 -81.82%
470.lbm (unexplained diff vs previous) (percentage) 0.050 -83.33%
471.omnetpp 50.000 -3.85%
471.omnetpp (avg) 50.000 -3.85%
471.omnetpp (avg) (percentage) 1.840 -
471.omnetpp (max) 90.000 -9.09%
471.omnetpp (max) (percentage) 3.270 -6.30%
471.omnetpp (unexplained diff vs previous) 74.000 89.74%
471.omnetpp (unexplained diff vs previous) (percentage) 2.670 94.89%
473.astar 68.000 -1.45%
473.astar (avg) 68.000 -1.45%
473.astar (avg) (percentage) 1.420 -
473.astar (max) 64.000 -20.00%
473.astar (max) (percentage) 1.330 -18.90%
473.astar (unexplained diff vs previous) 65.000 -17.72%
473.astar (unexplained diff vs previous) (percentage) 1.340 -16.77%
481.wrf (avg) (percentage) 1.850 -
481.wrf (max) (percentage) 0.260 -
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.570 -
482.sphinx3 (max) 14.000 27.27%
482.sphinx3 (max) (percentage) 1.640 30.16%
482.sphinx3 (unexplained diff vs previous) 15.000 36.36%
482.sphinx3 (unexplained diff vs previous) (percentage) 1.670 39.17%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.500 -
Geometric Mean 4.363 -20.71%