Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=fb0894b2b3e62366b66d6e20f61721c0e9e07263 (+ 28 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/57446d1bc9757ee1fb030600d38fa9487231f2a4 (+ 134 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=1d44530a5be2442e064baa48139adc9fdfb1fc6b (+ 10 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=428d764c681607c504c3274cdf38b863f34e9793 (+ 161 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/57/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0057 (4-changed-components) 2023-06-12T21:44:19 0:00:00
Previous 0056 (3-changed-components) 2023-06-09T18:31:58 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 46 89
Performance Improvements 131 136
Added Tests 4 0
Unchanged Tests 1055 1011
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf 26.80% 944.000 1197.000 - -89.79% -
436.cactusADM 1.11% 2607.000 2636.000 - -89.19% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
454.calculix (unexplained diff vs previous) (percentage) 214.77% 1.490 4.690 - 525.33% -
444.namd (unexplained diff vs previous) (percentage) 209.09% 0.110 0.340 - -93.50% -
444.namd (unexplained diff vs previous) 200.00% 1.000 3.000 - -99.64% -
454.calculix (unexplained diff vs previous) 200.00% 1.000 3.000 - -98.77% -
465.tonto (unexplained diff vs previous) 131.48% 54.000 125.000 - -15.54% -
465.tonto (unexplained diff vs previous) (percentage) 130.95% 0.420 0.970 - -14.16% -
482.sphinx3 (unexplained diff vs previous) (percentage) 125.00% 1.080 2.430 - -20.59% -
482.sphinx3 (unexplained diff vs previous) 120.00% 10.000 22.000 - -98.03% -
454.calculix (max) 100.00% 1.000 2.000 - -99.26% -
454.calculix (max) (percentage) 91.45% 2.340 4.480 - 433.33% -
465.tonto (max) (percentage) 70.18% 0.570 0.970 - -43.27% -
465.tonto (max) 69.86% 73.000 124.000 - -44.39% -
429.mcf (max) 67.54% 191.000 320.000 - 764.86% -
458.sjeng (unexplained diff vs previous) (percentage) 42.11% 0.760 1.080 - 260.00% -
458.sjeng (unexplained diff vs previous) 41.38% 58.000 82.000 - 70.83% -
429.mcf 34.18% 79.000 106.000 - 89.29% -
429.mcf (avg) 34.18% 79.000 106.000 - 89.29% -
429.mcf (max) (percentage) 32.41% 20.240 26.800 - 8275.00% -
444.namd (max) (percentage) 26.92% 0.260 0.330 - -94.01% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 26.90% 1.970 2.500 - 8.70% -
459.GemsFDTD (unexplained diff vs previous) 23.94% 142.000 176.000 - -69.86% -
462.libquantum (max) (percentage) 20.78% 1.540 1.860 - 2225.00% -
445.gobmk (max) (percentage) 20.00% 1.050 1.260 - 366.67% -
445.gobmk (max) 18.87% 53.000 63.000 - 90.91% -
458.sjeng (max) (percentage) 13.83% 0.940 1.070 - 256.67% -
481.wrf (unexplained diff vs previous) (percentage) 12.62% 3.090 3.480 - -14.50% -
458.sjeng (max) 12.50% 72.000 81.000 - 68.75% -
473.astar (unexplained diff vs previous) (percentage) 11.78% 2.970 3.320 - 606.38% -
470.lbm (unexplained diff vs previous) (percentage) 9.22% 1.410 1.540 - -69.44% -
481.wrf (unexplained diff vs previous) 8.63% 197.000 214.000 - -76.06% -
473.astar (unexplained diff vs previous) 8.28% 145.000 157.000 - 157.38% -
470.lbm (unexplained diff vs previous) 7.55% 53.000 57.000 - -93.07% -
464.h264ref (unexplained diff vs previous) (percentage) 7.02% 1.140 1.220 - 221.05% -
429.mcf (unexplained diff vs previous) 6.75% 237.000 253.000 - 2430.00% -
429.mcf (avg) (percentage) 5.93% 8.430 8.930 - 1760.42% -
464.h264ref (unexplained diff vs previous) 5.63% 71.000 75.000 - 29.31% -
481.wrf (max) (percentage) 5.33% 3.190 3.360 - -19.23% -
464.h264ref (max) (percentage) 5.22% 1.150 1.210 - 210.26% -
464.h264ref (max) 4.23% 71.000 74.000 - 25.42% -
435.gromacs (unexplained diff vs previous) (percentage) 3.98% 2.510 2.610 - 50.87% -
434.zeusmp (unexplained diff vs previous) (percentage) 3.70% 1.890 1.960 - -12.89% -
434.zeusmp (unexplained diff vs previous) 1.92% 52.000 53.000 - -88.72% -
435.gromacs (unexplained diff vs previous) 1.60% 187.000 190.000 - -34.93% -
481.wrf (max) 1.48% 203.000 206.000 - -77.46% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
454.calculix -4.48% 67.000 64.000 - -99.80% -
450.soplex -4.40% 591.000 565.000 - -96.40% -
481.wrf -3.36% 6367.000 6153.000 - -72.02% -
473.astar -3.22% 4879.000 4722.000 - -63.63% -
401.bzip2 -2.76% 2938.000 2857.000 - -83.17% -
435.gromacs -2.55% 7460.000 7270.000 - -56.86% -
459.GemsFDTD -2.44% 7208.000 7032.000 - -72.28% -
482.sphinx3 -2.37% 929.000 907.000 - -97.52% -
447.dealII -2.07% 1936.000 1896.000 - -79.17% -
433.milc -2.03% 1133.000 1110.000 - -91.88% -
434.zeusmp -1.93% 2752.000 2699.000 - -87.11% -
462.libquantum -1.86% 161.000 158.000 - -99.07% -
403.gcc -1.59% 63.000 62.000 - -99.30% -
470.lbm -1.52% 3753.000 3696.000 - -77.34% -
445.gobmk -1.26% 5083.000 5019.000 - -59.51% -
464.h264ref -1.21% 6222.000 6147.000 - -59.58% -
471.omnetpp -1.08% 2866.000 2835.000 - -69.40% -
458.sjeng -1.07% 7663.000 7581.000 - -52.67% -
410.bwaves -1.05% 5928.000 5866.000 - -60.22% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
437.leslie3d (unexplained diff vs previous) (percentage) -87.68% 3.490 0.430 - -58.25% -
437.leslie3d (unexplained diff vs previous) -87.56% 394.000 49.000 - -73.80% -
416.gamess (unexplained diff vs previous) (percentage) -61.11% 0.180 0.070 - -97.70% -
471.omnetpp (unexplained diff vs previous) -60.76% 79.000 31.000 - -50.00% -
471.omnetpp (unexplained diff vs previous) (percentage) -60.51% 2.760 1.090 - 62.69% -
416.gamess (unexplained diff vs previous) -57.69% 26.000 11.000 - -98.77% -
410.bwaves (unexplained diff vs previous) -36.73% 98.000 62.000 - -75.88% -
410.bwaves (unexplained diff vs previous) (percentage) -35.76% 1.650 1.060 - -39.08% -
436.cactusADM (unexplained diff vs previous) (percentage) -34.91% 1.690 1.100 - -77.13% -
436.cactusADM (unexplained diff vs previous) -34.09% 44.000 29.000 - -97.53% -
437.leslie3d (max) (percentage) -24.93% 3.370 2.530 - -18.39% -
437.leslie3d (max) -24.74% 380.000 286.000 - -48.93% -
453.povray (unexplained diff vs previous) (percentage) -20.88% 0.910 0.720 - -68.56% -
433.milc (unexplained diff vs previous) -20.69% 29.000 23.000 - -76.04% -
453.povray (unexplained diff vs previous) -20.00% 5.000 4.000 - -97.63% -
433.milc (unexplained diff vs previous) (percentage) -19.14% 2.560 2.070 - 195.71% -
450.soplex (unexplained diff vs previous) -16.13% 31.000 26.000 - -62.32% -
429.mcf (unexplained diff vs previous) (percentage) -15.81% 25.110 21.140 - 23388.89% -
471.omnetpp (max) -13.58% 81.000 70.000 - -59.30% -
450.soplex (unexplained diff vs previous) (percentage) -12.38% 5.250 4.600 - 945.45% -
471.omnetpp (max) (percentage) -12.37% 2.830 2.480 - 33.33% -
410.bwaves (max) -10.58% 104.000 93.000 - -64.23% -
400.perlbench (unexplained diff vs previous) -10.00% 10.000 9.000 - -74.29% -
453.povray (max) (percentage) -10.00% 0.900 0.810 - -68.73% -
410.bwaves (max) (percentage) -9.60% 1.770 1.600 - -9.60% -
400.perlbench (unexplained diff vs previous) (percentage) -9.52% 0.630 0.570 - 83.87% -
401.bzip2 (max) -9.30% 86.000 78.000 - 16.42% -
450.soplex (max) -9.30% 43.000 39.000 - -43.48% -
435.gromacs (max) -8.96% 279.000 254.000 - -14.19% -
482.sphinx3 (max) -8.70% 23.000 21.000 - -98.18% -
436.cactusADM (max) (percentage) -8.43% 1.660 1.520 - -69.90% -
473.astar (max) -7.87% 178.000 164.000 - 168.85% -
433.milc (max) -7.84% 51.000 47.000 - -51.55% -
436.cactusADM (max) -6.98% 43.000 40.000 - -96.75% -
470.lbm (max) -6.49% 77.000 72.000 - -91.94% -
435.gromacs (max) (percentage) -6.42% 3.740 3.500 - 98.86% -
400.perlbench (max) -6.25% 32.000 30.000 - -66.29% -
456.hmmer (max) -6.25% 16.000 15.000 - -84.69% -
456.hmmer (unexplained diff vs previous) -6.25% 16.000 15.000 - -84.69% -
401.bzip2 (max) (percentage) -6.12% 2.940 2.760 - 590.00% -
456.hmmer (max) (percentage) -5.56% 0.360 0.340 - -34.62% -
456.hmmer (unexplained diff vs previous) (percentage) -5.56% 0.360 0.340 - -34.62% -
447.dealII (max) -5.45% 55.000 52.000 - -65.33% -
433.milc (max) (percentage) -4.86% 4.530 4.310 - 507.04% -
416.gamess (max) -4.84% 62.000 59.000 - -93.61% -
470.lbm (max) (percentage) -4.83% 2.070 1.970 - -64.05% -
450.soplex (max) (percentage) -4.78% 7.320 6.970 - 1484.09% -
416.gamess (max) (percentage) -4.76% 0.420 0.400 - -87.26% -
482.sphinx3 (max) (percentage) -4.74% 2.530 2.410 - -23.49% -
473.astar (max) (percentage) -4.64% 3.660 3.490 - 642.55% -
400.perlbench (max) (percentage) -4.46% 2.020 1.930 - 141.25% -
444.namd -4.35% 23.000 22.000 - -94.69% -
444.namd (avg) -4.35% 23.000 22.000 - -94.69% -
447.dealII (max) (percentage) -4.18% 2.870 2.750 - 66.67% -
450.soplex -4.00% 25.000 24.000 - -35.14% -
450.soplex (avg) -4.00% 25.000 24.000 - -35.14% -
459.GemsFDTD (max) -3.93% 178.000 171.000 - -70.42% -
401.bzip2 (unexplained diff vs previous) -3.57% 84.000 81.000 - 17.39% -
482.sphinx3 -3.57% 28.000 27.000 - -96.86% -
482.sphinx3 (avg) -3.57% 28.000 27.000 - -96.86% -
481.wrf -3.54% 198.000 191.000 - -66.19% -
481.wrf (avg) -3.54% 198.000 191.000 - -66.19% -
471.omnetpp -3.26% 92.000 89.000 - -18.35% -
471.omnetpp (avg) -3.26% 92.000 89.000 - -18.35% -
401.bzip2 -3.19% 94.000 91.000 - 102.22% -
401.bzip2 (avg) -3.19% 94.000 91.000 - 102.22% -
459.GemsFDTD -3.16% 190.000 184.000 - -47.73% -
459.GemsFDTD (avg) -3.16% 190.000 184.000 - -47.73% -
433.milc -3.03% 33.000 32.000 - -65.96% -
433.milc (avg) -3.03% 33.000 32.000 - -65.96% -
434.zeusmp -2.82% 71.000 69.000 - -72.94% -
434.zeusmp (avg) -2.82% 71.000 69.000 - -72.94% -
435.gromacs -2.74% 219.000 213.000 - 33.12% -
435.gromacs (avg) -2.74% 219.000 213.000 - 33.12% -
444.namd (avg) (percentage) -2.65% 2.640 2.570 - -1.15% -
470.lbm -2.63% 114.000 111.000 - -81.74% -
470.lbm (avg) -2.63% 114.000 111.000 - -81.74% -
400.perlbench (avg) (percentage) -2.61% 2.680 2.610 - 569.23% -
464.h264ref -2.59% 116.000 113.000 - 91.53% -
464.h264ref (avg) -2.59% 116.000 113.000 - 91.53% -
437.leslie3d (avg) (percentage) -2.50% 3.600 3.510 - 110.18% -
473.astar -2.50% 120.000 117.000 - 129.41% -
473.astar (avg) -2.50% 120.000 117.000 - 129.41% -
416.gamess -2.39% 209.000 204.000 - -57.94% -
416.gamess (avg) -2.39% 209.000 204.000 - -57.94% -
400.perlbench -2.38% 42.000 41.000 - -4.65% -
400.perlbench (avg) -2.38% 42.000 41.000 - -4.65% -
445.gobmk -2.30% 87.000 85.000 - 214.81% -
445.gobmk (avg) -2.30% 87.000 85.000 - 214.81% -
453.povray (avg) (percentage) -2.26% 2.650 2.590 - 69.28% -
410.bwaves -2.22% 135.000 132.000 - -54.33% -
410.bwaves (avg) -2.22% 135.000 132.000 - -54.33% -
437.leslie3d -2.22% 406.000 397.000 - 31.89% -
437.leslie3d (avg) -2.22% 406.000 397.000 - 31.89% -
416.gamess (avg) (percentage) -2.13% 1.410 1.380 - -16.36% -
456.hmmer (avg) (percentage) -2.04% 1.470 1.440 - 380.00% -
447.dealII -1.96% 51.000 50.000 - -36.71% -
447.dealII (avg) -1.96% 51.000 50.000 - -36.71% -
434.zeusmp (max) -1.89% 53.000 52.000 - -89.19% -
458.sjeng -1.75% 114.000 112.000 - 211.11% -
458.sjeng (avg) -1.75% 114.000 112.000 - 211.11% -
403.gcc (max) (percentage) -1.69% 4.150 4.080 - 591.53% -
436.cactusADM (avg) (percentage) -1.61% 2.490 2.450 - 2.08% -
471.omnetpp (avg) (percentage) -1.55% 3.220 3.170 - 168.64% -
456.hmmer -1.54% 65.000 64.000 - 14.29% -
456.hmmer (avg) -1.54% 65.000 64.000 - 14.29% -
410.bwaves (avg) (percentage) -1.31% 2.290 2.260 - 15.31% -
470.lbm (avg) (percentage) -1.31% 3.060 3.020 - -19.03% -
459.GemsFDTD (max) (percentage) -1.21% 2.470 2.440 - 7.02% -
445.gobmk (avg) (percentage) -1.16% 1.730 1.710 - 677.27% -
462.libquantum (avg) (percentage) -1.11% 2.700 2.670 - 1305.26% -
464.h264ref (avg) (percentage) -1.07% 1.870 1.850 - 374.36% -

Added Tests - sample variability
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (unexplained diff vs previous) (percentage) 23388.89% 0.090 21.140 - -15.81% -
429.mcf (max) (percentage) 8275.00% 0.320 26.800 - 32.41% -
429.mcf (unexplained diff vs previous) 2430.00% 10.000 253.000 - 6.75% -
462.libquantum (unexplained diff vs previous) (percentage) 2275.00% 0.080 1.900 - 0.00% -
462.libquantum (max) (percentage) 2225.00% 0.080 1.860 - 20.78% -
429.mcf (avg) (percentage) 1760.42% 0.480 8.930 - 5.93% -
450.soplex (avg) (percentage) 1670.83% 0.240 4.250 - 0.00% -
450.soplex (max) (percentage) 1484.09% 0.440 6.970 - -4.78% -
462.libquantum (avg) (percentage) 1305.26% 0.190 2.670 - -1.11% -
401.bzip2 (avg) (percentage) 1085.19% 0.270 3.200 - -0.31% -
450.soplex (unexplained diff vs previous) (percentage) 945.45% 0.440 4.600 - -12.38% -
429.mcf (max) 764.86% 37.000 320.000 - 67.54% -
445.gobmk (unexplained diff vs previous) (percentage) 753.33% 0.150 1.280 - 0.00% -
445.gobmk (avg) (percentage) 677.27% 0.220 1.710 - -1.16% -
473.astar (max) (percentage) 642.55% 0.470 3.490 - -4.64% -
473.astar (unexplained diff vs previous) (percentage) 606.38% 0.470 3.320 - 11.78% -
401.bzip2 (unexplained diff vs previous) (percentage) 592.68% 0.410 2.840 - -0.70% -
403.gcc (max) (percentage) 591.53% 0.590 4.080 - -1.69% -
401.bzip2 (max) (percentage) 590.00% 0.400 2.760 - -6.12% -
400.perlbench (avg) (percentage) 569.23% 0.390 2.610 - -2.61% -
458.sjeng (avg) (percentage) 547.83% 0.230 1.490 - -0.67% -
454.calculix (unexplained diff vs previous) (percentage) 525.33% 0.750 4.690 - 214.77% -
473.astar (avg) (percentage) 520.00% 0.400 2.480 - 0.81% -
433.milc (max) (percentage) 507.04% 0.710 4.310 - -4.86% -
454.calculix (max) (percentage) 433.33% 0.840 4.480 - 91.45% -
403.gcc (avg) (percentage) 381.40% 0.430 2.070 - 0.00% -
456.hmmer (avg) (percentage) 380.00% 0.300 1.440 - -2.04% -
464.h264ref (avg) (percentage) 374.36% 0.390 1.850 - -1.07% -
445.gobmk (max) (percentage) 366.67% 0.270 1.260 - 20.00% -
433.milc (avg) (percentage) 327.54% 0.690 2.950 - -0.67% -
458.sjeng (unexplained diff vs previous) (percentage) 260.00% 0.300 1.080 - 42.11% -
458.sjeng (max) (percentage) 256.67% 0.300 1.070 - 13.83% -
445.gobmk (unexplained diff vs previous) 255.56% 18.000 64.000 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) 221.05% 0.380 1.220 - 7.02% -
445.gobmk 214.81% 27.000 85.000 - -2.30% -
445.gobmk (avg) 214.81% 27.000 85.000 - -2.30% -
458.sjeng 211.11% 36.000 112.000 - -1.75% -
458.sjeng (avg) 211.11% 36.000 112.000 - -1.75% -
464.h264ref (max) (percentage) 210.26% 0.390 1.210 - 5.22% -
435.gromacs (avg) (percentage) 208.42% 0.950 2.930 - -0.34% -
447.dealII (avg) (percentage) 208.05% 0.870 2.680 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) 195.71% 0.700 2.070 - -19.14% -
473.astar (max) 168.85% 61.000 164.000 - -7.87% -
471.omnetpp (avg) (percentage) 168.64% 1.180 3.170 - -1.55% -
473.astar (unexplained diff vs previous) 157.38% 61.000 157.000 - 8.28% -
400.perlbench (max) (percentage) 141.25% 0.800 1.930 - -4.46% -
473.astar 129.41% 51.000 117.000 - -2.50% -
473.astar (avg) 129.41% 51.000 117.000 - -2.50% -
434.zeusmp (avg) (percentage) 110.66% 1.220 2.570 - -0.77% -
437.leslie3d (avg) (percentage) 110.18% 1.670 3.510 - -2.50% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
401.bzip2 102.22% 45.000 91.000 - -3.19% -
401.bzip2 (avg) 102.22% 45.000 91.000 - -3.19% -
454.calculix (avg) (percentage) 101.97% 2.030 4.100 - 0.24% -
435.gromacs (max) (percentage) 98.86% 1.760 3.500 - -6.42% -
464.h264ref 91.53% 59.000 113.000 - -2.59% -
464.h264ref (avg) 91.53% 59.000 113.000 - -2.59% -
445.gobmk (max) 90.91% 33.000 63.000 - 18.87% -
429.mcf 89.29% 56.000 106.000 - 34.18% -
429.mcf (avg) 89.29% 56.000 106.000 - 34.18% -
459.GemsFDTD (avg) (percentage) 89.21% 1.390 2.630 - -0.38% -
400.perlbench (unexplained diff vs previous) (percentage) 83.87% 0.310 0.570 - -9.52% -
483.xalancbmk (max) (percentage) 79.35% 0.920 1.650 - 0.00% -
458.sjeng (unexplained diff vs previous) 70.83% 48.000 82.000 - 41.38% -
453.povray (avg) (percentage) 69.28% 1.530 2.590 - -2.26% -
458.sjeng (max) 68.75% 48.000 81.000 - 12.50% -
447.dealII (max) (percentage) 66.67% 1.650 2.750 - -4.18% -
471.omnetpp (unexplained diff vs previous) (percentage) 62.69% 0.670 1.090 - -60.51% -
435.gromacs (unexplained diff vs previous) (percentage) 50.87% 1.730 2.610 - 3.98% -
471.omnetpp (max) (percentage) 33.33% 1.860 2.480 - -12.37% -
435.gromacs 33.12% 160.000 213.000 - -2.74% -
435.gromacs (avg) 33.12% 160.000 213.000 - -2.74% -
437.leslie3d 31.89% 301.000 397.000 - -2.22% -
437.leslie3d (avg) 31.89% 301.000 397.000 - -2.22% -
482.sphinx3 (avg) (percentage) 29.79% 2.350 3.050 - -0.65% -
464.h264ref (unexplained diff vs previous) 29.31% 58.000 75.000 - 5.63% -
464.h264ref (max) 25.42% 59.000 74.000 - 4.23% -
481.wrf (avg) (percentage) 21.40% 2.570 3.120 - 0.32% -
401.bzip2 (unexplained diff vs previous) 17.39% 69.000 81.000 - -3.57% -
401.bzip2 (max) 16.42% 67.000 78.000 - -9.30% -
410.bwaves (avg) (percentage) 15.31% 1.960 2.260 - -1.31% -
456.hmmer 14.29% 56.000 64.000 - -1.54% -
456.hmmer (avg) 14.29% 56.000 64.000 - -1.54% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 8.70% 2.300 2.500 - 26.90% -
459.GemsFDTD (max) (percentage) 7.02% 2.280 2.440 - -1.21% -
436.cactusADM (avg) (percentage) 2.08% 2.400 2.450 - -1.61% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -
nb_total_tests -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.80% 32491.000 64.000 - -4.48% -
403.gcc -99.30% 8890.000 62.000 - -1.59% -
462.libquantum -99.07% 17005.000 158.000 - -1.86% -
482.sphinx3 -97.52% 36586.000 907.000 - -2.37% -
450.soplex -96.40% 15709.000 565.000 - -4.40% -
444.namd -94.40% 15957.000 894.000 - -0.33% -
453.povray -92.49% 7395.000 555.000 - 0.73% -
433.milc -91.88% 13668.000 1110.000 - -2.03% -
429.mcf -89.79% 11727.000 1197.000 - 26.80% -
436.cactusADM -89.19% 24396.000 2636.000 - 1.11% -
434.zeusmp -87.11% 20932.000 2699.000 - -1.93% -
400.perlbench -85.92% 11215.000 1579.000 - -0.57% -
401.bzip2 -83.17% 16971.000 2857.000 - -2.76% -
447.dealII -79.17% 9101.000 1896.000 - -2.07% -
470.lbm -77.34% 16312.000 3696.000 - -1.52% -
456.hmmer -76.53% 18995.000 4459.000 - -0.34% -
459.GemsFDTD -72.28% 25364.000 7032.000 - -2.44% -
481.wrf -72.02% 21991.000 6153.000 - -3.36% -
471.omnetpp -69.40% 9265.000 2835.000 - -1.08% -
473.astar -63.63% 12983.000 4722.000 - -3.22% -
410.bwaves -60.22% 14745.000 5866.000 - -1.05% -
464.h264ref -59.58% 15208.000 6147.000 - -1.21% -
445.gobmk -59.51% 12396.000 5019.000 - -1.26% -
435.gromacs -56.86% 16853.000 7270.000 - -2.55% -
458.sjeng -52.67% 16018.000 7581.000 - -1.07% -
416.gamess -49.54% 29409.000 14841.000 - 0.07% -
437.leslie3d -37.26% 18070.000 11338.000 - 0.43% -
465.tonto -1.69% 13044.000 12824.000 - -0.97% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.77% 879.000 2.000 - 0.00% -
454.calculix -99.70% 659.000 2.000 - 0.00% -
454.calculix (avg) -99.70% 659.000 2.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.64% 834.000 3.000 - 200.00% -
454.calculix (max) -99.26% 272.000 2.000 - 100.00% -
416.gamess (unexplained diff vs previous) -98.77% 896.000 11.000 - -57.69% -
454.calculix (unexplained diff vs previous) -98.77% 244.000 3.000 - 200.00% -
482.sphinx3 (max) -98.18% 1152.000 21.000 - -8.70% -
482.sphinx3 (unexplained diff vs previous) -98.03% 1118.000 22.000 - 120.00% -
453.povray (max) -97.91% 191.000 4.000 - 0.00% -
416.gamess (unexplained diff vs previous) (percentage) -97.70% 3.050 0.070 - -61.11% -
453.povray (unexplained diff vs previous) -97.63% 169.000 4.000 - -20.00% -
436.cactusADM (unexplained diff vs previous) -97.53% 1173.000 29.000 - -34.09% -
403.gcc -97.37% 38.000 1.000 - 0.00% -
403.gcc (avg) -97.37% 38.000 1.000 - 0.00% -
482.sphinx3 -96.86% 859.000 27.000 - -3.57% -
482.sphinx3 (avg) -96.86% 859.000 27.000 - -3.57% -
436.cactusADM (max) -96.75% 1231.000 40.000 - -6.98% -
403.gcc (max) -96.15% 52.000 2.000 - 0.00% -
444.namd -94.69% 414.000 22.000 - -4.35% -
444.namd (avg) -94.69% 414.000 22.000 - -4.35% -
444.namd (max) (percentage) -94.01% 5.510 0.330 - 26.92% -
416.gamess (max) -93.61% 923.000 59.000 - -4.84% -
444.namd (unexplained diff vs previous) (percentage) -93.50% 5.230 0.340 - 209.09% -
470.lbm (unexplained diff vs previous) -93.07% 822.000 57.000 - 7.55% -
470.lbm (max) -91.94% 893.000 72.000 - -6.49% -
434.zeusmp (max) -89.19% 481.000 52.000 - -1.89% -
436.cactusADM -89.06% 585.000 64.000 - 0.00% -
436.cactusADM (avg) -89.06% 585.000 64.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) -88.72% 470.000 53.000 - 1.92% -
453.povray -87.61% 113.000 14.000 - 0.00% -
453.povray (avg) -87.61% 113.000 14.000 - 0.00% -
462.libquantum -87.50% 32.000 4.000 - 0.00% -
462.libquantum (avg) -87.50% 32.000 4.000 - 0.00% -
416.gamess (max) (percentage) -87.26% 3.140 0.400 - -4.76% -
456.hmmer (max) -84.69% 98.000 15.000 - -6.25% -
456.hmmer (unexplained diff vs previous) -84.69% 98.000 15.000 - -6.25% -
462.libquantum (max) -84.62% 13.000 2.000 - 0.00% -
470.lbm -81.74% 608.000 111.000 - -2.63% -
470.lbm (avg) -81.74% 608.000 111.000 - -2.63% -
462.libquantum (unexplained diff vs previous) -78.57% 14.000 3.000 - 0.00% -
481.wrf (max) -77.46% 914.000 206.000 - 1.48% -
436.cactusADM (unexplained diff vs previous) (percentage) -77.13% 4.810 1.100 - -34.91% -
481.wrf (unexplained diff vs previous) -76.06% 894.000 214.000 - 8.63% -
433.milc (unexplained diff vs previous) -76.04% 96.000 23.000 - -20.69% -
410.bwaves (unexplained diff vs previous) -75.88% 257.000 62.000 - -36.73% -
400.perlbench (unexplained diff vs previous) -74.29% 35.000 9.000 - -10.00% -
437.leslie3d (unexplained diff vs previous) -73.80% 187.000 49.000 - -87.56% -
434.zeusmp -72.94% 255.000 69.000 - -2.82% -
434.zeusmp (avg) -72.94% 255.000 69.000 - -2.82% -
459.GemsFDTD (max) -70.42% 578.000 171.000 - -3.93% -
436.cactusADM (max) (percentage) -69.90% 5.050 1.520 - -8.43% -
459.GemsFDTD (unexplained diff vs previous) -69.86% 584.000 176.000 - 23.94% -
470.lbm (unexplained diff vs previous) (percentage) -69.44% 5.040 1.540 - 9.22% -
453.povray (max) (percentage) -68.73% 2.590 0.810 - -10.00% -
453.povray (unexplained diff vs previous) (percentage) -68.56% 2.290 0.720 - -20.88% -
400.perlbench (max) -66.29% 89.000 30.000 - -6.25% -
481.wrf -66.19% 565.000 191.000 - -3.54% -
481.wrf (avg) -66.19% 565.000 191.000 - -3.54% -
433.milc -65.96% 94.000 32.000 - -3.03% -
433.milc (avg) -65.96% 94.000 32.000 - -3.03% -
447.dealII (max) -65.33% 150.000 52.000 - -5.45% -
410.bwaves (max) -64.23% 260.000 93.000 - -10.58% -
470.lbm (max) (percentage) -64.05% 5.480 1.970 - -4.83% -
450.soplex (unexplained diff vs previous) -62.32% 69.000 26.000 - -16.13% -
471.omnetpp (max) -59.30% 172.000 70.000 - -13.58% -
437.leslie3d (unexplained diff vs previous) (percentage) -58.25% 1.030 0.430 - -87.68% -
416.gamess -57.94% 485.000 204.000 - -2.39% -
416.gamess (avg) -57.94% 485.000 204.000 - -2.39% -
410.bwaves -54.33% 289.000 132.000 - -2.22% -
410.bwaves (avg) -54.33% 289.000 132.000 - -2.22% -
433.milc (max) -51.55% 97.000 47.000 - -7.84% -
471.omnetpp (unexplained diff vs previous) -50.00% 62.000 31.000 - -60.76% -
437.leslie3d (max) -48.93% 560.000 286.000 - -24.74% -
459.GemsFDTD -47.73% 352.000 184.000 - -3.16% -
459.GemsFDTD (avg) -47.73% 352.000 184.000 - -3.16% -
465.tonto (max) -44.39% 223.000 124.000 - 69.86% -
450.soplex (max) -43.48% 69.000 39.000 - -9.30% -
465.tonto (max) (percentage) -43.27% 1.710 0.970 - 70.18% -
410.bwaves (unexplained diff vs previous) (percentage) -39.08% 1.740 1.060 - -35.76% -
447.dealII -36.71% 79.000 50.000 - -1.96% -
447.dealII (avg) -36.71% 79.000 50.000 - -1.96% -
450.soplex -35.14% 37.000 24.000 - -4.00% -
450.soplex (avg) -35.14% 37.000 24.000 - -4.00% -
435.gromacs (unexplained diff vs previous) -34.93% 292.000 190.000 - 1.60% -
456.hmmer (max) (percentage) -34.62% 0.520 0.340 - -5.56% -
456.hmmer (unexplained diff vs previous) (percentage) -34.62% 0.520 0.340 - -5.56% -
482.sphinx3 (max) (percentage) -23.49% 3.150 2.410 - -4.74% -
482.sphinx3 (unexplained diff vs previous) (percentage) -20.59% 3.060 2.430 - 125.00% -
481.wrf (max) (percentage) -19.23% 4.160 3.360 - 5.33% -
470.lbm (avg) (percentage) -19.03% 3.730 3.020 - -1.31% -
437.leslie3d (max) (percentage) -18.39% 3.100 2.530 - -24.93% -
471.omnetpp -18.35% 109.000 89.000 - -3.26% -
471.omnetpp (avg) -18.35% 109.000 89.000 - -3.26% -
416.gamess (avg) (percentage) -16.36% 1.650 1.380 - -2.13% -
434.zeusmp (max) (percentage) -16.09% 2.300 1.930 - 0.00% -
465.tonto -15.54% 148.000 125.000 - -0.79% -
465.tonto (avg) -15.54% 148.000 125.000 - -0.79% -
465.tonto (unexplained diff vs previous) -15.54% 148.000 125.000 - 131.48% -
481.wrf (unexplained diff vs previous) (percentage) -14.50% 4.070 3.480 - 12.62% -
435.gromacs (max) -14.19% 296.000 254.000 - -8.96% -
465.tonto (unexplained diff vs previous) (percentage) -14.16% 1.130 0.970 - 130.95% -
465.tonto (avg) (percentage) -14.04% 1.140 0.980 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -12.89% 2.250 1.960 - 3.70% -
410.bwaves (max) (percentage) -9.60% 1.770 1.600 - -9.60% -
400.perlbench -4.65% 43.000 41.000 - -2.38% -
400.perlbench (avg) -4.65% 43.000 41.000 - -2.38% -
444.namd (avg) (percentage) -1.15% 2.600 2.570 - -2.65% -

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1579.000 -
401.bzip2 2857.000 -2.76%
403.gcc 62.000 -1.59%
410.bwaves 5866.000 -1.05%
416.gamess 14841.000 -
429.mcf 1197.000 26.80%
433.milc 1110.000 -2.03%
434.zeusmp 2699.000 -1.93%
435.gromacs 7270.000 -2.55%
436.cactusADM 2636.000 1.11%
437.leslie3d 11338.000 -
444.namd 894.000 -
445.gobmk 5019.000 -1.26%
447.dealII 1896.000 -2.07%
450.soplex 565.000 -4.40%
453.povray 555.000 -
454.calculix 64.000 -4.48%
456.hmmer 4459.000 -
458.sjeng 7581.000 -1.07%
459.GemsFDTD 7032.000 -2.44%
462.libquantum 158.000 -1.86%
464.h264ref 6147.000 -1.21%
465.tonto 12824.000 -
470.lbm 3696.000 -1.52%
471.omnetpp 2835.000 -1.08%
473.astar 4722.000 -3.22%
481.wrf 6153.000 -3.36%
482.sphinx3 907.000 -2.37%
Geometric Mean 2170.368 -
Code Size (bytes)
Name Current %
400.perlbench 563490.000 -
401.bzip2 33652.000 -
403.gcc 1589658.000 -
410.bwaves 14722.000 -
416.gamess 4552138.000 -
429.mcf 7709.000 -
433.milc 47469.000 -
434.zeusmp 140944.000 -
435.gromacs 393954.000 -
436.cactusADM 256928.000 -
437.leslie3d 48001.000 -
444.namd 162594.000 -
445.gobmk 1088104.000 -
447.dealII 191598.000 -
450.soplex 135630.000 -
453.povray 488585.000 -
454.calculix 682263.000 -
456.hmmer 74921.000 -
458.sjeng 81181.000 -
459.GemsFDTD 196030.000 -
462.libquantum 13082.000 -
464.h264ref 301670.000 -
465.tonto 1435576.000 -
470.lbm 10845.000 -
471.omnetpp 296211.000 -
473.astar 15549.000 -
481.wrf 1539936.000 -
482.sphinx3 80066.000 -
Geometric Mean 157305.838 -
sample variability
Name Current %
400.perlbench 41.000 -2.38%
400.perlbench (avg) 41.000 -2.38%
400.perlbench (avg) (percentage) 2.610 -2.61%
400.perlbench (max) 30.000 -6.25%
400.perlbench (max) (percentage) 1.930 -4.46%
400.perlbench (unexplained diff vs previous) 9.000 -10.00%
400.perlbench (unexplained diff vs previous) (percentage) 0.570 -9.52%
401.bzip2 91.000 -3.19%
401.bzip2 (avg) 91.000 -3.19%
401.bzip2 (avg) (percentage) 3.200 -
401.bzip2 (max) 78.000 -9.30%
401.bzip2 (max) (percentage) 2.760 -6.12%
401.bzip2 (unexplained diff vs previous) 81.000 -3.57%
401.bzip2 (unexplained diff vs previous) (percentage) 2.840 -
403.gcc 1.000 -
403.gcc (avg) 1.000 -
403.gcc (avg) (percentage) 2.070 -
403.gcc (max) 2.000 -
403.gcc (max) (percentage) 4.080 -1.69%
410.bwaves 132.000 -2.22%
410.bwaves (avg) 132.000 -2.22%
410.bwaves (avg) (percentage) 2.260 -1.31%
410.bwaves (max) 93.000 -10.58%
410.bwaves (max) (percentage) 1.600 -9.60%
410.bwaves (unexplained diff vs previous) 62.000 -36.73%
410.bwaves (unexplained diff vs previous) (percentage) 1.060 -35.76%
416.gamess 204.000 -2.39%
416.gamess (avg) 204.000 -2.39%
416.gamess (avg) (percentage) 1.380 -2.13%
416.gamess (max) 59.000 -4.84%
416.gamess (max) (percentage) 0.400 -4.76%
416.gamess (unexplained diff vs previous) 11.000 -57.69%
416.gamess (unexplained diff vs previous) (percentage) 0.070 -61.11%
429.mcf 106.000 34.18%
429.mcf (avg) 106.000 34.18%
429.mcf (avg) (percentage) 8.930 5.93%
429.mcf (max) 320.000 67.54%
429.mcf (max) (percentage) 26.800 32.41%
429.mcf (unexplained diff vs previous) 253.000 6.75%
429.mcf (unexplained diff vs previous) (percentage) 21.140 -15.81%
433.milc 32.000 -3.03%
433.milc (avg) 32.000 -3.03%
433.milc (avg) (percentage) 2.950 -
433.milc (max) 47.000 -7.84%
433.milc (max) (percentage) 4.310 -4.86%
433.milc (unexplained diff vs previous) 23.000 -20.69%
433.milc (unexplained diff vs previous) (percentage) 2.070 -19.14%
434.zeusmp 69.000 -2.82%
434.zeusmp (avg) 69.000 -2.82%
434.zeusmp (avg) (percentage) 2.570 -
434.zeusmp (max) 52.000 -1.89%
434.zeusmp (max) (percentage) 1.930 -
434.zeusmp (unexplained diff vs previous) 53.000 1.92%
434.zeusmp (unexplained diff vs previous) (percentage) 1.960 3.70%
435.gromacs 213.000 -2.74%
435.gromacs (avg) 213.000 -2.74%
435.gromacs (avg) (percentage) 2.930 -
435.gromacs (max) 254.000 -8.96%
435.gromacs (max) (percentage) 3.500 -6.42%
435.gromacs (unexplained diff vs previous) 190.000 1.60%
435.gromacs (unexplained diff vs previous) (percentage) 2.610 3.98%
436.cactusADM 64.000 -
436.cactusADM (avg) 64.000 -
436.cactusADM (avg) (percentage) 2.450 -1.61%
436.cactusADM (max) 40.000 -6.98%
436.cactusADM (max) (percentage) 1.520 -8.43%
436.cactusADM (unexplained diff vs previous) 29.000 -34.09%
436.cactusADM (unexplained diff vs previous) (percentage) 1.100 -34.91%
437.leslie3d 397.000 -2.22%
437.leslie3d (avg) 397.000 -2.22%
437.leslie3d (avg) (percentage) 3.510 -2.50%
437.leslie3d (max) 286.000 -24.74%
437.leslie3d (max) (percentage) 2.530 -24.93%
437.leslie3d (unexplained diff vs previous) 49.000 -87.56%
437.leslie3d (unexplained diff vs previous) (percentage) 0.430 -87.68%
444.namd 22.000 -4.35%
444.namd (avg) 22.000 -4.35%
444.namd (avg) (percentage) 2.570 -2.65%
444.namd (max) 2.000 -
444.namd (max) (percentage) 0.330 26.92%
444.namd (unexplained diff vs previous) 3.000 200.00%
444.namd (unexplained diff vs previous) (percentage) 0.340 209.09%
445.gobmk 85.000 -2.30%
445.gobmk (avg) 85.000 -2.30%
445.gobmk (avg) (percentage) 1.710 -1.16%
445.gobmk (max) 63.000 18.87%
445.gobmk (max) (percentage) 1.260 20.00%
445.gobmk (unexplained diff vs previous) 64.000 -
445.gobmk (unexplained diff vs previous) (percentage) 1.280 -
447.dealII 50.000 -1.96%
447.dealII (avg) 50.000 -1.96%
447.dealII (avg) (percentage) 2.680 -
447.dealII (max) 52.000 -5.45%
447.dealII (max) (percentage) 2.750 -4.18%
450.soplex 24.000 -4.00%
450.soplex (avg) 24.000 -4.00%
450.soplex (avg) (percentage) 4.250 -
450.soplex (max) 39.000 -9.30%
450.soplex (max) (percentage) 6.970 -4.78%
450.soplex (unexplained diff vs previous) 26.000 -16.13%
450.soplex (unexplained diff vs previous) (percentage) 4.600 -12.38%
453.povray 14.000 -
453.povray (avg) 14.000 -
453.povray (avg) (percentage) 2.590 -2.26%
453.povray (max) 4.000 -
453.povray (max) (percentage) 0.810 -10.00%
453.povray (unexplained diff vs previous) 4.000 -20.00%
453.povray (unexplained diff vs previous) (percentage) 0.720 -20.88%
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 4.100 -
454.calculix (max) 2.000 100.00%
454.calculix (max) (percentage) 4.480 91.45%
454.calculix (unexplained diff vs previous) 3.000 200.00%
454.calculix (unexplained diff vs previous) (percentage) 4.690 214.77%
456.hmmer 64.000 -1.54%
456.hmmer (avg) 64.000 -1.54%
456.hmmer (avg) (percentage) 1.440 -2.04%
456.hmmer (max) 15.000 -6.25%
456.hmmer (max) (percentage) 0.340 -5.56%
456.hmmer (unexplained diff vs previous) 15.000 -6.25%
456.hmmer (unexplained diff vs previous) (percentage) 0.340 -5.56%
458.sjeng 112.000 -1.75%
458.sjeng (avg) 112.000 -1.75%
458.sjeng (avg) (percentage) 1.490 -
458.sjeng (max) 81.000 12.50%
458.sjeng (max) (percentage) 1.070 13.83%
458.sjeng (unexplained diff vs previous) 82.000 41.38%
458.sjeng (unexplained diff vs previous) (percentage) 1.080 42.11%
459.GemsFDTD 184.000 -3.16%
459.GemsFDTD (avg) 184.000 -3.16%
459.GemsFDTD (avg) (percentage) 2.630 -
459.GemsFDTD (max) 171.000 -3.93%
459.GemsFDTD (max) (percentage) 2.440 -1.21%
459.GemsFDTD (unexplained diff vs previous) 176.000 23.94%
459.GemsFDTD (unexplained diff vs previous) (percentage) 2.500 26.90%
462.libquantum 4.000 -
462.libquantum (avg) 4.000 -
462.libquantum (avg) (percentage) 2.670 -1.11%
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.860 20.78%
462.libquantum (unexplained diff vs previous) 3.000 -
462.libquantum (unexplained diff vs previous) (percentage) 1.900 -
464.h264ref 113.000 -2.59%
464.h264ref (avg) 113.000 -2.59%
464.h264ref (avg) (percentage) 1.850 -1.07%
464.h264ref (max) 74.000 4.23%
464.h264ref (max) (percentage) 1.210 5.22%
464.h264ref (unexplained diff vs previous) 75.000 5.63%
464.h264ref (unexplained diff vs previous) (percentage) 1.220 7.02%
465.tonto 125.000 -
465.tonto (avg) 125.000 -
465.tonto (avg) (percentage) 0.980 -
465.tonto (max) 124.000 69.86%
465.tonto (max) (percentage) 0.970 70.18%
465.tonto (unexplained diff vs previous) 125.000 131.48%
465.tonto (unexplained diff vs previous) (percentage) 0.970 130.95%
470.lbm 111.000 -2.63%
470.lbm (avg) 111.000 -2.63%
470.lbm (avg) (percentage) 3.020 -1.31%
470.lbm (max) 72.000 -6.49%
470.lbm (max) (percentage) 1.970 -4.83%
470.lbm (unexplained diff vs previous) 57.000 7.55%
470.lbm (unexplained diff vs previous) (percentage) 1.540 9.22%
471.omnetpp 89.000 -3.26%
471.omnetpp (avg) 89.000 -3.26%
471.omnetpp (avg) (percentage) 3.170 -1.55%
471.omnetpp (max) 70.000 -13.58%
471.omnetpp (max) (percentage) 2.480 -12.37%
471.omnetpp (unexplained diff vs previous) 31.000 -60.76%
471.omnetpp (unexplained diff vs previous) (percentage) 1.090 -60.51%
473.astar 117.000 -2.50%
473.astar (avg) 117.000 -2.50%
473.astar (avg) (percentage) 2.480 -
473.astar (max) 164.000 -7.87%
473.astar (max) (percentage) 3.490 -4.64%
473.astar (unexplained diff vs previous) 157.000 8.28%
473.astar (unexplained diff vs previous) (percentage) 3.320 11.78%
481.wrf 191.000 -3.54%
481.wrf (avg) 191.000 -3.54%
481.wrf (avg) (percentage) 3.120 -
481.wrf (max) 206.000 1.48%
481.wrf (max) (percentage) 3.360 5.33%
481.wrf (unexplained diff vs previous) 214.000 8.63%
481.wrf (unexplained diff vs previous) (percentage) 3.480 12.62%
482.sphinx3 27.000 -3.57%
482.sphinx3 (avg) 27.000 -3.57%
482.sphinx3 (avg) (percentage) 3.050 -
482.sphinx3 (max) 21.000 -8.70%
482.sphinx3 (max) (percentage) 2.410 -4.74%
482.sphinx3 (unexplained diff vs previous) 22.000 120.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 2.430 125.00%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.650 -
Geometric Mean 11.640 -2.35%