Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=936f94cf5fd29f8642cab4f1b03b711b9cab699f (+ 14 commits)
git_gcc https://gcc.gnu.org/git/gcc.git (e1009b3de2d05782ae1e0c62f9e81da14c4d6156) (+ 79 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=3c2b9dc41cd05da055fae6f793a355063156bdf3 (+ 6 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/11747c52c04a32dd5e83ebffaf8030e0155c6ee7 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/332/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0332 (3-changed-components) 2024-11-28T16:21:54 0:00:00
Previous 0332 (-00000-) 2022-11-14T03:13:22 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 54 12
Performance Improvements 141 178
Added Tests 2 2
Unchanged Tests 1039 1044
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
465.tonto 1.18% 1948611.000 1971631.000 - 1.31% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 31.91% 8437.000 11129.000 - 1.72% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
453.povray (unexplained diff vs previous) (percentage) 2175.00% 0.080 1.820 - -99.83% -
445.gobmk (unexplained diff vs previous) (percentage) 1850.00% 0.020 0.390 - -99.65% -
429.mcf (avg) (percentage) 1800.00% 0.250 4.750 - -38.07% -
403.gcc (avg) (percentage) 1397.06% 0.340 5.090 - -30.46% -
401.bzip2 (unexplained diff vs previous) (percentage) 1350.00% 0.040 0.580 - -99.84% -
473.astar (avg) (percentage) 881.25% 0.160 1.570 - -62.26% -
445.gobmk (unexplained diff vs previous) 850.00% 2.000 19.000 - -99.66% -
450.soplex (unexplained diff vs previous) (percentage) 720.00% 0.250 2.050 - -99.92% -
465.tonto (unexplained diff vs previous) 657.14% 7.000 53.000 - -97.01% -
462.libquantum (max) (percentage) 525.53% 0.470 2.940 - -97.03% -
462.libquantum (unexplained diff vs previous) (percentage) 508.51% 0.470 2.860 - -99.97% -
465.tonto (unexplained diff vs previous) (percentage) 500.00% 0.080 0.480 - -97.04% -
403.gcc (max) (percentage) 459.70% 0.670 3.750 - -96.23% -
462.libquantum (avg) (percentage) 327.91% 0.430 1.840 - -75.98% -
429.mcf (max) (percentage) 313.16% 0.380 1.570 - -98.31% -
429.mcf (unexplained diff vs previous) (percentage) 305.26% 0.380 1.540 - -99.88% -
473.astar 263.64% 22.000 80.000 - -62.44% -
473.astar (avg) 263.64% 22.000 80.000 - -62.44% -
483.xalancbmk (avg) (percentage) 236.36% 0.880 2.960 - -54.46% -
401.bzip2 (unexplained diff vs previous) 200.00% 6.000 18.000 - -99.83% -
458.sjeng (avg) (percentage) 195.00% 0.200 0.590 - -81.50% -
454.calculix (avg) (percentage) 188.00% 1.250 3.600 - -55.11% -
401.bzip2 (avg) (percentage) 179.59% 0.490 1.370 - -75.00% -
450.soplex (avg) (percentage) 131.43% 1.050 2.430 - -62.21% -
464.h264ref (avg) (percentage) 129.03% 0.310 0.710 - -86.05% -
429.mcf 127.91% 43.000 98.000 - -10.09% -
429.mcf (avg) 127.91% 43.000 98.000 - -10.09% -
444.namd (unexplained diff vs previous) (percentage) 109.30% 0.430 0.900 - -99.92% -
453.povray (unexplained diff vs previous) 100.00% 5.000 10.000 - -99.82% -
483.xalancbmk (max) (percentage) 96.67% 0.900 1.770 - -97.42% -
437.leslie3d (avg) (percentage) 80.08% 2.460 4.430 - 45.25% -
458.sjeng (max) (percentage) 72.73% 0.550 0.950 - -98.00% -
465.tonto 70.00% 80.000 136.000 - -62.74% -
465.tonto (avg) 70.00% 80.000 136.000 - -62.74% -
471.omnetpp (avg) (percentage) 69.42% 1.210 2.050 - -75.00% -
459.GemsFDTD (max) (percentage) 57.14% 1.540 2.420 - -96.61% -
458.sjeng 53.57% 28.000 43.000 - -82.16% -
458.sjeng (avg) 53.57% 28.000 43.000 - -82.16% -
450.soplex (max) (percentage) 48.51% 2.020 3.000 - -96.88% -
445.gobmk (avg) (percentage) 48.28% 0.580 0.860 - -78.01% -
435.gromacs (unexplained diff vs previous) (percentage) 41.38% 0.290 0.410 - -99.58% -
437.leslie3d 38.90% 347.000 482.000 - 35.77% -
437.leslie3d (avg) 38.90% 347.000 482.000 - 35.77% -
447.dealII (avg) (percentage) 35.29% 0.850 1.150 - -80.07% -
465.tonto (avg) (percentage) 29.47% 0.950 1.230 - -63.17% -
459.GemsFDTD (avg) (percentage) 25.74% 1.010 1.270 - -78.25% -
435.gromacs (avg) (percentage) 20.00% 1.450 1.740 - -65.61% -
401.bzip2 (max) (percentage) 7.25% 0.690 0.740 - -99.06% -
453.povray (avg) (percentage) 3.36% 1.490 1.540 - -80.28% -
400.perlbench (avg) (percentage) 1.57% 1.910 1.940 - -69.45% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -8.61% 59491.000 54370.000 - -8.59% -
434.zeusmp -4.44% 153713.000 146883.000 - -3.39% -
459.GemsFDTD -3.42% 245040.000 236656.000 - -3.30% -
447.dealII -2.00% 312518.000 306263.000 - -1.71% -
462.libquantum -1.99% 16117.000 15797.000 - -1.79% -
410.bwaves -1.70% 16748.000 16464.000 - -1.77% -
458.sjeng -1.63% 104004.000 102311.000 - -1.67% -
470.lbm -1.45% 9687.000 9547.000 - -1.49% -
435.gromacs -1.31% 500327.000 493760.000 - -1.10% -
450.soplex -1.30% 205068.000 202406.000 - -1.30% -
433.milc -1.21% 61222.000 60484.000 - -1.46% -
453.povray -1.20% 657755.000 649833.000 - -1.75% -
401.bzip2 -1.19% 44241.000 43715.000 - -1.17% -
444.namd -1.08% 188365.000 186336.000 - -1.09% -
471.omnetpp -1.07% 480176.000 475044.000 - -0.62% -
482.sphinx3 -1.02% 112839.000 111686.000 - -1.04% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.79% 11576.000 140.000 - 0.00% -
482.sphinx3 -97.24% 30488.000 841.000 - -11.01% -
450.soplex -96.29% 15769.000 585.000 - 3.17% -
453.povray -91.59% 6531.000 549.000 - 0.37% -
444.namd -91.37% 8990.000 776.000 - 3.88% -
433.milc -91.11% 13030.000 1158.000 - 0.61% -
429.mcf -87.98% 17253.000 2073.000 - 45.58% -
434.zeusmp -82.74% 13404.000 2314.000 - -7.96% -
447.dealII -82.03% 9046.000 1626.000 - -5.68% -
401.bzip2 -79.18% 14907.000 3104.000 - 5.54% -
459.GemsFDTD -77.53% 23720.000 5329.000 - -13.67% -
471.omnetpp -75.06% 10781.000 2689.000 - -1.28% -
470.lbm -70.66% 12849.000 3770.000 - -0.48% -
473.astar -64.12% 14246.000 5112.000 - -0.31% -
445.gobmk -57.29% 11521.000 4921.000 - -2.03% -
410.bwaves -56.82% 13732.000 5930.000 - 4.92% -
435.gromacs -53.53% 15269.000 7095.000 - -2.39% -
458.sjeng -48.50% 14218.000 7322.000 - -3.24% -
437.leslie3d -22.88% 14128.000 10895.000 - -6.40% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (max) -98.69% 993.000 13.000 - -98.57% -
444.namd (max) -98.09% 629.000 12.000 - -98.25% -
482.sphinx3 (unexplained diff vs previous) -97.89% 570.000 12.000 - -99.95% -
482.sphinx3 -97.62% 588.000 14.000 - -81.08% -
482.sphinx3 (avg) -97.62% 588.000 14.000 - -81.08% -
444.namd -97.49% 358.000 9.000 - -86.76% -
444.namd (avg) -97.49% 358.000 9.000 - -86.76% -
471.omnetpp (unexplained diff vs previous) -96.91% 97.000 3.000 - -99.97% -
433.milc (unexplained diff vs previous) -96.60% 206.000 7.000 - -99.94% -
410.bwaves (max) -96.34% 573.000 21.000 - -99.36% -
462.libquantum -95.92% 49.000 2.000 - -80.00% -
462.libquantum (avg) -95.92% 49.000 2.000 - -80.00% -
470.lbm (max) -95.51% 646.000 29.000 - -98.94% -
453.povray (max) -94.92% 197.000 10.000 - -98.00% -
450.soplex (max) -94.65% 318.000 17.000 - -96.88% -
433.milc -94.31% 246.000 14.000 - -79.71% -
433.milc (avg) -94.31% 246.000 14.000 - -79.71% -
434.zeusmp (max) -93.93% 280.000 17.000 - -99.17% -
470.lbm (unexplained diff vs previous) -93.33% 45.000 3.000 - -99.97% -
447.dealII (max) -93.23% 133.000 9.000 - -99.35% -
462.libquantum (max) -92.59% 54.000 4.000 - -97.10% -
462.libquantum (unexplained diff vs previous) -92.59% 54.000 4.000 - -99.97% -
453.povray -91.75% 97.000 8.000 - -80.95% -
453.povray (avg) -91.75% 97.000 8.000 - -80.95% -
433.milc (max) -91.69% 325.000 27.000 - -97.41% -
450.soplex -91.52% 165.000 14.000 - -61.11% -
450.soplex (avg) -91.52% 165.000 14.000 - -61.11% -
410.bwaves (max) (percentage) -91.39% 4.180 0.360 - -99.38% -
434.zeusmp -89.39% 198.000 21.000 - -85.71% -
434.zeusmp (avg) -89.39% 198.000 21.000 - -85.71% -
470.lbm -88.76% 267.000 30.000 - -87.45% -
470.lbm (avg) -88.76% 267.000 30.000 - -87.45% -
471.omnetpp (unexplained diff vs previous) (percentage) -87.78% 0.900 0.110 - -99.97% -
416.gamess (max) (percentage) -87.71% 3.010 0.370 - -99.04% -
471.omnetpp (max) -85.86% 198.000 28.000 - -98.71% -
447.dealII (unexplained diff vs previous) -85.51% 69.000 10.000 - -99.86% -
445.gobmk (max) -84.80% 125.000 19.000 - -99.28% -
470.lbm (max) (percentage) -84.29% 5.030 0.790 - -98.91% -
435.gromacs (max) -82.64% 363.000 63.000 - -98.24% -
459.GemsFDTD (unexplained diff vs previous) -82.20% 118.000 21.000 - -99.86% -
444.namd (unexplained diff vs previous) -82.05% 39.000 7.000 - -99.92% -
410.bwaves -82.01% 278.000 50.000 - -79.92% -
410.bwaves (avg) -82.01% 278.000 50.000 - -79.92% -
410.bwaves (unexplained diff vs previous) -81.37% 102.000 19.000 - -99.76% -
481.wrf (max) (percentage) -81.04% 4.220 0.800 - -98.84% -
401.bzip2 (max) -78.43% 102.000 22.000 - -99.05% -
444.namd (max) (percentage) -77.86% 7.000 1.550 - -98.31% -
470.lbm (unexplained diff vs previous) (percentage) -77.14% 0.350 0.080 - -99.97% -
447.dealII -76.32% 76.000 18.000 - -81.82% -
447.dealII (avg) -76.32% 76.000 18.000 - -81.82% -
454.calculix (max) (percentage) -75.95% 3.410 0.820 - -99.18% -
458.sjeng (unexplained diff vs previous) -73.91% 23.000 6.000 - -99.91% -
459.GemsFDTD -71.97% 239.000 67.000 - -81.39% -
459.GemsFDTD (avg) -71.97% 239.000 67.000 - -81.39% -
444.namd (avg) (percentage) -70.68% 3.990 1.170 - -87.31% -
473.astar (unexplained diff vs previous) -70.18% 57.000 17.000 - -99.83% -
450.soplex (unexplained diff vs previous) -69.23% 39.000 12.000 - -99.92% -
473.astar (max) -66.07% 56.000 19.000 - -99.44% -
459.GemsFDTD (max) -64.93% 365.000 128.000 - -97.10% -
445.gobmk (max) (percentage) -64.22% 1.090 0.390 - -99.26% -
434.zeusmp (max) (percentage) -63.16% 2.090 0.770 - -99.06% -
436.cactusADM (max) (percentage) -62.95% 2.510 0.930 - -98.95% -
435.gromacs (max) (percentage) -62.18% 2.380 0.900 - -98.17% -
433.milc (unexplained diff vs previous) (percentage) -62.03% 1.580 0.600 - -99.94% -
470.lbm (avg) (percentage) -61.06% 2.080 0.810 - -87.20% -
416.gamess (avg) (percentage) -60.00% 1.900 0.760 - -82.81% -
447.dealII (max) (percentage) -58.78% 1.480 0.610 - -99.24% -
410.bwaves (avg) (percentage) -58.13% 2.030 0.850 - -80.73% -
471.omnetpp -57.69% 130.000 55.000 - -75.34% -
471.omnetpp (avg) -57.69% 130.000 55.000 - -75.34% -
410.bwaves (unexplained diff vs previous) (percentage) -56.76% 0.740 0.320 - -99.77% -
429.mcf (unexplained diff vs previous) -51.52% 66.000 32.000 - -99.83% -
429.mcf (max) -50.77% 65.000 32.000 - -97.58% -
458.sjeng (unexplained diff vs previous) (percentage) -50.00% 0.160 0.080 - -99.91% -
482.sphinx3 (max) (percentage) -50.00% 3.260 1.630 - -98.31% -
435.gromacs -44.34% 221.000 123.000 - -66.49% -
435.gromacs (avg) -44.34% 221.000 123.000 - -66.49% -
456.hmmer (max) (percentage) -43.42% 0.760 0.430 - -99.42% -
401.bzip2 -42.47% 73.000 42.000 - -73.91% -
401.bzip2 (avg) -42.47% 73.000 42.000 - -73.91% -
471.omnetpp (max) (percentage) -41.85% 1.840 1.070 - -98.66% -
453.povray (max) (percentage) -38.41% 3.020 1.860 - -97.96% -
434.zeusmp (avg) (percentage) -37.84% 1.480 0.920 - -84.27% -
445.gobmk -36.36% 66.000 42.000 - -78.57% -
445.gobmk (avg) -36.36% 66.000 42.000 - -78.57% -
436.cactusADM (avg) (percentage) -36.36% 1.650 1.050 - -82.44% -
435.gromacs (unexplained diff vs previous) -35.56% 45.000 29.000 - -99.59% -
433.milc (avg) (percentage) -34.92% 1.890 1.230 - -79.67% -
437.leslie3d (max) -31.72% 558.000 381.000 - -80.25% -
437.leslie3d (unexplained diff vs previous) -31.42% 576.000 395.000 - -82.93% -
465.tonto (max) (percentage) -23.97% 1.460 1.110 - -94.25% -
482.sphinx3 (unexplained diff vs previous) (percentage) -23.53% 1.870 1.430 - -99.95% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -22.00% 0.500 0.390 - -99.84% -
447.dealII (unexplained diff vs previous) (percentage) -18.42% 0.760 0.620 - -99.85% -
473.astar (unexplained diff vs previous) (percentage) -17.50% 0.400 0.330 - -99.83% -
481.wrf (avg) (percentage) -17.05% 2.170 1.800 - -71.15% -
400.perlbench (max) (percentage) -14.80% 3.040 2.590 - -96.92% -
482.sphinx3 (avg) (percentage) -11.92% 1.930 1.700 - -78.56% -
458.sjeng (max) -11.54% 78.000 69.000 - -98.08% -
437.leslie3d (max) (percentage) -11.39% 3.950 3.500 - -78.89% -
437.leslie3d (unexplained diff vs previous) (percentage) -11.03% 4.080 3.630 - -81.74% -
464.h264ref (max) (percentage) -9.09% 0.440 0.400 - -99.26% -
456.hmmer (avg) (percentage) -3.70% 0.540 0.520 - -88.21% -
433.milc (max) (percentage) -3.60% 2.500 2.410 - -97.34% -
473.astar (max) (percentage) -2.50% 0.400 0.390 - -99.41% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.31% 1946163.000 1971631.000 - 1.18% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 45.58% 1424.000 2073.000 - -87.98% -
401.bzip2 5.54% 2941.000 3104.000 - -79.18% -
410.bwaves 4.92% 5652.000 5930.000 - -56.82% -
444.namd 3.88% 747.000 776.000 - -91.37% -
450.soplex 3.17% 567.000 585.000 - -96.29% -
465.tonto 1.72% 10941.000 11129.000 - 31.91% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 45.25% 3.050 4.430 - 80.08% -
437.leslie3d 35.77% 355.000 482.000 - 38.90% -
437.leslie3d (avg) 35.77% 355.000 482.000 - 38.90% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -8.59% 59479.000 54370.000 - -8.61% -
434.zeusmp -3.39% 152037.000 146883.000 - -4.44% -
459.GemsFDTD -3.30% 244728.000 236656.000 - -3.42% -
462.libquantum -1.79% 16085.000 15797.000 - -1.99% -
410.bwaves -1.77% 16760.000 16464.000 - -1.70% -
453.povray -1.75% 661389.000 649833.000 - -1.20% -
447.dealII -1.71% 311591.000 306263.000 - -2.00% -
458.sjeng -1.67% 104048.000 102311.000 - -1.63% -
470.lbm -1.49% 9691.000 9547.000 - -1.45% -
433.milc -1.46% 61382.000 60484.000 - -1.21% -
450.soplex -1.30% 205081.000 202406.000 - -1.30% -
401.bzip2 -1.17% 44233.000 43715.000 - -1.19% -
435.gromacs -1.10% 499261.000 493760.000 - -1.31% -
444.namd -1.09% 188385.000 186336.000 - -1.08% -
482.sphinx3 -1.04% 112859.000 111686.000 - -1.02% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -13.67% 6173.000 5329.000 - -77.53% -
482.sphinx3 -11.01% 945.000 841.000 - -97.24% -
434.zeusmp -7.96% 2514.000 2314.000 - -82.74% -
437.leslie3d -6.40% 11640.000 10895.000 - -22.88% -
447.dealII -5.68% 1724.000 1626.000 - -82.03% -
458.sjeng -3.24% 7567.000 7322.000 - -48.50% -
435.gromacs -2.39% 7269.000 7095.000 - -53.53% -
445.gobmk -2.03% 5023.000 4921.000 - -57.29% -
471.omnetpp -1.28% 2724.000 2689.000 - -75.06% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
471.omnetpp (unexplained diff vs previous) (percentage) -99.97% 391.300 0.110 - -87.78% -
471.omnetpp (unexplained diff vs previous) -99.97% 10659.000 3.000 - -96.91% -
462.libquantum (unexplained diff vs previous) -99.97% 13518.000 4.000 - -92.59% -
462.libquantum (unexplained diff vs previous) (percentage) -99.97% 9655.710 2.860 - 508.51% -
470.lbm (unexplained diff vs previous) -99.97% 9990.000 3.000 - -93.33% -
470.lbm (unexplained diff vs previous) (percentage) -99.97% 263.730 0.080 - -77.14% -
482.sphinx3 (unexplained diff vs previous) -99.95% 26073.000 12.000 - -97.89% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.95% 2759.050 1.430 - -23.53% -
433.milc (unexplained diff vs previous) (percentage) -99.94% 980.800 0.600 - -62.03% -
433.milc (unexplained diff vs previous) -99.94% 11289.000 7.000 - -96.60% -
450.soplex (unexplained diff vs previous) (percentage) -99.92% 2557.320 2.050 - 720.00% -
444.namd (unexplained diff vs previous) (percentage) -99.92% 1118.610 0.900 - 109.30% -
450.soplex (unexplained diff vs previous) -99.92% 14500.000 12.000 - -69.23% -
444.namd (unexplained diff vs previous) -99.92% 8356.000 7.000 - -82.05% -
458.sjeng (unexplained diff vs previous) -99.91% 6876.000 6.000 - -73.91% -
458.sjeng (unexplained diff vs previous) (percentage) -99.91% 90.870 0.080 - -50.00% -
429.mcf (unexplained diff vs previous) (percentage) -99.88% 1305.690 1.540 - 305.26% -
459.GemsFDTD (unexplained diff vs previous) -99.86% 15458.000 21.000 - -82.20% -
447.dealII (unexplained diff vs previous) -99.86% 7017.000 10.000 - -85.51% -
447.dealII (unexplained diff vs previous) (percentage) -99.85% 407.020 0.620 - -18.42% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.84% 250.410 0.390 - -22.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.84% 368.340 0.580 - 1350.00% -
401.bzip2 (unexplained diff vs previous) -99.83% 10833.000 18.000 - 200.00% -
473.astar (unexplained diff vs previous) (percentage) -99.83% 197.850 0.330 - -17.50% -
473.astar (unexplained diff vs previous) -99.83% 10146.000 17.000 - -70.18% -
429.mcf (unexplained diff vs previous) -99.83% 18593.000 32.000 - -51.52% -
453.povray (unexplained diff vs previous) (percentage) -99.83% 1043.330 1.820 - 2175.00% -
453.povray (unexplained diff vs previous) -99.82% 5707.000 10.000 - 100.00% -
410.bwaves (unexplained diff vs previous) (percentage) -99.77% 137.900 0.320 - -56.76% -
410.bwaves (unexplained diff vs previous) -99.76% 7794.000 19.000 - -81.37% -
445.gobmk (unexplained diff vs previous) -99.66% 5581.000 19.000 - 850.00% -
445.gobmk (unexplained diff vs previous) (percentage) -99.65% 111.110 0.390 - 1850.00% -
435.gromacs (unexplained diff vs previous) -99.59% 7028.000 29.000 - -35.56% -
435.gromacs (unexplained diff vs previous) (percentage) -99.58% 96.680 0.410 - 41.38% -
473.astar (max) -99.44% 3406.000 19.000 - -66.07% -
456.hmmer (max) (percentage) -99.42% 73.590 0.430 - -43.42% -
473.astar (max) (percentage) -99.41% 66.430 0.390 - -2.50% -
410.bwaves (max) (percentage) -99.38% 57.970 0.360 - -91.39% -
410.bwaves (max) -99.36% 3276.000 21.000 - -96.34% -
447.dealII (max) -99.35% 1384.000 9.000 - -93.23% -
445.gobmk (max) -99.28% 2643.000 19.000 - -84.80% -
464.h264ref (max) (percentage) -99.26% 54.290 0.400 - -9.09% -
445.gobmk (max) (percentage) -99.26% 52.630 0.390 - -64.22% -
447.dealII (max) (percentage) -99.24% 80.280 0.610 - -58.78% -
454.calculix (max) (percentage) -99.18% 99.730 0.820 - -75.95% -
434.zeusmp (max) -99.17% 2055.000 17.000 - -93.93% -
401.bzip2 (max) (percentage) -99.06% 78.650 0.740 - 7.25% -
434.zeusmp (max) (percentage) -99.06% 81.770 0.770 - -63.16% -
401.bzip2 (max) -99.05% 2313.000 22.000 - -78.43% -
416.gamess (max) (percentage) -99.04% 38.550 0.370 - -87.71% -
436.cactusADM (max) (percentage) -98.95% 88.730 0.930 - -62.95% -
470.lbm (max) -98.94% 2746.000 29.000 - -95.51% -
470.lbm (max) (percentage) -98.91% 72.510 0.790 - -84.29% -
481.wrf (max) (percentage) -98.84% 68.700 0.800 - -81.04% -
471.omnetpp (max) -98.71% 2169.000 28.000 - -85.86% -
471.omnetpp (max) (percentage) -98.66% 79.650 1.070 - -41.85% -
482.sphinx3 (max) -98.57% 911.000 13.000 - -98.69% -
444.namd (max) (percentage) -98.31% 91.790 1.550 - -77.86% -
482.sphinx3 (max) (percentage) -98.31% 96.500 1.630 - -50.00% -
429.mcf (max) (percentage) -98.31% 92.890 1.570 - 313.16% -
444.namd (max) -98.25% 685.000 12.000 - -98.09% -
435.gromacs (max) -98.24% 3573.000 63.000 - -82.64% -
435.gromacs (max) (percentage) -98.17% 49.160 0.900 - -62.18% -
458.sjeng (max) -98.08% 3602.000 69.000 - -11.54% -
458.sjeng (max) (percentage) -98.00% 47.610 0.950 - 72.73% -
453.povray (max) -98.00% 499.000 10.000 - -94.92% -
453.povray (max) (percentage) -97.96% 91.250 1.860 - -38.41% -
429.mcf (max) -97.58% 1322.000 32.000 - -50.77% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 96.67% -
433.milc (max) -97.41% 1044.000 27.000 - -91.69% -
433.milc (max) (percentage) -97.34% 90.750 2.410 - -3.60% -
462.libquantum (max) -97.10% 138.000 4.000 - -92.59% -
459.GemsFDTD (max) -97.10% 4411.000 128.000 - -64.93% -
465.tonto (unexplained diff vs previous) (percentage) -97.04% 16.190 0.480 - 500.00% -
462.libquantum (max) (percentage) -97.03% 98.970 2.940 - 525.53% -
465.tonto (unexplained diff vs previous) -97.01% 1771.000 53.000 - 657.14% -
400.perlbench (max) (percentage) -96.92% 84.060 2.590 - -14.80% -
450.soplex (max) (percentage) -96.88% 96.240 3.000 - 48.51% -
450.soplex (max) -96.88% 545.000 17.000 - -94.65% -
459.GemsFDTD (max) (percentage) -96.61% 71.460 2.420 - 57.14% -
403.gcc (max) (percentage) -96.23% 99.360 3.750 - 459.70% -
465.tonto (max) (percentage) -94.25% 19.310 1.110 - -23.97% -
465.tonto (max) -94.18% 2112.000 123.000 - 0.00% -
456.hmmer (avg) (percentage) -88.21% 4.410 0.520 - -3.70% -
470.lbm -87.45% 239.000 30.000 - -88.76% -
470.lbm (avg) -87.45% 239.000 30.000 - -88.76% -
444.namd (avg) (percentage) -87.31% 9.220 1.170 - -70.68% -
470.lbm (avg) (percentage) -87.20% 6.330 0.810 - -61.06% -
444.namd -86.76% 68.000 9.000 - -97.49% -
444.namd (avg) -86.76% 68.000 9.000 - -97.49% -
464.h264ref (avg) (percentage) -86.05% 5.090 0.710 - 129.03% -
434.zeusmp -85.71% 147.000 21.000 - -89.39% -
434.zeusmp (avg) -85.71% 147.000 21.000 - -89.39% -
434.zeusmp (avg) (percentage) -84.27% 5.850 0.920 - -37.84% -
437.leslie3d (unexplained diff vs previous) -82.93% 2314.000 395.000 - -31.42% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - -60.00% -
436.cactusADM (avg) (percentage) -82.44% 5.980 1.050 - -36.36% -
458.sjeng -82.16% 241.000 43.000 - 53.57% -
458.sjeng (avg) -82.16% 241.000 43.000 - 53.57% -
447.dealII -81.82% 99.000 18.000 - -76.32% -
447.dealII (avg) -81.82% 99.000 18.000 - -76.32% -
437.leslie3d (unexplained diff vs previous) (percentage) -81.74% 19.880 3.630 - -11.03% -
458.sjeng (avg) (percentage) -81.50% 3.190 0.590 - 195.00% -
459.GemsFDTD -81.39% 360.000 67.000 - -71.97% -
459.GemsFDTD (avg) -81.39% 360.000 67.000 - -71.97% -
482.sphinx3 -81.08% 74.000 14.000 - -97.62% -
482.sphinx3 (avg) -81.08% 74.000 14.000 - -97.62% -
453.povray -80.95% 42.000 8.000 - -91.75% -
453.povray (avg) -80.95% 42.000 8.000 - -91.75% -
410.bwaves (avg) (percentage) -80.73% 4.410 0.850 - -58.13% -
453.povray (avg) (percentage) -80.28% 7.810 1.540 - 3.36% -
437.leslie3d (max) -80.25% 1929.000 381.000 - -31.72% -
447.dealII (avg) (percentage) -80.07% 5.770 1.150 - 35.29% -
462.libquantum -80.00% 10.000 2.000 - -95.92% -
462.libquantum (avg) -80.00% 10.000 2.000 - -95.92% -
410.bwaves -79.92% 249.000 50.000 - -82.01% -
410.bwaves (avg) -79.92% 249.000 50.000 - -82.01% -
433.milc -79.71% 69.000 14.000 - -94.31% -
433.milc (avg) -79.71% 69.000 14.000 - -94.31% -
433.milc (avg) (percentage) -79.67% 6.050 1.230 - -34.92% -
437.leslie3d (max) (percentage) -78.89% 16.580 3.500 - -11.39% -
445.gobmk -78.57% 196.000 42.000 - -36.36% -
445.gobmk (avg) -78.57% 196.000 42.000 - -36.36% -
482.sphinx3 (avg) (percentage) -78.56% 7.930 1.700 - -11.92% -
459.GemsFDTD (avg) (percentage) -78.25% 5.840 1.270 - 25.74% -
445.gobmk (avg) (percentage) -78.01% 3.910 0.860 - 48.28% -
462.libquantum (avg) (percentage) -75.98% 7.660 1.840 - 327.91% -
471.omnetpp -75.34% 223.000 55.000 - -57.69% -
471.omnetpp (avg) -75.34% 223.000 55.000 - -57.69% -
401.bzip2 (avg) (percentage) -75.00% 5.480 1.370 - 179.59% -
471.omnetpp (avg) (percentage) -75.00% 8.200 2.050 - 69.42% -
401.bzip2 -73.91% 161.000 42.000 - -42.47% -
401.bzip2 (avg) -73.91% 161.000 42.000 - -42.47% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - -17.05% -
400.perlbench (avg) (percentage) -69.45% 6.350 1.940 - 1.57% -
435.gromacs -66.49% 367.000 123.000 - -44.34% -
435.gromacs (avg) -66.49% 367.000 123.000 - -44.34% -
435.gromacs (avg) (percentage) -65.61% 5.060 1.740 - 20.00% -
465.tonto (avg) (percentage) -63.17% 3.340 1.230 - 29.47% -
465.tonto -62.74% 365.000 136.000 - 70.00% -
465.tonto (avg) -62.74% 365.000 136.000 - 70.00% -
473.astar -62.44% 213.000 80.000 - 263.64% -
473.astar (avg) -62.44% 213.000 80.000 - 263.64% -
473.astar (avg) (percentage) -62.26% 4.160 1.570 - 881.25% -
450.soplex (avg) (percentage) -62.21% 6.430 2.430 - 131.43% -
450.soplex -61.11% 36.000 14.000 - -91.52% -
450.soplex (avg) -61.11% 36.000 14.000 - -91.52% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 188.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 236.36% -
429.mcf (avg) (percentage) -38.07% 7.670 4.750 - 1800.00% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 1397.06% -
429.mcf -10.09% 109.000 98.000 - 127.91% -
429.mcf (avg) -10.09% 109.000 98.000 - 127.91% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
401.bzip2 3104.000 -79.18%
410.bwaves 5930.000 -56.82%
429.mcf 2073.000 -87.98%
433.milc 1158.000 -91.11%
434.zeusmp 2314.000 -82.74%
435.gromacs 7095.000 -53.53%
437.leslie3d 10895.000 -22.88%
444.namd 776.000 -91.37%
445.gobmk 4921.000 -57.29%
447.dealII 1626.000 -82.03%
450.soplex 585.000 -96.29%
453.povray 549.000 -91.59%
458.sjeng 7322.000 -48.50%
459.GemsFDTD 5329.000 -77.53%
462.libquantum 140.000 -98.79%
465.tonto 11129.000 31.91%
470.lbm 3770.000 -70.66%
471.omnetpp 2689.000 -75.06%
473.astar 5112.000 -64.12%
482.sphinx3 841.000 -97.24%
Geometric Mean 2404.576 -81.77%
Code Size (bytes)
Name Current %
401.bzip2 43715.000 -1.19%
410.bwaves 16464.000 -1.70%
429.mcf 9146.000 -
433.milc 60484.000 -1.21%
434.zeusmp 146883.000 -4.44%
435.gromacs 493760.000 -1.31%
437.leslie3d 54370.000 -8.61%
444.namd 186336.000 -1.08%
445.gobmk 1375086.000 -
447.dealII 306263.000 -2.00%
450.soplex 202406.000 -1.30%
453.povray 649833.000 -1.20%
458.sjeng 102311.000 -1.63%
459.GemsFDTD 236656.000 -3.42%
462.libquantum 15797.000 -1.99%
465.tonto 1971631.000 1.18%
470.lbm 9547.000 -1.45%
471.omnetpp 475044.000 -1.07%
473.astar 22752.000 -
482.sphinx3 111686.000 -1.02%
Geometric Mean 114139.112 -1.73%
Score
Name Current %
nb_compile_successful 20.000 -31.03%
nb_execution_successful 20.000 -31.03%
nb_total_tests 38.000 31.03%
Geometric Mean 24.771 -14.58%
sample variability
Name Current %
400.perlbench (avg) (percentage) 1.940 1.57%
400.perlbench (max) (percentage) 2.590 -14.80%
401.bzip2 42.000 -42.47%
401.bzip2 (avg) 42.000 -42.47%
401.bzip2 (avg) (percentage) 1.370 179.59%
401.bzip2 (max) 22.000 -78.43%
401.bzip2 (max) (percentage) 0.740 7.25%
401.bzip2 (unexplained diff vs previous) 18.000 200.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.580 1350.00%
403.gcc (avg) (percentage) 5.090 1397.06%
403.gcc (max) (percentage) 3.750 459.70%
410.bwaves 50.000 -82.01%
410.bwaves (avg) 50.000 -82.01%
410.bwaves (avg) (percentage) 0.850 -58.13%
410.bwaves (max) 21.000 -96.34%
410.bwaves (max) (percentage) 0.360 -91.39%
410.bwaves (unexplained diff vs previous) 19.000 -81.37%
410.bwaves (unexplained diff vs previous) (percentage) 0.320 -56.76%
416.gamess (avg) (percentage) 0.760 -60.00%
416.gamess (max) (percentage) 0.370 -87.71%
429.mcf 98.000 127.91%
429.mcf (avg) 98.000 127.91%
429.mcf (avg) (percentage) 4.750 1800.00%
429.mcf (max) 32.000 -50.77%
429.mcf (max) (percentage) 1.570 313.16%
429.mcf (unexplained diff vs previous) 32.000 -51.52%
429.mcf (unexplained diff vs previous) (percentage) 1.540 305.26%
433.milc 14.000 -94.31%
433.milc (avg) 14.000 -94.31%
433.milc (avg) (percentage) 1.230 -34.92%
433.milc (max) 27.000 -91.69%
433.milc (max) (percentage) 2.410 -3.60%
433.milc (unexplained diff vs previous) 7.000 -96.60%
433.milc (unexplained diff vs previous) (percentage) 0.600 -62.03%
434.zeusmp 21.000 -89.39%
434.zeusmp (avg) 21.000 -89.39%
434.zeusmp (avg) (percentage) 0.920 -37.84%
434.zeusmp (max) 17.000 -93.93%
434.zeusmp (max) (percentage) 0.770 -63.16%
434.zeusmp (unexplained diff vs previous) 0.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 0.000 -
435.gromacs 123.000 -44.34%
435.gromacs (avg) 123.000 -44.34%
435.gromacs (avg) (percentage) 1.740 20.00%
435.gromacs (max) 63.000 -82.64%
435.gromacs (max) (percentage) 0.900 -62.18%
435.gromacs (unexplained diff vs previous) 29.000 -35.56%
435.gromacs (unexplained diff vs previous) (percentage) 0.410 41.38%
436.cactusADM (avg) (percentage) 1.050 -36.36%
436.cactusADM (max) (percentage) 0.930 -62.95%
437.leslie3d 482.000 38.90%
437.leslie3d (avg) 482.000 38.90%
437.leslie3d (avg) (percentage) 4.430 80.08%
437.leslie3d (max) 381.000 -31.72%
437.leslie3d (max) (percentage) 3.500 -11.39%
437.leslie3d (unexplained diff vs previous) 395.000 -31.42%
437.leslie3d (unexplained diff vs previous) (percentage) 3.630 -11.03%
444.namd 9.000 -97.49%
444.namd (avg) 9.000 -97.49%
444.namd (avg) (percentage) 1.170 -70.68%
444.namd (max) 12.000 -98.09%
444.namd (max) (percentage) 1.550 -77.86%
444.namd (unexplained diff vs previous) 7.000 -82.05%
444.namd (unexplained diff vs previous) (percentage) 0.900 109.30%
445.gobmk 42.000 -36.36%
445.gobmk (avg) 42.000 -36.36%
445.gobmk (avg) (percentage) 0.860 48.28%
445.gobmk (max) 19.000 -84.80%
445.gobmk (max) (percentage) 0.390 -64.22%
445.gobmk (unexplained diff vs previous) 19.000 850.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.390 1850.00%
447.dealII 18.000 -76.32%
447.dealII (avg) 18.000 -76.32%
447.dealII (avg) (percentage) 1.150 35.29%
447.dealII (max) 9.000 -93.23%
447.dealII (max) (percentage) 0.610 -58.78%
447.dealII (unexplained diff vs previous) 10.000 -85.51%
447.dealII (unexplained diff vs previous) (percentage) 0.620 -18.42%
450.soplex 14.000 -91.52%
450.soplex (avg) 14.000 -91.52%
450.soplex (avg) (percentage) 2.430 131.43%
450.soplex (max) 17.000 -94.65%
450.soplex (max) (percentage) 3.000 48.51%
450.soplex (unexplained diff vs previous) 12.000 -69.23%
450.soplex (unexplained diff vs previous) (percentage) 2.050 720.00%
453.povray 8.000 -91.75%
453.povray (avg) 8.000 -91.75%
453.povray (avg) (percentage) 1.540 3.36%
453.povray (max) 10.000 -94.92%
453.povray (max) (percentage) 1.860 -38.41%
453.povray (unexplained diff vs previous) 10.000 100.00%
453.povray (unexplained diff vs previous) (percentage) 1.820 2175.00%
454.calculix (avg) (percentage) 3.600 188.00%
454.calculix (max) (percentage) 0.820 -75.95%
456.hmmer (avg) (percentage) 0.520 -3.70%
456.hmmer (max) (percentage) 0.430 -43.42%
458.sjeng 43.000 53.57%
458.sjeng (avg) 43.000 53.57%
458.sjeng (avg) (percentage) 0.590 195.00%
458.sjeng (max) 69.000 -11.54%
458.sjeng (max) (percentage) 0.950 72.73%
458.sjeng (unexplained diff vs previous) 6.000 -73.91%
458.sjeng (unexplained diff vs previous) (percentage) 0.080 -50.00%
459.GemsFDTD 67.000 -71.97%
459.GemsFDTD (avg) 67.000 -71.97%
459.GemsFDTD (avg) (percentage) 1.270 25.74%
459.GemsFDTD (max) 128.000 -64.93%
459.GemsFDTD (max) (percentage) 2.420 57.14%
459.GemsFDTD (unexplained diff vs previous) 21.000 -82.20%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.390 -22.00%
462.libquantum 2.000 -95.92%
462.libquantum (avg) 2.000 -95.92%
462.libquantum (avg) (percentage) 1.840 327.91%
462.libquantum (max) 4.000 -92.59%
462.libquantum (max) (percentage) 2.940 525.53%
462.libquantum (unexplained diff vs previous) 4.000 -92.59%
462.libquantum (unexplained diff vs previous) (percentage) 2.860 508.51%
464.h264ref (avg) (percentage) 0.710 129.03%
464.h264ref (max) (percentage) 0.400 -9.09%
465.tonto 136.000 70.00%
465.tonto (avg) 136.000 70.00%
465.tonto (avg) (percentage) 1.230 29.47%
465.tonto (max) 123.000 -
465.tonto (max) (percentage) 1.110 -23.97%
465.tonto (unexplained diff vs previous) 53.000 657.14%
465.tonto (unexplained diff vs previous) (percentage) 0.480 500.00%
470.lbm 30.000 -88.76%
470.lbm (avg) 30.000 -88.76%
470.lbm (avg) (percentage) 0.810 -61.06%
470.lbm (max) 29.000 -95.51%
470.lbm (max) (percentage) 0.790 -84.29%
470.lbm (unexplained diff vs previous) 3.000 -93.33%
470.lbm (unexplained diff vs previous) (percentage) 0.080 -77.14%
471.omnetpp 55.000 -57.69%
471.omnetpp (avg) 55.000 -57.69%
471.omnetpp (avg) (percentage) 2.050 69.42%
471.omnetpp (max) 28.000 -85.86%
471.omnetpp (max) (percentage) 1.070 -41.85%
471.omnetpp (unexplained diff vs previous) 3.000 -96.91%
471.omnetpp (unexplained diff vs previous) (percentage) 0.110 -87.78%
473.astar 80.000 263.64%
473.astar (avg) 80.000 263.64%
473.astar (avg) (percentage) 1.570 881.25%
473.astar (max) 19.000 -66.07%
473.astar (max) (percentage) 0.390 -2.50%
473.astar (unexplained diff vs previous) 17.000 -70.18%
473.astar (unexplained diff vs previous) (percentage) 0.330 -17.50%
481.wrf (avg) (percentage) 1.800 -17.05%
481.wrf (max) (percentage) 0.800 -81.04%
482.sphinx3 14.000 -97.62%
482.sphinx3 (avg) 14.000 -97.62%
482.sphinx3 (avg) (percentage) 1.700 -11.92%
482.sphinx3 (max) 13.000 -98.69%
482.sphinx3 (max) (percentage) 1.630 -50.00%
482.sphinx3 (unexplained diff vs previous) 12.000 -97.89%
482.sphinx3 (unexplained diff vs previous) (percentage) 1.430 -23.53%
483.xalancbmk (avg) (percentage) 2.960 236.36%
483.xalancbmk (max) (percentage) 1.770 96.67%
Geometric Mean 4.644 -55.92%