Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=18c4b05e40ebf5537f7171fadb2fdff86149faa3 (+ 105 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/2531f014fb2364777fb1ce09641db85bda5883b7 (+ 235 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=b300078d97a6892cb2fa1c63a46333754db60555 (+ 48 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/25cd1b6c4571a4d82346fe97e81e280f64129355 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/299/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0299 (3-changed-components) 2024-09-22T04:42:50 0:00:00
Previous 0298 (-00000-) 2022-09-12T00:16:07 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 43 13
Performance Improvements 144 166
Unchanged Tests 1049 1057
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
458.sjeng 2.26% 104028.000 106376.000 - 2.24% -
465.tonto 1.60% 1945443.000 1976555.000 - 1.56% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 26.67% 8789.000 11133.000 - 1.75% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
470.lbm (unexplained diff vs previous) (percentage) 2000.00% 0.020 0.420 0.000 -99.84% 0.000
403.gcc (avg) (percentage) 1397.06% 0.340 5.090 - -30.46% -
403.gcc (max) (percentage) 1026.47% 0.340 3.830 - -96.15% -
483.xalancbmk (avg) (percentage) 745.71% 0.350 2.960 - -54.46% -
454.calculix (avg) (percentage) 605.88% 0.510 3.600 - -55.11% -
429.mcf (max) (percentage) 542.31% 0.260 1.670 - -98.20% -
401.bzip2 (max) (percentage) 460.00% 0.250 1.400 - -98.22% -
470.lbm (unexplained diff vs previous) 433.33% 3.000 16.000 0.000 -99.84% 0.000
462.libquantum (max) (percentage) 386.96% 0.690 3.360 - -96.61% -
483.xalancbmk (max) (percentage) 342.50% 0.400 1.770 - -97.42% -
429.mcf (avg) (percentage) 334.21% 1.140 4.950 - -35.46% -
437.leslie3d (max) (percentage) 225.74% 4.390 14.300 - -13.75% -
471.omnetpp (max) (percentage) 213.16% 0.380 1.190 - -98.51% -
473.astar (avg) (percentage) 182.76% 0.580 1.640 - -60.58% -
458.sjeng (avg) (percentage) 173.91% 0.230 0.630 - -80.25% -
437.leslie3d (max) 158.84% 622.000 1610.000 - -16.54% -
471.omnetpp (unexplained diff vs previous) (percentage) 157.14% 0.350 0.900 0.000 -99.77% 0.000
401.bzip2 (avg) (percentage) 123.53% 0.680 1.520 - -72.26% -
450.soplex (avg) (percentage) 91.67% 1.320 2.530 - -60.65% -
437.leslie3d (avg) (percentage) 74.62% 2.600 4.540 - 48.85% -
400.perlbench (avg) (percentage) 45.71% 1.400 2.040 - -67.87% -
465.tonto 44.68% 94.000 136.000 - -62.74% -
465.tonto (avg) 44.68% 94.000 136.000 - -62.74% -
458.sjeng 43.75% 32.000 46.000 - -80.91% -
458.sjeng (avg) 43.75% 32.000 46.000 - -80.91% -
445.gobmk (avg) (percentage) 43.55% 0.620 0.890 - -77.24% -
437.leslie3d 38.86% 368.000 511.000 - 43.94% -
437.leslie3d (avg) 38.86% 368.000 511.000 - 43.94% -
400.perlbench (max) (percentage) 32.42% 1.820 2.410 - -97.13% -
462.libquantum (avg) (percentage) 30.50% 1.410 1.840 - -75.98% -
433.milc (unexplained diff vs previous) (percentage) 26.47% 0.340 0.430 0.000 -99.96% 0.000
473.astar (max) (percentage) 23.64% 3.680 4.550 - -93.15% -
471.omnetpp (avg) (percentage) 20.00% 1.800 2.160 - -73.66% -
465.tonto (avg) (percentage) 14.95% 1.070 1.230 - -63.17% -
401.bzip2 (max) 13.51% 37.000 42.000 - -98.18% -
435.gromacs (avg) (percentage) 8.64% 1.620 1.760 - -65.22% -
447.dealII (avg) (percentage) 7.02% 1.140 1.220 - -78.86% -
464.h264ref (avg) (percentage) 5.56% 0.720 0.760 - -85.07% -
473.astar 1.22% 82.000 83.000 - -61.03% -
473.astar (avg) 1.22% 82.000 83.000 - -61.03% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -7.77% 59403.000 54786.000 - -7.89% -
459.GemsFDTD -6.13% 247940.000 232732.000 - -4.90% -
434.zeusmp -4.70% 153897.000 146667.000 - -3.53% -
450.soplex -4.04% 211092.000 202558.000 - -1.23% -
435.gromacs -2.09% 500203.000 489745.000 - -1.91% -
464.h264ref -2.03% 387228.000 379348.000 - -2.17% -
410.bwaves -1.81% 16768.000 16464.000 - -1.77% -
470.lbm -1.69% 9711.000 9547.000 - -1.49% -
447.dealII -1.61% 312944.000 307900.000 - -1.18% -
471.omnetpp -1.47% 482480.000 475384.000 - -0.55% -
444.namd -1.35% 188301.000 185768.000 - -1.39% -
456.hmmer -1.28% 108631.000 107239.000 - -1.34% -
436.cactusADM -1.27% 352831.000 348359.000 - -1.28% -
453.povray -1.22% 661959.000 653881.000 - -1.14% -
433.milc -1.18% 61230.000 60509.000 - -1.42% -
401.bzip2 -1.14% 44221.000 43715.000 - -1.17% -
400.perlbench -1.05% 897104.000 887723.000 - -0.69% -
482.sphinx3 -1.03% 112831.000 111671.000 - -1.05% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.83% 11536.000 135.000 - -3.57% -
482.sphinx3 -97.25% 30419.000 836.000 - -11.53% -
450.soplex -96.31% 15869.000 585.000 - 3.17% -
453.povray -91.75% 6590.000 544.000 - -0.55% -
444.namd -91.31% 9014.000 783.000 - 4.82% -
433.milc -91.06% 12922.000 1155.000 - 0.35% -
429.mcf -88.99% 17291.000 1904.000 - 33.71% -
436.cactusADM -88.01% 17703.000 2122.000 - 1.87% -
400.perlbench -86.24% 10962.000 1508.000 - -9.32% -
447.dealII -81.56% 8884.000 1638.000 - -4.99% -
434.zeusmp -81.20% 13484.000 2535.000 - 0.84% -
401.bzip2 -79.79% 14935.000 3019.000 - 2.65% -
459.GemsFDTD -77.23% 23565.000 5366.000 - -13.07% -
456.hmmer -74.15% 11121.000 2875.000 - -0.24% -
471.omnetpp -74.03% 10710.000 2781.000 - 2.09% -
470.lbm -70.43% 12801.000 3785.000 - -0.08% -
473.astar -63.86% 14152.000 5114.000 - -0.27% -
410.bwaves -59.19% 14435.000 5891.000 - 4.23% -
464.h264ref -57.83% 13679.000 5769.000 - 0.14% -
445.gobmk -57.10% 11535.000 4948.000 - -1.49% -
435.gromacs -53.13% 15239.000 7143.000 - -1.73% -
458.sjeng -47.22% 14020.000 7400.000 - -2.21% -
437.leslie3d -20.66% 14191.000 11259.000 - -3.27% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (max) -99.65% 1706.000 6.000 - -99.34% -
444.namd (max) -99.52% 629.000 3.000 - -99.56% -
450.soplex (max) -99.12% 682.000 6.000 - -98.90% -
462.libquantum -98.77% 162.000 2.000 - -80.00% -
462.libquantum (avg) -98.77% 162.000 2.000 - -80.00% -
434.zeusmp (max) -98.07% 311.000 6.000 - -99.71% -
444.namd -98.05% 462.000 9.000 - -86.76% -
444.namd (avg) -98.05% 462.000 9.000 - -86.76% -
482.sphinx3 -98.01% 754.000 15.000 - -79.73% -
482.sphinx3 (avg) -98.01% 754.000 15.000 - -79.73% -
456.hmmer (max) -96.67% 60.000 2.000 - -99.91% -
433.milc (max) -95.78% 308.000 13.000 - -98.75% -
436.cactusADM (max) -95.31% 405.000 19.000 - -98.97% -
462.libquantum (max) -94.94% 79.000 4.000 - -97.10% -
433.milc -94.49% 254.000 14.000 - -79.71% -
433.milc (avg) -94.49% 254.000 14.000 - -79.71% -
470.lbm (max) -94.20% 414.000 24.000 - -99.13% -
453.povray -93.80% 129.000 8.000 - -80.95% -
453.povray (avg) -93.80% 129.000 8.000 - -80.95% -
450.soplex -93.30% 209.000 14.000 - -61.11% -
450.soplex (avg) -93.30% 209.000 14.000 - -61.11% -
453.povray (max) -93.23% 133.000 9.000 - -98.20% -
436.cactusADM -93.18% 352.000 24.000 - -80.65% -
436.cactusADM (avg) -93.18% 352.000 24.000 - -80.65% -
444.namd (max) (percentage) -92.84% 6.980 0.500 - -99.46% -
470.lbm -91.36% 382.000 33.000 - -86.19% -
470.lbm (avg) -91.36% 382.000 33.000 - -86.19% -
434.zeusmp -90.60% 266.000 25.000 - -82.99% -
434.zeusmp (avg) -90.60% 266.000 25.000 - -82.99% -
459.GemsFDTD (max) -89.58% 259.000 27.000 - -99.39% -
447.dealII (max) -88.93% 262.000 29.000 - -97.90% -
434.zeusmp (max) (percentage) -88.74% 2.310 0.260 - -99.68% -
433.milc (unexplained diff vs previous) -88.64% 44.000 5.000 0.000 -99.96% 0.000
410.bwaves (max) -88.31% 402.000 47.000 - -98.57% -
482.sphinx3 (max) (percentage) -86.45% 5.610 0.760 - -99.21% -
416.gamess (max) (percentage) -85.98% 2.710 0.380 - -99.01% -
401.bzip2 (unexplained diff vs previous) -84.38% 32.000 5.000 0.000 -99.95% 0.000
410.bwaves (unexplained diff vs previous) -84.00% 125.000 20.000 0.000 -99.74% 0.000
456.hmmer (max) (percentage) -83.33% 0.540 0.090 - -99.88% -
400.perlbench (max) -81.91% 199.000 36.000 - -97.42% -
447.dealII -81.19% 101.000 19.000 - -80.81% -
447.dealII (avg) -81.19% 101.000 19.000 - -80.81% -
400.perlbench -80.39% 153.000 30.000 - -71.43% -
400.perlbench (avg) -80.39% 153.000 30.000 - -71.43% -
470.lbm (max) (percentage) -79.63% 3.240 0.660 - -99.09% -
459.GemsFDTD -79.31% 348.000 72.000 - -80.00% -
459.GemsFDTD (avg) -79.31% 348.000 72.000 - -80.00% -
481.wrf (max) (percentage) -77.63% 3.710 0.830 - -98.79% -
435.gromacs (max) -77.40% 385.000 87.000 - -97.57% -
456.hmmer -76.56% 64.000 15.000 - -88.19% -
456.hmmer (avg) -76.56% 64.000 15.000 - -88.19% -
444.namd (avg) (percentage) -75.83% 5.130 1.240 - -86.55% -
410.bwaves -75.78% 223.000 54.000 - -78.31% -
410.bwaves (avg) -75.78% 223.000 54.000 - -78.31% -
450.soplex (max) (percentage) -74.42% 4.300 1.100 - -98.86% -
445.gobmk (max) -74.40% 125.000 32.000 - -98.79% -
410.bwaves (max) (percentage) -71.33% 2.790 0.800 - -98.62% -
470.lbm (avg) (percentage) -70.23% 2.990 0.890 - -85.94% -
458.sjeng (max) -69.70% 33.000 10.000 - -99.72% -
471.omnetpp -68.75% 192.000 60.000 - -73.09% -
471.omnetpp (avg) -68.75% 192.000 60.000 - -73.09% -
464.h264ref (max) -66.67% 153.000 51.000 - -98.37% -
465.tonto (max) (percentage) -65.35% 2.020 0.700 - -96.37% -
410.bwaves (unexplained diff vs previous) (percentage) -60.92% 0.870 0.340 0.000 -99.75% 0.000
436.cactusADM (max) (percentage) -58.95% 2.290 0.940 - -98.94% -
465.tonto (max) -56.50% 177.000 77.000 - -96.35% -
464.h264ref -56.12% 98.000 43.000 - -85.32% -
464.h264ref (avg) -56.12% 98.000 43.000 - -85.32% -
401.bzip2 -55.45% 101.000 45.000 - -72.05% -
401.bzip2 (avg) -55.45% 101.000 45.000 - -72.05% -
473.astar (max) -55.38% 520.000 232.000 - -93.19% -
459.GemsFDTD (max) (percentage) -52.73% 1.100 0.520 - -99.27% -
433.milc (max) (percentage) -52.30% 2.390 1.140 - -98.74% -
429.mcf -52.28% 197.000 94.000 - -13.76% -
429.mcf (avg) -52.28% 197.000 94.000 - -13.76% -
435.gromacs (max) (percentage) -51.78% 2.530 1.220 - -97.52% -
434.zeusmp (avg) (percentage) -50.00% 1.980 0.990 - -83.08% -
435.gromacs -49.19% 246.000 125.000 - -65.94% -
435.gromacs (avg) -49.19% 246.000 125.000 - -65.94% -
416.gamess (avg) (percentage) -43.70% 1.350 0.760 - -82.81% -
436.cactusADM (avg) (percentage) -42.71% 1.990 1.140 - -80.94% -
458.sjeng (max) (percentage) -41.67% 0.240 0.140 - -99.71% -
410.bwaves (avg) (percentage) -40.65% 1.550 0.920 - -79.14% -
445.gobmk (max) (percentage) -40.37% 1.090 0.650 - -98.76% -
447.dealII (max) (percentage) -39.53% 2.960 1.790 - -97.77% -
445.gobmk -38.03% 71.000 44.000 - -77.55% -
445.gobmk (avg) -38.03% 71.000 44.000 - -77.55% -
433.milc (avg) (percentage) -37.06% 1.970 1.240 - -79.50% -
471.omnetpp (unexplained diff vs previous) -32.43% 37.000 25.000 0.000 -99.77% 0.000
429.mcf (max) -29.55% 44.000 31.000 - -97.66% -
482.sphinx3 (avg) (percentage) -27.02% 2.480 1.810 - -77.18% -
453.povray (avg) (percentage) -21.83% 1.970 1.540 - -80.28% -
454.calculix (max) (percentage) -20.19% 1.040 0.830 - -99.17% -
464.h264ref (max) (percentage) -19.64% 1.120 0.900 - -98.34% -
401.bzip2 (unexplained diff vs previous) (percentage) -19.05% 0.210 0.170 0.000 -99.95% 0.000
471.omnetpp (max) -17.50% 40.000 33.000 - -98.48% -
453.povray (max) (percentage) -11.39% 2.020 1.790 - -98.04% -
459.GemsFDTD (avg) (percentage) -8.78% 1.480 1.350 - -76.88% -
456.hmmer (avg) (percentage) -6.90% 0.580 0.540 - -87.76% -
481.wrf (avg) (percentage) -4.26% 1.880 1.800 - -71.15% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 134.48% 29.000 68.000 - 134.48% -
nb_compile_successful 65.52% 29.000 48.000 - 65.52% -
nb_execution_successful 65.52% 29.000 48.000 - 65.52% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng 2.24% 104048.000 106376.000 - 2.26% -
465.tonto 1.56% 1946163.000 1976555.000 - 1.60% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 33.71% 1424.000 1904.000 - -88.99% -
444.namd 4.82% 747.000 783.000 - -91.31% -
410.bwaves 4.23% 5652.000 5891.000 - -59.19% -
450.soplex 3.17% 567.000 585.000 - -96.31% -
401.bzip2 2.65% 2941.000 3019.000 - -79.79% -
471.omnetpp 2.09% 2724.000 2781.000 - -74.03% -
436.cactusADM 1.87% 2083.000 2122.000 - -88.01% -
465.tonto 1.75% 10941.000 11133.000 - 26.67% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 48.85% 3.050 4.540 - 74.62% -
437.leslie3d 43.94% 355.000 511.000 - 38.86% -
437.leslie3d (avg) 43.94% 355.000 511.000 - 38.86% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -7.89% 59479.000 54786.000 - -7.77% -
459.GemsFDTD -4.90% 244728.000 232732.000 - -6.13% -
434.zeusmp -3.53% 152037.000 146667.000 - -4.70% -
464.h264ref -2.17% 387748.000 379348.000 - -2.03% -
435.gromacs -1.91% 499261.000 489745.000 - -2.09% -
410.bwaves -1.77% 16760.000 16464.000 - -1.81% -
470.lbm -1.49% 9691.000 9547.000 - -1.69% -
433.milc -1.42% 61382.000 60509.000 - -1.18% -
444.namd -1.39% 188385.000 185768.000 - -1.35% -
456.hmmer -1.34% 108691.000 107239.000 - -1.28% -
436.cactusADM -1.28% 352891.000 348359.000 - -1.27% -
450.soplex -1.23% 205081.000 202558.000 - -4.04% -
462.libquantum -1.19% 16085.000 15893.000 - -0.95% -
447.dealII -1.18% 311591.000 307900.000 - -1.61% -
401.bzip2 -1.17% 44233.000 43715.000 - -1.14% -
453.povray -1.14% 661389.000 653881.000 - -1.22% -
482.sphinx3 -1.05% 112859.000 111671.000 - -1.03% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -13.07% 6173.000 5366.000 - -77.23% -
482.sphinx3 -11.53% 945.000 836.000 - -97.25% -
400.perlbench -9.32% 1663.000 1508.000 - -86.24% -
447.dealII -4.99% 1724.000 1638.000 - -81.56% -
462.libquantum -3.57% 140.000 135.000 - -98.83% -
437.leslie3d -3.27% 11640.000 11259.000 - -20.66% -
458.sjeng -2.21% 7567.000 7400.000 - -47.22% -
435.gromacs -1.73% 7269.000 7143.000 - -53.13% -
445.gobmk -1.49% 5023.000 4948.000 - -57.10% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
433.milc (unexplained diff vs previous) (percentage) -99.96% 980.800 0.430 0.000 26.47% 0.000
433.milc (unexplained diff vs previous) -99.96% 11289.000 5.000 0.000 -88.64% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) -99.95% 368.340 0.170 0.000 -19.05% 0.000
401.bzip2 (unexplained diff vs previous) -99.95% 10833.000 5.000 0.000 -84.38% 0.000
456.hmmer (max) -99.91% 2120.000 2.000 - -96.67% -
456.hmmer (max) (percentage) -99.88% 73.590 0.090 - -83.33% -
470.lbm (unexplained diff vs previous) (percentage) -99.84% 263.730 0.420 0.000 2000.00% 0.000
470.lbm (unexplained diff vs previous) -99.84% 9990.000 16.000 0.000 433.33% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) -99.77% 391.300 0.900 0.000 157.14% 0.000
471.omnetpp (unexplained diff vs previous) -99.77% 10659.000 25.000 0.000 -32.43% 0.000
410.bwaves (unexplained diff vs previous) (percentage) -99.75% 137.900 0.340 0.000 -60.92% 0.000
410.bwaves (unexplained diff vs previous) -99.74% 7794.000 20.000 0.000 -84.00% 0.000
458.sjeng (max) -99.72% 3602.000 10.000 - -69.70% -
434.zeusmp (max) -99.71% 2055.000 6.000 - -98.07% -
458.sjeng (max) (percentage) -99.71% 47.610 0.140 - -41.67% -
434.zeusmp (max) (percentage) -99.68% 81.770 0.260 - -88.74% -
444.namd (max) -99.56% 685.000 3.000 - -99.52% -
444.namd (max) (percentage) -99.46% 91.790 0.500 - -92.84% -
459.GemsFDTD (max) -99.39% 4411.000 27.000 - -89.58% -
482.sphinx3 (max) -99.34% 911.000 6.000 - -99.65% -
459.GemsFDTD (max) (percentage) -99.27% 71.460 0.520 - -52.73% -
482.sphinx3 (max) (percentage) -99.21% 96.500 0.760 - -86.45% -
454.calculix (max) (percentage) -99.17% 99.730 0.830 - -20.19% -
470.lbm (max) -99.13% 2746.000 24.000 - -94.20% -
470.lbm (max) (percentage) -99.09% 72.510 0.660 - -79.63% -
416.gamess (max) (percentage) -99.01% 38.550 0.380 - -85.98% -
436.cactusADM (max) -98.97% 1848.000 19.000 - -95.31% -
436.cactusADM (max) (percentage) -98.94% 88.730 0.940 - -58.95% -
450.soplex (max) -98.90% 545.000 6.000 - -99.12% -
450.soplex (max) (percentage) -98.86% 96.240 1.100 - -74.42% -
481.wrf (max) (percentage) -98.79% 68.700 0.830 - -77.63% -
445.gobmk (max) -98.79% 2643.000 32.000 - -74.40% -
445.gobmk (max) (percentage) -98.76% 52.630 0.650 - -40.37% -
433.milc (max) -98.75% 1044.000 13.000 - -95.78% -
433.milc (max) (percentage) -98.74% 90.750 1.140 - -52.30% -
410.bwaves (max) (percentage) -98.62% 57.970 0.800 - -71.33% -
410.bwaves (max) -98.57% 3276.000 47.000 - -88.31% -
471.omnetpp (max) (percentage) -98.51% 79.650 1.190 - 213.16% -
471.omnetpp (max) -98.48% 2169.000 33.000 - -17.50% -
464.h264ref (max) -98.37% 3127.000 51.000 - -66.67% -
464.h264ref (max) (percentage) -98.34% 54.290 0.900 - -19.64% -
401.bzip2 (max) (percentage) -98.22% 78.650 1.400 - 460.00% -
429.mcf (max) (percentage) -98.20% 92.890 1.670 - 542.31% -
453.povray (max) -98.20% 499.000 9.000 - -93.23% -
401.bzip2 (max) -98.18% 2313.000 42.000 - 13.51% -
453.povray (max) (percentage) -98.04% 91.250 1.790 - -11.39% -
447.dealII (max) -97.90% 1384.000 29.000 - -88.93% -
447.dealII (max) (percentage) -97.77% 80.280 1.790 - -39.53% -
429.mcf (max) -97.66% 1322.000 31.000 - -29.55% -
435.gromacs (max) -97.57% 3573.000 87.000 - -77.40% -
435.gromacs (max) (percentage) -97.52% 49.160 1.220 - -51.78% -
400.perlbench (max) -97.42% 1397.000 36.000 - -81.91% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 342.50% -
400.perlbench (max) (percentage) -97.13% 84.060 2.410 - 32.42% -
462.libquantum (max) -97.10% 138.000 4.000 - -94.94% -
462.libquantum (max) (percentage) -96.61% 98.970 3.360 - 386.96% -
465.tonto (max) (percentage) -96.37% 19.310 0.700 - -65.35% -
465.tonto (max) -96.35% 2112.000 77.000 - -56.50% -
403.gcc (max) (percentage) -96.15% 99.360 3.830 - 1026.47% -
473.astar (max) -93.19% 3406.000 232.000 - -55.38% -
473.astar (max) (percentage) -93.15% 66.430 4.550 - 23.64% -
456.hmmer -88.19% 127.000 15.000 - -76.56% -
456.hmmer (avg) -88.19% 127.000 15.000 - -76.56% -
456.hmmer (avg) (percentage) -87.76% 4.410 0.540 - -6.90% -
444.namd -86.76% 68.000 9.000 - -98.05% -
444.namd (avg) -86.76% 68.000 9.000 - -98.05% -
444.namd (avg) (percentage) -86.55% 9.220 1.240 - -75.83% -
470.lbm -86.19% 239.000 33.000 - -91.36% -
470.lbm (avg) -86.19% 239.000 33.000 - -91.36% -
470.lbm (avg) (percentage) -85.94% 6.330 0.890 - -70.23% -
464.h264ref -85.32% 293.000 43.000 - -56.12% -
464.h264ref (avg) -85.32% 293.000 43.000 - -56.12% -
464.h264ref (avg) (percentage) -85.07% 5.090 0.760 - 5.56% -
434.zeusmp (avg) (percentage) -83.08% 5.850 0.990 - -50.00% -
434.zeusmp -82.99% 147.000 25.000 - -90.60% -
434.zeusmp (avg) -82.99% 147.000 25.000 - -90.60% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - -43.70% -
453.povray -80.95% 42.000 8.000 - -93.80% -
453.povray (avg) -80.95% 42.000 8.000 - -93.80% -
436.cactusADM (avg) (percentage) -80.94% 5.980 1.140 - -42.71% -
458.sjeng -80.91% 241.000 46.000 - 43.75% -
458.sjeng (avg) -80.91% 241.000 46.000 - 43.75% -
447.dealII -80.81% 99.000 19.000 - -81.19% -
447.dealII (avg) -80.81% 99.000 19.000 - -81.19% -
436.cactusADM -80.65% 124.000 24.000 - -93.18% -
436.cactusADM (avg) -80.65% 124.000 24.000 - -93.18% -
453.povray (avg) (percentage) -80.28% 7.810 1.540 - -21.83% -
458.sjeng (avg) (percentage) -80.25% 3.190 0.630 - 173.91% -
459.GemsFDTD -80.00% 360.000 72.000 - -79.31% -
459.GemsFDTD (avg) -80.00% 360.000 72.000 - -79.31% -
462.libquantum -80.00% 10.000 2.000 - -98.77% -
462.libquantum (avg) -80.00% 10.000 2.000 - -98.77% -
482.sphinx3 -79.73% 74.000 15.000 - -98.01% -
482.sphinx3 (avg) -79.73% 74.000 15.000 - -98.01% -
433.milc -79.71% 69.000 14.000 - -94.49% -
433.milc (avg) -79.71% 69.000 14.000 - -94.49% -
433.milc (avg) (percentage) -79.50% 6.050 1.240 - -37.06% -
410.bwaves (avg) (percentage) -79.14% 4.410 0.920 - -40.65% -
447.dealII (avg) (percentage) -78.86% 5.770 1.220 - 7.02% -
410.bwaves -78.31% 249.000 54.000 - -75.78% -
410.bwaves (avg) -78.31% 249.000 54.000 - -75.78% -
445.gobmk -77.55% 196.000 44.000 - -38.03% -
445.gobmk (avg) -77.55% 196.000 44.000 - -38.03% -
445.gobmk (avg) (percentage) -77.24% 3.910 0.890 - 43.55% -
482.sphinx3 (avg) (percentage) -77.18% 7.930 1.810 - -27.02% -
459.GemsFDTD (avg) (percentage) -76.88% 5.840 1.350 - -8.78% -
462.libquantum (avg) (percentage) -75.98% 7.660 1.840 - 30.50% -
471.omnetpp (avg) (percentage) -73.66% 8.200 2.160 - 20.00% -
471.omnetpp -73.09% 223.000 60.000 - -68.75% -
471.omnetpp (avg) -73.09% 223.000 60.000 - -68.75% -
401.bzip2 (avg) (percentage) -72.26% 5.480 1.520 - 123.53% -
401.bzip2 -72.05% 161.000 45.000 - -55.45% -
401.bzip2 (avg) -72.05% 161.000 45.000 - -55.45% -
400.perlbench -71.43% 105.000 30.000 - -80.39% -
400.perlbench (avg) -71.43% 105.000 30.000 - -80.39% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - -4.26% -
400.perlbench (avg) (percentage) -67.87% 6.350 2.040 - 45.71% -
435.gromacs -65.94% 367.000 125.000 - -49.19% -
435.gromacs (avg) -65.94% 367.000 125.000 - -49.19% -
435.gromacs (avg) (percentage) -65.22% 5.060 1.760 - 8.64% -
465.tonto (avg) (percentage) -63.17% 3.340 1.230 - 14.95% -
465.tonto -62.74% 365.000 136.000 - 44.68% -
465.tonto (avg) -62.74% 365.000 136.000 - 44.68% -
450.soplex -61.11% 36.000 14.000 - -93.30% -
450.soplex (avg) -61.11% 36.000 14.000 - -93.30% -
473.astar -61.03% 213.000 83.000 - 1.22% -
473.astar (avg) -61.03% 213.000 83.000 - 1.22% -
450.soplex (avg) (percentage) -60.65% 6.430 2.530 - 91.67% -
473.astar (avg) (percentage) -60.58% 4.160 1.640 - 182.76% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 605.88% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 745.71% -
429.mcf (avg) (percentage) -35.46% 7.670 4.950 - 334.21% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 1397.06% -
437.leslie3d (max) -16.54% 1929.000 1610.000 - 158.84% -
429.mcf -13.76% 109.000 94.000 - -52.28% -
429.mcf (avg) -13.76% 109.000 94.000 - -52.28% -
437.leslie3d (max) (percentage) -13.75% 16.580 14.300 - 225.74% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 134.48% -
nb_compile_successful 65.52% 29.000 48.000 - 65.52% -
nb_execution_successful 65.52% 29.000 48.000 - 65.52% -

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1508.000 -86.24%
401.bzip2 3019.000 -79.79%
410.bwaves 5891.000 -59.19%
429.mcf 1904.000 -88.99%
433.milc 1155.000 -91.06%
434.zeusmp 2535.000 -81.20%
435.gromacs 7143.000 -53.13%
436.cactusADM 2122.000 -88.01%
437.leslie3d 11259.000 -20.66%
444.namd 783.000 -91.31%
445.gobmk 4948.000 -57.10%
447.dealII 1638.000 -81.56%
450.soplex 585.000 -96.31%
453.povray 544.000 -91.75%
456.hmmer 2875.000 -74.15%
458.sjeng 7400.000 -47.22%
459.GemsFDTD 5366.000 -77.23%
462.libquantum 135.000 -98.83%
464.h264ref 5769.000 -57.83%
465.tonto 11133.000 26.67%
470.lbm 3785.000 -70.43%
471.omnetpp 2781.000 -74.03%
473.astar 5114.000 -63.86%
482.sphinx3 836.000 -97.25%
Geometric Mean 2454.913 -81.41%
Code Size (bytes)
Name Current %
400.perlbench 887723.000 -1.05%
401.bzip2 43715.000 -1.14%
410.bwaves 16464.000 -1.81%
429.mcf 9119.000 -
433.milc 60509.000 -1.18%
434.zeusmp 146667.000 -4.70%
435.gromacs 489745.000 -2.09%
436.cactusADM 348359.000 -1.27%
437.leslie3d 54786.000 -7.77%
444.namd 185768.000 -1.35%
445.gobmk 1374950.000 -
447.dealII 307900.000 -1.61%
450.soplex 202558.000 -4.04%
453.povray 653881.000 -1.22%
456.hmmer 107239.000 -1.28%
458.sjeng 106376.000 2.26%
459.GemsFDTD 232732.000 -6.13%
462.libquantum 15893.000 -
464.h264ref 379348.000 -2.03%
465.tonto 1976555.000 1.60%
470.lbm 9547.000 -1.69%
471.omnetpp 475384.000 -1.47%
473.astar 22760.000 -
482.sphinx3 111671.000 -1.03%
Geometric Mean 136774.909 -1.73%
Score
Name Current %
nb_compile_successful 48.000 65.52%
nb_execution_successful 48.000 65.52%
nb_total_tests 68.000 134.48%
Geometric Mean 53.909 85.89%
sample variability
Name Current %
400.perlbench 30.000 -80.39%
400.perlbench (avg) 30.000 -80.39%
400.perlbench (avg) (percentage) 2.040 45.71%
400.perlbench (max) 36.000 -81.91%
400.perlbench (max) (percentage) 2.410 32.42%
401.bzip2 45.000 -55.45%
401.bzip2 (avg) 45.000 -55.45%
401.bzip2 (avg) (percentage) 1.520 123.53%
401.bzip2 (max) 42.000 13.51%
401.bzip2 (max) (percentage) 1.400 460.00%
401.bzip2 (unexplained diff vs previous) 5.000 -84.38%
401.bzip2 (unexplained diff vs previous) (percentage) 0.170 -19.05%
403.gcc (avg) (percentage) 5.090 1397.06%
403.gcc (max) (percentage) 3.830 1026.47%
410.bwaves 54.000 -75.78%
410.bwaves (avg) 54.000 -75.78%
410.bwaves (avg) (percentage) 0.920 -40.65%
410.bwaves (max) 47.000 -88.31%
410.bwaves (max) (percentage) 0.800 -71.33%
410.bwaves (unexplained diff vs previous) 20.000 -84.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.340 -60.92%
416.gamess (avg) (percentage) 0.760 -43.70%
416.gamess (max) (percentage) 0.380 -85.98%
429.mcf 94.000 -52.28%
429.mcf (avg) 94.000 -52.28%
429.mcf (avg) (percentage) 4.950 334.21%
429.mcf (max) 31.000 -29.55%
429.mcf (max) (percentage) 1.670 542.31%
433.milc 14.000 -94.49%
433.milc (avg) 14.000 -94.49%
433.milc (avg) (percentage) 1.240 -37.06%
433.milc (max) 13.000 -95.78%
433.milc (max) (percentage) 1.140 -52.30%
433.milc (unexplained diff vs previous) 5.000 -88.64%
433.milc (unexplained diff vs previous) (percentage) 0.430 26.47%
434.zeusmp 25.000 -90.60%
434.zeusmp (avg) 25.000 -90.60%
434.zeusmp (avg) (percentage) 0.990 -50.00%
434.zeusmp (max) 6.000 -98.07%
434.zeusmp (max) (percentage) 0.260 -88.74%
435.gromacs 125.000 -49.19%
435.gromacs (avg) 125.000 -49.19%
435.gromacs (avg) (percentage) 1.760 8.64%
435.gromacs (max) 87.000 -77.40%
435.gromacs (max) (percentage) 1.220 -51.78%
436.cactusADM 24.000 -93.18%
436.cactusADM (avg) 24.000 -93.18%
436.cactusADM (avg) (percentage) 1.140 -42.71%
436.cactusADM (max) 19.000 -95.31%
436.cactusADM (max) (percentage) 0.940 -58.95%
437.leslie3d 511.000 38.86%
437.leslie3d (avg) 511.000 38.86%
437.leslie3d (avg) (percentage) 4.540 74.62%
437.leslie3d (max) 1610.000 158.84%
437.leslie3d (max) (percentage) 14.300 225.74%
444.namd 9.000 -98.05%
444.namd (avg) 9.000 -98.05%
444.namd (avg) (percentage) 1.240 -75.83%
444.namd (max) 3.000 -99.52%
444.namd (max) (percentage) 0.500 -92.84%
445.gobmk 44.000 -38.03%
445.gobmk (avg) 44.000 -38.03%
445.gobmk (avg) (percentage) 0.890 43.55%
445.gobmk (max) 32.000 -74.40%
445.gobmk (max) (percentage) 0.650 -40.37%
447.dealII 19.000 -81.19%
447.dealII (avg) 19.000 -81.19%
447.dealII (avg) (percentage) 1.220 7.02%
447.dealII (max) 29.000 -88.93%
447.dealII (max) (percentage) 1.790 -39.53%
450.soplex 14.000 -93.30%
450.soplex (avg) 14.000 -93.30%
450.soplex (avg) (percentage) 2.530 91.67%
450.soplex (max) 6.000 -99.12%
450.soplex (max) (percentage) 1.100 -74.42%
453.povray 8.000 -93.80%
453.povray (avg) 8.000 -93.80%
453.povray (avg) (percentage) 1.540 -21.83%
453.povray (max) 9.000 -93.23%
453.povray (max) (percentage) 1.790 -11.39%
454.calculix (avg) (percentage) 3.600 605.88%
454.calculix (max) (percentage) 0.830 -20.19%
456.hmmer 15.000 -76.56%
456.hmmer (avg) 15.000 -76.56%
456.hmmer (avg) (percentage) 0.540 -6.90%
456.hmmer (max) 2.000 -96.67%
456.hmmer (max) (percentage) 0.090 -83.33%
458.sjeng 46.000 43.75%
458.sjeng (avg) 46.000 43.75%
458.sjeng (avg) (percentage) 0.630 173.91%
458.sjeng (max) 10.000 -69.70%
458.sjeng (max) (percentage) 0.140 -41.67%
459.GemsFDTD 72.000 -79.31%
459.GemsFDTD (avg) 72.000 -79.31%
459.GemsFDTD (avg) (percentage) 1.350 -8.78%
459.GemsFDTD (max) 27.000 -89.58%
459.GemsFDTD (max) (percentage) 0.520 -52.73%
462.libquantum 2.000 -98.77%
462.libquantum (avg) 2.000 -98.77%
462.libquantum (avg) (percentage) 1.840 30.50%
462.libquantum (max) 4.000 -94.94%
462.libquantum (max) (percentage) 3.360 386.96%
464.h264ref 43.000 -56.12%
464.h264ref (avg) 43.000 -56.12%
464.h264ref (avg) (percentage) 0.760 5.56%
464.h264ref (max) 51.000 -66.67%
464.h264ref (max) (percentage) 0.900 -19.64%
465.tonto 136.000 44.68%
465.tonto (avg) 136.000 44.68%
465.tonto (avg) (percentage) 1.230 14.95%
465.tonto (max) 77.000 -56.50%
465.tonto (max) (percentage) 0.700 -65.35%
470.lbm 33.000 -91.36%
470.lbm (avg) 33.000 -91.36%
470.lbm (avg) (percentage) 0.890 -70.23%
470.lbm (max) 24.000 -94.20%
470.lbm (max) (percentage) 0.660 -79.63%
470.lbm (unexplained diff vs previous) 16.000 433.33%
470.lbm (unexplained diff vs previous) (percentage) 0.420 2000.00%
471.omnetpp 60.000 -68.75%
471.omnetpp (avg) 60.000 -68.75%
471.omnetpp (avg) (percentage) 2.160 20.00%
471.omnetpp (max) 33.000 -17.50%
471.omnetpp (max) (percentage) 1.190 213.16%
471.omnetpp (unexplained diff vs previous) 25.000 -32.43%
471.omnetpp (unexplained diff vs previous) (percentage) 0.900 157.14%
473.astar 83.000 1.22%
473.astar (avg) 83.000 1.22%
473.astar (avg) (percentage) 1.640 182.76%
473.astar (max) 232.000 -55.38%
473.astar (max) (percentage) 4.550 23.64%
481.wrf (avg) (percentage) 1.800 -4.26%
481.wrf (max) (percentage) 0.830 -77.63%
482.sphinx3 15.000 -98.01%
482.sphinx3 (avg) 15.000 -98.01%
482.sphinx3 (avg) (percentage) 1.810 -27.02%
482.sphinx3 (max) 6.000 -99.65%
482.sphinx3 (max) (percentage) 0.760 -86.45%
483.xalancbmk (avg) (percentage) 2.960 745.71%
483.xalancbmk (max) (percentage) 1.770 342.50%
Geometric Mean 6.634 -62.13%