Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=878e894801169461adc00129ad33947ad541420d (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/9715c545d33b3a32ddc1ae817ba9356ade1fb9df (+ 1 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=9c72830eb3e12c1d57ac7d1992c96ee69ccd835d (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=f9f042df0c5a1377b526200359b1d1124e1cd1ff (unchanged)
regression GNU-1057
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/204/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0204 (basepoints/gcc-14-06037-g9715c545d33) 2024-01-29T14:09:01 0:00:00
Previous 0203 (basepoints/gcc-14-06036-g4ecfa6c4a9d) 2024-01-29T07:57:32 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 38 11
Performance Improvements 119 186
Added Tests 2 2
Unchanged Tests 1077 1037
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 4.86% 10710.000 11231.000 - -3.51% -
400.perlbench 2.20% 1546.000 1580.000 - -4.99% -
447.dealII 1.18% 1691.000 1711.000 - -0.75% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
464.h264ref (unexplained diff vs previous) 240.00% 5.000 17.000 - -99.75% -
437.leslie3d (unexplained diff vs previous) 223.60% 161.000 521.000 - -77.48% -
464.h264ref (unexplained diff vs previous) (percentage) 222.22% 0.090 0.290 - -99.76% -
437.leslie3d (unexplained diff vs previous) (percentage) 209.33% 1.500 4.640 - -76.66% -
482.sphinx3 (unexplained diff vs previous) (percentage) 204.76% 0.210 0.640 - -99.98% -
482.sphinx3 (unexplained diff vs previous) 200.00% 2.000 6.000 - -99.98% -
444.namd (unexplained diff vs previous) 150.00% 2.000 5.000 - -99.94% -
444.namd (unexplained diff vs previous) (percentage) 148.15% 0.270 0.670 - -99.94% -
410.bwaves (unexplained diff vs previous) 146.67% 15.000 37.000 - -99.53% -
410.bwaves (unexplained diff vs previous) (percentage) 140.74% 0.270 0.650 - -99.53% -
437.leslie3d (max) 140.09% 227.000 545.000 - -71.75% -
437.leslie3d (max) (percentage) 129.25% 2.120 4.860 - -70.69% -
450.soplex (unexplained diff vs previous) 100.00% 1.000 2.000 - -99.99% -
450.soplex (unexplained diff vs previous) (percentage) 100.00% 0.170 0.340 - -99.99% -
458.sjeng (unexplained diff vs previous) (percentage) 83.33% 0.120 0.220 - -99.76% -
458.sjeng (unexplained diff vs previous) 77.78% 9.000 16.000 - -99.77% -
470.lbm (unexplained diff vs previous) (percentage) 52.38% 0.210 0.320 - -99.88% -
470.lbm (unexplained diff vs previous) 50.00% 8.000 12.000 - -99.88% -
444.namd (max) (percentage) 48.89% 0.450 0.670 - -99.27% -
444.namd (max) 33.33% 3.000 4.000 - -99.42% -
429.mcf (unexplained diff vs previous) (percentage) 33.33% 0.930 1.240 - -99.91% -
429.mcf (unexplained diff vs previous) 30.77% 13.000 17.000 - -99.91% -
459.GemsFDTD (unexplained diff vs previous) 17.24% 29.000 34.000 - -99.78% -
456.hmmer (unexplained diff vs previous) (percentage) 16.67% 0.240 0.280 - -99.90% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 14.89% 0.470 0.540 - -99.78% -
456.hmmer (unexplained diff vs previous) 14.29% 7.000 8.000 - -99.90% -
437.leslie3d 4.84% 517.000 542.000 - 52.68% -
437.leslie3d (avg) 4.84% 517.000 542.000 - 52.68% -
447.dealII 4.00% 25.000 26.000 - -73.74% -
447.dealII (avg) 4.00% 25.000 26.000 - -73.74% -
400.perlbench 2.78% 36.000 37.000 - -64.76% -
400.perlbench (avg) 2.78% 36.000 37.000 - -64.76% -
462.libquantum (unexplained diff vs previous) (percentage) 1.39% 0.720 0.730 - -99.99% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -3.85% 26.000 25.000 - -13.79% -
nb_execution_successful -3.85% 26.000 25.000 - -13.79% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf -1.22% 1393.000 1376.000 - -3.37% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
473.astar (unexplained diff vs previous) -83.78% 37.000 6.000 - -99.94% -
473.astar (unexplained diff vs previous) (percentage) -83.33% 0.720 0.120 - -99.94% -
445.gobmk (unexplained diff vs previous) -79.41% 34.000 7.000 - -99.87% -
445.gobmk (unexplained diff vs previous) (percentage) -79.41% 0.680 0.140 - -99.87% -
416.gamess (unexplained diff vs previous) (percentage) -76.47% 0.170 0.040 - -99.94% -
416.gamess (unexplained diff vs previous) -76.19% 21.000 5.000 - -99.94% -
453.povray (unexplained diff vs previous) (percentage) -75.34% 0.730 0.180 - -99.98% -
453.povray (unexplained diff vs previous) -75.00% 4.000 1.000 - -99.98% -
465.tonto (unexplained diff vs previous) (percentage) -56.12% 1.390 0.610 - -96.23% -
465.tonto (unexplained diff vs previous) -55.70% 158.000 70.000 - -96.05% -
471.omnetpp (unexplained diff vs previous) -52.00% 25.000 12.000 - -99.89% -
471.omnetpp (unexplained diff vs previous) (percentage) -51.65% 0.910 0.440 - -99.89% -
435.gromacs (unexplained diff vs previous) (percentage) -44.90% 0.490 0.270 - -99.72% -
435.gromacs (unexplained diff vs previous) -44.44% 36.000 20.000 - -99.72% -
401.bzip2 (unexplained diff vs previous) -41.18% 17.000 10.000 - -99.91% -
401.bzip2 (unexplained diff vs previous) (percentage) -41.07% 0.560 0.330 - -99.91% -
433.milc (unexplained diff vs previous) -37.50% 8.000 5.000 - -99.96% -
433.milc (unexplained diff vs previous) (percentage) -37.14% 0.700 0.440 - -99.96% -
416.gamess (max) (percentage) -35.00% 0.200 0.130 - -99.66% -
416.gamess (max) -33.33% 24.000 16.000 - -99.67% -
436.cactusADM (unexplained diff vs previous) -33.33% 3.000 2.000 - -99.99% -
436.cactusADM (unexplained diff vs previous) (percentage) -28.57% 0.140 0.100 - -99.99% -
447.dealII (unexplained diff vs previous) (percentage) -26.88% 1.600 1.170 - -99.71% -
447.dealII (unexplained diff vs previous) -25.93% 27.000 20.000 - -99.71% -
435.gromacs (max) -10.10% 208.000 187.000 - -94.77% -
400.perlbench (max) (percentage) -10.02% 4.090 3.680 - -95.62% -
453.povray -10.00% 10.000 9.000 - -78.57% -
453.povray (avg) -10.00% 10.000 9.000 - -78.57% -
482.sphinx3 (max) -10.00% 20.000 18.000 - -98.02% -
435.gromacs (max) (percentage) -9.89% 2.830 2.550 - -94.81% -
436.cactusADM (max) (percentage) -9.80% 1.020 0.920 - -98.96% -
436.cactusADM (max) -9.52% 21.000 19.000 - -98.97% -
434.zeusmp (max) -9.09% 33.000 30.000 - -98.54% -
456.hmmer (max) -9.09% 11.000 10.000 - -99.53% -
450.soplex (max) -8.33% 36.000 33.000 - -93.94% -
447.dealII (max) (percentage) -8.28% 1.570 1.440 - -98.21% -
471.omnetpp (max) -8.14% 86.000 79.000 - -96.36% -
400.perlbench (max) -7.94% 63.000 58.000 - -95.85% -
459.GemsFDTD (max) (percentage) -7.77% 1.930 1.780 - -97.51% -
456.hmmer (max) (percentage) -7.69% 0.390 0.360 - -99.51% -
447.dealII (max) -7.69% 26.000 24.000 - -98.27% -
482.sphinx3 (max) (percentage) -7.41% 2.160 2.000 - -97.93% -
450.soplex (max) (percentage) -7.41% 6.210 5.750 - -94.03% -
473.astar (max) -7.41% 216.000 200.000 - -94.13% -
473.astar (max) (percentage) -7.40% 4.190 3.880 - -94.16% -
465.tonto (max) (percentage) -7.39% 2.840 2.630 - -86.38% -
433.milc (max) (percentage) -7.36% 2.580 2.390 - -97.37% -
401.bzip2 (max) (percentage) -7.34% 2.860 2.650 - -96.63% -
429.mcf (max) -7.32% 314.000 291.000 - -77.99% -
471.omnetpp (max) (percentage) -7.30% 3.150 2.920 - -96.33% -
470.lbm (max) -7.14% 42.000 39.000 - -98.58% -
453.povray (max) (percentage) -7.14% 0.980 0.910 - -99.00% -
458.sjeng (max) -7.06% 85.000 79.000 - -97.81% -
401.bzip2 (max) -6.98% 86.000 80.000 - -96.54% -
458.sjeng (max) (percentage) -6.96% 1.150 1.070 - -97.75% -
434.zeusmp (max) (percentage) -6.92% 1.300 1.210 - -98.52% -
433.milc (max) -6.90% 29.000 27.000 - -97.41% -
465.tonto (max) -6.81% 323.000 301.000 - -85.75% -
459.GemsFDTD (max) -6.72% 119.000 111.000 - -97.48% -
464.h264ref (max) -6.67% 45.000 42.000 - -98.66% -
445.gobmk (max) (percentage) -6.60% 1.970 1.840 - -96.50% -
410.bwaves (max) (percentage) -6.38% 0.940 0.880 - -98.48% -
464.h264ref (max) (percentage) -6.33% 0.790 0.740 - -98.64% -
470.lbm (max) (percentage) -6.31% 1.110 1.040 - -98.57% -
429.mcf (max) (percentage) -6.25% 22.570 21.160 - -77.22% -
445.gobmk (max) -6.12% 98.000 92.000 - -96.52% -
410.bwaves (max) -5.77% 52.000 49.000 - -98.50% -
462.libquantum (max) (percentage) -5.38% 1.860 1.760 - -98.22% -
482.sphinx3 -5.00% 20.000 19.000 - -74.32% -
482.sphinx3 (avg) -5.00% 20.000 19.000 - -74.32% -
454.calculix (max) (percentage) -4.20% 1.190 1.140 - -98.86% -
456.hmmer -4.17% 24.000 23.000 - -81.89% -
456.hmmer (avg) -4.17% 24.000 23.000 - -81.89% -
416.gamess (avg) (percentage) -3.49% 0.860 0.830 - -81.22% -
416.gamess -2.83% 106.000 103.000 - -81.27% -
416.gamess (avg) -2.83% 106.000 103.000 - -81.27% -
434.zeusmp -2.70% 37.000 36.000 - -75.51% -
434.zeusmp (avg) -2.70% 37.000 36.000 - -75.51% -
403.gcc (max) (percentage) -2.69% 5.570 5.420 - -94.55% -
435.gromacs -2.58% 155.000 151.000 - -58.86% -
435.gromacs (avg) -2.58% 155.000 151.000 - -58.86% -
400.perlbench (unexplained diff vs previous) (percentage) -2.27% 2.200 2.150 - -99.59% -
429.mcf -2.13% 94.000 92.000 - -15.60% -
429.mcf (avg) -2.13% 94.000 92.000 - -15.60% -
434.zeusmp (avg) (percentage) -2.04% 1.470 1.440 - -75.38% -
470.lbm -2.00% 50.000 49.000 - -79.50% -
470.lbm (avg) -2.00% 50.000 49.000 - -79.50% -
473.astar -1.96% 102.000 100.000 - -53.05% -
473.astar (avg) -1.96% 102.000 100.000 - -53.05% -
435.gromacs (avg) (percentage) -1.90% 2.110 2.070 - -59.09% -
436.cactusADM (avg) (percentage) -1.75% 1.710 1.680 - -71.91% -
445.gobmk (avg) (percentage) -1.75% 1.140 1.120 - -71.36% -
464.h264ref (avg) (percentage) -1.75% 1.140 1.120 - -78.00% -
450.soplex (avg) (percentage) -1.68% 2.980 2.930 - -54.43% -
481.wrf (max) (percentage) -1.67% 1.200 1.180 - -98.28% -
453.povray (avg) (percentage) -1.64% 1.830 1.800 - -76.95% -
458.sjeng -1.64% 61.000 60.000 - -75.10% -
458.sjeng (avg) -1.64% 61.000 60.000 - -75.10% -
401.bzip2 -1.61% 62.000 61.000 - -62.11% -
401.bzip2 (avg) -1.61% 62.000 61.000 - -62.11% -
464.h264ref -1.54% 65.000 64.000 - -78.16% -
464.h264ref (avg) -1.54% 65.000 64.000 - -78.16% -
473.astar (avg) (percentage) -1.52% 1.980 1.950 - -53.12% -
471.omnetpp -1.52% 66.000 65.000 - -70.85% -
471.omnetpp (avg) -1.52% 66.000 65.000 - -70.85% -
470.lbm (avg) (percentage) -1.50% 1.330 1.310 - -79.30% -
401.bzip2 (avg) (percentage) -1.46% 2.050 2.020 - -63.14% -
482.sphinx3 (avg) (percentage) -1.40% 2.150 2.120 - -73.27% -
429.mcf (avg) (percentage) -1.32% 6.800 6.710 - -12.52% -
465.tonto (avg) (percentage) -1.28% 1.560 1.540 - -53.89% -
471.omnetpp (avg) (percentage) -1.23% 2.430 2.400 - -70.73% -
458.sjeng (avg) (percentage) -1.20% 0.830 0.820 - -74.29% -
456.hmmer (avg) (percentage) -1.19% 0.840 0.830 - -81.18% -
459.GemsFDTD (avg) (percentage) -1.18% 1.690 1.670 - -71.40% -
433.milc (avg) (percentage) -1.14% 1.750 1.730 - -71.40% -
444.namd (avg) (percentage) -1.11% 1.800 1.780 - -80.69% -
462.libquantum (avg) (percentage) -1.07% 1.870 1.850 - -75.85% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 3.12% 32.000 33.000 - 13.79% -

Added Tests - sample variability
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 1.72% 205081.000 208601.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 4.75% 10941.000 11461.000 - 0.61% -
450.soplex 3.70% 567.000 588.000 - -0.34% -
401.bzip2 2.72% 2941.000 3021.000 - -0.33% -
459.GemsFDTD 1.21% 6173.000 6248.000 - 0.55% -
434.zeusmp 1.15% 2514.000 2543.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 58.36% 3.050 4.830 - 0.00% -
437.leslie3d 52.68% 355.000 542.000 - 4.84% -
437.leslie3d (avg) 52.68% 355.000 542.000 - 4.84% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -13.79% 29.000 25.000 - -3.85% -
nb_execution_successful -13.79% 29.000 25.000 - -3.85% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -1.45% 244728.000 241184.000 - 0.00% -
410.bwaves -1.22% 16760.000 16556.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
400.perlbench -4.99% 1663.000 1580.000 - 2.20% -
437.leslie3d -3.51% 11640.000 11231.000 - 4.86% -
429.mcf -3.37% 1424.000 1376.000 - -1.22% -
462.libquantum -2.14% 140.000 137.000 - -0.72% -
458.sjeng -1.97% 7567.000 7418.000 - 0.22% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) -99.99% 13518.000 1.000 - 0.00% -
462.libquantum (unexplained diff vs previous) (percentage) -99.99% 9655.710 0.730 - 1.39% -
436.cactusADM (unexplained diff vs previous) -99.99% 16396.000 2.000 - -33.33% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.99% 787.130 0.100 - -28.57% -
450.soplex (unexplained diff vs previous) (percentage) -99.99% 2557.320 0.340 - 100.00% -
450.soplex (unexplained diff vs previous) -99.99% 14500.000 2.000 - 100.00% -
453.povray (unexplained diff vs previous) (percentage) -99.98% 1043.330 0.180 - -75.34% -
453.povray (unexplained diff vs previous) -99.98% 5707.000 1.000 - -75.00% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 6.000 - 200.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.98% 2759.050 0.640 - 204.76% -
433.milc (unexplained diff vs previous) -99.96% 11289.000 5.000 - -37.50% -
433.milc (unexplained diff vs previous) (percentage) -99.96% 980.800 0.440 - -37.14% -
473.astar (unexplained diff vs previous) -99.94% 10146.000 6.000 - -83.78% -
444.namd (unexplained diff vs previous) -99.94% 8356.000 5.000 - 150.00% -
444.namd (unexplained diff vs previous) (percentage) -99.94% 1118.610 0.670 - 148.15% -
473.astar (unexplained diff vs previous) (percentage) -99.94% 197.850 0.120 - -83.33% -
416.gamess (unexplained diff vs previous) (percentage) -99.94% 62.730 0.040 - -76.47% -
416.gamess (unexplained diff vs previous) -99.94% 7819.000 5.000 - -76.19% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.91% 368.340 0.330 - -41.07% -
429.mcf (unexplained diff vs previous) -99.91% 18593.000 17.000 - 30.77% -
401.bzip2 (unexplained diff vs previous) -99.91% 10833.000 10.000 - -41.18% -
429.mcf (unexplained diff vs previous) (percentage) -99.91% 1305.690 1.240 - 33.33% -
456.hmmer (unexplained diff vs previous) -99.90% 8030.000 8.000 - 14.29% -
456.hmmer (unexplained diff vs previous) (percentage) -99.90% 278.630 0.280 - 16.67% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.89% 391.300 0.440 - -51.65% -
471.omnetpp (unexplained diff vs previous) -99.89% 10659.000 12.000 - -52.00% -
470.lbm (unexplained diff vs previous) -99.88% 9990.000 12.000 - 50.00% -
470.lbm (unexplained diff vs previous) (percentage) -99.88% 263.730 0.320 - 52.38% -
445.gobmk (unexplained diff vs previous) -99.87% 5581.000 7.000 - -79.41% -
445.gobmk (unexplained diff vs previous) (percentage) -99.87% 111.110 0.140 - -79.41% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.78% 250.410 0.540 - 14.89% -
459.GemsFDTD (unexplained diff vs previous) -99.78% 15458.000 34.000 - 17.24% -
458.sjeng (unexplained diff vs previous) -99.77% 6876.000 16.000 - 77.78% -
458.sjeng (unexplained diff vs previous) (percentage) -99.76% 90.870 0.220 - 83.33% -
464.h264ref (unexplained diff vs previous) (percentage) -99.76% 118.780 0.290 - 222.22% -
464.h264ref (unexplained diff vs previous) -99.75% 6843.000 17.000 - 240.00% -
435.gromacs (unexplained diff vs previous) (percentage) -99.72% 96.680 0.270 - -44.90% -
435.gromacs (unexplained diff vs previous) -99.72% 7028.000 20.000 - -44.44% -
447.dealII (unexplained diff vs previous) -99.71% 7017.000 20.000 - -25.93% -
447.dealII (unexplained diff vs previous) (percentage) -99.71% 407.020 1.170 - -26.88% -
416.gamess (max) -99.67% 4805.000 16.000 - -33.33% -
416.gamess (max) (percentage) -99.66% 38.550 0.130 - -35.00% -
400.perlbench (unexplained diff vs previous) -99.61% 8772.000 34.000 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) -99.59% 527.480 2.150 - -2.27% -
410.bwaves (unexplained diff vs previous) (percentage) -99.53% 137.900 0.650 - 140.74% -
456.hmmer (max) -99.53% 2120.000 10.000 - -9.09% -
410.bwaves (unexplained diff vs previous) -99.53% 7794.000 37.000 - 146.67% -
456.hmmer (max) (percentage) -99.51% 73.590 0.360 - -7.69% -
444.namd (max) -99.42% 685.000 4.000 - 33.33% -
444.namd (max) (percentage) -99.27% 91.790 0.670 - 48.89% -
453.povray (max) (percentage) -99.00% 91.250 0.910 - -7.14% -
453.povray (max) -99.00% 499.000 5.000 - 0.00% -
436.cactusADM (max) -98.97% 1848.000 19.000 - -9.52% -
436.cactusADM (max) (percentage) -98.96% 88.730 0.920 - -9.80% -
454.calculix (max) (percentage) -98.86% 99.730 1.140 - -4.20% -
464.h264ref (max) -98.66% 3127.000 42.000 - -6.67% -
464.h264ref (max) (percentage) -98.64% 54.290 0.740 - -6.33% -
470.lbm (max) -98.58% 2746.000 39.000 - -7.14% -
470.lbm (max) (percentage) -98.57% 72.510 1.040 - -6.31% -
462.libquantum (max) -98.55% 138.000 2.000 - 0.00% -
434.zeusmp (max) -98.54% 2055.000 30.000 - -9.09% -
434.zeusmp (max) (percentage) -98.52% 81.770 1.210 - -6.92% -
410.bwaves (max) -98.50% 3276.000 49.000 - -5.77% -
410.bwaves (max) (percentage) -98.48% 57.970 0.880 - -6.38% -
481.wrf (max) (percentage) -98.28% 68.700 1.180 - -1.67% -
447.dealII (max) -98.27% 1384.000 24.000 - -7.69% -
462.libquantum (max) (percentage) -98.22% 98.970 1.760 - -5.38% -
447.dealII (max) (percentage) -98.21% 80.280 1.440 - -8.28% -
482.sphinx3 (max) -98.02% 911.000 18.000 - -10.00% -
482.sphinx3 (max) (percentage) -97.93% 96.500 2.000 - -7.41% -
458.sjeng (max) -97.81% 3602.000 79.000 - -7.06% -
458.sjeng (max) (percentage) -97.75% 47.610 1.070 - -6.96% -
459.GemsFDTD (max) (percentage) -97.51% 71.460 1.780 - -7.77% -
459.GemsFDTD (max) -97.48% 4411.000 111.000 - -6.72% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
433.milc (max) -97.41% 1044.000 27.000 - -6.90% -
433.milc (max) (percentage) -97.37% 90.750 2.390 - -7.36% -
401.bzip2 (max) (percentage) -96.63% 78.650 2.650 - -7.34% -
401.bzip2 (max) -96.54% 2313.000 80.000 - -6.98% -
445.gobmk (max) -96.52% 2643.000 92.000 - -6.12% -
445.gobmk (max) (percentage) -96.50% 52.630 1.840 - -6.60% -
471.omnetpp (max) -96.36% 2169.000 79.000 - -8.14% -
471.omnetpp (max) (percentage) -96.33% 79.650 2.920 - -7.30% -
465.tonto (unexplained diff vs previous) (percentage) -96.23% 16.190 0.610 - -56.12% -
465.tonto (unexplained diff vs previous) -96.05% 1771.000 70.000 - -55.70% -
400.perlbench (max) -95.85% 1397.000 58.000 - -7.94% -
400.perlbench (max) (percentage) -95.62% 84.060 3.680 - -10.02% -
435.gromacs (max) (percentage) -94.81% 49.160 2.550 - -9.89% -
435.gromacs (max) -94.77% 3573.000 187.000 - -10.10% -
403.gcc (max) (percentage) -94.55% 99.360 5.420 - -2.69% -
473.astar (max) (percentage) -94.16% 66.430 3.880 - -7.40% -
473.astar (max) -94.13% 3406.000 200.000 - -7.41% -
450.soplex (max) (percentage) -94.03% 96.240 5.750 - -7.41% -
450.soplex (max) -93.94% 545.000 33.000 - -8.33% -
465.tonto (max) (percentage) -86.38% 19.310 2.630 - -7.39% -
465.tonto (max) -85.75% 2112.000 301.000 - -6.81% -
456.hmmer -81.89% 127.000 23.000 - -4.17% -
456.hmmer (avg) -81.89% 127.000 23.000 - -4.17% -
416.gamess -81.27% 550.000 103.000 - -2.83% -
416.gamess (avg) -81.27% 550.000 103.000 - -2.83% -
416.gamess (avg) (percentage) -81.22% 4.420 0.830 - -3.49% -
456.hmmer (avg) (percentage) -81.18% 4.410 0.830 - -1.19% -
444.namd -80.88% 68.000 13.000 - 0.00% -
444.namd (avg) -80.88% 68.000 13.000 - 0.00% -
444.namd (avg) (percentage) -80.69% 9.220 1.780 - -1.11% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
470.lbm -79.50% 239.000 49.000 - -2.00% -
470.lbm (avg) -79.50% 239.000 49.000 - -2.00% -
470.lbm (avg) (percentage) -79.30% 6.330 1.310 - -1.50% -
453.povray -78.57% 42.000 9.000 - -10.00% -
453.povray (avg) -78.57% 42.000 9.000 - -10.00% -
464.h264ref -78.16% 293.000 64.000 - -1.54% -
464.h264ref (avg) -78.16% 293.000 64.000 - -1.54% -
464.h264ref (avg) (percentage) -78.00% 5.090 1.120 - -1.75% -
429.mcf (max) -77.99% 1322.000 291.000 - -7.32% -
437.leslie3d (unexplained diff vs previous) -77.48% 2314.000 521.000 - 223.60% -
429.mcf (max) (percentage) -77.22% 92.890 21.160 - -6.25% -
453.povray (avg) (percentage) -76.95% 7.810 1.800 - -1.64% -
437.leslie3d (unexplained diff vs previous) (percentage) -76.66% 19.880 4.640 - 209.33% -
462.libquantum (avg) (percentage) -75.85% 7.660 1.850 - -1.07% -
434.zeusmp -75.51% 147.000 36.000 - -2.70% -
434.zeusmp (avg) -75.51% 147.000 36.000 - -2.70% -
434.zeusmp (avg) (percentage) -75.38% 5.850 1.440 - -2.04% -
458.sjeng -75.10% 241.000 60.000 - -1.64% -
458.sjeng (avg) -75.10% 241.000 60.000 - -1.64% -
482.sphinx3 -74.32% 74.000 19.000 - -5.00% -
482.sphinx3 (avg) -74.32% 74.000 19.000 - -5.00% -
458.sjeng (avg) (percentage) -74.29% 3.190 0.820 - -1.20% -
447.dealII -73.74% 99.000 26.000 - 4.00% -
447.dealII (avg) -73.74% 99.000 26.000 - 4.00% -
447.dealII (avg) (percentage) -73.66% 5.770 1.520 - 0.00% -
482.sphinx3 (avg) (percentage) -73.27% 7.930 2.120 - -1.40% -
433.milc -72.46% 69.000 19.000 - 0.00% -
433.milc (avg) -72.46% 69.000 19.000 - 0.00% -
436.cactusADM (avg) (percentage) -71.91% 5.980 1.680 - -1.75% -
410.bwaves -71.89% 249.000 70.000 - 0.00% -
410.bwaves (avg) -71.89% 249.000 70.000 - 0.00% -
436.cactusADM -71.77% 124.000 35.000 - 0.00% -
436.cactusADM (avg) -71.77% 124.000 35.000 - 0.00% -
437.leslie3d (max) -71.75% 1929.000 545.000 - 140.09% -
410.bwaves (avg) (percentage) -71.66% 4.410 1.250 - -0.79% -
445.gobmk -71.43% 196.000 56.000 - 0.00% -
445.gobmk (avg) -71.43% 196.000 56.000 - 0.00% -
433.milc (avg) (percentage) -71.40% 6.050 1.730 - -1.14% -
459.GemsFDTD (avg) (percentage) -71.40% 5.840 1.670 - -1.18% -
445.gobmk (avg) (percentage) -71.36% 3.910 1.120 - -1.75% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
459.GemsFDTD -71.11% 360.000 104.000 - -0.95% -
459.GemsFDTD (avg) -71.11% 360.000 104.000 - -0.95% -
471.omnetpp -70.85% 223.000 65.000 - -1.52% -
471.omnetpp (avg) -70.85% 223.000 65.000 - -1.52% -
471.omnetpp (avg) (percentage) -70.73% 8.200 2.400 - -1.23% -
437.leslie3d (max) (percentage) -70.69% 16.580 4.860 - 129.25% -
400.perlbench -64.76% 105.000 37.000 - 2.78% -
400.perlbench (avg) -64.76% 105.000 37.000 - 2.78% -
401.bzip2 (avg) (percentage) -63.14% 5.480 2.020 - -1.46% -
400.perlbench (avg) (percentage) -62.36% 6.350 2.390 - 0.00% -
401.bzip2 -62.11% 161.000 61.000 - -1.61% -
401.bzip2 (avg) -62.11% 161.000 61.000 - -1.61% -
435.gromacs (avg) (percentage) -59.09% 5.060 2.070 - -1.90% -
435.gromacs -58.86% 367.000 151.000 - -2.58% -
435.gromacs (avg) -58.86% 367.000 151.000 - -2.58% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
450.soplex (avg) (percentage) -54.43% 6.430 2.930 - -1.68% -
465.tonto (avg) (percentage) -53.89% 3.340 1.540 - -1.28% -
473.astar (avg) (percentage) -53.12% 4.160 1.950 - -1.52% -
473.astar -53.05% 213.000 100.000 - -1.96% -
473.astar (avg) -53.05% 213.000 100.000 - -1.96% -
450.soplex -52.78% 36.000 17.000 - 0.00% -
450.soplex (avg) -52.78% 36.000 17.000 - 0.00% -
465.tonto -51.78% 365.000 176.000 - -0.56% -
465.tonto (avg) -51.78% 365.000 176.000 - -0.56% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf -15.60% 109.000 92.000 - -2.13% -
429.mcf (avg) -15.60% 109.000 92.000 - -2.13% -
429.mcf (avg) (percentage) -12.52% 7.670 6.710 - -1.32% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 13.79% 29.000 33.000 - 3.12% -

Added Tests - sample variability
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1580.000 2.20%
401.bzip2 3021.000 -
410.bwaves 5667.000 -
416.gamess 12425.000 -
429.mcf 1376.000 -1.22%
433.milc 1145.000 -
434.zeusmp 2543.000 -
435.gromacs 7338.000 -
436.cactusADM 2103.000 -
437.leslie3d 11231.000 4.86%
444.namd 745.000 -
445.gobmk 5006.000 -
447.dealII 1711.000 1.18%
450.soplex 588.000 -
453.povray 551.000 -
456.hmmer 2888.000 -
458.sjeng 7418.000 -
459.GemsFDTD 6248.000 -
462.libquantum 137.000 -
464.h264ref 5798.000 -
465.tonto 11461.000 -
470.lbm 3791.000 -
471.omnetpp 2739.000 -
473.astar 5163.000 -
482.sphinx3 942.000 -
Geometric Mean 2623.647 -
Code Size (bytes)
Name Current %
400.perlbench 887707.000 -
401.bzip2 44007.000 -
410.bwaves 16556.000 -
416.gamess 5731769.000 -
429.mcf 9119.000 -
433.milc 61117.000 -
434.zeusmp 153211.000 -
435.gromacs 498121.000 -
436.cactusADM 351609.000 -
437.leslie3d 58926.000 -
444.namd 187924.000 -
445.gobmk 1379062.000 -
447.dealII 310927.000 -
450.soplex 208601.000 -
453.povray 658313.000 -
456.hmmer 108399.000 -
458.sjeng 103492.000 -
459.GemsFDTD 241184.000 -
462.libquantum 16117.000 -
464.h264ref 386364.000 -
465.tonto 1949594.000 -
470.lbm 9707.000 -
471.omnetpp 480478.000 -
473.astar 22996.000 -
482.sphinx3 112287.000 -
Geometric Mean 160773.412 -
Score
Name Current %
nb_compile_successful 25.000 -3.85%
nb_execution_successful 25.000 -3.85%
nb_total_tests 33.000 3.12%
Geometric Mean 27.424 -1.58%
sample variability
Name Current %
400.perlbench 37.000 2.78%
400.perlbench (avg) 37.000 2.78%
400.perlbench (avg) (percentage) 2.390 -
400.perlbench (max) 58.000 -7.94%
400.perlbench (max) (percentage) 3.680 -10.02%
400.perlbench (unexplained diff vs previous) 34.000 -
400.perlbench (unexplained diff vs previous) (percentage) 2.150 -2.27%
401.bzip2 61.000 -1.61%
401.bzip2 (avg) 61.000 -1.61%
401.bzip2 (avg) (percentage) 2.020 -1.46%
401.bzip2 (max) 80.000 -6.98%
401.bzip2 (max) (percentage) 2.650 -7.34%
401.bzip2 (unexplained diff vs previous) 10.000 -41.18%
401.bzip2 (unexplained diff vs previous) (percentage) 0.330 -41.07%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 5.420 -2.69%
410.bwaves 70.000 -
410.bwaves (avg) 70.000 -
410.bwaves (avg) (percentage) 1.250 -
410.bwaves (max) 49.000 -5.77%
410.bwaves (max) (percentage) 0.880 -6.38%
410.bwaves (unexplained diff vs previous) 37.000 146.67%
410.bwaves (unexplained diff vs previous) (percentage) 0.650 140.74%
416.gamess 103.000 -2.83%
416.gamess (avg) 103.000 -2.83%
416.gamess (avg) (percentage) 0.830 -3.49%
416.gamess (max) 16.000 -33.33%
416.gamess (max) (percentage) 0.130 -35.00%
416.gamess (unexplained diff vs previous) 5.000 -76.19%
416.gamess (unexplained diff vs previous) (percentage) 0.040 -76.47%
429.mcf 92.000 -2.13%
429.mcf (avg) 92.000 -2.13%
429.mcf (avg) (percentage) 6.710 -1.32%
429.mcf (max) 291.000 -7.32%
429.mcf (max) (percentage) 21.160 -6.25%
429.mcf (unexplained diff vs previous) 17.000 30.77%
429.mcf (unexplained diff vs previous) (percentage) 1.240 33.33%
433.milc 19.000 -
433.milc (avg) 19.000 -
433.milc (avg) (percentage) 1.730 -1.14%
433.milc (max) 27.000 -6.90%
433.milc (max) (percentage) 2.390 -7.36%
433.milc (unexplained diff vs previous) 5.000 -37.50%
433.milc (unexplained diff vs previous) (percentage) 0.440 -37.14%
434.zeusmp 36.000 -2.70%
434.zeusmp (avg) 36.000 -2.70%
434.zeusmp (avg) (percentage) 1.440 -2.04%
434.zeusmp (max) 30.000 -9.09%
434.zeusmp (max) (percentage) 1.210 -6.92%
434.zeusmp (unexplained diff vs previous) 0.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 0.000 -
435.gromacs 151.000 -2.58%
435.gromacs (avg) 151.000 -2.58%
435.gromacs (avg) (percentage) 2.070 -1.90%
435.gromacs (max) 187.000 -10.10%
435.gromacs (max) (percentage) 2.550 -9.89%
435.gromacs (unexplained diff vs previous) 20.000 -44.44%
435.gromacs (unexplained diff vs previous) (percentage) 0.270 -44.90%
436.cactusADM 35.000 -
436.cactusADM (avg) 35.000 -
436.cactusADM (avg) (percentage) 1.680 -1.75%
436.cactusADM (max) 19.000 -9.52%
436.cactusADM (max) (percentage) 0.920 -9.80%
436.cactusADM (unexplained diff vs previous) 2.000 -33.33%
436.cactusADM (unexplained diff vs previous) (percentage) 0.100 -28.57%
437.leslie3d 542.000 4.84%
437.leslie3d (avg) 542.000 4.84%
437.leslie3d (avg) (percentage) 4.830 -
437.leslie3d (max) 545.000 140.09%
437.leslie3d (max) (percentage) 4.860 129.25%
437.leslie3d (unexplained diff vs previous) 521.000 223.60%
437.leslie3d (unexplained diff vs previous) (percentage) 4.640 209.33%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.780 -1.11%
444.namd (max) 4.000 33.33%
444.namd (max) (percentage) 0.670 48.89%
444.namd (unexplained diff vs previous) 5.000 150.00%
444.namd (unexplained diff vs previous) (percentage) 0.670 148.15%
445.gobmk 56.000 -
445.gobmk (avg) 56.000 -
445.gobmk (avg) (percentage) 1.120 -1.75%
445.gobmk (max) 92.000 -6.12%
445.gobmk (max) (percentage) 1.840 -6.60%
445.gobmk (unexplained diff vs previous) 7.000 -79.41%
445.gobmk (unexplained diff vs previous) (percentage) 0.140 -79.41%
447.dealII 26.000 4.00%
447.dealII (avg) 26.000 4.00%
447.dealII (avg) (percentage) 1.520 -
447.dealII (max) 24.000 -7.69%
447.dealII (max) (percentage) 1.440 -8.28%
447.dealII (unexplained diff vs previous) 20.000 -25.93%
447.dealII (unexplained diff vs previous) (percentage) 1.170 -26.88%
450.soplex 17.000 -
450.soplex (avg) 17.000 -
450.soplex (avg) (percentage) 2.930 -1.68%
450.soplex (max) 33.000 -8.33%
450.soplex (max) (percentage) 5.750 -7.41%
450.soplex (unexplained diff vs previous) 2.000 100.00%
450.soplex (unexplained diff vs previous) (percentage) 0.340 100.00%
453.povray 9.000 -10.00%
453.povray (avg) 9.000 -10.00%
453.povray (avg) (percentage) 1.800 -1.64%
453.povray (max) 5.000 -
453.povray (max) (percentage) 0.910 -7.14%
453.povray (unexplained diff vs previous) 1.000 -75.00%
453.povray (unexplained diff vs previous) (percentage) 0.180 -75.34%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 1.140 -4.20%
456.hmmer 23.000 -4.17%
456.hmmer (avg) 23.000 -4.17%
456.hmmer (avg) (percentage) 0.830 -1.19%
456.hmmer (max) 10.000 -9.09%
456.hmmer (max) (percentage) 0.360 -7.69%
456.hmmer (unexplained diff vs previous) 8.000 14.29%
456.hmmer (unexplained diff vs previous) (percentage) 0.280 16.67%
458.sjeng 60.000 -1.64%
458.sjeng (avg) 60.000 -1.64%
458.sjeng (avg) (percentage) 0.820 -1.20%
458.sjeng (max) 79.000 -7.06%
458.sjeng (max) (percentage) 1.070 -6.96%
458.sjeng (unexplained diff vs previous) 16.000 77.78%
458.sjeng (unexplained diff vs previous) (percentage) 0.220 83.33%
459.GemsFDTD 104.000 -
459.GemsFDTD (avg) 104.000 -
459.GemsFDTD (avg) (percentage) 1.670 -1.18%
459.GemsFDTD (max) 111.000 -6.72%
459.GemsFDTD (max) (percentage) 1.780 -7.77%
459.GemsFDTD (unexplained diff vs previous) 34.000 17.24%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.540 14.89%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.850 -1.07%
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.760 -5.38%
462.libquantum (unexplained diff vs previous) 1.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.730 1.39%
464.h264ref 64.000 -1.54%
464.h264ref (avg) 64.000 -1.54%
464.h264ref (avg) (percentage) 1.120 -1.75%
464.h264ref (max) 42.000 -6.67%
464.h264ref (max) (percentage) 0.740 -6.33%
464.h264ref (unexplained diff vs previous) 17.000 240.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.290 222.22%
465.tonto 176.000 -
465.tonto (avg) 176.000 -
465.tonto (avg) (percentage) 1.540 -1.28%
465.tonto (max) 301.000 -6.81%
465.tonto (max) (percentage) 2.630 -7.39%
465.tonto (unexplained diff vs previous) 70.000 -55.70%
465.tonto (unexplained diff vs previous) (percentage) 0.610 -56.12%
470.lbm 49.000 -2.00%
470.lbm (avg) 49.000 -2.00%
470.lbm (avg) (percentage) 1.310 -1.50%
470.lbm (max) 39.000 -7.14%
470.lbm (max) (percentage) 1.040 -6.31%
470.lbm (unexplained diff vs previous) 12.000 50.00%
470.lbm (unexplained diff vs previous) (percentage) 0.320 52.38%
471.omnetpp 65.000 -1.52%
471.omnetpp (avg) 65.000 -1.52%
471.omnetpp (avg) (percentage) 2.400 -1.23%
471.omnetpp (max) 79.000 -8.14%
471.omnetpp (max) (percentage) 2.920 -7.30%
471.omnetpp (unexplained diff vs previous) 12.000 -52.00%
471.omnetpp (unexplained diff vs previous) (percentage) 0.440 -51.65%
473.astar 100.000 -1.96%
473.astar (avg) 100.000 -1.96%
473.astar (avg) (percentage) 1.950 -1.52%
473.astar (max) 200.000 -7.41%
473.astar (max) (percentage) 3.880 -7.40%
473.astar (unexplained diff vs previous) 6.000 -83.78%
473.astar (unexplained diff vs previous) (percentage) 0.120 -83.33%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 1.180 -1.67%
482.sphinx3 19.000 -5.00%
482.sphinx3 (avg) 19.000 -5.00%
482.sphinx3 (avg) (percentage) 2.120 -1.40%
482.sphinx3 (max) 18.000 -10.00%
482.sphinx3 (max) (percentage) 2.000 -7.41%
482.sphinx3 (unexplained diff vs previous) 6.000 200.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.640 204.76%
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 6.212 -15.23%