Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=b757e3c1acbec5173f5ca04eb0cc7cfad7c7cb7e (+ 23 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/dbc9b45a3c2468ad134b3a9bd4961f7ae6bc1e67 (+ 65 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=85472c20a55ea2a49a7fbdf71652b4009118b0ae (+ 21 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=30e92e8838a078df359a97e36d048f06bbd5bec1 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/247/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0247 (3-changed-components) 2024-05-21T05:57:18 0:00:00
Previous 0249 (4-changed-components) 2024-05-25T17:45:02 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 115 17
Performance Improvements 33 192
Added Tests 4 2
Unchanged Tests 1084 1025
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
459.GemsFDTD 1.79% 236832.000 241060.000 - -1.50% -
444.namd 1.75% 186296.000 189548.000 - 0.62% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
435.gromacs 1.83% 7150.000 7281.000 - 0.17% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (unexplained diff vs previous) (percentage) 2066.67% 0.060 1.300 0.000 -99.75% 0.000
400.perlbench (unexplained diff vs previous) 1900.00% 1.000 20.000 0.000 -99.77% 0.000
465.tonto (unexplained diff vs previous) (percentage) 891.30% 0.230 2.280 0.000 -85.92% 0.000
465.tonto (unexplained diff vs previous) 877.78% 27.000 264.000 0.000 -85.09% 0.000
444.namd (unexplained diff vs previous) 700.00% 1.000 8.000 0.000 -99.90% 0.000
444.namd (unexplained diff vs previous) (percentage) 692.31% 0.130 1.030 0.000 -99.91% 0.000
410.bwaves (unexplained diff vs previous) (percentage) 566.67% 0.030 0.200 0.000 -99.85% 0.000
410.bwaves (unexplained diff vs previous) 500.00% 2.000 12.000 0.000 -99.85% 0.000
473.astar (unexplained diff vs previous) (percentage) 480.77% 0.260 1.510 0.000 -99.24% 0.000
473.astar (unexplained diff vs previous) 471.43% 14.000 80.000 0.000 -99.21% 0.000
453.povray (unexplained diff vs previous) (percentage) 351.35% 0.370 1.670 0.000 -99.84% 0.000
453.povray (unexplained diff vs previous) 350.00% 2.000 9.000 0.000 -99.84% 0.000
401.bzip2 (unexplained diff vs previous) 340.00% 5.000 22.000 0.000 -99.80% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) 335.29% 0.170 0.740 0.000 -99.80% 0.000
434.zeusmp (unexplained diff vs previous) 240.00% 5.000 17.000 0.000 -99.85% 0.000
434.zeusmp (unexplained diff vs previous) (percentage) 235.00% 0.200 0.670 0.000 -99.85% 0.000
471.omnetpp (unexplained diff vs previous) 150.00% 24.000 60.000 0.000 -99.44% 0.000
471.omnetpp (unexplained diff vs previous) (percentage) 149.43% 0.870 2.170 0.000 -99.45% 0.000
462.libquantum (unexplained diff vs previous) (percentage) 103.40% 1.470 2.990 0.000 -99.97% 0.000
462.libquantum (unexplained diff vs previous) 100.00% 2.000 4.000 0.000 -99.97% 0.000
447.dealII (unexplained diff vs previous) 85.71% 7.000 13.000 0.000 -99.81% 0.000
447.dealII (unexplained diff vs previous) (percentage) 83.72% 0.430 0.790 0.000 -99.81% 0.000
462.libquantum (max) (percentage) 50.26% 1.930 2.900 - -97.07% -
462.libquantum (max) 50.00% 2.000 3.000 - -97.83% -
433.milc (max) (percentage) 38.46% 0.260 0.360 - -99.60% -
433.milc (max) 33.33% 3.000 4.000 - -99.62% -
444.namd (max) 33.33% 6.000 8.000 - -98.83% -
444.namd (max) (percentage) 33.33% 0.780 1.040 - -98.87% -
447.dealII (max) 30.00% 10.000 13.000 - -99.06% -
447.dealII (max) (percentage) 25.40% 0.630 0.790 - -99.02% -
473.astar (max) (percentage) 25.21% 1.190 1.490 - -97.76% -
401.bzip2 (max) (percentage) 25.00% 0.600 0.750 - -99.05% -
473.astar (max) 23.81% 63.000 78.000 - -97.71% -
435.gromacs (unexplained diff vs previous) 23.42% 111.000 137.000 0.000 -98.05% 0.000
401.bzip2 (max) 22.22% 18.000 22.000 - -99.05% -
435.gromacs (unexplained diff vs previous) (percentage) 21.29% 1.550 1.880 0.000 -98.06% 0.000
435.gromacs (max) 18.80% 117.000 139.000 - -96.11% -
470.lbm (max) (percentage) 18.18% 0.110 0.130 - -99.82% -
436.cactusADM (max) 16.67% 12.000 14.000 - -99.24% -
435.gromacs (max) (percentage) 16.36% 1.650 1.920 - -96.09% -
453.povray (max) (percentage) 16.31% 1.410 1.640 - -98.20% -
453.povray (max) 14.29% 7.000 8.000 - -98.40% -
434.zeusmp (max) 13.33% 15.000 17.000 - -99.17% -
456.hmmer (max) (percentage) 13.16% 0.380 0.430 - -99.42% -
436.cactusADM (max) (percentage) 13.11% 0.610 0.690 - -99.22% -
471.omnetpp (max) 12.66% 79.000 89.000 - -95.90% -
471.omnetpp (max) (percentage) 12.50% 2.880 3.240 - -95.93% -
465.tonto (max) 11.57% 242.000 270.000 - -87.22% -
410.bwaves (max) 11.54% 26.000 29.000 - -99.11% -
410.bwaves (max) (percentage) 11.36% 0.440 0.490 - -99.15% -
465.tonto (max) (percentage) 10.95% 2.100 2.330 - -87.93% -
400.perlbench (max) (percentage) 10.57% 1.230 1.360 - -98.38% -
458.sjeng (max) 10.53% 19.000 21.000 - -99.42% -
456.hmmer (max) 9.09% 11.000 12.000 - -99.43% -
437.leslie3d (max) 8.86% 1084.000 1180.000 - -38.83% -
437.leslie3d (max) (percentage) 8.00% 9.620 10.390 - -37.33% -
434.zeusmp (max) (percentage) 7.94% 0.630 0.680 - -99.17% -
458.sjeng (max) (percentage) 7.41% 0.270 0.290 - -99.39% -
482.sphinx3 7.14% 14.000 15.000 - -79.73% -
482.sphinx3 (avg) 7.14% 14.000 15.000 - -79.73% -
429.mcf (max) (percentage) 6.93% 2.310 2.470 - -97.34% -
429.mcf (max) 6.82% 44.000 47.000 - -96.44% -
433.milc 6.67% 15.000 16.000 - -76.81% -
433.milc (avg) 6.67% 15.000 16.000 - -76.81% -
450.soplex (max) (percentage) 6.59% 0.910 0.970 - -98.99% -
445.gobmk (max) (percentage) 6.25% 0.640 0.680 - -98.71% -
445.gobmk (max) 6.25% 32.000 34.000 - -98.71% -
482.sphinx3 (max) (percentage) 5.66% 0.530 0.560 - -99.42% -
400.perlbench (max) 5.26% 19.000 20.000 - -98.57% -
434.zeusmp 3.45% 29.000 30.000 - -79.59% -
434.zeusmp (avg) 3.45% 29.000 30.000 - -79.59% -
436.cactusADM 3.45% 29.000 30.000 - -75.81% -
436.cactusADM (avg) 3.45% 29.000 30.000 - -75.81% -
410.bwaves (avg) (percentage) 2.91% 1.030 1.060 - -75.96% -
434.zeusmp (avg) (percentage) 2.59% 1.160 1.190 - -79.66% -
470.lbm 2.56% 39.000 40.000 - -83.26% -
470.lbm (avg) 2.56% 39.000 40.000 - -83.26% -
473.astar (avg) (percentage) 2.53% 1.580 1.620 - -61.06% -
465.tonto 2.50% 160.000 164.000 - -55.07% -
465.tonto (avg) 2.50% 160.000 164.000 - -55.07% -
473.astar 2.41% 83.000 85.000 - -60.09% -
473.astar (avg) 2.41% 83.000 85.000 - -60.09% -
435.gromacs 2.34% 128.000 131.000 - -64.31% -
435.gromacs (avg) 2.34% 128.000 131.000 - -64.31% -
416.gamess (max) (percentage) 2.27% 0.440 0.450 - -98.83% -
433.milc (avg) (percentage) 2.19% 1.370 1.400 - -76.86% -
465.tonto (avg) (percentage) 2.16% 1.390 1.420 - -57.49% -
436.cactusADM (avg) (percentage) 2.14% 1.400 1.430 - -76.09% -
481.wrf (max) (percentage) 2.13% 0.940 0.960 - -98.60% -
470.lbm (avg) (percentage) 1.90% 1.050 1.070 - -83.10% -
400.perlbench (avg) (percentage) 1.83% 2.180 2.220 - -65.04% -
453.povray (avg) (percentage) 1.79% 1.680 1.710 - -78.10% -
429.mcf 1.75% 114.000 116.000 - 6.42% -
429.mcf (avg) 1.75% 114.000 116.000 - 6.42% -
401.bzip2 (avg) (percentage) 1.73% 1.730 1.760 - -67.88% -
482.sphinx3 (avg) (percentage) 1.69% 1.770 1.800 - -77.30% -
410.bwaves 1.64% 61.000 62.000 - -75.10% -
410.bwaves (avg) 1.64% 61.000 62.000 - -75.10% -
471.omnetpp 1.64% 61.000 62.000 - -72.20% -
471.omnetpp (avg) 1.64% 61.000 62.000 - -72.20% -
437.leslie3d 1.56% 514.000 522.000 - 47.04% -
437.leslie3d (avg) 1.56% 514.000 522.000 - 47.04% -
447.dealII (avg) (percentage) 1.49% 1.340 1.360 - -76.43% -
456.hmmer (avg) (percentage) 1.45% 0.690 0.700 - -84.13% -
429.mcf (avg) (percentage) 1.35% 5.920 6.000 - -21.77% -
471.omnetpp (avg) (percentage) 1.35% 2.220 2.250 - -72.56% -
444.namd (avg) (percentage) 1.34% 1.490 1.510 - -83.62% -
450.soplex (avg) (percentage) 1.24% 2.420 2.450 - -61.90% -
403.gcc (max) (percentage) 1.18% 4.240 4.290 - -95.68% -
454.calculix (max) (percentage) 1.14% 0.880 0.890 - -99.11% -
464.h264ref (avg) (percentage) 1.11% 0.900 0.910 - -82.12% -
445.gobmk (avg) (percentage) 1.05% 0.950 0.960 - -75.45% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
453.povray -1.25% 660325.000 652073.000 - -1.41% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
400.perlbench -1.91% 1571.000 1541.000 - -7.34% -
450.soplex -1.70% 589.000 579.000 - 2.12% -
401.bzip2 -1.49% 3018.000 2973.000 - 1.09% -
462.libquantum -1.47% 136.000 134.000 - -4.29% -
453.povray -1.28% 547.000 540.000 - -1.28% -
459.GemsFDTD -1.15% 5376.000 5314.000 - -13.92% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
445.gobmk (unexplained diff vs previous) -84.62% 26.000 4.000 0.000 -99.93% 0.000
445.gobmk (unexplained diff vs previous) (percentage) -84.31% 0.510 0.080 0.000 -99.93% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) -75.44% 0.570 0.140 0.000 -99.98% 0.000
436.cactusADM (unexplained diff vs previous) -75.00% 12.000 3.000 0.000 -99.98% 0.000
456.hmmer (unexplained diff vs previous) (percentage) -70.00% 0.100 0.030 0.000 -99.99% 0.000
456.hmmer (unexplained diff vs previous) -66.67% 3.000 1.000 0.000 -99.99% 0.000
482.sphinx3 (unexplained diff vs previous) -66.67% 3.000 1.000 0.000 -100.00% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) -66.67% 0.360 0.120 0.000 -100.00% 0.000
450.soplex (unexplained diff vs previous) -50.00% 4.000 2.000 0.000 -99.99% 0.000
470.lbm (unexplained diff vs previous) -50.00% 2.000 1.000 0.000 -99.99% 0.000
450.soplex (unexplained diff vs previous) (percentage) -48.53% 0.680 0.350 0.000 -99.99% 0.000
459.GemsFDTD (unexplained diff vs previous) -48.08% 52.000 27.000 0.000 -99.83% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) -47.42% 0.970 0.510 0.000 -99.80% 0.000
459.GemsFDTD (max) -46.15% 52.000 28.000 - -99.37% -
459.GemsFDTD (max) (percentage) -45.92% 0.980 0.530 - -99.26% -
429.mcf (unexplained diff vs previous) -42.86% 7.000 4.000 0.000 -99.98% 0.000
429.mcf (unexplained diff vs previous) (percentage) -41.67% 0.360 0.210 0.000 -99.98% 0.000
470.lbm (unexplained diff vs previous) (percentage) -40.00% 0.050 0.030 0.000 -99.99% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) -19.59% 1.940 1.560 0.000 -92.15% 0.000
437.leslie3d (unexplained diff vs previous) -19.18% 219.000 177.000 0.000 -92.35% 0.000
464.h264ref (unexplained diff vs previous) (percentage) -12.12% 0.330 0.290 0.000 -99.76% 0.000
464.h264ref (max) -10.53% 19.000 17.000 - -99.46% -
464.h264ref (unexplained diff vs previous) -10.53% 19.000 17.000 0.000 -99.75% 0.000
464.h264ref (max) (percentage) -9.09% 0.330 0.300 - -99.45% -
459.GemsFDTD -1.32% 76.000 75.000 - -79.17% -
459.GemsFDTD (avg) -1.32% 76.000 75.000 - -79.17% -

Added Tests - sample variability
433.milc (unexplained diff vs previous)
433.milc (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
400.perlbench 1.46% 893848.000 906859.000 - -0.01% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 36.17% 1424.000 1939.000 - 0.15% -
465.tonto 5.92% 10941.000 11589.000 - 0.55% -
410.bwaves 5.11% 5652.000 5941.000 - -0.10% -
444.namd 4.28% 747.000 779.000 - 0.26% -
473.astar 3.14% 5128.000 5289.000 - -0.30% -
450.soplex 2.12% 567.000 579.000 - -1.70% -
436.cactusADM 1.78% 2083.000 2120.000 - -0.05% -
471.omnetpp 1.36% 2724.000 2761.000 - 0.22% -
456.hmmer 1.25% 2882.000 2918.000 - -0.21% -
401.bzip2 1.09% 2941.000 2973.000 - -1.49% -
433.milc 1.04% 1151.000 1163.000 - 0.35% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 50.82% 3.050 4.600 - 0.88% -
437.leslie3d 47.04% 355.000 522.000 - 1.56% -
437.leslie3d (avg) 47.04% 355.000 522.000 - 1.56% -
429.mcf 6.42% 109.000 116.000 - 1.75% -
429.mcf (avg) 6.42% 109.000 116.000 - 1.75% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
434.zeusmp -2.48% 152037.000 148271.000 - 0.00% -
437.leslie3d -1.63% 59479.000 58510.000 - 0.00% -
462.libquantum -1.59% 16085.000 15829.000 - 0.00% -
464.h264ref -1.51% 387748.000 381880.000 - -0.01% -
459.GemsFDTD -1.50% 244728.000 241060.000 - 1.79% -
470.lbm -1.44% 9691.000 9551.000 - 0.00% -
453.povray -1.41% 661389.000 652073.000 - -1.25% -
410.bwaves -1.26% 16760.000 16548.000 - 0.00% -
435.gromacs -1.12% 499261.000 493649.000 - -0.01% -
433.milc -1.10% 61382.000 60709.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -13.92% 6173.000 5314.000 - -1.15% -
482.sphinx3 -11.22% 945.000 839.000 - -0.36% -
400.perlbench -7.34% 1663.000 1541.000 - -1.91% -
462.libquantum -4.29% 140.000 134.000 - -1.47% -
447.dealII -3.94% 1724.000 1656.000 - 0.67% -
458.sjeng -2.42% 7567.000 7384.000 - 0.07% -
437.leslie3d -2.41% 11640.000 11360.000 - 0.74% -
453.povray -1.28% 547.000 540.000 - -1.28% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
482.sphinx3 (unexplained diff vs previous) -100.00% 26073.000 1.000 0.000 -66.67% 0.000
482.sphinx3 (unexplained diff vs previous) (percentage) -100.00% 2759.050 0.120 0.000 -66.67% 0.000
470.lbm (unexplained diff vs previous) -99.99% 9990.000 1.000 0.000 -50.00% 0.000
456.hmmer (unexplained diff vs previous) (percentage) -99.99% 278.630 0.030 0.000 -70.00% 0.000
470.lbm (unexplained diff vs previous) (percentage) -99.99% 263.730 0.030 0.000 -40.00% 0.000
456.hmmer (unexplained diff vs previous) -99.99% 8030.000 1.000 0.000 -66.67% 0.000
450.soplex (unexplained diff vs previous) (percentage) -99.99% 2557.320 0.350 0.000 -48.53% 0.000
450.soplex (unexplained diff vs previous) -99.99% 14500.000 2.000 0.000 -50.00% 0.000
429.mcf (unexplained diff vs previous) (percentage) -99.98% 1305.690 0.210 0.000 -41.67% 0.000
436.cactusADM (unexplained diff vs previous) (percentage) -99.98% 787.130 0.140 0.000 -75.44% 0.000
436.cactusADM (unexplained diff vs previous) -99.98% 16396.000 3.000 0.000 -75.00% 0.000
429.mcf (unexplained diff vs previous) -99.98% 18593.000 4.000 0.000 -42.86% 0.000
462.libquantum (unexplained diff vs previous) -99.97% 13518.000 4.000 0.000 100.00% 0.000
462.libquantum (unexplained diff vs previous) (percentage) -99.97% 9655.710 2.990 0.000 103.40% 0.000
445.gobmk (unexplained diff vs previous) -99.93% 5581.000 4.000 0.000 -84.62% 0.000
445.gobmk (unexplained diff vs previous) (percentage) -99.93% 111.110 0.080 0.000 -84.31% 0.000
444.namd (unexplained diff vs previous) (percentage) -99.91% 1118.610 1.030 0.000 692.31% 0.000
444.namd (unexplained diff vs previous) -99.90% 8356.000 8.000 0.000 700.00% 0.000
410.bwaves (unexplained diff vs previous) (percentage) -99.85% 137.900 0.200 0.000 566.67% 0.000
470.lbm (max) -99.85% 2746.000 4.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.85% 448.690 0.670 0.000 235.00% 0.000
434.zeusmp (unexplained diff vs previous) -99.85% 11280.000 17.000 0.000 240.00% 0.000
410.bwaves (unexplained diff vs previous) -99.85% 7794.000 12.000 0.000 500.00% 0.000
453.povray (unexplained diff vs previous) -99.84% 5707.000 9.000 0.000 350.00% 0.000
453.povray (unexplained diff vs previous) (percentage) -99.84% 1043.330 1.670 0.000 351.35% 0.000
459.GemsFDTD (unexplained diff vs previous) -99.83% 15458.000 27.000 0.000 -48.08% 0.000
470.lbm (max) (percentage) -99.82% 72.510 0.130 - 18.18% -
447.dealII (unexplained diff vs previous) -99.81% 7017.000 13.000 0.000 85.71% 0.000
447.dealII (unexplained diff vs previous) (percentage) -99.81% 407.020 0.790 0.000 83.72% 0.000
401.bzip2 (unexplained diff vs previous) (percentage) -99.80% 368.340 0.740 0.000 335.29% 0.000
401.bzip2 (unexplained diff vs previous) -99.80% 10833.000 22.000 0.000 340.00% 0.000
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.80% 250.410 0.510 0.000 -47.42% 0.000
400.perlbench (unexplained diff vs previous) -99.77% 8772.000 20.000 0.000 1900.00% 0.000
464.h264ref (unexplained diff vs previous) (percentage) -99.76% 118.780 0.290 0.000 -12.12% 0.000
400.perlbench (unexplained diff vs previous) (percentage) -99.75% 527.480 1.300 0.000 2066.67% 0.000
464.h264ref (unexplained diff vs previous) -99.75% 6843.000 17.000 0.000 -10.53% 0.000
458.sjeng (unexplained diff vs previous) -99.74% 6876.000 18.000 0.000 0.00% 0.000
458.sjeng (unexplained diff vs previous) (percentage) -99.74% 90.870 0.240 0.000 0.00% 0.000
433.milc (max) -99.62% 1044.000 4.000 - 33.33% -
433.milc (max) (percentage) -99.60% 90.750 0.360 - 38.46% -
482.sphinx3 (max) -99.56% 911.000 4.000 - 0.00% -
464.h264ref (max) -99.46% 3127.000 17.000 - -10.53% -
464.h264ref (max) (percentage) -99.45% 54.290 0.300 - -9.09% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.45% 391.300 2.170 0.000 149.43% 0.000
471.omnetpp (unexplained diff vs previous) -99.44% 10659.000 60.000 0.000 150.00% 0.000
456.hmmer (max) -99.43% 2120.000 12.000 - 9.09% -
482.sphinx3 (max) (percentage) -99.42% 96.500 0.560 - 5.66% -
458.sjeng (max) -99.42% 3602.000 21.000 - 10.53% -
456.hmmer (max) (percentage) -99.42% 73.590 0.430 - 13.16% -
458.sjeng (max) (percentage) -99.39% 47.610 0.290 - 7.41% -
459.GemsFDTD (max) -99.37% 4411.000 28.000 - -46.15% -
459.GemsFDTD (max) (percentage) -99.26% 71.460 0.530 - -45.92% -
436.cactusADM (max) -99.24% 1848.000 14.000 - 16.67% -
473.astar (unexplained diff vs previous) (percentage) -99.24% 197.850 1.510 0.000 480.77% 0.000
436.cactusADM (max) (percentage) -99.22% 88.730 0.690 - 13.11% -
473.astar (unexplained diff vs previous) -99.21% 10146.000 80.000 0.000 471.43% 0.000
434.zeusmp (max) -99.17% 2055.000 17.000 - 13.33% -
434.zeusmp (max) (percentage) -99.17% 81.770 0.680 - 7.94% -
410.bwaves (max) (percentage) -99.15% 57.970 0.490 - 11.36% -
410.bwaves (max) -99.11% 3276.000 29.000 - 11.54% -
454.calculix (max) (percentage) -99.11% 99.730 0.890 - 1.14% -
450.soplex (max) -99.08% 545.000 5.000 - 0.00% -
447.dealII (max) -99.06% 1384.000 13.000 - 30.00% -
401.bzip2 (max) -99.05% 2313.000 22.000 - 22.22% -
401.bzip2 (max) (percentage) -99.05% 78.650 0.750 - 25.00% -
447.dealII (max) (percentage) -99.02% 80.280 0.790 - 25.40% -
450.soplex (max) (percentage) -98.99% 96.240 0.970 - 6.59% -
444.namd (max) (percentage) -98.87% 91.790 1.040 - 33.33% -
416.gamess (max) (percentage) -98.83% 38.550 0.450 - 2.27% -
444.namd (max) -98.83% 685.000 8.000 - 33.33% -
445.gobmk (max) -98.71% 2643.000 34.000 - 6.25% -
445.gobmk (max) (percentage) -98.71% 52.630 0.680 - 6.25% -
481.wrf (max) (percentage) -98.60% 68.700 0.960 - 2.13% -
400.perlbench (max) -98.57% 1397.000 20.000 - 5.26% -
453.povray (max) -98.40% 499.000 8.000 - 14.29% -
400.perlbench (max) (percentage) -98.38% 84.060 1.360 - 10.57% -
453.povray (max) (percentage) -98.20% 91.250 1.640 - 16.31% -
435.gromacs (unexplained diff vs previous) (percentage) -98.06% 96.680 1.880 0.000 21.29% 0.000
435.gromacs (unexplained diff vs previous) -98.05% 7028.000 137.000 0.000 23.42% 0.000
462.libquantum (max) -97.83% 138.000 3.000 - 50.00% -
473.astar (max) (percentage) -97.76% 66.430 1.490 - 25.21% -
473.astar (max) -97.71% 3406.000 78.000 - 23.81% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
429.mcf (max) (percentage) -97.34% 92.890 2.470 - 6.93% -
462.libquantum (max) (percentage) -97.07% 98.970 2.900 - 50.26% -
429.mcf (max) -96.44% 1322.000 47.000 - 6.82% -
435.gromacs (max) -96.11% 3573.000 139.000 - 18.80% -
435.gromacs (max) (percentage) -96.09% 49.160 1.920 - 16.36% -
471.omnetpp (max) (percentage) -95.93% 79.650 3.240 - 12.50% -
471.omnetpp (max) -95.90% 2169.000 89.000 - 12.66% -
403.gcc (max) (percentage) -95.68% 99.360 4.290 - 1.18% -
437.leslie3d (unexplained diff vs previous) -92.35% 2314.000 177.000 0.000 -19.18% 0.000
437.leslie3d (unexplained diff vs previous) (percentage) -92.15% 19.880 1.560 0.000 -19.59% 0.000
465.tonto (max) (percentage) -87.93% 19.310 2.330 - 10.95% -
465.tonto (max) -87.22% 2112.000 270.000 - 11.57% -
465.tonto (unexplained diff vs previous) (percentage) -85.92% 16.190 2.280 0.000 891.30% 0.000
465.tonto (unexplained diff vs previous) -85.09% 1771.000 264.000 0.000 877.78% 0.000
456.hmmer -84.25% 127.000 20.000 - 0.00% -
456.hmmer (avg) -84.25% 127.000 20.000 - 0.00% -
456.hmmer (avg) (percentage) -84.13% 4.410 0.700 - 1.45% -
444.namd -83.82% 68.000 11.000 - 0.00% -
444.namd (avg) -83.82% 68.000 11.000 - 0.00% -
444.namd (avg) (percentage) -83.62% 9.220 1.510 - 1.34% -
470.lbm -83.26% 239.000 40.000 - 2.56% -
470.lbm (avg) -83.26% 239.000 40.000 - 2.56% -
470.lbm (avg) (percentage) -83.10% 6.330 1.070 - 1.90% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - 0.00% -
464.h264ref -82.25% 293.000 52.000 - 0.00% -
464.h264ref (avg) -82.25% 293.000 52.000 - 0.00% -
464.h264ref (avg) (percentage) -82.12% 5.090 0.910 - 1.11% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
482.sphinx3 -79.73% 74.000 15.000 - 7.14% -
482.sphinx3 (avg) -79.73% 74.000 15.000 - 7.14% -
434.zeusmp (avg) (percentage) -79.66% 5.850 1.190 - 2.59% -
434.zeusmp -79.59% 147.000 30.000 - 3.45% -
434.zeusmp (avg) -79.59% 147.000 30.000 - 3.45% -
459.GemsFDTD -79.17% 360.000 75.000 - -1.32% -
459.GemsFDTD (avg) -79.17% 360.000 75.000 - -1.32% -
453.povray -78.57% 42.000 9.000 - 0.00% -
453.povray (avg) -78.57% 42.000 9.000 - 0.00% -
458.sjeng -78.42% 241.000 52.000 - 0.00% -
458.sjeng (avg) -78.42% 241.000 52.000 - 0.00% -
453.povray (avg) (percentage) -78.10% 7.810 1.710 - 1.79% -
447.dealII -77.78% 99.000 22.000 - 0.00% -
447.dealII (avg) -77.78% 99.000 22.000 - 0.00% -
458.sjeng (avg) (percentage) -77.74% 3.190 0.710 - 0.00% -
482.sphinx3 (avg) (percentage) -77.30% 7.930 1.800 - 1.69% -
462.libquantum (avg) (percentage) -76.89% 7.660 1.770 - 0.00% -
433.milc (avg) (percentage) -76.86% 6.050 1.400 - 2.19% -
433.milc -76.81% 69.000 16.000 - 6.67% -
433.milc (avg) -76.81% 69.000 16.000 - 6.67% -
447.dealII (avg) (percentage) -76.43% 5.770 1.360 - 1.49% -
436.cactusADM (avg) (percentage) -76.09% 5.980 1.430 - 2.14% -
410.bwaves (avg) (percentage) -75.96% 4.410 1.060 - 2.91% -
436.cactusADM -75.81% 124.000 30.000 - 3.45% -
436.cactusADM (avg) -75.81% 124.000 30.000 - 3.45% -
459.GemsFDTD (avg) (percentage) -75.51% 5.840 1.430 - 0.70% -
445.gobmk -75.51% 196.000 48.000 - 0.00% -
445.gobmk (avg) -75.51% 196.000 48.000 - 0.00% -
445.gobmk (avg) (percentage) -75.45% 3.910 0.960 - 1.05% -
410.bwaves -75.10% 249.000 62.000 - 1.64% -
410.bwaves (avg) -75.10% 249.000 62.000 - 1.64% -
471.omnetpp (avg) (percentage) -72.56% 8.200 2.250 - 1.35% -
471.omnetpp -72.20% 223.000 62.000 - 1.64% -
471.omnetpp (avg) -72.20% 223.000 62.000 - 1.64% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
401.bzip2 (avg) (percentage) -67.88% 5.480 1.760 - 1.73% -
401.bzip2 -67.70% 161.000 52.000 - 0.00% -
401.bzip2 (avg) -67.70% 161.000 52.000 - 0.00% -
400.perlbench -67.62% 105.000 34.000 - 0.00% -
400.perlbench (avg) -67.62% 105.000 34.000 - 0.00% -
400.perlbench (avg) (percentage) -65.04% 6.350 2.220 - 1.83% -
435.gromacs (avg) (percentage) -64.43% 5.060 1.800 - 0.00% -
435.gromacs -64.31% 367.000 131.000 - 2.34% -
435.gromacs (avg) -64.31% 367.000 131.000 - 2.34% -
450.soplex (avg) (percentage) -61.90% 6.430 2.450 - 1.24% -
450.soplex -61.11% 36.000 14.000 - 0.00% -
450.soplex (avg) -61.11% 36.000 14.000 - 0.00% -
473.astar (avg) (percentage) -61.06% 4.160 1.620 - 2.53% -
473.astar -60.09% 213.000 85.000 - 2.41% -
473.astar (avg) -60.09% 213.000 85.000 - 2.41% -
465.tonto (avg) (percentage) -57.49% 3.340 1.420 - 2.16% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
465.tonto -55.07% 365.000 164.000 - 2.50% -
465.tonto (avg) -55.07% 365.000 164.000 - 2.50% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
437.leslie3d (max) -38.83% 1929.000 1180.000 - 8.86% -
437.leslie3d (max) (percentage) -37.33% 16.580 10.390 - 8.00% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf (avg) (percentage) -21.77% 7.670 6.000 - 1.35% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 0.00% -
nb_compile_successful 65.52% 29.000 48.000 - 0.00% -
nb_execution_successful 65.52% 29.000 48.000 - 0.00% -

Added Tests - sample variability
433.milc (unexplained diff vs previous)
433.milc (unexplained diff vs previous) (percentage)

Report Time: 0.41s
Execution (seconds)
Name Current %
400.perlbench 1541.000 -1.91%
401.bzip2 2973.000 -1.49%
410.bwaves 5941.000 -
429.mcf 1939.000 -
433.milc 1163.000 -
434.zeusmp 2524.000 -
435.gromacs 7281.000 1.83%
436.cactusADM 2120.000 -
437.leslie3d 11360.000 -
444.namd 779.000 -
445.gobmk 5057.000 -
447.dealII 1656.000 -
450.soplex 579.000 -1.70%
453.povray 540.000 -1.28%
456.hmmer 2918.000 -
458.sjeng 7384.000 -
459.GemsFDTD 5314.000 -1.15%
462.libquantum 134.000 -1.47%
464.h264ref 5768.000 -
465.tonto 11589.000 -
470.lbm 3790.000 -
471.omnetpp 2761.000 -
473.astar 5289.000 -
482.sphinx3 839.000 -
Geometric Mean 2469.209 -
Code Size (bytes)
Name Current %
400.perlbench 906859.000 -
401.bzip2 43835.000 -
410.bwaves 16548.000 -
429.mcf 9055.000 -
433.milc 60709.000 -
434.zeusmp 148271.000 -
435.gromacs 493649.000 -
436.cactusADM 350575.000 -
437.leslie3d 58510.000 -
444.namd 189548.000 1.75%
445.gobmk 1383162.000 -
447.dealII 309588.000 -
450.soplex 205231.000 -
453.povray 652073.000 -1.25%
456.hmmer 108227.000 -
458.sjeng 103124.000 -
459.GemsFDTD 241060.000 1.79%
462.libquantum 15829.000 -
464.h264ref 381880.000 -
465.tonto 1936910.000 -
470.lbm 9551.000 -
471.omnetpp 476222.000 -
473.astar 22888.000 -
482.sphinx3 112175.000 -
Geometric Mean 137725.195 -
sample variability
Name Current %
400.perlbench 34.000 -
400.perlbench (avg) 34.000 -
400.perlbench (avg) (percentage) 2.220 1.83%
400.perlbench (max) 20.000 5.26%
400.perlbench (max) (percentage) 1.360 10.57%
400.perlbench (unexplained diff vs previous) 20.000 1900.00%
400.perlbench (unexplained diff vs previous) (percentage) 1.300 2066.67%
401.bzip2 52.000 -
401.bzip2 (avg) 52.000 -
401.bzip2 (avg) (percentage) 1.760 1.73%
401.bzip2 (max) 22.000 22.22%
401.bzip2 (max) (percentage) 0.750 25.00%
401.bzip2 (unexplained diff vs previous) 22.000 340.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.740 335.29%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 4.290 1.18%
410.bwaves 62.000 1.64%
410.bwaves (avg) 62.000 1.64%
410.bwaves (avg) (percentage) 1.060 2.91%
410.bwaves (max) 29.000 11.54%
410.bwaves (max) (percentage) 0.490 11.36%
410.bwaves (unexplained diff vs previous) 12.000 500.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.200 566.67%
416.gamess (avg) (percentage) 0.760 -
416.gamess (max) (percentage) 0.450 2.27%
429.mcf 116.000 1.75%
429.mcf (avg) 116.000 1.75%
429.mcf (avg) (percentage) 6.000 1.35%
429.mcf (max) 47.000 6.82%
429.mcf (max) (percentage) 2.470 6.93%
429.mcf (unexplained diff vs previous) 4.000 -42.86%
429.mcf (unexplained diff vs previous) (percentage) 0.210 -41.67%
433.milc 16.000 6.67%
433.milc (avg) 16.000 6.67%
433.milc (avg) (percentage) 1.400 2.19%
433.milc (max) 4.000 33.33%
433.milc (max) (percentage) 0.360 38.46%
433.milc (unexplained diff vs previous) 0.000 -
433.milc (unexplained diff vs previous) (percentage) 0.000 -
434.zeusmp 30.000 3.45%
434.zeusmp (avg) 30.000 3.45%
434.zeusmp (avg) (percentage) 1.190 2.59%
434.zeusmp (max) 17.000 13.33%
434.zeusmp (max) (percentage) 0.680 7.94%
434.zeusmp (unexplained diff vs previous) 17.000 240.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.670 235.00%
435.gromacs 131.000 2.34%
435.gromacs (avg) 131.000 2.34%
435.gromacs (avg) (percentage) 1.800 -
435.gromacs (max) 139.000 18.80%
435.gromacs (max) (percentage) 1.920 16.36%
435.gromacs (unexplained diff vs previous) 137.000 23.42%
435.gromacs (unexplained diff vs previous) (percentage) 1.880 21.29%
436.cactusADM 30.000 3.45%
436.cactusADM (avg) 30.000 3.45%
436.cactusADM (avg) (percentage) 1.430 2.14%
436.cactusADM (max) 14.000 16.67%
436.cactusADM (max) (percentage) 0.690 13.11%
436.cactusADM (unexplained diff vs previous) 3.000 -75.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.140 -75.44%
437.leslie3d 522.000 1.56%
437.leslie3d (avg) 522.000 1.56%
437.leslie3d (avg) (percentage) 4.600 -
437.leslie3d (max) 1180.000 8.86%
437.leslie3d (max) (percentage) 10.390 8.00%
437.leslie3d (unexplained diff vs previous) 177.000 -19.18%
437.leslie3d (unexplained diff vs previous) (percentage) 1.560 -19.59%
444.namd 11.000 -
444.namd (avg) 11.000 -
444.namd (avg) (percentage) 1.510 1.34%
444.namd (max) 8.000 33.33%
444.namd (max) (percentage) 1.040 33.33%
444.namd (unexplained diff vs previous) 8.000 700.00%
444.namd (unexplained diff vs previous) (percentage) 1.030 692.31%
445.gobmk 48.000 -
445.gobmk (avg) 48.000 -
445.gobmk (avg) (percentage) 0.960 1.05%
445.gobmk (max) 34.000 6.25%
445.gobmk (max) (percentage) 0.680 6.25%
445.gobmk (unexplained diff vs previous) 4.000 -84.62%
445.gobmk (unexplained diff vs previous) (percentage) 0.080 -84.31%
447.dealII 22.000 -
447.dealII (avg) 22.000 -
447.dealII (avg) (percentage) 1.360 1.49%
447.dealII (max) 13.000 30.00%
447.dealII (max) (percentage) 0.790 25.40%
447.dealII (unexplained diff vs previous) 13.000 85.71%
447.dealII (unexplained diff vs previous) (percentage) 0.790 83.72%
450.soplex 14.000 -
450.soplex (avg) 14.000 -
450.soplex (avg) (percentage) 2.450 1.24%
450.soplex (max) 5.000 -
450.soplex (max) (percentage) 0.970 6.59%
450.soplex (unexplained diff vs previous) 2.000 -50.00%
450.soplex (unexplained diff vs previous) (percentage) 0.350 -48.53%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.710 1.79%
453.povray (max) 8.000 14.29%
453.povray (max) (percentage) 1.640 16.31%
453.povray (unexplained diff vs previous) 9.000 350.00%
453.povray (unexplained diff vs previous) (percentage) 1.670 351.35%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 0.890 1.14%
456.hmmer 20.000 -
456.hmmer (avg) 20.000 -
456.hmmer (avg) (percentage) 0.700 1.45%
456.hmmer (max) 12.000 9.09%
456.hmmer (max) (percentage) 0.430 13.16%
456.hmmer (unexplained diff vs previous) 1.000 -66.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.030 -70.00%
458.sjeng 52.000 -
458.sjeng (avg) 52.000 -
458.sjeng (avg) (percentage) 0.710 -
458.sjeng (max) 21.000 10.53%
458.sjeng (max) (percentage) 0.290 7.41%
458.sjeng (unexplained diff vs previous) 18.000 -
458.sjeng (unexplained diff vs previous) (percentage) 0.240 -
459.GemsFDTD 75.000 -1.32%
459.GemsFDTD (avg) 75.000 -1.32%
459.GemsFDTD (avg) (percentage) 1.430 -
459.GemsFDTD (max) 28.000 -46.15%
459.GemsFDTD (max) (percentage) 0.530 -45.92%
459.GemsFDTD (unexplained diff vs previous) 27.000 -48.08%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.510 -47.42%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.770 -
462.libquantum (max) 3.000 50.00%
462.libquantum (max) (percentage) 2.900 50.26%
462.libquantum (unexplained diff vs previous) 4.000 100.00%
462.libquantum (unexplained diff vs previous) (percentage) 2.990 103.40%
464.h264ref 52.000 -
464.h264ref (avg) 52.000 -
464.h264ref (avg) (percentage) 0.910 1.11%
464.h264ref (max) 17.000 -10.53%
464.h264ref (max) (percentage) 0.300 -9.09%
464.h264ref (unexplained diff vs previous) 17.000 -10.53%
464.h264ref (unexplained diff vs previous) (percentage) 0.290 -12.12%
465.tonto 164.000 2.50%
465.tonto (avg) 164.000 2.50%
465.tonto (avg) (percentage) 1.420 2.16%
465.tonto (max) 270.000 11.57%
465.tonto (max) (percentage) 2.330 10.95%
465.tonto (unexplained diff vs previous) 264.000 877.78%
465.tonto (unexplained diff vs previous) (percentage) 2.280 891.30%
470.lbm 40.000 2.56%
470.lbm (avg) 40.000 2.56%
470.lbm (avg) (percentage) 1.070 1.90%
470.lbm (max) 4.000 -
470.lbm (max) (percentage) 0.130 18.18%
470.lbm (unexplained diff vs previous) 1.000 -50.00%
470.lbm (unexplained diff vs previous) (percentage) 0.030 -40.00%
471.omnetpp 62.000 1.64%
471.omnetpp (avg) 62.000 1.64%
471.omnetpp (avg) (percentage) 2.250 1.35%
471.omnetpp (max) 89.000 12.66%
471.omnetpp (max) (percentage) 3.240 12.50%
471.omnetpp (unexplained diff vs previous) 60.000 150.00%
471.omnetpp (unexplained diff vs previous) (percentage) 2.170 149.43%
473.astar 85.000 2.41%
473.astar (avg) 85.000 2.41%
473.astar (avg) (percentage) 1.620 2.53%
473.astar (max) 78.000 23.81%
473.astar (max) (percentage) 1.490 25.21%
473.astar (unexplained diff vs previous) 80.000 471.43%
473.astar (unexplained diff vs previous) (percentage) 1.510 480.77%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 0.960 2.13%
482.sphinx3 15.000 7.14%
482.sphinx3 (avg) 15.000 7.14%
482.sphinx3 (avg) (percentage) 1.800 1.69%
482.sphinx3 (max) 4.000 -
482.sphinx3 (max) (percentage) 0.560 5.66%
482.sphinx3 (unexplained diff vs previous) 1.000 -66.67%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.120 -66.67%
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 4.951 -