Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e1e4078ac59740a79cd709d61872abe15aba0087 (unchanged)
git_gcc https://gcc.gnu.org/git/gcc.git (6543a214485c0a722152e34de37424f9a0104bbb) (+ 97 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=e442e8376dcb14c904523c78994157cb5b502ff1 (unchanged)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/bb1d93eedc25f5d3a1db5e2183a6cbc5f3b5888a (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/318/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0318 (basepoints/gcc-15-04960-g6543a214485) 2024-11-04T19:24:12 0:00:00
Previous 0317 (basepoints/gcc-15-04863-g1fb467dbcc2) 2024-11-04T01:36:17 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 38 12
Performance Improvements 85 201
Added Tests 6 2
Unchanged Tests 1107 1021
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 5.88% 10953.000 11597.000 - -0.37% -
400.perlbench 1.95% 1535.000 1565.000 - -5.89% -
471.omnetpp 1.62% 2648.000 2691.000 - -1.21% -
450.soplex 1.56% 578.000 587.000 - 3.53% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
464.h264ref (unexplained diff vs previous) 2600.00% 1.000 27.000 - -99.61% -
464.h264ref (unexplained diff vs previous) (percentage) 2200.00% 0.020 0.460 - -99.61% -
400.perlbench (unexplained diff vs previous) 1400.00% 2.000 30.000 - -99.66% -
400.perlbench (unexplained diff vs previous) (percentage) 1376.92% 0.130 1.920 - -99.64% -
447.dealII (unexplained diff vs previous) (percentage) 616.67% 0.180 1.290 - -99.68% -
447.dealII (unexplained diff vs previous) 600.00% 3.000 21.000 - -99.70% -
482.sphinx3 (unexplained diff vs previous) 400.00% 1.000 5.000 - -99.98% -
482.sphinx3 (unexplained diff vs previous) (percentage) 400.00% 0.120 0.600 - -99.98% -
458.sjeng (unexplained diff vs previous) 220.00% 5.000 16.000 - -99.77% -
458.sjeng (unexplained diff vs previous) (percentage) 214.29% 0.070 0.220 - -99.76% -
450.soplex (unexplained diff vs previous) 125.00% 4.000 9.000 - -99.94% -
450.soplex (unexplained diff vs previous) (percentage) 121.74% 0.690 1.530 - -99.94% -
473.astar (unexplained diff vs previous) (percentage) 112.00% 0.250 0.530 - -99.73% -
473.astar (unexplained diff vs previous) 107.69% 13.000 27.000 - -99.73% -
444.namd (unexplained diff vs previous) 100.00% 1.000 2.000 - -99.98% -
444.namd (unexplained diff vs previous) (percentage) 100.00% 0.130 0.260 - -99.98% -
429.mcf (unexplained diff vs previous) (percentage) 89.43% 1.230 2.330 - -99.82% -
429.mcf (unexplained diff vs previous) 84.62% 26.000 48.000 - -99.74% -
471.omnetpp (unexplained diff vs previous) 59.26% 27.000 43.000 - -99.60% -
445.gobmk (unexplained diff vs previous) 59.09% 22.000 35.000 - -99.37% -
445.gobmk (unexplained diff vs previous) (percentage) 59.09% 0.440 0.700 - -99.37% -
471.omnetpp (unexplained diff vs previous) (percentage) 56.86% 1.020 1.600 - -99.59% -
401.bzip2 (unexplained diff vs previous) 50.00% 4.000 6.000 - -99.94% -
401.bzip2 (unexplained diff vs previous) (percentage) 42.86% 0.140 0.200 - -99.95% -
471.omnetpp (max) 30.30% 33.000 43.000 - -98.02% -
471.omnetpp (max) (percentage) 27.56% 1.270 1.620 - -97.97% -
436.cactusADM (unexplained diff vs previous) (percentage) 26.32% 0.190 0.240 - -99.97% -
436.cactusADM (unexplained diff vs previous) 25.00% 4.000 5.000 - -99.97% -
437.leslie3d 6.29% 477.000 507.000 - 42.82% -
437.leslie3d (avg) 6.29% 477.000 507.000 - 42.82% -
400.perlbench 3.45% 29.000 30.000 - -71.43% -
400.perlbench (avg) 3.45% 29.000 30.000 - -71.43% -
471.omnetpp 1.79% 56.000 57.000 - -74.44% -
471.omnetpp (avg) 1.79% 56.000 57.000 - -74.44% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf -2.28% 2109.000 2061.000 - 44.73% -
435.gromacs -2.00% 7311.000 7165.000 - -1.43% -
447.dealII -1.27% 1651.000 1630.000 - -5.45% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
456.hmmer (unexplained diff vs previous) (percentage) -92.86% 0.420 0.030 - -99.99% -
456.hmmer (unexplained diff vs previous) -91.67% 12.000 1.000 - -99.99% -
434.zeusmp (max) (percentage) -77.85% 1.580 0.350 - -99.57% -
434.zeusmp (max) -77.78% 36.000 8.000 - -99.61% -
410.bwaves (max) -68.25% 63.000 20.000 - -99.39% -
410.bwaves (max) (percentage) -68.22% 1.070 0.340 - -99.41% -
453.povray (max) -64.71% 17.000 6.000 - -98.80% -
470.lbm (unexplained diff vs previous) -64.00% 25.000 9.000 - -99.91% -
453.povray (max) (percentage) -63.99% 3.110 1.120 - -98.77% -
470.lbm (unexplained diff vs previous) (percentage) -63.64% 0.660 0.240 - -99.91% -
453.povray (unexplained diff vs previous) -50.00% 2.000 1.000 - -99.98% -
453.povray (unexplained diff vs previous) (percentage) -50.00% 0.360 0.180 - -99.98% -
462.libquantum (max) -40.00% 5.000 3.000 - -97.83% -
462.libquantum (max) (percentage) -39.84% 3.790 2.280 - -97.70% -
470.lbm (max) -39.39% 33.000 20.000 - -99.27% -
456.hmmer (max) (percentage) -38.60% 0.570 0.350 - -99.52% -
470.lbm (max) (percentage) -38.20% 0.890 0.550 - -99.24% -
456.hmmer (max) -37.50% 16.000 10.000 - -99.53% -
435.gromacs (max) -36.73% 226.000 143.000 - -96.00% -
435.gromacs (max) (percentage) -35.48% 3.100 2.000 - -95.93% -
434.zeusmp (unexplained diff vs previous) (percentage) -20.93% 0.430 0.340 - -99.92% -
435.gromacs (unexplained diff vs previous) -20.65% 184.000 146.000 - -97.92% -
434.zeusmp (unexplained diff vs previous) -20.00% 10.000 8.000 - -99.93% -
437.leslie3d (unexplained diff vs previous) (percentage) -19.80% 6.920 5.550 - -72.08% -
435.gromacs (unexplained diff vs previous) (percentage) -19.05% 2.520 2.040 - -97.89% -
410.bwaves (unexplained diff vs previous) -16.67% 24.000 20.000 - -99.74% -
437.leslie3d (unexplained diff vs previous) -15.04% 758.000 644.000 - -72.17% -
410.bwaves (unexplained diff vs previous) (percentage) -15.00% 0.400 0.340 - -99.75% -
444.namd (max) -11.11% 9.000 8.000 - -98.83% -
433.milc (max) -9.68% 31.000 28.000 - -97.32% -
437.leslie3d (max) (percentage) -9.12% 6.470 5.880 - -64.54% -
429.mcf (max) -7.50% 240.000 222.000 - -83.21% -
459.GemsFDTD (max) -7.14% 98.000 91.000 - -97.94% -
456.hmmer -6.67% 15.000 14.000 - -88.98% -
456.hmmer (avg) -6.67% 15.000 14.000 - -88.98% -
459.GemsFDTD (max) (percentage) -6.59% 1.820 1.700 - -97.62% -
433.milc (max) (percentage) -6.34% 2.680 2.510 - -97.23% -
450.soplex (max) (percentage) -6.16% 4.220 3.960 - -95.89% -
458.sjeng (max) (percentage) -6.09% 1.150 1.080 - -97.73% -
458.sjeng (max) -5.95% 84.000 79.000 - -97.81% -
473.astar (max) -5.29% 208.000 197.000 - -94.22% -
429.mcf (max) (percentage) -5.18% 11.390 10.800 - -88.37% -
400.perlbench (max) -5.00% 40.000 38.000 - -97.28% -
401.bzip2 (max) -5.00% 60.000 57.000 - -97.54% -
482.sphinx3 (max) -5.00% 20.000 19.000 - -97.91% -
401.bzip2 (max) (percentage) -4.85% 2.060 1.960 - -97.51% -
482.sphinx3 (max) (percentage) -4.78% 2.510 2.390 - -97.52% -
447.dealII (max) -4.76% 21.000 20.000 - -98.55% -
473.astar (max) (percentage) -4.69% 4.050 3.860 - -94.19% -
400.perlbench (max) (percentage) -4.60% 2.610 2.490 - -97.04% -
444.namd (max) (percentage) -4.17% 1.200 1.150 - -98.75% -
450.soplex (max) -4.17% 24.000 23.000 - -95.78% -
464.h264ref (max) (percentage) -4.00% 1.000 0.960 - -98.23% -
465.tonto (max) -3.90% 205.000 197.000 - -90.67% -
445.gobmk (max) -3.90% 77.000 74.000 - -97.20% -
445.gobmk (max) (percentage) -3.87% 1.550 1.490 - -97.17% -
437.leslie3d (max) -3.81% 708.000 681.000 - -64.70% -
465.tonto (max) (percentage) -3.78% 1.850 1.780 - -90.78% -
436.cactusADM (max) (percentage) -3.54% 1.130 1.090 - -98.77% -
464.h264ref (max) -3.51% 57.000 55.000 - -98.24% -
429.mcf -2.97% 101.000 98.000 - -10.09% -
429.mcf (avg) -2.97% 101.000 98.000 - -10.09% -
435.gromacs -2.33% 129.000 126.000 - -65.67% -
435.gromacs (avg) -2.33% 129.000 126.000 - -65.67% -
458.sjeng -2.22% 45.000 44.000 - -81.74% -
458.sjeng (avg) -2.22% 45.000 44.000 - -81.74% -
465.tonto -2.16% 139.000 136.000 - -62.74% -
465.tonto (avg) -2.16% 139.000 136.000 - -62.74% -
456.hmmer (avg) (percentage) -1.89% 0.530 0.520 - -88.21% -
444.namd (avg) (percentage) -1.68% 1.190 1.170 - -87.31% -
458.sjeng (avg) (percentage) -1.61% 0.620 0.610 - -80.88% -
465.tonto (avg) (percentage) -1.60% 1.250 1.230 - -63.17% -
447.dealII (max) (percentage) -1.55% 1.290 1.270 - -98.42% -
459.GemsFDTD -1.41% 71.000 70.000 - -80.56% -
459.GemsFDTD (avg) -1.41% 71.000 70.000 - -80.56% -
401.bzip2 (avg) (percentage) -1.38% 1.450 1.430 - -73.91% -
464.h264ref (avg) (percentage) -1.35% 0.740 0.730 - -85.66% -
473.astar -1.20% 83.000 82.000 - -61.50% -
473.astar (avg) -1.20% 83.000 82.000 - -61.50% -
410.bwaves (avg) (percentage) -1.12% 0.890 0.880 - -80.05% -
462.libquantum (avg) (percentage) -1.08% 1.860 1.840 - -75.98% -
434.zeusmp (avg) (percentage) -1.03% 0.970 0.960 - -83.59% -

Added Tests - sample variability
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.52% 1946163.000 1975683.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 44.73% 1424.000 2061.000 - -2.28% -
444.namd 4.69% 747.000 782.000 - 0.26% -
410.bwaves 4.64% 5652.000 5914.000 - -0.34% -
450.soplex 3.53% 567.000 587.000 - 1.56% -
436.cactusADM 1.54% 2083.000 2115.000 - 0.24% -
465.tonto 1.35% 10941.000 11089.000 - -0.30% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 43.61% 3.050 4.380 - 0.46% -
437.leslie3d 42.82% 355.000 507.000 - 6.29% -
437.leslie3d (avg) 42.82% 355.000 507.000 - 6.29% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - 0.00% -
nb_execution_successful -17.24% 29.000 24.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -7.89% 59479.000 54786.000 - 0.00% -
434.zeusmp -3.39% 152037.000 146883.000 - 0.00% -
459.GemsFDTD -3.30% 244728.000 236656.000 - 0.00% -
464.h264ref -2.14% 387748.000 379440.000 - 0.00% -
435.gromacs -1.92% 499261.000 489677.000 - 0.00% -
410.bwaves -1.77% 16760.000 16464.000 - 0.00% -
453.povray -1.75% 661389.000 649821.000 - 0.00% -
458.sjeng -1.70% 104048.000 102284.000 - 0.00% -
470.lbm -1.49% 9691.000 9547.000 - 0.00% -
447.dealII -1.48% 311591.000 306976.000 - 0.00% -
433.milc -1.44% 61382.000 60501.000 - 0.00% -
444.namd -1.39% 188385.000 185768.000 - 0.00% -
456.hmmer -1.38% 108691.000 107195.000 - 0.00% -
450.soplex -1.22% 205081.000 202574.000 - 0.00% -
462.libquantum -1.19% 16085.000 15893.000 - 0.00% -
401.bzip2 -1.16% 44233.000 43719.000 - 0.00% -
482.sphinx3 -1.06% 112859.000 111663.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -13.19% 6173.000 5359.000 - -0.50% -
482.sphinx3 -11.53% 945.000 836.000 - 0.60% -
434.zeusmp -7.68% 2514.000 2321.000 - 0.35% -
400.perlbench -5.89% 1663.000 1565.000 - 1.95% -
447.dealII -5.45% 1724.000 1630.000 - -1.27% -
458.sjeng -2.70% 7567.000 7363.000 - 0.22% -
435.gromacs -1.43% 7269.000 7165.000 - -2.00% -
462.libquantum -1.43% 140.000 138.000 - 0.00% -
471.omnetpp -1.21% 2724.000 2691.000 - 1.62% -
445.gobmk -1.06% 5023.000 4970.000 - -0.70% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
456.hmmer (unexplained diff vs previous) (percentage) -99.99% 278.630 0.030 - -92.86% -
456.hmmer (unexplained diff vs previous) -99.99% 8030.000 1.000 - -91.67% -
453.povray (unexplained diff vs previous) (percentage) -99.98% 1043.330 0.180 - -50.00% -
453.povray (unexplained diff vs previous) -99.98% 5707.000 1.000 - -50.00% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 5.000 - 400.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.98% 2759.050 0.600 - 400.00% -
444.namd (unexplained diff vs previous) (percentage) -99.98% 1118.610 0.260 - 100.00% -
444.namd (unexplained diff vs previous) -99.98% 8356.000 2.000 - 100.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.97% 787.130 0.240 - 26.32% -
436.cactusADM (unexplained diff vs previous) -99.97% 16396.000 5.000 - 25.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.95% 368.340 0.200 - 42.86% -
401.bzip2 (unexplained diff vs previous) -99.94% 10833.000 6.000 - 50.00% -
450.soplex (unexplained diff vs previous) (percentage) -99.94% 2557.320 1.530 - 121.74% -
433.milc (unexplained diff vs previous) -99.94% 11289.000 7.000 - 0.00% -
450.soplex (unexplained diff vs previous) -99.94% 14500.000 9.000 - 125.00% -
433.milc (unexplained diff vs previous) (percentage) -99.94% 980.800 0.610 - 0.00% -
434.zeusmp (unexplained diff vs previous) -99.93% 11280.000 8.000 - -20.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.92% 448.690 0.340 - -20.93% -
470.lbm (unexplained diff vs previous) -99.91% 9990.000 9.000 - -64.00% -
470.lbm (unexplained diff vs previous) (percentage) -99.91% 263.730 0.240 - -63.64% -
459.GemsFDTD (unexplained diff vs previous) -99.83% 15458.000 27.000 - 0.00% -
429.mcf (unexplained diff vs previous) (percentage) -99.82% 1305.690 2.330 - 89.43% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.80% 250.410 0.500 - 0.00% -
458.sjeng (unexplained diff vs previous) -99.77% 6876.000 16.000 - 220.00% -
458.sjeng (unexplained diff vs previous) (percentage) -99.76% 90.870 0.220 - 214.29% -
410.bwaves (unexplained diff vs previous) (percentage) -99.75% 137.900 0.340 - -15.00% -
410.bwaves (unexplained diff vs previous) -99.74% 7794.000 20.000 - -16.67% -
429.mcf (unexplained diff vs previous) -99.74% 18593.000 48.000 - 84.62% -
473.astar (unexplained diff vs previous) -99.73% 10146.000 27.000 - 107.69% -
473.astar (unexplained diff vs previous) (percentage) -99.73% 197.850 0.530 - 112.00% -
447.dealII (unexplained diff vs previous) -99.70% 7017.000 21.000 - 600.00% -
447.dealII (unexplained diff vs previous) (percentage) -99.68% 407.020 1.290 - 616.67% -
400.perlbench (unexplained diff vs previous) -99.66% 8772.000 30.000 - 1400.00% -
400.perlbench (unexplained diff vs previous) (percentage) -99.64% 527.480 1.920 - 1376.92% -
464.h264ref (unexplained diff vs previous) (percentage) -99.61% 118.780 0.460 - 2200.00% -
434.zeusmp (max) -99.61% 2055.000 8.000 - -77.78% -
464.h264ref (unexplained diff vs previous) -99.61% 6843.000 27.000 - 2600.00% -
471.omnetpp (unexplained diff vs previous) -99.60% 10659.000 43.000 - 59.26% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.59% 391.300 1.600 - 56.86% -
434.zeusmp (max) (percentage) -99.57% 81.770 0.350 - -77.85% -
456.hmmer (max) -99.53% 2120.000 10.000 - -37.50% -
456.hmmer (max) (percentage) -99.52% 73.590 0.350 - -38.60% -
410.bwaves (max) (percentage) -99.41% 57.970 0.340 - -68.22% -
410.bwaves (max) -99.39% 3276.000 20.000 - -68.25% -
445.gobmk (unexplained diff vs previous) -99.37% 5581.000 35.000 - 59.09% -
445.gobmk (unexplained diff vs previous) (percentage) -99.37% 111.110 0.700 - 59.09% -
470.lbm (max) -99.27% 2746.000 20.000 - -39.39% -
470.lbm (max) (percentage) -99.24% 72.510 0.550 - -38.20% -
454.calculix (max) (percentage) -99.17% 99.730 0.830 - 0.00% -
416.gamess (max) (percentage) -99.04% 38.550 0.370 - 0.00% -
444.namd (max) -98.83% 685.000 8.000 - -11.11% -
481.wrf (max) (percentage) -98.81% 68.700 0.820 - 0.00% -
453.povray (max) -98.80% 499.000 6.000 - -64.71% -
453.povray (max) (percentage) -98.77% 91.250 1.120 - -63.99% -
436.cactusADM (max) (percentage) -98.77% 88.730 1.090 - -3.54% -
436.cactusADM (max) -98.76% 1848.000 23.000 - 0.00% -
444.namd (max) (percentage) -98.75% 91.790 1.150 - -4.17% -
447.dealII (max) -98.55% 1384.000 20.000 - -4.76% -
447.dealII (max) (percentage) -98.42% 80.280 1.270 - -1.55% -
464.h264ref (max) -98.24% 3127.000 55.000 - -3.51% -
464.h264ref (max) (percentage) -98.23% 54.290 0.960 - -4.00% -
465.tonto (unexplained diff vs previous) (percentage) -98.15% 16.190 0.300 - 0.00% -
465.tonto (unexplained diff vs previous) -98.14% 1771.000 33.000 - 0.00% -
471.omnetpp (max) -98.02% 2169.000 43.000 - 30.30% -
471.omnetpp (max) (percentage) -97.97% 79.650 1.620 - 27.56% -
459.GemsFDTD (max) -97.94% 4411.000 91.000 - -7.14% -
435.gromacs (unexplained diff vs previous) -97.92% 7028.000 146.000 - -20.65% -
482.sphinx3 (max) -97.91% 911.000 19.000 - -5.00% -
435.gromacs (unexplained diff vs previous) (percentage) -97.89% 96.680 2.040 - -19.05% -
462.libquantum (max) -97.83% 138.000 3.000 - -40.00% -
458.sjeng (max) -97.81% 3602.000 79.000 - -5.95% -
458.sjeng (max) (percentage) -97.73% 47.610 1.080 - -6.09% -
462.libquantum (max) (percentage) -97.70% 98.970 2.280 - -39.84% -
459.GemsFDTD (max) (percentage) -97.62% 71.460 1.700 - -6.59% -
401.bzip2 (max) -97.54% 2313.000 57.000 - -5.00% -
482.sphinx3 (max) (percentage) -97.52% 96.500 2.390 - -4.78% -
401.bzip2 (max) (percentage) -97.51% 78.650 1.960 - -4.85% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
433.milc (max) -97.32% 1044.000 28.000 - -9.68% -
400.perlbench (max) -97.28% 1397.000 38.000 - -5.00% -
433.milc (max) (percentage) -97.23% 90.750 2.510 - -6.34% -
445.gobmk (max) -97.20% 2643.000 74.000 - -3.90% -
445.gobmk (max) (percentage) -97.17% 52.630 1.490 - -3.87% -
400.perlbench (max) (percentage) -97.04% 84.060 2.490 - -4.60% -
403.gcc (max) (percentage) -96.20% 99.360 3.780 - -0.26% -
435.gromacs (max) -96.00% 3573.000 143.000 - -36.73% -
435.gromacs (max) (percentage) -95.93% 49.160 2.000 - -35.48% -
450.soplex (max) (percentage) -95.89% 96.240 3.960 - -6.16% -
450.soplex (max) -95.78% 545.000 23.000 - -4.17% -
473.astar (max) -94.22% 3406.000 197.000 - -5.29% -
473.astar (max) (percentage) -94.19% 66.430 3.860 - -4.69% -
465.tonto (max) (percentage) -90.78% 19.310 1.780 - -3.78% -
465.tonto (max) -90.67% 2112.000 197.000 - -3.90% -
456.hmmer -88.98% 127.000 14.000 - -6.67% -
456.hmmer (avg) -88.98% 127.000 14.000 - -6.67% -
429.mcf (max) (percentage) -88.37% 92.890 10.800 - -5.18% -
456.hmmer (avg) (percentage) -88.21% 4.410 0.520 - -1.89% -
444.namd (avg) (percentage) -87.31% 9.220 1.170 - -1.68% -
444.namd -86.76% 68.000 9.000 - 0.00% -
444.namd (avg) -86.76% 68.000 9.000 - 0.00% -
470.lbm -86.61% 239.000 32.000 - 0.00% -
470.lbm (avg) -86.61% 239.000 32.000 - 0.00% -
470.lbm (avg) (percentage) -86.57% 6.330 0.850 - 0.00% -
464.h264ref -85.67% 293.000 42.000 - 0.00% -
464.h264ref (avg) -85.67% 293.000 42.000 - 0.00% -
464.h264ref (avg) (percentage) -85.66% 5.090 0.730 - -1.35% -
434.zeusmp -85.03% 147.000 22.000 - 0.00% -
434.zeusmp (avg) -85.03% 147.000 22.000 - 0.00% -
434.zeusmp (avg) (percentage) -83.59% 5.850 0.960 - -1.03% -
429.mcf (max) -83.21% 1322.000 222.000 - -7.50% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - 0.00% -
436.cactusADM -82.26% 124.000 22.000 - 0.00% -
436.cactusADM (avg) -82.26% 124.000 22.000 - 0.00% -
436.cactusADM (avg) (percentage) -82.11% 5.980 1.070 - -0.93% -
458.sjeng -81.74% 241.000 44.000 - -2.22% -
458.sjeng (avg) -81.74% 241.000 44.000 - -2.22% -
482.sphinx3 -81.08% 74.000 14.000 - 0.00% -
482.sphinx3 (avg) -81.08% 74.000 14.000 - 0.00% -
453.povray -80.95% 42.000 8.000 - 0.00% -
453.povray (avg) -80.95% 42.000 8.000 - 0.00% -
458.sjeng (avg) (percentage) -80.88% 3.190 0.610 - -1.61% -
447.dealII -80.81% 99.000 19.000 - 0.00% -
447.dealII (avg) -80.81% 99.000 19.000 - 0.00% -
459.GemsFDTD -80.56% 360.000 70.000 - -1.41% -
459.GemsFDTD (avg) -80.56% 360.000 70.000 - -1.41% -
410.bwaves (avg) (percentage) -80.05% 4.410 0.880 - -1.12% -
453.povray (avg) (percentage) -80.03% 7.810 1.560 - -0.64% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
433.milc -79.71% 69.000 14.000 - 0.00% -
433.milc (avg) -79.71% 69.000 14.000 - 0.00% -
433.milc (avg) (percentage) -79.67% 6.050 1.230 - 0.00% -
447.dealII (avg) (percentage) -79.38% 5.770 1.190 - 0.00% -
410.bwaves -79.12% 249.000 52.000 - 0.00% -
410.bwaves (avg) -79.12% 249.000 52.000 - 0.00% -
482.sphinx3 (avg) (percentage) -78.06% 7.930 1.740 - -0.57% -
459.GemsFDTD (avg) (percentage) -77.57% 5.840 1.310 - -0.76% -
445.gobmk -77.55% 196.000 44.000 - 0.00% -
445.gobmk (avg) -77.55% 196.000 44.000 - 0.00% -
445.gobmk (avg) (percentage) -77.24% 3.910 0.890 - 0.00% -
462.libquantum (avg) (percentage) -75.98% 7.660 1.840 - -1.08% -
471.omnetpp -74.44% 223.000 57.000 - 1.79% -
471.omnetpp (avg) -74.44% 223.000 57.000 - 1.79% -
401.bzip2 -73.91% 161.000 42.000 - 0.00% -
401.bzip2 (avg) -73.91% 161.000 42.000 - 0.00% -
401.bzip2 (avg) (percentage) -73.91% 5.480 1.430 - -1.38% -
471.omnetpp (avg) (percentage) -73.90% 8.200 2.140 - 0.00% -
437.leslie3d (unexplained diff vs previous) -72.17% 2314.000 644.000 - -15.04% -
437.leslie3d (unexplained diff vs previous) (percentage) -72.08% 19.880 5.550 - -19.80% -
400.perlbench -71.43% 105.000 30.000 - 3.45% -
400.perlbench (avg) -71.43% 105.000 30.000 - 3.45% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
400.perlbench (avg) (percentage) -69.45% 6.350 1.940 - 0.00% -
435.gromacs -65.67% 367.000 126.000 - -2.33% -
435.gromacs (avg) -65.67% 367.000 126.000 - -2.33% -
435.gromacs (avg) (percentage) -65.02% 5.060 1.770 - 0.00% -
437.leslie3d (max) -64.70% 1929.000 681.000 - -3.81% -
437.leslie3d (max) (percentage) -64.54% 16.580 5.880 - -9.12% -
465.tonto (avg) (percentage) -63.17% 3.340 1.230 - -1.60% -
465.tonto -62.74% 365.000 136.000 - -2.16% -
465.tonto (avg) -62.74% 365.000 136.000 - -2.16% -
473.astar -61.50% 213.000 82.000 - -1.20% -
473.astar (avg) -61.50% 213.000 82.000 - -1.20% -
450.soplex (avg) (percentage) -61.12% 6.430 2.500 - -0.40% -
450.soplex -61.11% 36.000 14.000 - 0.00% -
450.soplex (avg) -61.11% 36.000 14.000 - 0.00% -
473.astar (avg) (percentage) -61.06% 4.160 1.620 - -0.61% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
429.mcf (avg) (percentage) -37.68% 7.670 4.780 - -0.62% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf -10.09% 109.000 98.000 - -2.97% -
429.mcf (avg) -10.09% 109.000 98.000 - -2.97% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1565.000 1.95%
401.bzip2 2951.000 -
410.bwaves 5914.000 -
429.mcf 2061.000 -2.28%
433.milc 1150.000 -
434.zeusmp 2321.000 -
435.gromacs 7165.000 -2.00%
436.cactusADM 2115.000 -
437.leslie3d 11597.000 5.88%
444.namd 782.000 -
445.gobmk 4970.000 -
447.dealII 1630.000 -1.27%
450.soplex 587.000 1.56%
453.povray 547.000 -
456.hmmer 2880.000 -
458.sjeng 7363.000 -
459.GemsFDTD 5359.000 -
462.libquantum 138.000 -
464.h264ref 5816.000 -
465.tonto 11089.000 -
470.lbm 3784.000 -
471.omnetpp 2691.000 1.62%
473.astar 5119.000 -
482.sphinx3 836.000 -
Geometric Mean 2458.067 -
Code Size (bytes)
Name Current %
400.perlbench 892379.000 -
401.bzip2 43719.000 -
410.bwaves 16464.000 -
429.mcf 9119.000 -
433.milc 60501.000 -
434.zeusmp 146883.000 -
435.gromacs 489677.000 -
436.cactusADM 352411.000 -
437.leslie3d 54786.000 -
444.namd 185768.000 -
445.gobmk 1379170.000 -
447.dealII 306976.000 -
450.soplex 202574.000 -
453.povray 649821.000 -
456.hmmer 107195.000 -
458.sjeng 102284.000 -
459.GemsFDTD 236656.000 -
462.libquantum 15893.000 -
464.h264ref 379440.000 -
465.tonto 1975683.000 -
470.lbm 9547.000 -
471.omnetpp 475388.000 -
473.astar 22760.000 -
482.sphinx3 111663.000 -
Geometric Mean 136711.200 -
sample variability
Name Current %
400.perlbench 30.000 3.45%
400.perlbench (avg) 30.000 3.45%
400.perlbench (avg) (percentage) 1.940 -
400.perlbench (max) 38.000 -5.00%
400.perlbench (max) (percentage) 2.490 -4.60%
400.perlbench (unexplained diff vs previous) 30.000 1400.00%
400.perlbench (unexplained diff vs previous) (percentage) 1.920 1376.92%
401.bzip2 42.000 -
401.bzip2 (avg) 42.000 -
401.bzip2 (avg) (percentage) 1.430 -1.38%
401.bzip2 (max) 57.000 -5.00%
401.bzip2 (max) (percentage) 1.960 -4.85%
401.bzip2 (unexplained diff vs previous) 6.000 50.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.200 42.86%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 3.780 -
410.bwaves 52.000 -
410.bwaves (avg) 52.000 -
410.bwaves (avg) (percentage) 0.880 -1.12%
410.bwaves (max) 20.000 -68.25%
410.bwaves (max) (percentage) 0.340 -68.22%
410.bwaves (unexplained diff vs previous) 20.000 -16.67%
410.bwaves (unexplained diff vs previous) (percentage) 0.340 -15.00%
416.gamess (avg) (percentage) 0.760 -
416.gamess (max) (percentage) 0.370 -
429.mcf 98.000 -2.97%
429.mcf (avg) 98.000 -2.97%
429.mcf (avg) (percentage) 4.780 -
429.mcf (max) 222.000 -7.50%
429.mcf (max) (percentage) 10.800 -5.18%
429.mcf (unexplained diff vs previous) 48.000 84.62%
429.mcf (unexplained diff vs previous) (percentage) 2.330 89.43%
433.milc 14.000 -
433.milc (avg) 14.000 -
433.milc (avg) (percentage) 1.230 -
433.milc (max) 28.000 -9.68%
433.milc (max) (percentage) 2.510 -6.34%
433.milc (unexplained diff vs previous) 7.000 -
433.milc (unexplained diff vs previous) (percentage) 0.610 -
434.zeusmp 22.000 -
434.zeusmp (avg) 22.000 -
434.zeusmp (avg) (percentage) 0.960 -1.03%
434.zeusmp (max) 8.000 -77.78%
434.zeusmp (max) (percentage) 0.350 -77.85%
434.zeusmp (unexplained diff vs previous) 8.000 -20.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.340 -20.93%
435.gromacs 126.000 -2.33%
435.gromacs (avg) 126.000 -2.33%
435.gromacs (avg) (percentage) 1.770 -
435.gromacs (max) 143.000 -36.73%
435.gromacs (max) (percentage) 2.000 -35.48%
435.gromacs (unexplained diff vs previous) 146.000 -20.65%
435.gromacs (unexplained diff vs previous) (percentage) 2.040 -19.05%
436.cactusADM 22.000 -
436.cactusADM (avg) 22.000 -
436.cactusADM (avg) (percentage) 1.070 -
436.cactusADM (max) 23.000 -
436.cactusADM (max) (percentage) 1.090 -3.54%
436.cactusADM (unexplained diff vs previous) 5.000 25.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.240 26.32%
437.leslie3d 507.000 6.29%
437.leslie3d (avg) 507.000 6.29%
437.leslie3d (avg) (percentage) 4.380 -
437.leslie3d (max) 681.000 -3.81%
437.leslie3d (max) (percentage) 5.880 -9.12%
437.leslie3d (unexplained diff vs previous) 644.000 -15.04%
437.leslie3d (unexplained diff vs previous) (percentage) 5.550 -19.80%
444.namd 9.000 -
444.namd (avg) 9.000 -
444.namd (avg) (percentage) 1.170 -1.68%
444.namd (max) 8.000 -11.11%
444.namd (max) (percentage) 1.150 -4.17%
444.namd (unexplained diff vs previous) 2.000 100.00%
444.namd (unexplained diff vs previous) (percentage) 0.260 100.00%
445.gobmk 44.000 -
445.gobmk (avg) 44.000 -
445.gobmk (avg) (percentage) 0.890 -
445.gobmk (max) 74.000 -3.90%
445.gobmk (max) (percentage) 1.490 -3.87%
445.gobmk (unexplained diff vs previous) 35.000 59.09%
445.gobmk (unexplained diff vs previous) (percentage) 0.700 59.09%
447.dealII 19.000 -
447.dealII (avg) 19.000 -
447.dealII (avg) (percentage) 1.190 -
447.dealII (max) 20.000 -4.76%
447.dealII (max) (percentage) 1.270 -1.55%
447.dealII (unexplained diff vs previous) 21.000 600.00%
447.dealII (unexplained diff vs previous) (percentage) 1.290 616.67%
450.soplex 14.000 -
450.soplex (avg) 14.000 -
450.soplex (avg) (percentage) 2.500 -
450.soplex (max) 23.000 -4.17%
450.soplex (max) (percentage) 3.960 -6.16%
450.soplex (unexplained diff vs previous) 9.000 125.00%
450.soplex (unexplained diff vs previous) (percentage) 1.530 121.74%
453.povray 8.000 -
453.povray (avg) 8.000 -
453.povray (avg) (percentage) 1.560 -
453.povray (max) 6.000 -64.71%
453.povray (max) (percentage) 1.120 -63.99%
453.povray (unexplained diff vs previous) 1.000 -50.00%
453.povray (unexplained diff vs previous) (percentage) 0.180 -50.00%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 0.830 -
456.hmmer 14.000 -6.67%
456.hmmer (avg) 14.000 -6.67%
456.hmmer (avg) (percentage) 0.520 -1.89%
456.hmmer (max) 10.000 -37.50%
456.hmmer (max) (percentage) 0.350 -38.60%
456.hmmer (unexplained diff vs previous) 1.000 -91.67%
456.hmmer (unexplained diff vs previous) (percentage) 0.030 -92.86%
458.sjeng 44.000 -2.22%
458.sjeng (avg) 44.000 -2.22%
458.sjeng (avg) (percentage) 0.610 -1.61%
458.sjeng (max) 79.000 -5.95%
458.sjeng (max) (percentage) 1.080 -6.09%
458.sjeng (unexplained diff vs previous) 16.000 220.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.220 214.29%
459.GemsFDTD 70.000 -1.41%
459.GemsFDTD (avg) 70.000 -1.41%
459.GemsFDTD (avg) (percentage) 1.310 -
459.GemsFDTD (max) 91.000 -7.14%
459.GemsFDTD (max) (percentage) 1.700 -6.59%
459.GemsFDTD (unexplained diff vs previous) 27.000 -
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.500 -
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.840 -1.08%
462.libquantum (max) 3.000 -40.00%
462.libquantum (max) (percentage) 2.280 -39.84%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 42.000 -
464.h264ref (avg) 42.000 -
464.h264ref (avg) (percentage) 0.730 -1.35%
464.h264ref (max) 55.000 -3.51%
464.h264ref (max) (percentage) 0.960 -4.00%
464.h264ref (unexplained diff vs previous) 27.000 2600.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.460 2200.00%
465.tonto 136.000 -2.16%
465.tonto (avg) 136.000 -2.16%
465.tonto (avg) (percentage) 1.230 -1.60%
465.tonto (max) 197.000 -3.90%
465.tonto (max) (percentage) 1.780 -3.78%
465.tonto (unexplained diff vs previous) 33.000 -
465.tonto (unexplained diff vs previous) (percentage) 0.300 -
470.lbm 32.000 -
470.lbm (avg) 32.000 -
470.lbm (avg) (percentage) 0.850 -
470.lbm (max) 20.000 -39.39%
470.lbm (max) (percentage) 0.550 -38.20%
470.lbm (unexplained diff vs previous) 9.000 -64.00%
470.lbm (unexplained diff vs previous) (percentage) 0.240 -63.64%
471.omnetpp 57.000 1.79%
471.omnetpp (avg) 57.000 1.79%
471.omnetpp (avg) (percentage) 2.140 -
471.omnetpp (max) 43.000 30.30%
471.omnetpp (max) (percentage) 1.620 27.56%
471.omnetpp (unexplained diff vs previous) 43.000 59.26%
471.omnetpp (unexplained diff vs previous) (percentage) 1.600 56.86%
473.astar 82.000 -1.20%
473.astar (avg) 82.000 -1.20%
473.astar (avg) (percentage) 1.620 -
473.astar (max) 197.000 -5.29%
473.astar (max) (percentage) 3.860 -4.69%
473.astar (unexplained diff vs previous) 27.000 107.69%
473.astar (unexplained diff vs previous) (percentage) 0.530 112.00%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 0.820 -
482.sphinx3 14.000 -
482.sphinx3 (avg) 14.000 -
482.sphinx3 (avg) (percentage) 1.740 -
482.sphinx3 (max) 19.000 -5.00%
482.sphinx3 (max) (percentage) 2.390 -4.78%
482.sphinx3 (unexplained diff vs previous) 5.000 400.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.600 400.00%
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 5.617 1.84%