Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=61ced226a4fc2e6df7836cd9c0f7e1ad47af2440 (+ 13 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/ecd2c373720af214f20671f9e7d760548b51ef4a (+ 27 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=dc1a77269c971652a8a5167ec366792eae052e65 (+ 11 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=378e271eb0f3f02d1e81bd813ec97c106238613e (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/211/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0211 (3-changed-components) 2024-03-22T06:41:17 0:00:00
Previous 0210 (4-changed-components) 2024-03-19T04:27:09 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 48 15
Performance Improvements 91 197
Added Tests 2 2
Unchanged Tests 1095 1022
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 4.76% 11262.000 11798.000 - 1.36% -
471.omnetpp 2.19% 2735.000 2795.000 - 2.61% -
447.dealII 1.77% 1640.000 1669.000 - -3.19% -
462.libquantum 1.49% 134.000 136.000 - -2.86% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
445.gobmk (unexplained diff vs previous) (percentage) 1650.00% 0.020 0.350 - -99.68% -
445.gobmk (unexplained diff vs previous) 1600.00% 1.000 17.000 - -99.70% -
470.lbm (unexplained diff vs previous) 466.67% 3.000 17.000 - -99.83% -
470.lbm (unexplained diff vs previous) (percentage) 462.50% 0.080 0.450 - -99.83% -
433.milc (unexplained diff vs previous) 400.00% 1.000 5.000 - -99.96% -
444.namd (unexplained diff vs previous) 400.00% 1.000 5.000 - -99.94% -
444.namd (unexplained diff vs previous) (percentage) 400.00% 0.130 0.650 - -99.94% -
433.milc (unexplained diff vs previous) (percentage) 377.78% 0.090 0.430 - -99.96% -
465.tonto (unexplained diff vs previous) (percentage) 287.50% 0.160 0.620 - -96.17% -
465.tonto (unexplained diff vs previous) 278.95% 19.000 72.000 - -95.93% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 209.09% 0.110 0.340 - -99.86% -
459.GemsFDTD (unexplained diff vs previous) 200.00% 6.000 18.000 - -99.88% -
437.leslie3d (unexplained diff vs previous) 174.87% 195.000 536.000 - -76.84% -
437.leslie3d (unexplained diff vs previous) (percentage) 162.43% 1.730 4.540 - -77.16% -
473.astar (unexplained diff vs previous) 138.46% 13.000 31.000 - -99.69% -
473.astar (unexplained diff vs previous) (percentage) 136.00% 0.250 0.590 - -99.70% -
435.gromacs (unexplained diff vs previous) (percentage) 135.48% 0.620 1.460 - -98.49% -
435.gromacs (unexplained diff vs previous) 131.11% 45.000 104.000 - -98.52% -
435.gromacs (max) (percentage) 111.76% 0.680 1.440 - -97.07% -
435.gromacs (max) 108.16% 49.000 102.000 - -97.15% -
429.mcf (unexplained diff vs previous) 100.00% 1.000 2.000 - -99.99% -
429.mcf (unexplained diff vs previous) (percentage) 100.00% 0.070 0.140 - -99.99% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - -99.99% -
462.libquantum (unexplained diff vs previous) (percentage) 96.00% 0.750 1.470 - -99.98% -
473.astar (max) 82.35% 17.000 31.000 - -99.09% -
473.astar (max) (percentage) 81.82% 0.330 0.600 - -99.10% -
436.cactusADM (unexplained diff vs previous) (percentage) 71.43% 0.140 0.240 - -99.97% -
436.cactusADM (unexplained diff vs previous) 66.67% 3.000 5.000 - -99.97% -
410.bwaves (max) 44.12% 34.000 49.000 - -98.50% -
410.bwaves (max) (percentage) 40.68% 0.590 0.830 - -98.57% -
410.bwaves (unexplained diff vs previous) 40.00% 35.000 49.000 - -99.37% -
410.bwaves (unexplained diff vs previous) (percentage) 38.98% 0.590 0.820 - -99.41% -
471.omnetpp (unexplained diff vs previous) 25.00% 48.000 60.000 - -99.44% -
471.omnetpp (unexplained diff vs previous) (percentage) 22.16% 1.760 2.150 - -99.45% -
470.lbm (max) 21.43% 14.000 17.000 - -99.38% -
470.lbm (max) (percentage) 15.38% 0.390 0.450 - -99.38% -
437.leslie3d 4.66% 536.000 561.000 - 58.03% -
437.leslie3d (avg) 4.66% 536.000 561.000 - 58.03% -
447.dealII 4.35% 23.000 24.000 - -75.76% -
447.dealII (avg) 4.35% 23.000 24.000 - -75.76% -
447.dealII (max) 3.57% 28.000 29.000 - -97.90% -
447.dealII (max) (percentage) 1.72% 1.740 1.770 - -97.80% -
471.omnetpp 1.56% 64.000 65.000 - -70.85% -
471.omnetpp (avg) 1.56% 64.000 65.000 - -70.85% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
400.perlbench -2.17% 1570.000 1536.000 - -7.64% -
435.gromacs -1.44% 7214.000 7110.000 - -2.19% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
434.zeusmp (unexplained diff vs previous) -76.92% 26.000 6.000 - -99.95% -
434.zeusmp (unexplained diff vs previous) (percentage) -76.47% 1.020 0.240 - -99.95% -
401.bzip2 (unexplained diff vs previous) (percentage) -67.48% 1.630 0.530 - -99.86% -
401.bzip2 (unexplained diff vs previous) -67.35% 49.000 16.000 - -99.85% -
416.gamess (unexplained diff vs previous) (percentage) -61.76% 0.680 0.260 - -99.59% -
416.gamess (unexplained diff vs previous) -61.45% 83.000 32.000 - -99.59% -
453.povray (max) (percentage) -60.22% 1.860 0.740 - -99.19% -
453.povray (max) -60.00% 10.000 4.000 - -99.20% -
464.h264ref (unexplained diff vs previous) (percentage) -56.25% 0.160 0.070 - -99.94% -
464.h264ref (unexplained diff vs previous) -55.56% 9.000 4.000 - -99.94% -
456.hmmer (unexplained diff vs previous) (percentage) -51.85% 0.270 0.130 - -99.95% -
450.soplex (unexplained diff vs previous) -50.00% 2.000 1.000 - -99.99% -
450.soplex (unexplained diff vs previous) (percentage) -50.00% 0.340 0.170 - -99.99% -
456.hmmer (unexplained diff vs previous) -50.00% 8.000 4.000 - -99.95% -
458.sjeng (unexplained diff vs previous) -50.00% 12.000 6.000 - -99.91% -
458.sjeng (unexplained diff vs previous) (percentage) -50.00% 0.160 0.080 - -99.91% -
462.libquantum (max) -33.33% 3.000 2.000 - -98.55% -
400.perlbench (unexplained diff vs previous) -27.66% 47.000 34.000 - -99.61% -
434.zeusmp (max) -26.92% 26.000 19.000 - -99.08% -
400.perlbench (unexplained diff vs previous) (percentage) -26.09% 2.990 2.210 - -99.58% -
434.zeusmp (max) (percentage) -25.00% 1.040 0.780 - -99.05% -
453.povray (unexplained diff vs previous) (percentage) -20.65% 0.920 0.730 - -99.93% -
453.povray (unexplained diff vs previous) -20.00% 5.000 4.000 - -99.93% -
456.hmmer (max) (percentage) -18.52% 0.270 0.220 - -99.70% -
456.hmmer (max) -14.29% 7.000 6.000 - -99.72% -
429.mcf (max) -11.11% 9.000 8.000 - -99.39% -
400.perlbench (max) -10.42% 48.000 43.000 - -96.92% -
458.sjeng (max) (percentage) -10.34% 0.290 0.260 - -99.45% -
433.milc (max) -10.00% 10.000 9.000 - -99.14% -
462.libquantum (max) (percentage) -10.00% 2.400 2.160 - -97.82% -
458.sjeng (max) -9.52% 21.000 19.000 - -99.47% -
400.perlbench (max) (percentage) -8.41% 3.090 2.830 - -96.63% -
464.h264ref (max) -8.00% 25.000 23.000 - -99.26% -
437.leslie3d (max) (percentage) -7.52% 5.320 4.920 - -70.33% -
416.gamess (max) (percentage) -7.35% 0.680 0.630 - -98.37% -
416.gamess (max) -7.32% 82.000 76.000 - -98.42% -
401.bzip2 (max) -7.14% 70.000 65.000 - -97.19% -
464.h264ref (max) (percentage) -6.82% 0.440 0.410 - -99.24% -
482.sphinx3 (max) (percentage) -6.03% 1.160 1.090 - -98.87% -
433.milc (max) (percentage) -5.62% 0.890 0.840 - -99.07% -
401.bzip2 (max) (percentage) -5.58% 2.330 2.200 - -97.20% -
471.omnetpp (max) (percentage) -5.37% 2.420 2.290 - -97.12% -
450.soplex (max) (percentage) -5.26% 0.570 0.540 - -99.44% -
465.tonto (max) (percentage) -4.94% 0.810 0.770 - -96.01% -
429.mcf (max) (percentage) -4.62% 0.650 0.620 - -99.33% -
444.namd (max) (percentage) -4.40% 0.910 0.870 - -99.05% -
459.GemsFDTD (max) (percentage) -4.35% 0.460 0.440 - -99.38% -
456.hmmer -4.35% 23.000 22.000 - -82.68% -
456.hmmer (avg) -4.35% 23.000 22.000 - -82.68% -
465.tonto (max) -4.30% 93.000 89.000 - -95.79% -
459.GemsFDTD (max) -4.17% 24.000 23.000 - -99.48% -
436.cactusADM (max) (percentage) -4.00% 0.250 0.240 - -99.73% -
437.leslie3d (max) -3.17% 599.000 580.000 - -69.93% -
416.gamess -3.09% 97.000 94.000 - -82.91% -
416.gamess (avg) -3.09% 97.000 94.000 - -82.91% -
471.omnetpp (max) -3.03% 66.000 64.000 - -97.05% -
435.gromacs -2.82% 142.000 138.000 - -62.40% -
435.gromacs (avg) -2.82% 142.000 138.000 - -62.40% -
416.gamess (avg) (percentage) -2.50% 0.800 0.780 - -82.35% -
464.h264ref (avg) (percentage) -1.90% 1.050 1.030 - -79.76% -
445.gobmk (avg) (percentage) -1.89% 1.060 1.040 - -73.40% -
459.GemsFDTD (avg) (percentage) -1.89% 1.590 1.560 - -73.29% -
481.wrf (max) (percentage) -1.85% 1.080 1.060 - -98.46% -
401.bzip2 -1.72% 58.000 57.000 - -64.60% -
401.bzip2 (avg) -1.72% 58.000 57.000 - -64.60% -
458.sjeng -1.72% 58.000 57.000 - -76.35% -
458.sjeng (avg) -1.72% 58.000 57.000 - -76.35% -
447.dealII (unexplained diff vs previous) (percentage) -1.69% 1.770 1.740 - -99.57% -
464.h264ref -1.67% 60.000 59.000 - -79.86% -
464.h264ref (avg) -1.67% 60.000 59.000 - -79.86% -
429.mcf (avg) (percentage) -1.61% 6.230 6.130 - -20.08% -
482.sphinx3 (avg) (percentage) -1.50% 2.000 1.970 - -75.16% -
434.zeusmp (avg) (percentage) -1.49% 1.340 1.320 - -77.44% -
450.soplex (avg) (percentage) -1.47% 2.730 2.690 - -58.16% -
403.gcc (max) (percentage) -1.45% 4.830 4.760 - -95.21% -
465.tonto (avg) (percentage) -1.35% 1.480 1.460 - -56.29% -
436.cactusADM (avg) (percentage) -1.28% 1.560 1.540 - -74.25% -
456.hmmer (avg) (percentage) -1.28% 0.780 0.770 - -82.54% -
458.sjeng (avg) (percentage) -1.28% 0.780 0.770 - -75.86% -
433.milc (avg) (percentage) -1.23% 1.630 1.610 - -73.39% -
459.GemsFDTD -1.20% 83.000 82.000 - -77.22% -
459.GemsFDTD (avg) -1.20% 83.000 82.000 - -77.22% -
429.mcf -1.16% 86.000 85.000 - -22.02% -
429.mcf (avg) -1.16% 86.000 85.000 - -22.02% -
453.povray (avg) (percentage) -1.14% 1.760 1.740 - -77.72% -
473.astar (avg) (percentage) -1.09% 1.830 1.810 - -56.49% -
401.bzip2 (avg) (percentage) -1.03% 1.940 1.920 - -64.96% -
454.calculix (max) (percentage) -1.02% 0.980 0.970 - -99.03% -
435.gromacs (avg) (percentage) -1.02% 1.970 1.950 - -61.46% -

Added Tests - sample variability
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 6.72% 10941.000 11676.000 - 0.62% -
410.bwaves 5.15% 5652.000 5943.000 - 0.83% -
444.namd 3.48% 747.000 773.000 - 0.65% -
450.soplex 3.00% 567.000 584.000 - 0.17% -
456.hmmer 2.88% 2882.000 2965.000 - 0.14% -
471.omnetpp 2.61% 2724.000 2795.000 - 2.19% -
401.bzip2 1.80% 2941.000 2994.000 - -0.53% -
473.astar 1.66% 5128.000 5213.000 - 0.60% -
436.cactusADM 1.39% 2083.000 2112.000 - 0.24% -
437.leslie3d 1.36% 11640.000 11798.000 - 4.76% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d 58.03% 355.000 561.000 - 4.66% -
437.leslie3d (avg) 58.03% 355.000 561.000 - 4.66% -
437.leslie3d (avg) (percentage) 56.07% 3.050 4.760 - 0.00% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -13.79% 29.000 25.000 - 0.00% -
nb_execution_successful -13.79% 29.000 25.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -2.65% 244728.000 238232.000 - 0.00% -
434.zeusmp -2.56% 152037.000 148143.000 - 0.00% -
410.bwaves -1.67% 16760.000 16480.000 - 0.00% -
444.namd -1.61% 188385.000 185348.000 - 0.00% -
464.h264ref -1.46% 387748.000 382096.000 - 0.00% -
470.lbm -1.44% 9691.000 9551.000 - 0.00% -
435.gromacs -1.39% 499261.000 492297.000 - 0.00% -
437.leslie3d -1.33% 59479.000 58690.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -14.71% 6173.000 5265.000 - 0.34% -
482.sphinx3 -12.17% 945.000 830.000 - 0.00% -
400.perlbench -7.64% 1663.000 1536.000 - -2.17% -
447.dealII -3.19% 1724.000 1669.000 - 1.77% -
462.libquantum -2.86% 140.000 136.000 - 1.49% -
416.gamess -2.37% 12465.000 12169.000 - 0.26% -
445.gobmk -2.21% 5023.000 4912.000 - 0.35% -
435.gromacs -2.19% 7269.000 7110.000 - -1.44% -
429.mcf -2.11% 1424.000 1394.000 - 0.14% -
458.sjeng -1.64% 7567.000 7443.000 - -0.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
450.soplex (unexplained diff vs previous) (percentage) -99.99% 2557.320 0.170 - -50.00% -
450.soplex (unexplained diff vs previous) -99.99% 14500.000 1.000 - -50.00% -
429.mcf (unexplained diff vs previous) (percentage) -99.99% 1305.690 0.140 - 100.00% -
429.mcf (unexplained diff vs previous) -99.99% 18593.000 2.000 - 100.00% -
462.libquantum (unexplained diff vs previous) -99.99% 13518.000 2.000 - 100.00% -
462.libquantum (unexplained diff vs previous) (percentage) -99.98% 9655.710 1.470 - 96.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.97% 787.130 0.240 - 71.43% -
436.cactusADM (unexplained diff vs previous) -99.97% 16396.000 5.000 - 66.67% -
433.milc (unexplained diff vs previous) (percentage) -99.96% 980.800 0.430 - 377.78% -
433.milc (unexplained diff vs previous) -99.96% 11289.000 5.000 - 400.00% -
456.hmmer (unexplained diff vs previous) (percentage) -99.95% 278.630 0.130 - -51.85% -
456.hmmer (unexplained diff vs previous) -99.95% 8030.000 4.000 - -50.00% -
434.zeusmp (unexplained diff vs previous) -99.95% 11280.000 6.000 - -76.92% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.95% 448.690 0.240 - -76.47% -
444.namd (unexplained diff vs previous) (percentage) -99.94% 1118.610 0.650 - 400.00% -
464.h264ref (unexplained diff vs previous) -99.94% 6843.000 4.000 - -55.56% -
464.h264ref (unexplained diff vs previous) (percentage) -99.94% 118.780 0.070 - -56.25% -
444.namd (unexplained diff vs previous) -99.94% 8356.000 5.000 - 400.00% -
453.povray (unexplained diff vs previous) (percentage) -99.93% 1043.330 0.730 - -20.65% -
453.povray (unexplained diff vs previous) -99.93% 5707.000 4.000 - -20.00% -
458.sjeng (unexplained diff vs previous) -99.91% 6876.000 6.000 - -50.00% -
458.sjeng (unexplained diff vs previous) (percentage) -99.91% 90.870 0.080 - -50.00% -
459.GemsFDTD (unexplained diff vs previous) -99.88% 15458.000 18.000 - 200.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.86% 250.410 0.340 - 209.09% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.86% 368.340 0.530 - -67.48% -
401.bzip2 (unexplained diff vs previous) -99.85% 10833.000 16.000 - -67.35% -
470.lbm (unexplained diff vs previous) -99.83% 9990.000 17.000 - 466.67% -
470.lbm (unexplained diff vs previous) (percentage) -99.83% 263.730 0.450 - 462.50% -
436.cactusADM (max) (percentage) -99.73% 88.730 0.240 - -4.00% -
436.cactusADM (max) -99.73% 1848.000 5.000 - 0.00% -
456.hmmer (max) -99.72% 2120.000 6.000 - -14.29% -
473.astar (unexplained diff vs previous) (percentage) -99.70% 197.850 0.590 - 136.00% -
456.hmmer (max) (percentage) -99.70% 73.590 0.220 - -18.52% -
445.gobmk (unexplained diff vs previous) -99.70% 5581.000 17.000 - 1600.00% -
473.astar (unexplained diff vs previous) -99.69% 10146.000 31.000 - 138.46% -
445.gobmk (unexplained diff vs previous) (percentage) -99.68% 111.110 0.350 - 1650.00% -
400.perlbench (unexplained diff vs previous) -99.61% 8772.000 34.000 - -27.66% -
416.gamess (unexplained diff vs previous) -99.59% 7819.000 32.000 - -61.45% -
447.dealII (unexplained diff vs previous) -99.59% 7017.000 29.000 - 0.00% -
416.gamess (unexplained diff vs previous) (percentage) -99.59% 62.730 0.260 - -61.76% -
400.perlbench (unexplained diff vs previous) (percentage) -99.58% 527.480 2.210 - -26.09% -
447.dealII (unexplained diff vs previous) (percentage) -99.57% 407.020 1.740 - -1.69% -
459.GemsFDTD (max) -99.48% 4411.000 23.000 - -4.17% -
458.sjeng (max) -99.47% 3602.000 19.000 - -9.52% -
458.sjeng (max) (percentage) -99.45% 47.610 0.260 - -10.34% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.45% 391.300 2.150 - 22.16% -
450.soplex (max) -99.45% 545.000 3.000 - 0.00% -
450.soplex (max) (percentage) -99.44% 96.240 0.540 - -5.26% -
471.omnetpp (unexplained diff vs previous) -99.44% 10659.000 60.000 - 25.00% -
410.bwaves (unexplained diff vs previous) (percentage) -99.41% 137.900 0.820 - 38.98% -
429.mcf (max) -99.39% 1322.000 8.000 - -11.11% -
459.GemsFDTD (max) (percentage) -99.38% 71.460 0.440 - -4.35% -
470.lbm (max) -99.38% 2746.000 17.000 - 21.43% -
470.lbm (max) (percentage) -99.38% 72.510 0.450 - 15.38% -
410.bwaves (unexplained diff vs previous) -99.37% 7794.000 49.000 - 40.00% -
429.mcf (max) (percentage) -99.33% 92.890 0.620 - -4.62% -
464.h264ref (max) -99.26% 3127.000 23.000 - -8.00% -
464.h264ref (max) (percentage) -99.24% 54.290 0.410 - -6.82% -
453.povray (max) -99.20% 499.000 4.000 - -60.00% -
453.povray (max) (percentage) -99.19% 91.250 0.740 - -60.22% -
433.milc (max) -99.14% 1044.000 9.000 - -10.00% -
444.namd (max) -99.12% 685.000 6.000 - 0.00% -
473.astar (max) (percentage) -99.10% 66.430 0.600 - 81.82% -
473.astar (max) -99.09% 3406.000 31.000 - 82.35% -
434.zeusmp (max) -99.08% 2055.000 19.000 - -26.92% -
433.milc (max) (percentage) -99.07% 90.750 0.840 - -5.62% -
444.namd (max) (percentage) -99.05% 91.790 0.870 - -4.40% -
434.zeusmp (max) (percentage) -99.05% 81.770 0.780 - -25.00% -
454.calculix (max) (percentage) -99.03% 99.730 0.970 - -1.02% -
482.sphinx3 (max) -99.01% 911.000 9.000 - 0.00% -
482.sphinx3 (max) (percentage) -98.87% 96.500 1.090 - -6.03% -
410.bwaves (max) (percentage) -98.57% 57.970 0.830 - 40.68% -
462.libquantum (max) -98.55% 138.000 2.000 - -33.33% -
435.gromacs (unexplained diff vs previous) -98.52% 7028.000 104.000 - 131.11% -
410.bwaves (max) -98.50% 3276.000 49.000 - 44.12% -
435.gromacs (unexplained diff vs previous) (percentage) -98.49% 96.680 1.460 - 135.48% -
445.gobmk (max) -98.49% 2643.000 40.000 - 0.00% -
481.wrf (max) (percentage) -98.46% 68.700 1.060 - -1.85% -
445.gobmk (max) (percentage) -98.44% 52.630 0.820 - 0.00% -
416.gamess (max) -98.42% 4805.000 76.000 - -7.32% -
416.gamess (max) (percentage) -98.37% 38.550 0.630 - -7.35% -
447.dealII (max) -97.90% 1384.000 29.000 - 3.57% -
462.libquantum (max) (percentage) -97.82% 98.970 2.160 - -10.00% -
447.dealII (max) (percentage) -97.80% 80.280 1.770 - 1.72% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
401.bzip2 (max) (percentage) -97.20% 78.650 2.200 - -5.58% -
401.bzip2 (max) -97.19% 2313.000 65.000 - -7.14% -
435.gromacs (max) -97.15% 3573.000 102.000 - 108.16% -
471.omnetpp (max) (percentage) -97.12% 79.650 2.290 - -5.37% -
435.gromacs (max) (percentage) -97.07% 49.160 1.440 - 111.76% -
471.omnetpp (max) -97.05% 2169.000 64.000 - -3.03% -
400.perlbench (max) -96.92% 1397.000 43.000 - -10.42% -
400.perlbench (max) (percentage) -96.63% 84.060 2.830 - -8.41% -
465.tonto (unexplained diff vs previous) (percentage) -96.17% 16.190 0.620 - 287.50% -
465.tonto (max) (percentage) -96.01% 19.310 0.770 - -4.94% -
465.tonto (unexplained diff vs previous) -95.93% 1771.000 72.000 - 278.95% -
465.tonto (max) -95.79% 2112.000 89.000 - -4.30% -
403.gcc (max) (percentage) -95.21% 99.360 4.760 - -1.45% -
416.gamess -82.91% 550.000 94.000 - -3.09% -
416.gamess (avg) -82.91% 550.000 94.000 - -3.09% -
456.hmmer -82.68% 127.000 22.000 - -4.35% -
456.hmmer (avg) -82.68% 127.000 22.000 - -4.35% -
456.hmmer (avg) (percentage) -82.54% 4.410 0.770 - -1.28% -
416.gamess (avg) (percentage) -82.35% 4.420 0.780 - -2.50% -
444.namd -82.35% 68.000 12.000 - 0.00% -
444.namd (avg) -82.35% 68.000 12.000 - 0.00% -
444.namd (avg) (percentage) -81.89% 9.220 1.670 - -0.60% -
470.lbm -80.75% 239.000 46.000 - 0.00% -
470.lbm (avg) -80.75% 239.000 46.000 - 0.00% -
470.lbm (avg) (percentage) -80.57% 6.330 1.230 - -0.81% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
464.h264ref -79.86% 293.000 59.000 - -1.67% -
464.h264ref (avg) -79.86% 293.000 59.000 - -1.67% -
464.h264ref (avg) (percentage) -79.76% 5.090 1.030 - -1.90% -
453.povray -78.57% 42.000 9.000 - 0.00% -
453.povray (avg) -78.57% 42.000 9.000 - 0.00% -
482.sphinx3 -78.38% 74.000 16.000 - 0.00% -
482.sphinx3 (avg) -78.38% 74.000 16.000 - 0.00% -
453.povray (avg) (percentage) -77.72% 7.810 1.740 - -1.14% -
434.zeusmp -77.55% 147.000 33.000 - 0.00% -
434.zeusmp (avg) -77.55% 147.000 33.000 - 0.00% -
434.zeusmp (avg) (percentage) -77.44% 5.850 1.320 - -1.49% -
459.GemsFDTD -77.22% 360.000 82.000 - -1.20% -
459.GemsFDTD (avg) -77.22% 360.000 82.000 - -1.20% -
437.leslie3d (unexplained diff vs previous) (percentage) -77.16% 19.880 4.540 - 162.43% -
437.leslie3d (unexplained diff vs previous) -76.84% 2314.000 536.000 - 174.87% -
462.libquantum (avg) (percentage) -76.37% 7.660 1.810 - 0.00% -
458.sjeng -76.35% 241.000 57.000 - -1.72% -
458.sjeng (avg) -76.35% 241.000 57.000 - -1.72% -
458.sjeng (avg) (percentage) -75.86% 3.190 0.770 - -1.28% -
447.dealII -75.76% 99.000 24.000 - 4.35% -
447.dealII (avg) -75.76% 99.000 24.000 - 4.35% -
482.sphinx3 (avg) (percentage) -75.16% 7.930 1.970 - -1.50% -
447.dealII (avg) (percentage) -74.52% 5.770 1.470 - 0.68% -
436.cactusADM (avg) (percentage) -74.25% 5.980 1.540 - -1.28% -
436.cactusADM -74.19% 124.000 32.000 - 0.00% -
436.cactusADM (avg) -74.19% 124.000 32.000 - 0.00% -
445.gobmk -73.98% 196.000 51.000 - 0.00% -
445.gobmk (avg) -73.98% 196.000 51.000 - 0.00% -
433.milc -73.91% 69.000 18.000 - 0.00% -
433.milc (avg) -73.91% 69.000 18.000 - 0.00% -
410.bwaves (avg) (percentage) -73.70% 4.410 1.160 - 0.00% -
445.gobmk (avg) (percentage) -73.40% 3.910 1.040 - -1.89% -
433.milc (avg) (percentage) -73.39% 6.050 1.610 - -1.23% -
459.GemsFDTD (avg) (percentage) -73.29% 5.840 1.560 - -1.89% -
410.bwaves -72.69% 249.000 68.000 - 0.00% -
410.bwaves (avg) -72.69% 249.000 68.000 - 0.00% -
471.omnetpp (avg) (percentage) -71.22% 8.200 2.360 - 0.00% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
471.omnetpp -70.85% 223.000 65.000 - 1.56% -
471.omnetpp (avg) -70.85% 223.000 65.000 - 1.56% -
437.leslie3d (max) (percentage) -70.33% 16.580 4.920 - -7.52% -
437.leslie3d (max) -69.93% 1929.000 580.000 - -3.17% -
400.perlbench -65.71% 105.000 36.000 - 0.00% -
400.perlbench (avg) -65.71% 105.000 36.000 - 0.00% -
401.bzip2 (avg) (percentage) -64.96% 5.480 1.920 - -1.03% -
401.bzip2 -64.60% 161.000 57.000 - -1.72% -
401.bzip2 (avg) -64.60% 161.000 57.000 - -1.72% -
400.perlbench (avg) (percentage) -62.99% 6.350 2.350 - 0.00% -
435.gromacs -62.40% 367.000 138.000 - -2.82% -
435.gromacs (avg) -62.40% 367.000 138.000 - -2.82% -
435.gromacs (avg) (percentage) -61.46% 5.060 1.950 - -1.02% -
450.soplex -58.33% 36.000 15.000 - 0.00% -
450.soplex (avg) -58.33% 36.000 15.000 - 0.00% -
450.soplex (avg) (percentage) -58.16% 6.430 2.690 - -1.47% -
473.astar (avg) (percentage) -56.49% 4.160 1.810 - -1.09% -
465.tonto (avg) (percentage) -56.29% 3.340 1.460 - -1.35% -
473.astar -55.87% 213.000 94.000 - 0.00% -
473.astar (avg) -55.87% 213.000 94.000 - 0.00% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
465.tonto -53.42% 365.000 170.000 - -0.58% -
465.tonto (avg) -53.42% 365.000 170.000 - -0.58% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf -22.02% 109.000 85.000 - -1.16% -
429.mcf (avg) -22.02% 109.000 85.000 - -1.16% -
429.mcf (avg) (percentage) -20.08% 7.670 6.130 - -1.61% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 13.79% 29.000 33.000 - 0.00% -

Added Tests - sample variability
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1536.000 -2.17%
401.bzip2 2994.000 -
410.bwaves 5943.000 -
416.gamess 12169.000 -
429.mcf 1394.000 -
433.milc 1153.000 -
434.zeusmp 2531.000 -
435.gromacs 7110.000 -1.44%
436.cactusADM 2112.000 -
437.leslie3d 11798.000 4.76%
444.namd 773.000 -
445.gobmk 4912.000 -
447.dealII 1669.000 1.77%
450.soplex 584.000 -
453.povray 546.000 -
456.hmmer 2965.000 -
458.sjeng 7443.000 -
459.GemsFDTD 5265.000 -
462.libquantum 136.000 1.49%
464.h264ref 5757.000 -
465.tonto 11676.000 -
470.lbm 3805.000 -
471.omnetpp 2795.000 2.19%
473.astar 5213.000 -
482.sphinx3 830.000 -
Geometric Mean 2599.983 -
Code Size (bytes)
Name Current %
400.perlbench 890587.000 -
401.bzip2 43919.000 -
410.bwaves 16480.000 -
416.gamess 5700921.000 -
429.mcf 9055.000 -
433.milc 60805.000 -
434.zeusmp 148143.000 -
435.gromacs 492297.000 -
436.cactusADM 350637.000 -
437.leslie3d 58690.000 -
444.namd 185348.000 -
445.gobmk 1383066.000 -
447.dealII 310444.000 -
450.soplex 205303.000 -
453.povray 656269.000 -
456.hmmer 108215.000 -
458.sjeng 103292.000 -
459.GemsFDTD 238232.000 -
462.libquantum 16085.000 -
464.h264ref 382096.000 -
465.tonto 1939926.000 -
470.lbm 9551.000 -
471.omnetpp 479794.000 -
473.astar 22880.000 -
482.sphinx3 112243.000 -
Geometric Mean 159736.912 -
sample variability
Name Current %
400.perlbench 36.000 -
400.perlbench (avg) 36.000 -
400.perlbench (avg) (percentage) 2.350 -
400.perlbench (max) 43.000 -10.42%
400.perlbench (max) (percentage) 2.830 -8.41%
400.perlbench (unexplained diff vs previous) 34.000 -27.66%
400.perlbench (unexplained diff vs previous) (percentage) 2.210 -26.09%
401.bzip2 57.000 -1.72%
401.bzip2 (avg) 57.000 -1.72%
401.bzip2 (avg) (percentage) 1.920 -1.03%
401.bzip2 (max) 65.000 -7.14%
401.bzip2 (max) (percentage) 2.200 -5.58%
401.bzip2 (unexplained diff vs previous) 16.000 -67.35%
401.bzip2 (unexplained diff vs previous) (percentage) 0.530 -67.48%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 4.760 -1.45%
410.bwaves 68.000 -
410.bwaves (avg) 68.000 -
410.bwaves (avg) (percentage) 1.160 -
410.bwaves (max) 49.000 44.12%
410.bwaves (max) (percentage) 0.830 40.68%
410.bwaves (unexplained diff vs previous) 49.000 40.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.820 38.98%
416.gamess 94.000 -3.09%
416.gamess (avg) 94.000 -3.09%
416.gamess (avg) (percentage) 0.780 -2.50%
416.gamess (max) 76.000 -7.32%
416.gamess (max) (percentage) 0.630 -7.35%
416.gamess (unexplained diff vs previous) 32.000 -61.45%
416.gamess (unexplained diff vs previous) (percentage) 0.260 -61.76%
429.mcf 85.000 -1.16%
429.mcf (avg) 85.000 -1.16%
429.mcf (avg) (percentage) 6.130 -1.61%
429.mcf (max) 8.000 -11.11%
429.mcf (max) (percentage) 0.620 -4.62%
429.mcf (unexplained diff vs previous) 2.000 100.00%
429.mcf (unexplained diff vs previous) (percentage) 0.140 100.00%
433.milc 18.000 -
433.milc (avg) 18.000 -
433.milc (avg) (percentage) 1.610 -1.23%
433.milc (max) 9.000 -10.00%
433.milc (max) (percentage) 0.840 -5.62%
433.milc (unexplained diff vs previous) 5.000 400.00%
433.milc (unexplained diff vs previous) (percentage) 0.430 377.78%
434.zeusmp 33.000 -
434.zeusmp (avg) 33.000 -
434.zeusmp (avg) (percentage) 1.320 -1.49%
434.zeusmp (max) 19.000 -26.92%
434.zeusmp (max) (percentage) 0.780 -25.00%
434.zeusmp (unexplained diff vs previous) 6.000 -76.92%
434.zeusmp (unexplained diff vs previous) (percentage) 0.240 -76.47%
435.gromacs 138.000 -2.82%
435.gromacs (avg) 138.000 -2.82%
435.gromacs (avg) (percentage) 1.950 -1.02%
435.gromacs (max) 102.000 108.16%
435.gromacs (max) (percentage) 1.440 111.76%
435.gromacs (unexplained diff vs previous) 104.000 131.11%
435.gromacs (unexplained diff vs previous) (percentage) 1.460 135.48%
436.cactusADM 32.000 -
436.cactusADM (avg) 32.000 -
436.cactusADM (avg) (percentage) 1.540 -1.28%
436.cactusADM (max) 5.000 -
436.cactusADM (max) (percentage) 0.240 -4.00%
436.cactusADM (unexplained diff vs previous) 5.000 66.67%
436.cactusADM (unexplained diff vs previous) (percentage) 0.240 71.43%
437.leslie3d 561.000 4.66%
437.leslie3d (avg) 561.000 4.66%
437.leslie3d (avg) (percentage) 4.760 -
437.leslie3d (max) 580.000 -3.17%
437.leslie3d (max) (percentage) 4.920 -7.52%
437.leslie3d (unexplained diff vs previous) 536.000 174.87%
437.leslie3d (unexplained diff vs previous) (percentage) 4.540 162.43%
444.namd 12.000 -
444.namd (avg) 12.000 -
444.namd (avg) (percentage) 1.670 -
444.namd (max) 6.000 -
444.namd (max) (percentage) 0.870 -4.40%
444.namd (unexplained diff vs previous) 5.000 400.00%
444.namd (unexplained diff vs previous) (percentage) 0.650 400.00%
445.gobmk 51.000 -
445.gobmk (avg) 51.000 -
445.gobmk (avg) (percentage) 1.040 -1.89%
445.gobmk (max) 40.000 -
445.gobmk (max) (percentage) 0.820 -
445.gobmk (unexplained diff vs previous) 17.000 1600.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.350 1650.00%
447.dealII 24.000 4.35%
447.dealII (avg) 24.000 4.35%
447.dealII (avg) (percentage) 1.470 -
447.dealII (max) 29.000 3.57%
447.dealII (max) (percentage) 1.770 1.72%
447.dealII (unexplained diff vs previous) 29.000 -
447.dealII (unexplained diff vs previous) (percentage) 1.740 -1.69%
450.soplex 15.000 -
450.soplex (avg) 15.000 -
450.soplex (avg) (percentage) 2.690 -1.47%
450.soplex (max) 3.000 -
450.soplex (max) (percentage) 0.540 -5.26%
450.soplex (unexplained diff vs previous) 1.000 -50.00%
450.soplex (unexplained diff vs previous) (percentage) 0.170 -50.00%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.740 -1.14%
453.povray (max) 4.000 -60.00%
453.povray (max) (percentage) 0.740 -60.22%
453.povray (unexplained diff vs previous) 4.000 -20.00%
453.povray (unexplained diff vs previous) (percentage) 0.730 -20.65%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 0.970 -1.02%
456.hmmer 22.000 -4.35%
456.hmmer (avg) 22.000 -4.35%
456.hmmer (avg) (percentage) 0.770 -1.28%
456.hmmer (max) 6.000 -14.29%
456.hmmer (max) (percentage) 0.220 -18.52%
456.hmmer (unexplained diff vs previous) 4.000 -50.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.130 -51.85%
458.sjeng 57.000 -1.72%
458.sjeng (avg) 57.000 -1.72%
458.sjeng (avg) (percentage) 0.770 -1.28%
458.sjeng (max) 19.000 -9.52%
458.sjeng (max) (percentage) 0.260 -10.34%
458.sjeng (unexplained diff vs previous) 6.000 -50.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.080 -50.00%
459.GemsFDTD 82.000 -1.20%
459.GemsFDTD (avg) 82.000 -1.20%
459.GemsFDTD (avg) (percentage) 1.560 -1.89%
459.GemsFDTD (max) 23.000 -4.17%
459.GemsFDTD (max) (percentage) 0.440 -4.35%
459.GemsFDTD (unexplained diff vs previous) 18.000 200.00%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.340 209.09%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.810 -
462.libquantum (max) 2.000 -33.33%
462.libquantum (max) (percentage) 2.160 -10.00%
462.libquantum (unexplained diff vs previous) 2.000 100.00%
462.libquantum (unexplained diff vs previous) (percentage) 1.470 96.00%
464.h264ref 59.000 -1.67%
464.h264ref (avg) 59.000 -1.67%
464.h264ref (avg) (percentage) 1.030 -1.90%
464.h264ref (max) 23.000 -8.00%
464.h264ref (max) (percentage) 0.410 -6.82%
464.h264ref (unexplained diff vs previous) 4.000 -55.56%
464.h264ref (unexplained diff vs previous) (percentage) 0.070 -56.25%
465.tonto 170.000 -
465.tonto (avg) 170.000 -
465.tonto (avg) (percentage) 1.460 -1.35%
465.tonto (max) 89.000 -4.30%
465.tonto (max) (percentage) 0.770 -4.94%
465.tonto (unexplained diff vs previous) 72.000 278.95%
465.tonto (unexplained diff vs previous) (percentage) 0.620 287.50%
470.lbm 46.000 -
470.lbm (avg) 46.000 -
470.lbm (avg) (percentage) 1.230 -
470.lbm (max) 17.000 21.43%
470.lbm (max) (percentage) 0.450 15.38%
470.lbm (unexplained diff vs previous) 17.000 466.67%
470.lbm (unexplained diff vs previous) (percentage) 0.450 462.50%
471.omnetpp 65.000 1.56%
471.omnetpp (avg) 65.000 1.56%
471.omnetpp (avg) (percentage) 2.360 -
471.omnetpp (max) 64.000 -3.03%
471.omnetpp (max) (percentage) 2.290 -5.37%
471.omnetpp (unexplained diff vs previous) 60.000 25.00%
471.omnetpp (unexplained diff vs previous) (percentage) 2.150 22.16%
473.astar 94.000 -
473.astar (avg) 94.000 -
473.astar (avg) (percentage) 1.810 -1.09%
473.astar (max) 31.000 82.35%
473.astar (max) (percentage) 0.600 81.82%
473.astar (unexplained diff vs previous) 31.000 138.46%
473.astar (unexplained diff vs previous) (percentage) 0.590 136.00%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 1.060 -1.85%
482.sphinx3 16.000 -
482.sphinx3 (avg) 16.000 -
482.sphinx3 (avg) (percentage) 1.970 -1.50%
482.sphinx3 (max) 9.000 -
482.sphinx3 (max) (percentage) 1.090 -6.03%
482.sphinx3 (unexplained diff vs previous) 0.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.000 -
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 5.267 -3.66%