Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=f5e259f32f011dc563b3da9e279acaee3aafba8e (unchanged)
git_gcc https://gcc.gnu.org/git/gcc.git (e30bc91e96f710cb1a15a32c4c53ddfb40f0f282) (+ 24 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=7f045c0b48633b198b42bebdff0024d7cfab3901 (unchanged)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/ecd929caff09448ce7a2830a27611e63ddc823e9 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/328/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0328 (basepoints/gcc-15-05350-ge30bc91e96f) 2024-11-18T10:18:12 0:00:00
Previous 0328 (3-changed-components) 2022-11-09T17:16:33 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 54 9
Performance Improvements 119 174
Added Tests 12 2
Unchanged Tests 1051 1051
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
465.tonto 1.39% 1948611.000 1975683.000 - 1.52% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 28.54% 8365.000 10752.000 - -1.73% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
473.astar (max) (percentage) 4633.33% 0.090 4.260 - -93.59% -
429.mcf (max) (percentage) 2475.00% 0.400 10.300 - -88.91% -
429.mcf (avg) (percentage) 1891.67% 0.240 4.780 - -37.68% -
473.astar (unexplained diff vs previous) (percentage) 1716.67% 0.060 1.090 - -99.45% -
473.astar (max) 1625.00% 12.000 207.000 - -93.92% -
403.gcc (avg) (percentage) 1541.94% 0.310 5.090 - -30.46% -
403.gcc (max) (percentage) 1153.33% 0.300 3.760 - -96.22% -
473.astar (avg) (percentage) 973.33% 0.150 1.610 - -61.30% -
473.astar (unexplained diff vs previous) 562.50% 8.000 53.000 - -99.48% -
462.libquantum (max) (percentage) 494.74% 0.380 2.260 - -97.72% -
462.libquantum (unexplained diff vs previous) (percentage) 481.58% 0.380 2.210 - -99.98% -
458.sjeng (unexplained diff vs previous) (percentage) 462.50% 0.080 0.450 - -99.50% -
462.libquantum (avg) (percentage) 330.23% 0.430 1.850 - -75.85% -
473.astar 271.43% 21.000 78.000 - -63.38% -
473.astar (avg) 271.43% 21.000 78.000 - -63.38% -
454.calculix (avg) (percentage) 227.27% 1.100 3.600 - -55.11% -
483.xalancbmk (avg) (percentage) 211.58% 0.950 2.960 - -54.46% -
458.sjeng (unexplained diff vs previous) 200.00% 11.000 33.000 - -99.52% -
437.leslie3d (max) (percentage) 192.77% 3.460 10.130 - -38.90% -
458.sjeng (avg) (percentage) 185.71% 0.210 0.600 - -81.19% -
429.mcf (max) 177.94% 68.000 189.000 - -85.70% -
401.bzip2 (avg) (percentage) 154.55% 0.550 1.400 - -74.45% -
450.soplex (avg) (percentage) 139.22% 1.020 2.440 - -62.05% -
437.leslie3d (avg) (percentage) 116.91% 2.070 4.490 - 47.21% -
429.mcf 114.63% 41.000 88.000 - -19.27% -
429.mcf (avg) 114.63% 41.000 88.000 - -19.27% -
437.leslie3d (max) 114.50% 531.000 1139.000 - -40.95% -
450.soplex (max) (percentage) 106.83% 1.610 3.330 - -96.54% -
464.h264ref (avg) (percentage) 102.86% 0.350 0.710 - -86.05% -
465.tonto (max) 102.84% 141.000 286.000 - -86.46% -
437.leslie3d 58.81% 318.000 505.000 - 42.25% -
437.leslie3d (avg) 58.81% 318.000 505.000 - 42.25% -
471.omnetpp (avg) (percentage) 57.58% 1.320 2.080 - -74.63% -
465.tonto (max) (percentage) 57.40% 1.690 2.660 - -86.22% -
458.sjeng 48.28% 29.000 43.000 - -82.16% -
458.sjeng (avg) 48.28% 29.000 43.000 - -82.16% -
456.hmmer (max) (percentage) 45.95% 0.370 0.540 - -99.27% -
445.gobmk (avg) (percentage) 45.00% 0.600 0.870 - -77.75% -
447.dealII (avg) (percentage) 43.75% 0.800 1.150 - -80.07% -
465.tonto 41.49% 94.000 133.000 - -63.56% -
465.tonto (avg) 41.49% 94.000 133.000 - -63.56% -
459.GemsFDTD (avg) (percentage) 29.00% 1.000 1.290 - -77.91% -
482.sphinx3 (max) (percentage) 27.06% 1.700 2.160 - -97.76% -
459.GemsFDTD (max) (percentage) 25.56% 1.800 2.260 - -96.84% -
458.sjeng (max) (percentage) 24.56% 0.570 0.710 - -98.51% -
435.gromacs (avg) (percentage) 20.41% 1.470 1.770 - -65.02% -
465.tonto (avg) (percentage) 9.73% 1.130 1.240 - -62.87% -
471.omnetpp (unexplained diff vs previous) (percentage) 6.14% 1.140 1.210 - -99.69% -
453.povray (avg) (percentage) 5.52% 1.450 1.530 - -80.41% -
456.hmmer (avg) (percentage) 4.00% 0.500 0.520 - -88.21% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -8.61% 59491.000 54370.000 - -8.59% -
434.zeusmp -4.44% 153713.000 146883.000 - -3.39% -
459.GemsFDTD -3.42% 245040.000 236656.000 - -3.30% -
435.gromacs -2.13% 500327.000 489684.000 - -1.92% -
462.libquantum -1.99% 16117.000 15797.000 - -1.79% -
447.dealII -1.75% 312518.000 307034.000 - -1.46% -
410.bwaves -1.70% 16748.000 16464.000 - -1.77% -
458.sjeng -1.63% 104004.000 102311.000 - -1.67% -
470.lbm -1.45% 9687.000 9547.000 - -1.49% -
444.namd -1.38% 188365.000 185768.000 - -1.39% -
450.soplex -1.22% 205068.000 202574.000 - -1.22% -
453.povray -1.21% 657755.000 649821.000 - -1.75% -
433.milc -1.21% 61222.000 60484.000 - -1.46% -
401.bzip2 -1.18% 44241.000 43719.000 - -1.16% -
482.sphinx3 -1.02% 112839.000 111690.000 - -1.04% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.82% 11490.000 136.000 - -2.86% -
482.sphinx3 -97.21% 28903.000 805.000 - -14.81% -
450.soplex -96.59% 16084.000 548.000 - -3.35% -
444.namd -92.14% 9747.000 766.000 - 2.54% -
453.povray -91.61% 6328.000 531.000 - -2.93% -
433.milc -91.52% 13221.000 1121.000 - -2.61% -
429.mcf -89.30% 17219.000 1842.000 - 29.35% -
434.zeusmp -83.36% 13724.000 2283.000 - -9.19% -
447.dealII -82.01% 9098.000 1637.000 - -5.05% -
401.bzip2 -80.90% 14944.000 2854.000 - -2.96% -
459.GemsFDTD -77.84% 23235.000 5150.000 - -16.57% -
471.omnetpp -75.23% 10700.000 2650.000 - -2.72% -
470.lbm -69.32% 12138.000 3724.000 - -1.69% -
473.astar -65.70% 14185.000 4865.000 - -5.13% -
445.gobmk -58.96% 11646.000 4779.000 - -4.86% -
410.bwaves -57.71% 13735.000 5809.000 - 2.78% -
435.gromacs -55.71% 15708.000 6957.000 - -4.29% -
458.sjeng -48.93% 14223.000 7264.000 - -4.00% -
437.leslie3d -26.80% 15367.000 11248.000 - -3.37% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (unexplained diff vs previous) -98.82% 255.000 3.000 - -99.98% -
444.namd -97.95% 391.000 8.000 - -88.24% -
444.namd (avg) -97.95% 391.000 8.000 - -88.24% -
434.zeusmp (unexplained diff vs previous) -97.55% 245.000 6.000 - -99.95% -
434.zeusmp (max) -97.49% 279.000 7.000 - -99.66% -
482.sphinx3 -97.47% 514.000 13.000 - -82.43% -
482.sphinx3 (avg) -97.47% 514.000 13.000 - -82.43% -
444.namd (max) -97.32% 710.000 19.000 - -97.23% -
453.povray (max) -96.88% 192.000 6.000 - -98.80% -
482.sphinx3 (max) -96.54% 491.000 17.000 - -98.13% -
462.libquantum -95.92% 49.000 2.000 - -80.00% -
462.libquantum (avg) -95.92% 49.000 2.000 - -80.00% -
410.bwaves (unexplained diff vs previous) -95.91% 758.000 31.000 - -99.60% -
433.milc -94.58% 240.000 13.000 - -81.16% -
433.milc (avg) -94.58% 240.000 13.000 - -81.16% -
459.GemsFDTD (unexplained diff vs previous) -93.88% 425.000 26.000 - -99.83% -
433.milc (max) -93.43% 335.000 22.000 - -97.89% -
462.libquantum (unexplained diff vs previous) -93.18% 44.000 3.000 - -99.98% -
450.soplex (max) -93.02% 258.000 18.000 - -96.70% -
462.libquantum (max) -93.02% 43.000 3.000 - -97.83% -
447.dealII (max) -93.01% 143.000 10.000 - -99.28% -
450.soplex -92.07% 164.000 13.000 - -63.89% -
450.soplex (avg) -92.07% 164.000 13.000 - -63.89% -
470.lbm (max) -91.89% 407.000 33.000 - -98.80% -
453.povray -91.21% 91.000 8.000 - -80.95% -
453.povray (avg) -91.21% 91.000 8.000 - -80.95% -
410.bwaves (unexplained diff vs previous) (percentage) -90.40% 5.520 0.530 - -99.62% -
410.bwaves (max) -89.97% 718.000 72.000 - -97.80% -
434.zeusmp -89.86% 207.000 21.000 - -85.71% -
434.zeusmp (avg) -89.86% 207.000 21.000 - -85.71% -
401.bzip2 (max) -88.79% 116.000 13.000 - -99.44% -
416.gamess (max) (percentage) -88.22% 3.140 0.370 - -99.04% -
470.lbm -87.18% 234.000 30.000 - -87.45% -
470.lbm (avg) -87.18% 234.000 30.000 - -87.45% -
434.zeusmp (unexplained diff vs previous) (percentage) -85.47% 1.790 0.260 - -99.94% -
471.omnetpp (max) -84.78% 230.000 35.000 - -98.39% -
410.bwaves -84.04% 307.000 49.000 - -80.32% -
410.bwaves (avg) -84.04% 307.000 49.000 - -80.32% -
434.zeusmp (max) (percentage) -83.82% 2.040 0.330 - -99.60% -
481.wrf (max) (percentage) -78.80% 3.820 0.810 - -98.82% -
410.bwaves (max) (percentage) -76.10% 5.230 1.250 - -97.84% -
454.calculix (max) (percentage) -75.67% 3.370 0.820 - -99.18% -
447.dealII -75.00% 72.000 18.000 - -81.82% -
447.dealII (avg) -75.00% 72.000 18.000 - -81.82% -
471.omnetpp (unexplained diff vs previous) -73.77% 122.000 32.000 - -99.70% -
470.lbm (max) (percentage) -73.51% 3.360 0.890 - -98.77% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -72.68% 1.830 0.500 - -99.80% -
459.GemsFDTD (max) -72.25% 418.000 116.000 - -97.37% -
459.GemsFDTD -71.55% 232.000 66.000 - -81.67% -
459.GemsFDTD (avg) -71.55% 232.000 66.000 - -81.67% -
444.namd (avg) (percentage) -71.14% 4.020 1.160 - -87.42% -
450.soplex (unexplained diff vs previous) (percentage) -65.41% 1.590 0.550 - -99.98% -
483.xalancbmk (max) (percentage) -65.09% 5.070 1.770 - -97.42% -
444.namd (max) (percentage) -64.75% 7.290 2.570 - -97.20% -
445.gobmk (max) -64.38% 146.000 52.000 - -98.03% -
416.gamess (avg) (percentage) -62.38% 2.020 0.760 - -82.81% -
410.bwaves (avg) (percentage) -61.61% 2.240 0.860 - -80.50% -
435.gromacs (max) -61.46% 384.000 148.000 - -95.86% -
471.omnetpp -60.99% 141.000 55.000 - -75.34% -
471.omnetpp (avg) -60.99% 141.000 55.000 - -75.34% -
436.cactusADM (max) (percentage) -60.90% 2.660 1.040 - -98.83% -
447.dealII (max) (percentage) -59.49% 1.580 0.640 - -99.20% -
453.povray (max) (percentage) -57.24% 3.040 1.300 - -98.58% -
470.lbm (avg) (percentage) -56.99% 1.930 0.830 - -86.89% -
401.bzip2 -52.44% 82.000 39.000 - -75.78% -
401.bzip2 (avg) -52.44% 82.000 39.000 - -75.78% -
435.gromacs -46.52% 230.000 123.000 - -66.49% -
435.gromacs (avg) -46.52% 230.000 123.000 - -66.49% -
445.gobmk -40.58% 69.000 41.000 - -79.08% -
445.gobmk (avg) -40.58% 69.000 41.000 - -79.08% -
401.bzip2 (max) (percentage) -39.74% 0.780 0.470 - -99.40% -
434.zeusmp (avg) (percentage) -37.75% 1.510 0.940 - -83.93% -
471.omnetpp (max) (percentage) -37.67% 2.150 1.340 - -98.32% -
458.sjeng (max) -37.04% 81.000 51.000 - -98.58% -
436.cactusADM (avg) (percentage) -36.36% 1.650 1.050 - -82.44% -
433.milc (avg) (percentage) -31.87% 1.820 1.240 - -79.50% -
433.milc (max) (percentage) -20.87% 2.540 2.010 - -97.79% -
464.h264ref (max) (percentage) -19.64% 0.560 0.450 - -99.17% -
481.wrf (avg) (percentage) -14.69% 2.110 1.800 - -71.15% -
400.perlbench (max) (percentage) -13.91% 3.450 2.970 - -96.47% -
435.gromacs (max) (percentage) -13.06% 2.450 2.130 - -95.67% -
445.gobmk (max) (percentage) -12.70% 1.260 1.100 - -97.91% -
400.perlbench (avg) (percentage) -5.83% 2.060 1.940 - -69.45% -
482.sphinx3 (avg) (percentage) -3.93% 1.780 1.710 - -78.44% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
401.bzip2 (unexplained diff vs previous)
401.bzip2 (unexplained diff vs previous) (percentage)
437.leslie3d (unexplained diff vs previous)
437.leslie3d (unexplained diff vs previous) (percentage)
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.52% 1946163.000 1975683.000 - 1.39% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 29.35% 1424.000 1842.000 - -89.30% -
410.bwaves 2.78% 5652.000 5809.000 - -57.71% -
444.namd 2.54% 747.000 766.000 - -92.14% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 47.21% 3.050 4.490 - 116.91% -
437.leslie3d 42.25% 355.000 505.000 - 58.81% -
437.leslie3d (avg) 42.25% 355.000 505.000 - 58.81% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -31.03% 29.000 20.000 - -31.03% -
nb_execution_successful -31.03% 29.000 20.000 - -31.03% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -8.59% 59479.000 54370.000 - -8.61% -
434.zeusmp -3.39% 152037.000 146883.000 - -4.44% -
459.GemsFDTD -3.30% 244728.000 236656.000 - -3.42% -
435.gromacs -1.92% 499261.000 489684.000 - -2.13% -
462.libquantum -1.79% 16085.000 15797.000 - -1.99% -
410.bwaves -1.77% 16760.000 16464.000 - -1.70% -
453.povray -1.75% 661389.000 649821.000 - -1.21% -
458.sjeng -1.67% 104048.000 102311.000 - -1.63% -
470.lbm -1.49% 9691.000 9547.000 - -1.45% -
433.milc -1.46% 61382.000 60484.000 - -1.21% -
447.dealII -1.46% 311591.000 307034.000 - -1.75% -
444.namd -1.39% 188385.000 185768.000 - -1.38% -
450.soplex -1.22% 205081.000 202574.000 - -1.22% -
401.bzip2 -1.16% 44233.000 43719.000 - -1.18% -
482.sphinx3 -1.04% 112859.000 111690.000 - -1.02% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -16.57% 6173.000 5150.000 - -77.84% -
482.sphinx3 -14.81% 945.000 805.000 - -97.21% -
434.zeusmp -9.19% 2514.000 2283.000 - -83.36% -
473.astar -5.13% 5128.000 4865.000 - -65.70% -
447.dealII -5.05% 1724.000 1637.000 - -82.01% -
445.gobmk -4.86% 5023.000 4779.000 - -58.96% -
435.gromacs -4.29% 7269.000 6957.000 - -55.71% -
458.sjeng -4.00% 7567.000 7264.000 - -48.93% -
437.leslie3d -3.37% 11640.000 11248.000 - -26.80% -
450.soplex -3.35% 567.000 548.000 - -96.59% -
401.bzip2 -2.96% 2941.000 2854.000 - -80.90% -
453.povray -2.93% 547.000 531.000 - -91.61% -
462.libquantum -2.86% 140.000 136.000 - -98.82% -
471.omnetpp -2.72% 2724.000 2650.000 - -75.23% -
433.milc -2.61% 1151.000 1121.000 - -91.52% -
465.tonto -1.73% 10941.000 10752.000 - 28.54% -
470.lbm -1.69% 3788.000 3724.000 - -69.32% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 (unexplained diff vs previous) -99.99% 10833.000 1.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.99% 368.340 0.040 - 0.00% -
450.soplex (unexplained diff vs previous) -99.98% 14500.000 3.000 - -98.82% -
450.soplex (unexplained diff vs previous) (percentage) -99.98% 2557.320 0.550 - -65.41% -
462.libquantum (unexplained diff vs previous) -99.98% 13518.000 3.000 - -93.18% -
462.libquantum (unexplained diff vs previous) (percentage) -99.98% 9655.710 2.210 - 481.58% -
434.zeusmp (unexplained diff vs previous) -99.95% 11280.000 6.000 - -97.55% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.94% 448.690 0.260 - -85.47% -
453.povray (unexplained diff vs previous) -99.88% 5707.000 7.000 - 0.00% -
453.povray (unexplained diff vs previous) (percentage) -99.87% 1043.330 1.320 - 0.00% -
470.lbm (unexplained diff vs previous) -99.86% 9990.000 14.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -99.86% 263.730 0.380 - 0.00% -
459.GemsFDTD (unexplained diff vs previous) -99.83% 15458.000 26.000 - -93.88% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.80% 250.410 0.500 - -72.68% -
471.omnetpp (unexplained diff vs previous) -99.70% 10659.000 32.000 - -73.77% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.69% 391.300 1.210 - 6.14% -
434.zeusmp (max) -99.66% 2055.000 7.000 - -97.49% -
410.bwaves (unexplained diff vs previous) (percentage) -99.62% 137.900 0.530 - -90.40% -
410.bwaves (unexplained diff vs previous) -99.60% 7794.000 31.000 - -95.91% -
434.zeusmp (max) (percentage) -99.60% 81.770 0.330 - -83.82% -
458.sjeng (unexplained diff vs previous) -99.52% 6876.000 33.000 - 200.00% -
458.sjeng (unexplained diff vs previous) (percentage) -99.50% 90.870 0.450 - 462.50% -
473.astar (unexplained diff vs previous) -99.48% 10146.000 53.000 - 562.50% -
473.astar (unexplained diff vs previous) (percentage) -99.45% 197.850 1.090 - 1716.67% -
401.bzip2 (max) -99.44% 2313.000 13.000 - -88.79% -
401.bzip2 (max) (percentage) -99.40% 78.650 0.470 - -39.74% -
447.dealII (max) -99.28% 1384.000 10.000 - -93.01% -
456.hmmer (max) (percentage) -99.27% 73.590 0.540 - 45.95% -
447.dealII (max) (percentage) -99.20% 80.280 0.640 - -59.49% -
454.calculix (max) (percentage) -99.18% 99.730 0.820 - -75.67% -
464.h264ref (max) (percentage) -99.17% 54.290 0.450 - -19.64% -
416.gamess (max) (percentage) -99.04% 38.550 0.370 - -88.22% -
436.cactusADM (max) (percentage) -98.83% 88.730 1.040 - -60.90% -
481.wrf (max) (percentage) -98.82% 68.700 0.810 - -78.80% -
470.lbm (max) -98.80% 2746.000 33.000 - -91.89% -
453.povray (max) -98.80% 499.000 6.000 - -96.88% -
470.lbm (max) (percentage) -98.77% 72.510 0.890 - -73.51% -
458.sjeng (max) -98.58% 3602.000 51.000 - -37.04% -
453.povray (max) (percentage) -98.58% 91.250 1.300 - -57.24% -
458.sjeng (max) (percentage) -98.51% 47.610 0.710 - 24.56% -
471.omnetpp (max) -98.39% 2169.000 35.000 - -84.78% -
471.omnetpp (max) (percentage) -98.32% 79.650 1.340 - -37.67% -
482.sphinx3 (max) -98.13% 911.000 17.000 - -96.54% -
445.gobmk (max) -98.03% 2643.000 52.000 - -64.38% -
445.gobmk (max) (percentage) -97.91% 52.630 1.100 - -12.70% -
433.milc (max) -97.89% 1044.000 22.000 - -93.43% -
410.bwaves (max) (percentage) -97.84% 57.970 1.250 - -76.10% -
462.libquantum (max) -97.83% 138.000 3.000 - -93.02% -
410.bwaves (max) -97.80% 3276.000 72.000 - -89.97% -
433.milc (max) (percentage) -97.79% 90.750 2.010 - -20.87% -
482.sphinx3 (max) (percentage) -97.76% 96.500 2.160 - 27.06% -
462.libquantum (max) (percentage) -97.72% 98.970 2.260 - 494.74% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - -65.09% -
459.GemsFDTD (max) -97.37% 4411.000 116.000 - -72.25% -
444.namd (max) -97.23% 685.000 19.000 - -97.32% -
444.namd (max) (percentage) -97.20% 91.790 2.570 - -64.75% -
459.GemsFDTD (max) (percentage) -96.84% 71.460 2.260 - 25.56% -
450.soplex (max) -96.70% 545.000 18.000 - -93.02% -
450.soplex (max) (percentage) -96.54% 96.240 3.330 - 106.83% -
400.perlbench (max) (percentage) -96.47% 84.060 2.970 - -13.91% -
403.gcc (max) (percentage) -96.22% 99.360 3.760 - 1153.33% -
435.gromacs (max) -95.86% 3573.000 148.000 - -61.46% -
435.gromacs (max) (percentage) -95.67% 49.160 2.130 - -13.06% -
473.astar (max) -93.92% 3406.000 207.000 - 1625.00% -
473.astar (max) (percentage) -93.59% 66.430 4.260 - 4633.33% -
465.tonto (unexplained diff vs previous) -89.72% 1771.000 182.000 - 0.00% -
465.tonto (unexplained diff vs previous) (percentage) -89.56% 16.190 1.690 - 0.00% -
429.mcf (max) (percentage) -88.91% 92.890 10.300 - 2475.00% -
437.leslie3d (unexplained diff vs previous) -88.85% 2314.000 258.000 - 0.00% -
437.leslie3d (unexplained diff vs previous) (percentage) -88.48% 19.880 2.290 - 0.00% -
444.namd -88.24% 68.000 8.000 - -97.95% -
444.namd (avg) -88.24% 68.000 8.000 - -97.95% -
456.hmmer (avg) (percentage) -88.21% 4.410 0.520 - 4.00% -
470.lbm -87.45% 239.000 30.000 - -87.18% -
470.lbm (avg) -87.45% 239.000 30.000 - -87.18% -
444.namd (avg) (percentage) -87.42% 9.220 1.160 - -71.14% -
470.lbm (avg) (percentage) -86.89% 6.330 0.830 - -56.99% -
465.tonto (max) -86.46% 2112.000 286.000 - 102.84% -
465.tonto (max) (percentage) -86.22% 19.310 2.660 - 57.40% -
464.h264ref (avg) (percentage) -86.05% 5.090 0.710 - 102.86% -
434.zeusmp -85.71% 147.000 21.000 - -89.86% -
434.zeusmp (avg) -85.71% 147.000 21.000 - -89.86% -
429.mcf (max) -85.70% 1322.000 189.000 - 177.94% -
434.zeusmp (avg) (percentage) -83.93% 5.850 0.940 - -37.75% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - -62.38% -
436.cactusADM (avg) (percentage) -82.44% 5.980 1.050 - -36.36% -
482.sphinx3 -82.43% 74.000 13.000 - -97.47% -
482.sphinx3 (avg) -82.43% 74.000 13.000 - -97.47% -
458.sjeng -82.16% 241.000 43.000 - 48.28% -
458.sjeng (avg) -82.16% 241.000 43.000 - 48.28% -
447.dealII -81.82% 99.000 18.000 - -75.00% -
447.dealII (avg) -81.82% 99.000 18.000 - -75.00% -
459.GemsFDTD -81.67% 360.000 66.000 - -71.55% -
459.GemsFDTD (avg) -81.67% 360.000 66.000 - -71.55% -
458.sjeng (avg) (percentage) -81.19% 3.190 0.600 - 185.71% -
433.milc -81.16% 69.000 13.000 - -94.58% -
433.milc (avg) -81.16% 69.000 13.000 - -94.58% -
453.povray -80.95% 42.000 8.000 - -91.21% -
453.povray (avg) -80.95% 42.000 8.000 - -91.21% -
410.bwaves (avg) (percentage) -80.50% 4.410 0.860 - -61.61% -
453.povray (avg) (percentage) -80.41% 7.810 1.530 - 5.52% -
410.bwaves -80.32% 249.000 49.000 - -84.04% -
410.bwaves (avg) -80.32% 249.000 49.000 - -84.04% -
447.dealII (avg) (percentage) -80.07% 5.770 1.150 - 43.75% -
462.libquantum -80.00% 10.000 2.000 - -95.92% -
462.libquantum (avg) -80.00% 10.000 2.000 - -95.92% -
433.milc (avg) (percentage) -79.50% 6.050 1.240 - -31.87% -
445.gobmk -79.08% 196.000 41.000 - -40.58% -
445.gobmk (avg) -79.08% 196.000 41.000 - -40.58% -
482.sphinx3 (avg) (percentage) -78.44% 7.930 1.710 - -3.93% -
459.GemsFDTD (avg) (percentage) -77.91% 5.840 1.290 - 29.00% -
445.gobmk (avg) (percentage) -77.75% 3.910 0.870 - 45.00% -
462.libquantum (avg) (percentage) -75.85% 7.660 1.850 - 330.23% -
401.bzip2 -75.78% 161.000 39.000 - -52.44% -
401.bzip2 (avg) -75.78% 161.000 39.000 - -52.44% -
471.omnetpp -75.34% 223.000 55.000 - -60.99% -
471.omnetpp (avg) -75.34% 223.000 55.000 - -60.99% -
471.omnetpp (avg) (percentage) -74.63% 8.200 2.080 - 57.58% -
401.bzip2 (avg) (percentage) -74.45% 5.480 1.400 - 154.55% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - -14.69% -
400.perlbench (avg) (percentage) -69.45% 6.350 1.940 - -5.83% -
435.gromacs -66.49% 367.000 123.000 - -46.52% -
435.gromacs (avg) -66.49% 367.000 123.000 - -46.52% -
435.gromacs (avg) (percentage) -65.02% 5.060 1.770 - 20.41% -
450.soplex -63.89% 36.000 13.000 - -92.07% -
450.soplex (avg) -63.89% 36.000 13.000 - -92.07% -
465.tonto -63.56% 365.000 133.000 - 41.49% -
465.tonto (avg) -63.56% 365.000 133.000 - 41.49% -
473.astar -63.38% 213.000 78.000 - 271.43% -
473.astar (avg) -63.38% 213.000 78.000 - 271.43% -
465.tonto (avg) (percentage) -62.87% 3.340 1.240 - 9.73% -
450.soplex (avg) (percentage) -62.05% 6.430 2.440 - 139.22% -
473.astar (avg) (percentage) -61.30% 4.160 1.610 - 973.33% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 227.27% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 211.58% -
437.leslie3d (max) -40.95% 1929.000 1139.000 - 114.50% -
437.leslie3d (max) (percentage) -38.90% 16.580 10.130 - 192.77% -
429.mcf (avg) (percentage) -37.68% 7.670 4.780 - 1891.67% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 1541.94% -
429.mcf -19.27% 109.000 88.000 - 114.63% -
429.mcf (avg) -19.27% 109.000 88.000 - 114.63% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 31.03% 29.000 38.000 - 31.03% -

Added Tests - sample variability
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)

Report Time: 0.39s
Execution (seconds)
Name Current %
401.bzip2 2854.000 -80.90%
410.bwaves 5809.000 -57.71%
429.mcf 1842.000 -89.30%
433.milc 1121.000 -91.52%
434.zeusmp 2283.000 -83.36%
435.gromacs 6957.000 -55.71%
437.leslie3d 11248.000 -26.80%
444.namd 766.000 -92.14%
445.gobmk 4779.000 -58.96%
447.dealII 1637.000 -82.01%
450.soplex 548.000 -96.59%
453.povray 531.000 -91.61%
458.sjeng 7264.000 -48.93%
459.GemsFDTD 5150.000 -77.84%
462.libquantum 136.000 -98.82%
465.tonto 10752.000 28.54%
470.lbm 3724.000 -69.32%
471.omnetpp 2650.000 -75.23%
473.astar 4865.000 -65.70%
482.sphinx3 805.000 -97.21%
Geometric Mean 2331.602 -82.39%
Code Size (bytes)
Name Current %
401.bzip2 43719.000 -1.18%
410.bwaves 16464.000 -1.70%
429.mcf 9146.000 -
433.milc 60484.000 -1.21%
434.zeusmp 146883.000 -4.44%
435.gromacs 489684.000 -2.13%
437.leslie3d 54370.000 -8.61%
444.namd 185768.000 -1.38%
445.gobmk 1379206.000 -
447.dealII 307034.000 -1.75%
450.soplex 202574.000 -1.22%
453.povray 649821.000 -1.21%
458.sjeng 102311.000 -1.63%
459.GemsFDTD 236656.000 -3.42%
462.libquantum 15797.000 -1.99%
465.tonto 1975683.000 1.39%
470.lbm 9547.000 -1.45%
471.omnetpp 475388.000 -
473.astar 22760.000 -
482.sphinx3 111690.000 -1.02%
Geometric Mean 114129.016 -1.74%
Score
Name Current %
nb_compile_successful 20.000 -31.03%
nb_execution_successful 20.000 -31.03%
nb_total_tests 38.000 31.03%
Geometric Mean 24.771 -14.58%
sample variability
Name Current %
400.perlbench (avg) (percentage) 1.940 -5.83%
400.perlbench (max) (percentage) 2.970 -13.91%
401.bzip2 39.000 -52.44%
401.bzip2 (avg) 39.000 -52.44%
401.bzip2 (avg) (percentage) 1.400 154.55%
401.bzip2 (max) 13.000 -88.79%
401.bzip2 (max) (percentage) 0.470 -39.74%
401.bzip2 (unexplained diff vs previous) 1.000 -
401.bzip2 (unexplained diff vs previous) (percentage) 0.040 -
403.gcc (avg) (percentage) 5.090 1541.94%
403.gcc (max) (percentage) 3.760 1153.33%
410.bwaves 49.000 -84.04%
410.bwaves (avg) 49.000 -84.04%
410.bwaves (avg) (percentage) 0.860 -61.61%
410.bwaves (max) 72.000 -89.97%
410.bwaves (max) (percentage) 1.250 -76.10%
410.bwaves (unexplained diff vs previous) 31.000 -95.91%
410.bwaves (unexplained diff vs previous) (percentage) 0.530 -90.40%
416.gamess (avg) (percentage) 0.760 -62.38%
416.gamess (max) (percentage) 0.370 -88.22%
429.mcf 88.000 114.63%
429.mcf (avg) 88.000 114.63%
429.mcf (avg) (percentage) 4.780 1891.67%
429.mcf (max) 189.000 177.94%
429.mcf (max) (percentage) 10.300 2475.00%
433.milc 13.000 -94.58%
433.milc (avg) 13.000 -94.58%
433.milc (avg) (percentage) 1.240 -31.87%
433.milc (max) 22.000 -93.43%
433.milc (max) (percentage) 2.010 -20.87%
434.zeusmp 21.000 -89.86%
434.zeusmp (avg) 21.000 -89.86%
434.zeusmp (avg) (percentage) 0.940 -37.75%
434.zeusmp (max) 7.000 -97.49%
434.zeusmp (max) (percentage) 0.330 -83.82%
434.zeusmp (unexplained diff vs previous) 6.000 -97.55%
434.zeusmp (unexplained diff vs previous) (percentage) 0.260 -85.47%
435.gromacs 123.000 -46.52%
435.gromacs (avg) 123.000 -46.52%
435.gromacs (avg) (percentage) 1.770 20.41%
435.gromacs (max) 148.000 -61.46%
435.gromacs (max) (percentage) 2.130 -13.06%
436.cactusADM (avg) (percentage) 1.050 -36.36%
436.cactusADM (max) (percentage) 1.040 -60.90%
437.leslie3d 505.000 58.81%
437.leslie3d (avg) 505.000 58.81%
437.leslie3d (avg) (percentage) 4.490 116.91%
437.leslie3d (max) 1139.000 114.50%
437.leslie3d (max) (percentage) 10.130 192.77%
437.leslie3d (unexplained diff vs previous) 258.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 2.290 -
444.namd 8.000 -97.95%
444.namd (avg) 8.000 -97.95%
444.namd (avg) (percentage) 1.160 -71.14%
444.namd (max) 19.000 -97.32%
444.namd (max) (percentage) 2.570 -64.75%
444.namd (unexplained diff vs previous) 0.000 -
444.namd (unexplained diff vs previous) (percentage) 0.000 -
445.gobmk 41.000 -40.58%
445.gobmk (avg) 41.000 -40.58%
445.gobmk (avg) (percentage) 0.870 45.00%
445.gobmk (max) 52.000 -64.38%
445.gobmk (max) (percentage) 1.100 -12.70%
447.dealII 18.000 -75.00%
447.dealII (avg) 18.000 -75.00%
447.dealII (avg) (percentage) 1.150 43.75%
447.dealII (max) 10.000 -93.01%
447.dealII (max) (percentage) 0.640 -59.49%
450.soplex 13.000 -92.07%
450.soplex (avg) 13.000 -92.07%
450.soplex (avg) (percentage) 2.440 139.22%
450.soplex (max) 18.000 -93.02%
450.soplex (max) (percentage) 3.330 106.83%
450.soplex (unexplained diff vs previous) 3.000 -98.82%
450.soplex (unexplained diff vs previous) (percentage) 0.550 -65.41%
453.povray 8.000 -91.21%
453.povray (avg) 8.000 -91.21%
453.povray (avg) (percentage) 1.530 5.52%
453.povray (max) 6.000 -96.88%
453.povray (max) (percentage) 1.300 -57.24%
453.povray (unexplained diff vs previous) 7.000 -
453.povray (unexplained diff vs previous) (percentage) 1.320 -
454.calculix (avg) (percentage) 3.600 227.27%
454.calculix (max) (percentage) 0.820 -75.67%
456.hmmer (avg) (percentage) 0.520 4.00%
456.hmmer (max) (percentage) 0.540 45.95%
458.sjeng 43.000 48.28%
458.sjeng (avg) 43.000 48.28%
458.sjeng (avg) (percentage) 0.600 185.71%
458.sjeng (max) 51.000 -37.04%
458.sjeng (max) (percentage) 0.710 24.56%
458.sjeng (unexplained diff vs previous) 33.000 200.00%
458.sjeng (unexplained diff vs previous) (percentage) 0.450 462.50%
459.GemsFDTD 66.000 -71.55%
459.GemsFDTD (avg) 66.000 -71.55%
459.GemsFDTD (avg) (percentage) 1.290 29.00%
459.GemsFDTD (max) 116.000 -72.25%
459.GemsFDTD (max) (percentage) 2.260 25.56%
459.GemsFDTD (unexplained diff vs previous) 26.000 -93.88%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.500 -72.68%
462.libquantum 2.000 -95.92%
462.libquantum (avg) 2.000 -95.92%
462.libquantum (avg) (percentage) 1.850 330.23%
462.libquantum (max) 3.000 -93.02%
462.libquantum (max) (percentage) 2.260 494.74%
462.libquantum (unexplained diff vs previous) 3.000 -93.18%
462.libquantum (unexplained diff vs previous) (percentage) 2.210 481.58%
464.h264ref (avg) (percentage) 0.710 102.86%
464.h264ref (max) (percentage) 0.450 -19.64%
465.tonto 133.000 41.49%
465.tonto (avg) 133.000 41.49%
465.tonto (avg) (percentage) 1.240 9.73%
465.tonto (max) 286.000 102.84%
465.tonto (max) (percentage) 2.660 57.40%
465.tonto (unexplained diff vs previous) 182.000 -
465.tonto (unexplained diff vs previous) (percentage) 1.690 -
470.lbm 30.000 -87.18%
470.lbm (avg) 30.000 -87.18%
470.lbm (avg) (percentage) 0.830 -56.99%
470.lbm (max) 33.000 -91.89%
470.lbm (max) (percentage) 0.890 -73.51%
470.lbm (unexplained diff vs previous) 14.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.380 -
471.omnetpp 55.000 -60.99%
471.omnetpp (avg) 55.000 -60.99%
471.omnetpp (avg) (percentage) 2.080 57.58%
471.omnetpp (max) 35.000 -84.78%
471.omnetpp (max) (percentage) 1.340 -37.67%
471.omnetpp (unexplained diff vs previous) 32.000 -73.77%
471.omnetpp (unexplained diff vs previous) (percentage) 1.210 6.14%
473.astar 78.000 271.43%
473.astar (avg) 78.000 271.43%
473.astar (avg) (percentage) 1.610 973.33%
473.astar (max) 207.000 1625.00%
473.astar (max) (percentage) 4.260 4633.33%
473.astar (unexplained diff vs previous) 53.000 562.50%
473.astar (unexplained diff vs previous) (percentage) 1.090 1716.67%
481.wrf (avg) (percentage) 1.800 -14.69%
481.wrf (max) (percentage) 0.810 -78.80%
482.sphinx3 13.000 -97.47%
482.sphinx3 (avg) 13.000 -97.47%
482.sphinx3 (avg) (percentage) 1.710 -3.93%
482.sphinx3 (max) 17.000 -96.54%
482.sphinx3 (max) (percentage) 2.160 27.06%
483.xalancbmk (avg) (percentage) 2.960 211.58%
483.xalancbmk (max) (percentage) 1.770 -65.09%
Geometric Mean 5.519 -58.15%