Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=1001055e3552760015661dd9b081435051fe1340 (+ 30 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/bcccc3221b838ee7ae7848e7194603acb18294b3 (+ 132 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=0e8738a48c239ce6d010bba9ed7c38bd86307f32 (+ 10 commits)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/ddf92a2ebaa11a494946f6f153a23d36b7801d5b (+ 15835 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/303/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0303 (4-changed-components) 2024-10-06T13:10:41 0:00:00
Previous 0303 (-00000-) 2022-09-16T02:43:54 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 47 13
Performance Improvements 153 189
Added Tests 10 0
Unchanged Tests 1026 1034
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
465.tonto 1.61% 1945391.000 1976623.000 - 1.57% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 35.81% 8219.000 11162.000 - 2.02% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) (percentage) 12250.00% 0.020 2.470 - -99.81% -
473.astar (max) (percentage) 2136.84% 0.190 4.250 - -93.60% -
429.mcf (unexplained diff vs previous) 1600.00% 3.000 51.000 - -99.73% -
403.gcc (avg) (percentage) 1442.42% 0.330 5.090 - -30.46% -
429.mcf (max) (percentage) 1406.25% 0.160 2.410 - -97.41% -
483.xalancbmk (avg) (percentage) 1309.52% 0.210 2.960 - -54.46% -
403.gcc (max) (percentage) 1173.33% 0.300 3.820 - -96.16% -
429.mcf (avg) (percentage) 1013.64% 0.440 4.900 - -36.11% -
473.astar (max) 742.31% 26.000 219.000 - -93.57% -
454.calculix (avg) (percentage) 718.18% 0.440 3.600 - -55.11% -
483.xalancbmk (max) (percentage) 453.12% 0.320 1.770 - -97.42% -
473.astar (avg) (percentage) 403.12% 0.320 1.610 - -61.30% -
401.bzip2 (avg) (percentage) 325.71% 0.350 1.490 - -72.81% -
458.sjeng (avg) (percentage) 210.00% 0.200 0.620 - -80.56% -
437.leslie3d (max) (percentage) 188.42% 4.060 11.710 - -29.37% -
401.bzip2 (unexplained diff vs previous) (percentage) 185.71% 0.070 0.200 - -99.95% -
450.soplex (avg) (percentage) 115.38% 1.170 2.520 - -60.81% -
437.leslie3d (max) 115.10% 596.000 1282.000 - -33.54% -
462.libquantum (avg) (percentage) 111.63% 0.860 1.820 - -76.24% -
462.libquantum (max) (percentage) 106.08% 1.480 3.050 - -96.92% -
473.astar 84.44% 45.000 83.000 - -61.03% -
473.astar (avg) 84.44% 45.000 83.000 - -61.03% -
464.h264ref (avg) (percentage) 82.93% 0.410 0.750 - -85.27% -
429.mcf (max) 81.48% 27.000 49.000 - -96.29% -
465.tonto 73.08% 78.000 135.000 - -63.01% -
465.tonto (avg) 73.08% 78.000 135.000 - -63.01% -
458.sjeng 66.67% 27.000 45.000 - -81.33% -
458.sjeng (avg) 66.67% 27.000 45.000 - -81.33% -
437.leslie3d (avg) (percentage) 59.79% 2.810 4.490 - 47.21% -
454.calculix (max) (percentage) 59.62% 0.520 0.830 - -99.17% -
400.perlbench (avg) (percentage) 41.13% 1.410 1.990 - -68.66% -
447.dealII (avg) (percentage) 38.64% 0.880 1.220 - -78.86% -
429.mcf 32.89% 76.000 101.000 - -7.34% -
429.mcf (avg) 32.89% 76.000 101.000 - -7.34% -
465.tonto (avg) (percentage) 26.04% 0.960 1.210 - -63.77% -
401.bzip2 (max) (percentage) 21.21% 0.330 0.400 - -99.49% -
437.leslie3d 19.17% 412.000 491.000 - 38.31% -
437.leslie3d (avg) 19.17% 412.000 491.000 - 38.31% -
400.perlbench (max) (percentage) 18.75% 1.920 2.280 - -97.29% -
437.leslie3d (unexplained diff vs previous) (percentage) 11.08% 3.610 4.010 - -79.83% -
453.povray (max) (percentage) 10.61% 1.980 2.190 - -97.60% -
445.gobmk (avg) (percentage) 8.54% 0.820 0.890 - -77.24% -
459.GemsFDTD (avg) (percentage) 3.08% 1.300 1.340 - -77.05% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful -14.29% 28.000 24.000 - -17.24% -
nb_execution_successful -14.29% 28.000 24.000 - -17.24% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -7.77% 59403.000 54786.000 - -7.89% -
459.GemsFDTD -7.71% 252076.000 232648.000 - -4.94% -
434.zeusmp -4.70% 153897.000 146667.000 - -3.53% -
435.gromacs -2.88% 504291.000 489749.000 - -1.91% -
450.soplex -2.13% 206988.000 202574.000 - -1.22% -
464.h264ref -1.98% 387188.000 379532.000 - -2.12% -
453.povray -1.83% 661943.000 649825.000 - -1.75% -
410.bwaves -1.81% 16768.000 16464.000 - -1.77% -
470.lbm -1.69% 9711.000 9547.000 - -1.49% -
458.sjeng -1.67% 104024.000 102284.000 - -1.70% -
447.dealII -1.61% 312932.000 307896.000 - -1.19% -
471.omnetpp -1.47% 482480.000 475388.000 - -0.55% -
444.namd -1.35% 188301.000 185768.000 - -1.39% -
456.hmmer -1.33% 108631.000 107191.000 - -1.38% -
436.cactusADM -1.23% 352679.000 348331.000 - -1.29% -
433.milc -1.18% 61230.000 60509.000 - -1.42% -
401.bzip2 -1.14% 44221.000 43719.000 - -1.16% -
482.sphinx3 -1.04% 112831.000 111659.000 - -1.06% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.80% 11399.000 137.000 - -2.14% -
482.sphinx3 -97.10% 28867.000 838.000 - -11.32% -
450.soplex -96.33% 16067.000 589.000 - 3.88% -
444.namd -91.99% 9726.000 779.000 - 4.28% -
453.povray -91.42% 6366.000 546.000 - -0.18% -
433.milc -91.36% 13367.000 1155.000 - 0.35% -
436.cactusADM -88.41% 18199.000 2110.000 - 1.30% -
429.mcf -88.03% 17275.000 2068.000 - 45.22% -
400.perlbench -86.18% 11162.000 1543.000 - -7.22% -
447.dealII -81.88% 8980.000 1627.000 - -5.63% -
434.zeusmp -81.83% 13807.000 2509.000 - -0.20% -
401.bzip2 -80.02% 14913.000 2979.000 - 1.29% -
459.GemsFDTD -76.60% 22953.000 5371.000 - -12.99% -
456.hmmer -73.96% 11039.000 2875.000 - -0.24% -
471.omnetpp -72.78% 10030.000 2730.000 - 0.22% -
470.lbm -69.24% 12297.000 3783.000 - -0.13% -
473.astar -63.60% 14175.000 5160.000 - 0.62% -
464.h264ref -57.88% 13775.000 5802.000 - 0.71% -
445.gobmk -57.68% 11716.000 4958.000 - -1.29% -
410.bwaves -57.62% 14032.000 5947.000 - 5.22% -
435.gromacs -54.98% 15724.000 7079.000 - -2.61% -
458.sjeng -47.26% 13982.000 7374.000 - -2.55% -
437.leslie3d -25.39% 14682.000 10954.000 - -5.89% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (unexplained diff vs previous) -99.78% 465.000 1.000 - -99.99% -
482.sphinx3 (unexplained diff vs previous) -99.55% 1117.000 5.000 - -99.98% -
482.sphinx3 (max) -99.44% 1076.000 6.000 - -99.34% -
444.namd (max) -99.35% 768.000 5.000 - -99.27% -
444.namd (unexplained diff vs previous) -99.30% 712.000 5.000 - -99.94% -
462.libquantum (unexplained diff vs previous) -98.83% 171.000 2.000 - -99.99% -
450.soplex (max) -98.75% 639.000 8.000 - -98.53% -
444.namd -98.40% 562.000 9.000 - -86.76% -
444.namd (avg) -98.40% 562.000 9.000 - -86.76% -
482.sphinx3 -98.13% 750.000 14.000 - -81.08% -
482.sphinx3 (avg) -98.13% 750.000 14.000 - -81.08% -
462.libquantum -97.96% 98.000 2.000 - -80.00% -
462.libquantum (avg) -97.96% 98.000 2.000 - -80.00% -
433.milc (max) -97.71% 481.000 11.000 - -98.95% -
436.cactusADM (unexplained diff vs previous) -97.62% 504.000 12.000 - -99.93% -
462.libquantum (max) -97.62% 168.000 4.000 - -97.10% -
433.milc (unexplained diff vs previous) (percentage) -97.41% 3.480 0.090 - -99.99% -
436.cactusADM (max) -96.72% 518.000 17.000 - -99.08% -
470.lbm (unexplained diff vs previous) -96.46% 508.000 18.000 - -99.82% -
410.bwaves (unexplained diff vs previous) -96.06% 254.000 10.000 - -99.87% -
470.lbm (max) -95.90% 488.000 20.000 - -99.27% -
434.zeusmp (max) -95.89% 316.000 13.000 - -99.37% -
433.milc -95.87% 339.000 14.000 - -79.71% -
433.milc (avg) -95.87% 339.000 14.000 - -79.71% -
434.zeusmp (unexplained diff vs previous) -95.47% 309.000 14.000 - -99.88% -
456.hmmer (max) -94.48% 145.000 8.000 - -99.62% -
436.cactusADM -94.28% 402.000 23.000 - -81.45% -
436.cactusADM (avg) -94.28% 402.000 23.000 - -81.45% -
471.omnetpp (unexplained diff vs previous) -93.66% 678.000 43.000 - -99.60% -
453.povray -93.16% 117.000 8.000 - -80.95% -
453.povray (avg) -93.16% 117.000 8.000 - -80.95% -
450.soplex -92.51% 187.000 14.000 - -61.11% -
450.soplex (avg) -92.51% 187.000 14.000 - -61.11% -
444.namd (max) (percentage) -91.77% 7.900 0.650 - -99.29% -
453.povray (max) -91.27% 126.000 11.000 - -97.80% -
444.namd (unexplained diff vs previous) (percentage) -91.26% 7.320 0.640 - -99.94% -
470.lbm -91.25% 377.000 33.000 - -86.19% -
470.lbm (avg) -91.25% 377.000 33.000 - -86.19% -
447.dealII (max) -91.06% 246.000 22.000 - -98.41% -
410.bwaves (unexplained diff vs previous) (percentage) -90.61% 1.810 0.170 - -99.88% -
434.zeusmp -90.55% 254.000 24.000 - -83.67% -
434.zeusmp (avg) -90.55% 254.000 24.000 - -83.67% -
470.lbm (unexplained diff vs previous) (percentage) -88.38% 4.130 0.480 - -99.82% -
459.GemsFDTD (max) -88.11% 328.000 39.000 - -99.12% -
410.bwaves (max) -87.95% 249.000 30.000 - -99.08% -
416.gamess (max) (percentage) -87.83% 3.040 0.370 - -99.04% -
471.omnetpp (max) -87.22% 634.000 81.000 - -96.27% -
470.lbm (max) (percentage) -86.65% 3.970 0.530 - -99.27% -
482.sphinx3 (unexplained diff vs previous) (percentage) -84.50% 3.870 0.600 - -99.98% -
400.perlbench (max) -83.64% 214.000 35.000 - -97.49% -
456.hmmer -81.48% 81.000 15.000 - -88.19% -
456.hmmer (avg) -81.48% 81.000 15.000 - -88.19% -
464.h264ref (max) -80.95% 42.000 8.000 - -99.74% -
400.perlbench -80.89% 157.000 30.000 - -71.43% -
400.perlbench (avg) -80.89% 157.000 30.000 - -71.43% -
445.gobmk (max) -80.65% 124.000 24.000 - -99.09% -
482.sphinx3 (max) (percentage) -79.62% 3.730 0.760 - -99.21% -
436.cactusADM (unexplained diff vs previous) (percentage) -79.42% 2.770 0.570 - -99.93% -
444.namd (avg) (percentage) -78.89% 5.780 1.220 - -86.77% -
456.hmmer (max) (percentage) -78.03% 1.320 0.290 - -99.61% -
401.bzip2 (max) -77.55% 49.000 11.000 - -99.52% -
458.sjeng (max) -76.67% 30.000 7.000 - -99.81% -
471.omnetpp (unexplained diff vs previous) (percentage) -76.63% 6.760 1.580 - -99.60% -
459.GemsFDTD -76.17% 298.000 71.000 - -80.28% -
459.GemsFDTD (avg) -76.17% 298.000 71.000 - -80.28% -
434.zeusmp (max) (percentage) -75.98% 2.290 0.550 - -99.33% -
447.dealII -75.95% 79.000 19.000 - -80.81% -
447.dealII (avg) -75.95% 79.000 19.000 - -80.81% -
434.zeusmp (unexplained diff vs previous) (percentage) -75.00% 2.240 0.560 - -99.88% -
471.omnetpp -74.12% 228.000 59.000 - -73.54% -
471.omnetpp (avg) -74.12% 228.000 59.000 - -73.54% -
481.wrf (max) (percentage) -72.33% 3.000 0.830 - -98.79% -
435.gromacs (max) -71.82% 369.000 104.000 - -97.09% -
433.milc (max) (percentage) -71.67% 3.600 1.020 - -98.88% -
470.lbm (avg) (percentage) -71.34% 3.070 0.880 - -86.10% -
436.cactusADM (max) (percentage) -70.88% 2.850 0.830 - -99.06% -
410.bwaves (max) (percentage) -70.79% 1.780 0.520 - -99.10% -
465.tonto (max) (percentage) -70.37% 1.620 0.480 - -97.51% -
410.bwaves -65.16% 155.000 54.000 - -78.31% -
410.bwaves (avg) -65.16% 155.000 54.000 - -78.31% -
450.soplex (max) (percentage) -63.82% 3.980 1.440 - -98.50% -
416.gamess (avg) (percentage) -61.81% 1.990 0.760 - -82.81% -
465.tonto (max) -60.15% 133.000 53.000 - -97.49% -
435.gromacs -57.44% 289.000 123.000 - -66.49% -
435.gromacs (avg) -57.44% 289.000 123.000 - -66.49% -
458.sjeng (max) (percentage) -54.55% 0.220 0.100 - -99.79% -
445.gobmk -54.17% 96.000 44.000 - -77.55% -
445.gobmk (avg) -54.17% 96.000 44.000 - -77.55% -
445.gobmk (max) (percentage) -53.77% 1.060 0.490 - -99.07% -
471.omnetpp (max) (percentage) -52.61% 6.330 3.000 - -96.23% -
464.h264ref (max) (percentage) -51.61% 0.310 0.150 - -99.72% -
433.milc (avg) (percentage) -51.57% 2.540 1.230 - -79.67% -
447.dealII (max) (percentage) -49.27% 2.740 1.390 - -98.27% -
459.GemsFDTD (max) (percentage) -48.95% 1.430 0.730 - -98.98% -
436.cactusADM (avg) (percentage) -48.87% 2.210 1.130 - -81.10% -
434.zeusmp (avg) (percentage) -47.28% 1.840 0.970 - -83.42% -
401.bzip2 (unexplained diff vs previous) -40.00% 10.000 6.000 - -99.94% -
435.gromacs (max) (percentage) -37.45% 2.350 1.470 - -97.01% -
482.sphinx3 (avg) (percentage) -31.54% 2.600 1.780 - -77.55% -
456.hmmer (avg) (percentage) -27.03% 0.740 0.540 - -87.76% -
481.wrf (avg) (percentage) -24.69% 2.390 1.800 - -71.15% -
464.h264ref -23.21% 56.000 43.000 - -85.32% -
464.h264ref (avg) -23.21% 56.000 43.000 - -85.32% -
410.bwaves (avg) (percentage) -18.02% 1.110 0.910 - -79.37% -
437.leslie3d (unexplained diff vs previous) -17.17% 530.000 439.000 - -81.03% -
453.povray (avg) (percentage) -16.22% 1.850 1.550 - -80.15% -
401.bzip2 -15.38% 52.000 44.000 - -72.67% -
401.bzip2 (avg) -15.38% 52.000 44.000 - -72.67% -
435.gromacs (avg) (percentage) -5.43% 1.840 1.740 - -65.61% -
471.omnetpp (avg) (percentage) -3.95% 2.280 2.190 - -73.29% -
462.libquantum (unexplained diff vs previous) (percentage) -2.67% 1.500 1.460 - -99.98% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 17.24% 29.000 34.000 - 17.24% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
473.astar (unexplained diff vs previous)
473.astar (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.57% 1946163.000 1976623.000 - 1.61% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
429.mcf 45.22% 1424.000 2068.000 - -88.03% -
410.bwaves 5.22% 5652.000 5947.000 - -57.62% -
444.namd 4.28% 747.000 779.000 - -91.99% -
450.soplex 3.88% 567.000 589.000 - -96.33% -
465.tonto 2.02% 10941.000 11162.000 - 35.81% -
436.cactusADM 1.30% 2083.000 2110.000 - -88.41% -
401.bzip2 1.29% 2941.000 2979.000 - -80.02% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 47.21% 3.050 4.490 - 59.79% -
437.leslie3d 38.31% 355.000 491.000 - 19.17% -
437.leslie3d (avg) 38.31% 355.000 491.000 - 19.17% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - -14.29% -
nb_execution_successful -17.24% 29.000 24.000 - -14.29% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d -7.89% 59479.000 54786.000 - -7.77% -
459.GemsFDTD -4.94% 244728.000 232648.000 - -7.71% -
434.zeusmp -3.53% 152037.000 146667.000 - -4.70% -
464.h264ref -2.12% 387748.000 379532.000 - -1.98% -
435.gromacs -1.91% 499261.000 489749.000 - -2.88% -
410.bwaves -1.77% 16760.000 16464.000 - -1.81% -
453.povray -1.75% 661389.000 649825.000 - -1.83% -
458.sjeng -1.70% 104048.000 102284.000 - -1.67% -
470.lbm -1.49% 9691.000 9547.000 - -1.69% -
433.milc -1.42% 61382.000 60509.000 - -1.18% -
444.namd -1.39% 188385.000 185768.000 - -1.35% -
456.hmmer -1.38% 108691.000 107191.000 - -1.33% -
436.cactusADM -1.29% 352891.000 348331.000 - -1.23% -
450.soplex -1.22% 205081.000 202574.000 - -2.13% -
462.libquantum -1.19% 16085.000 15893.000 - -0.95% -
447.dealII -1.19% 311591.000 307896.000 - -1.61% -
401.bzip2 -1.16% 44233.000 43719.000 - -1.14% -
482.sphinx3 -1.06% 112859.000 111659.000 - -1.04% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -12.99% 6173.000 5371.000 - -76.60% -
482.sphinx3 -11.32% 945.000 838.000 - -97.10% -
400.perlbench -7.22% 1663.000 1543.000 - -86.18% -
437.leslie3d -5.89% 11640.000 10954.000 - -25.39% -
447.dealII -5.63% 1724.000 1627.000 - -81.88% -
435.gromacs -2.61% 7269.000 7079.000 - -54.98% -
458.sjeng -2.55% 7567.000 7374.000 - -47.26% -
462.libquantum -2.14% 140.000 137.000 - -98.80% -
445.gobmk -1.29% 5023.000 4958.000 - -57.68% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
433.milc (unexplained diff vs previous) -99.99% 11289.000 1.000 - -99.78% -
433.milc (unexplained diff vs previous) (percentage) -99.99% 980.800 0.090 - -97.41% -
462.libquantum (unexplained diff vs previous) -99.99% 13518.000 2.000 - -98.83% -
462.libquantum (unexplained diff vs previous) (percentage) -99.98% 9655.710 1.460 - -2.67% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 5.000 - -99.55% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.98% 2759.050 0.600 - -84.50% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.95% 368.340 0.200 - 185.71% -
401.bzip2 (unexplained diff vs previous) -99.94% 10833.000 6.000 - -40.00% -
444.namd (unexplained diff vs previous) (percentage) -99.94% 1118.610 0.640 - -91.26% -
444.namd (unexplained diff vs previous) -99.94% 8356.000 5.000 - -99.30% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.93% 787.130 0.570 - -79.42% -
436.cactusADM (unexplained diff vs previous) -99.93% 16396.000 12.000 - -97.62% -
400.perlbench (unexplained diff vs previous) -99.92% 8772.000 7.000 - 0.00% -
400.perlbench (unexplained diff vs previous) (percentage) -99.91% 527.480 0.450 - 0.00% -
453.povray (unexplained diff vs previous) -99.88% 5707.000 7.000 - 0.00% -
453.povray (unexplained diff vs previous) (percentage) -99.88% 1043.330 1.280 - 0.00% -
410.bwaves (unexplained diff vs previous) (percentage) -99.88% 137.900 0.170 - -90.61% -
434.zeusmp (unexplained diff vs previous) -99.88% 11280.000 14.000 - -95.47% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.88% 448.690 0.560 - -75.00% -
410.bwaves (unexplained diff vs previous) -99.87% 7794.000 10.000 - -96.06% -
470.lbm (unexplained diff vs previous) -99.82% 9990.000 18.000 - -96.46% -
470.lbm (unexplained diff vs previous) (percentage) -99.82% 263.730 0.480 - -88.38% -
429.mcf (unexplained diff vs previous) (percentage) -99.81% 1305.690 2.470 - 12250.00% -
458.sjeng (max) -99.81% 3602.000 7.000 - -76.67% -
458.sjeng (max) (percentage) -99.79% 47.610 0.100 - -54.55% -
464.h264ref (max) -99.74% 3127.000 8.000 - -80.95% -
429.mcf (unexplained diff vs previous) -99.73% 18593.000 51.000 - 1600.00% -
464.h264ref (max) (percentage) -99.72% 54.290 0.150 - -51.61% -
447.dealII (unexplained diff vs previous) -99.67% 7017.000 23.000 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) -99.65% 407.020 1.410 - 0.00% -
456.hmmer (max) -99.62% 2120.000 8.000 - -94.48% -
456.hmmer (max) (percentage) -99.61% 73.590 0.290 - -78.03% -
471.omnetpp (unexplained diff vs previous) -99.60% 10659.000 43.000 - -93.66% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.60% 391.300 1.580 - -76.63% -
473.astar (unexplained diff vs previous) (percentage) -99.56% 197.850 0.870 - 0.00% -
473.astar (unexplained diff vs previous) -99.56% 10146.000 45.000 - 0.00% -
401.bzip2 (max) -99.52% 2313.000 11.000 - -77.55% -
401.bzip2 (max) (percentage) -99.49% 78.650 0.400 - 21.21% -
435.gromacs (unexplained diff vs previous) -99.47% 7028.000 37.000 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) -99.46% 96.680 0.520 - 0.00% -
434.zeusmp (max) -99.37% 2055.000 13.000 - -95.89% -
482.sphinx3 (max) -99.34% 911.000 6.000 - -99.44% -
434.zeusmp (max) (percentage) -99.33% 81.770 0.550 - -75.98% -
444.namd (max) (percentage) -99.29% 91.790 0.650 - -91.77% -
470.lbm (max) -99.27% 2746.000 20.000 - -95.90% -
444.namd (max) -99.27% 685.000 5.000 - -99.35% -
470.lbm (max) (percentage) -99.27% 72.510 0.530 - -86.65% -
482.sphinx3 (max) (percentage) -99.21% 96.500 0.760 - -79.62% -
454.calculix (max) (percentage) -99.17% 99.730 0.830 - 59.62% -
459.GemsFDTD (max) -99.12% 4411.000 39.000 - -88.11% -
410.bwaves (max) (percentage) -99.10% 57.970 0.520 - -70.79% -
445.gobmk (max) -99.09% 2643.000 24.000 - -80.65% -
410.bwaves (max) -99.08% 3276.000 30.000 - -87.95% -
436.cactusADM (max) -99.08% 1848.000 17.000 - -96.72% -
445.gobmk (max) (percentage) -99.07% 52.630 0.490 - -53.77% -
436.cactusADM (max) (percentage) -99.06% 88.730 0.830 - -70.88% -
416.gamess (max) (percentage) -99.04% 38.550 0.370 - -87.83% -
459.GemsFDTD (max) (percentage) -98.98% 71.460 0.730 - -48.95% -
433.milc (max) -98.95% 1044.000 11.000 - -97.71% -
433.milc (max) (percentage) -98.88% 90.750 1.020 - -71.67% -
481.wrf (max) (percentage) -98.79% 68.700 0.830 - -72.33% -
450.soplex (max) -98.53% 545.000 8.000 - -98.75% -
450.soplex (max) (percentage) -98.50% 96.240 1.440 - -63.82% -
447.dealII (max) -98.41% 1384.000 22.000 - -91.06% -
447.dealII (max) (percentage) -98.27% 80.280 1.390 - -49.27% -
453.povray (max) -97.80% 499.000 11.000 - -91.27% -
453.povray (max) (percentage) -97.60% 91.250 2.190 - 10.61% -
465.tonto (max) (percentage) -97.51% 19.310 0.480 - -70.37% -
400.perlbench (max) -97.49% 1397.000 35.000 - -83.64% -
465.tonto (max) -97.49% 2112.000 53.000 - -60.15% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 453.12% -
429.mcf (max) (percentage) -97.41% 92.890 2.410 - 1406.25% -
400.perlbench (max) (percentage) -97.29% 84.060 2.280 - 18.75% -
462.libquantum (max) -97.10% 138.000 4.000 - -97.62% -
435.gromacs (max) -97.09% 3573.000 104.000 - -71.82% -
435.gromacs (max) (percentage) -97.01% 49.160 1.470 - -37.45% -
462.libquantum (max) (percentage) -96.92% 98.970 3.050 - 106.08% -
429.mcf (max) -96.29% 1322.000 49.000 - 81.48% -
471.omnetpp (max) -96.27% 2169.000 81.000 - -87.22% -
471.omnetpp (max) (percentage) -96.23% 79.650 3.000 - -52.61% -
403.gcc (max) (percentage) -96.16% 99.360 3.820 - 1173.33% -
473.astar (max) (percentage) -93.60% 66.430 4.250 - 2136.84% -
473.astar (max) -93.57% 3406.000 219.000 - 742.31% -
456.hmmer -88.19% 127.000 15.000 - -81.48% -
456.hmmer (avg) -88.19% 127.000 15.000 - -81.48% -
456.hmmer (avg) (percentage) -87.76% 4.410 0.540 - -27.03% -
444.namd (avg) (percentage) -86.77% 9.220 1.220 - -78.89% -
444.namd -86.76% 68.000 9.000 - -98.40% -
444.namd (avg) -86.76% 68.000 9.000 - -98.40% -
470.lbm -86.19% 239.000 33.000 - -91.25% -
470.lbm (avg) -86.19% 239.000 33.000 - -91.25% -
470.lbm (avg) (percentage) -86.10% 6.330 0.880 - -71.34% -
464.h264ref -85.32% 293.000 43.000 - -23.21% -
464.h264ref (avg) -85.32% 293.000 43.000 - -23.21% -
464.h264ref (avg) (percentage) -85.27% 5.090 0.750 - 82.93% -
434.zeusmp -83.67% 147.000 24.000 - -90.55% -
434.zeusmp (avg) -83.67% 147.000 24.000 - -90.55% -
434.zeusmp (avg) (percentage) -83.42% 5.850 0.970 - -47.28% -
416.gamess (avg) (percentage) -82.81% 4.420 0.760 - -61.81% -
436.cactusADM -81.45% 124.000 23.000 - -94.28% -
436.cactusADM (avg) -81.45% 124.000 23.000 - -94.28% -
458.sjeng -81.33% 241.000 45.000 - 66.67% -
458.sjeng (avg) -81.33% 241.000 45.000 - 66.67% -
436.cactusADM (avg) (percentage) -81.10% 5.980 1.130 - -48.87% -
482.sphinx3 -81.08% 74.000 14.000 - -98.13% -
482.sphinx3 (avg) -81.08% 74.000 14.000 - -98.13% -
437.leslie3d (unexplained diff vs previous) -81.03% 2314.000 439.000 - -17.17% -
453.povray -80.95% 42.000 8.000 - -93.16% -
453.povray (avg) -80.95% 42.000 8.000 - -93.16% -
447.dealII -80.81% 99.000 19.000 - -75.95% -
447.dealII (avg) -80.81% 99.000 19.000 - -75.95% -
458.sjeng (avg) (percentage) -80.56% 3.190 0.620 - 210.00% -
459.GemsFDTD -80.28% 360.000 71.000 - -76.17% -
459.GemsFDTD (avg) -80.28% 360.000 71.000 - -76.17% -
453.povray (avg) (percentage) -80.15% 7.810 1.550 - -16.22% -
462.libquantum -80.00% 10.000 2.000 - -97.96% -
462.libquantum (avg) -80.00% 10.000 2.000 - -97.96% -
437.leslie3d (unexplained diff vs previous) (percentage) -79.83% 19.880 4.010 - 11.08% -
433.milc -79.71% 69.000 14.000 - -95.87% -
433.milc (avg) -79.71% 69.000 14.000 - -95.87% -
433.milc (avg) (percentage) -79.67% 6.050 1.230 - -51.57% -
410.bwaves (avg) (percentage) -79.37% 4.410 0.910 - -18.02% -
447.dealII (avg) (percentage) -78.86% 5.770 1.220 - 38.64% -
410.bwaves -78.31% 249.000 54.000 - -65.16% -
410.bwaves (avg) -78.31% 249.000 54.000 - -65.16% -
482.sphinx3 (avg) (percentage) -77.55% 7.930 1.780 - -31.54% -
445.gobmk -77.55% 196.000 44.000 - -54.17% -
445.gobmk (avg) -77.55% 196.000 44.000 - -54.17% -
445.gobmk (avg) (percentage) -77.24% 3.910 0.890 - 8.54% -
459.GemsFDTD (avg) (percentage) -77.05% 5.840 1.340 - 3.08% -
462.libquantum (avg) (percentage) -76.24% 7.660 1.820 - 111.63% -
471.omnetpp -73.54% 223.000 59.000 - -74.12% -
471.omnetpp (avg) -73.54% 223.000 59.000 - -74.12% -
471.omnetpp (avg) (percentage) -73.29% 8.200 2.190 - -3.95% -
401.bzip2 (avg) (percentage) -72.81% 5.480 1.490 - 325.71% -
401.bzip2 -72.67% 161.000 44.000 - -15.38% -
401.bzip2 (avg) -72.67% 161.000 44.000 - -15.38% -
400.perlbench -71.43% 105.000 30.000 - -80.89% -
400.perlbench (avg) -71.43% 105.000 30.000 - -80.89% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - -24.69% -
400.perlbench (avg) (percentage) -68.66% 6.350 1.990 - 41.13% -
435.gromacs -66.49% 367.000 123.000 - -57.44% -
435.gromacs (avg) -66.49% 367.000 123.000 - -57.44% -
435.gromacs (avg) (percentage) -65.61% 5.060 1.740 - -5.43% -
465.tonto (avg) (percentage) -63.77% 3.340 1.210 - 26.04% -
465.tonto -63.01% 365.000 135.000 - 73.08% -
465.tonto (avg) -63.01% 365.000 135.000 - 73.08% -
473.astar (avg) (percentage) -61.30% 4.160 1.610 - 403.12% -
450.soplex -61.11% 36.000 14.000 - -92.51% -
450.soplex (avg) -61.11% 36.000 14.000 - -92.51% -
473.astar -61.03% 213.000 83.000 - 84.44% -
473.astar (avg) -61.03% 213.000 83.000 - 84.44% -
450.soplex (avg) (percentage) -60.81% 6.430 2.520 - 115.38% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 718.18% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 1309.52% -
429.mcf (avg) (percentage) -36.11% 7.670 4.900 - 1013.64% -
437.leslie3d (max) -33.54% 1929.000 1282.000 - 115.10% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 1442.42% -
437.leslie3d (max) (percentage) -29.37% 16.580 11.710 - 188.42% -
429.mcf -7.34% 109.000 101.000 - 32.89% -
429.mcf (avg) -7.34% 109.000 101.000 - 32.89% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 17.24% -

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1543.000 -86.18%
401.bzip2 2979.000 -80.02%
410.bwaves 5947.000 -57.62%
429.mcf 2068.000 -88.03%
433.milc 1155.000 -91.36%
434.zeusmp 2509.000 -81.83%
435.gromacs 7079.000 -54.98%
436.cactusADM 2110.000 -88.41%
437.leslie3d 10954.000 -25.39%
444.namd 779.000 -91.99%
445.gobmk 4958.000 -57.68%
447.dealII 1627.000 -81.88%
450.soplex 589.000 -96.33%
453.povray 546.000 -91.42%
456.hmmer 2875.000 -73.96%
458.sjeng 7374.000 -47.26%
459.GemsFDTD 5371.000 -76.60%
462.libquantum 137.000 -98.80%
464.h264ref 5802.000 -57.88%
465.tonto 11162.000 35.81%
470.lbm 3783.000 -69.24%
471.omnetpp 2730.000 -72.78%
473.astar 5160.000 -63.60%
482.sphinx3 838.000 -97.10%
Geometric Mean 2461.327 -81.33%
Code Size (bytes)
Name Current %
400.perlbench 895883.000 -
401.bzip2 43719.000 -1.14%
410.bwaves 16464.000 -1.81%
429.mcf 9119.000 -
433.milc 60509.000 -1.18%
434.zeusmp 146667.000 -4.70%
435.gromacs 489749.000 -2.88%
436.cactusADM 348331.000 -1.23%
437.leslie3d 54786.000 -7.77%
444.namd 185768.000 -1.35%
445.gobmk 1375078.000 -
447.dealII 307896.000 -1.61%
450.soplex 202574.000 -2.13%
453.povray 649825.000 -1.83%
456.hmmer 107191.000 -1.33%
458.sjeng 102284.000 -1.67%
459.GemsFDTD 232648.000 -7.71%
462.libquantum 15893.000 -
464.h264ref 379532.000 -1.98%
465.tonto 1976623.000 1.61%
470.lbm 9547.000 -1.69%
471.omnetpp 475388.000 -1.47%
473.astar 22760.000 -
482.sphinx3 111659.000 -1.04%
Geometric Mean 136567.004 -1.88%
Score
Name Current %
nb_compile_successful 24.000 -14.29%
nb_execution_successful 24.000 -14.29%
nb_total_tests 34.000 17.24%
Geometric Mean 26.955 -4.85%
sample variability
Name Current %
400.perlbench 30.000 -80.89%
400.perlbench (avg) 30.000 -80.89%
400.perlbench (avg) (percentage) 1.990 41.13%
400.perlbench (max) 35.000 -83.64%
400.perlbench (max) (percentage) 2.280 18.75%
400.perlbench (unexplained diff vs previous) 7.000 -
400.perlbench (unexplained diff vs previous) (percentage) 0.450 -
401.bzip2 44.000 -15.38%
401.bzip2 (avg) 44.000 -15.38%
401.bzip2 (avg) (percentage) 1.490 325.71%
401.bzip2 (max) 11.000 -77.55%
401.bzip2 (max) (percentage) 0.400 21.21%
401.bzip2 (unexplained diff vs previous) 6.000 -40.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.200 185.71%
403.gcc (avg) (percentage) 5.090 1442.42%
403.gcc (max) (percentage) 3.820 1173.33%
410.bwaves 54.000 -65.16%
410.bwaves (avg) 54.000 -65.16%
410.bwaves (avg) (percentage) 0.910 -18.02%
410.bwaves (max) 30.000 -87.95%
410.bwaves (max) (percentage) 0.520 -70.79%
410.bwaves (unexplained diff vs previous) 10.000 -96.06%
410.bwaves (unexplained diff vs previous) (percentage) 0.170 -90.61%
416.gamess (avg) (percentage) 0.760 -61.81%
416.gamess (max) (percentage) 0.370 -87.83%
429.mcf 101.000 32.89%
429.mcf (avg) 101.000 32.89%
429.mcf (avg) (percentage) 4.900 1013.64%
429.mcf (max) 49.000 81.48%
429.mcf (max) (percentage) 2.410 1406.25%
429.mcf (unexplained diff vs previous) 51.000 1600.00%
429.mcf (unexplained diff vs previous) (percentage) 2.470 12250.00%
433.milc 14.000 -95.87%
433.milc (avg) 14.000 -95.87%
433.milc (avg) (percentage) 1.230 -51.57%
433.milc (max) 11.000 -97.71%
433.milc (max) (percentage) 1.020 -71.67%
433.milc (unexplained diff vs previous) 1.000 -99.78%
433.milc (unexplained diff vs previous) (percentage) 0.090 -97.41%
434.zeusmp 24.000 -90.55%
434.zeusmp (avg) 24.000 -90.55%
434.zeusmp (avg) (percentage) 0.970 -47.28%
434.zeusmp (max) 13.000 -95.89%
434.zeusmp (max) (percentage) 0.550 -75.98%
434.zeusmp (unexplained diff vs previous) 14.000 -95.47%
434.zeusmp (unexplained diff vs previous) (percentage) 0.560 -75.00%
435.gromacs 123.000 -57.44%
435.gromacs (avg) 123.000 -57.44%
435.gromacs (avg) (percentage) 1.740 -5.43%
435.gromacs (max) 104.000 -71.82%
435.gromacs (max) (percentage) 1.470 -37.45%
435.gromacs (unexplained diff vs previous) 37.000 -
435.gromacs (unexplained diff vs previous) (percentage) 0.520 -
436.cactusADM 23.000 -94.28%
436.cactusADM (avg) 23.000 -94.28%
436.cactusADM (avg) (percentage) 1.130 -48.87%
436.cactusADM (max) 17.000 -96.72%
436.cactusADM (max) (percentage) 0.830 -70.88%
436.cactusADM (unexplained diff vs previous) 12.000 -97.62%
436.cactusADM (unexplained diff vs previous) (percentage) 0.570 -79.42%
437.leslie3d 491.000 19.17%
437.leslie3d (avg) 491.000 19.17%
437.leslie3d (avg) (percentage) 4.490 59.79%
437.leslie3d (max) 1282.000 115.10%
437.leslie3d (max) (percentage) 11.710 188.42%
437.leslie3d (unexplained diff vs previous) 439.000 -17.17%
437.leslie3d (unexplained diff vs previous) (percentage) 4.010 11.08%
444.namd 9.000 -98.40%
444.namd (avg) 9.000 -98.40%
444.namd (avg) (percentage) 1.220 -78.89%
444.namd (max) 5.000 -99.35%
444.namd (max) (percentage) 0.650 -91.77%
444.namd (unexplained diff vs previous) 5.000 -99.30%
444.namd (unexplained diff vs previous) (percentage) 0.640 -91.26%
445.gobmk 44.000 -54.17%
445.gobmk (avg) 44.000 -54.17%
445.gobmk (avg) (percentage) 0.890 8.54%
445.gobmk (max) 24.000 -80.65%
445.gobmk (max) (percentage) 0.490 -53.77%
447.dealII 19.000 -75.95%
447.dealII (avg) 19.000 -75.95%
447.dealII (avg) (percentage) 1.220 38.64%
447.dealII (max) 22.000 -91.06%
447.dealII (max) (percentage) 1.390 -49.27%
447.dealII (unexplained diff vs previous) 23.000 -
447.dealII (unexplained diff vs previous) (percentage) 1.410 -
450.soplex 14.000 -92.51%
450.soplex (avg) 14.000 -92.51%
450.soplex (avg) (percentage) 2.520 115.38%
450.soplex (max) 8.000 -98.75%
450.soplex (max) (percentage) 1.440 -63.82%
453.povray 8.000 -93.16%
453.povray (avg) 8.000 -93.16%
453.povray (avg) (percentage) 1.550 -16.22%
453.povray (max) 11.000 -91.27%
453.povray (max) (percentage) 2.190 10.61%
453.povray (unexplained diff vs previous) 7.000 -
453.povray (unexplained diff vs previous) (percentage) 1.280 -
454.calculix (avg) (percentage) 3.600 718.18%
454.calculix (max) (percentage) 0.830 59.62%
456.hmmer 15.000 -81.48%
456.hmmer (avg) 15.000 -81.48%
456.hmmer (avg) (percentage) 0.540 -27.03%
456.hmmer (max) 8.000 -94.48%
456.hmmer (max) (percentage) 0.290 -78.03%
458.sjeng 45.000 66.67%
458.sjeng (avg) 45.000 66.67%
458.sjeng (avg) (percentage) 0.620 210.00%
458.sjeng (max) 7.000 -76.67%
458.sjeng (max) (percentage) 0.100 -54.55%
459.GemsFDTD 71.000 -76.17%
459.GemsFDTD (avg) 71.000 -76.17%
459.GemsFDTD (avg) (percentage) 1.340 3.08%
459.GemsFDTD (max) 39.000 -88.11%
459.GemsFDTD (max) (percentage) 0.730 -48.95%
462.libquantum 2.000 -97.96%
462.libquantum (avg) 2.000 -97.96%
462.libquantum (avg) (percentage) 1.820 111.63%
462.libquantum (max) 4.000 -97.62%
462.libquantum (max) (percentage) 3.050 106.08%
462.libquantum (unexplained diff vs previous) 2.000 -98.83%
462.libquantum (unexplained diff vs previous) (percentage) 1.460 -2.67%
464.h264ref 43.000 -23.21%
464.h264ref (avg) 43.000 -23.21%
464.h264ref (avg) (percentage) 0.750 82.93%
464.h264ref (max) 8.000 -80.95%
464.h264ref (max) (percentage) 0.150 -51.61%
465.tonto 135.000 73.08%
465.tonto (avg) 135.000 73.08%
465.tonto (avg) (percentage) 1.210 26.04%
465.tonto (max) 53.000 -60.15%
465.tonto (max) (percentage) 0.480 -70.37%
470.lbm 33.000 -91.25%
470.lbm (avg) 33.000 -91.25%
470.lbm (avg) (percentage) 0.880 -71.34%
470.lbm (max) 20.000 -95.90%
470.lbm (max) (percentage) 0.530 -86.65%
470.lbm (unexplained diff vs previous) 18.000 -96.46%
470.lbm (unexplained diff vs previous) (percentage) 0.480 -88.38%
471.omnetpp 59.000 -74.12%
471.omnetpp (avg) 59.000 -74.12%
471.omnetpp (avg) (percentage) 2.190 -3.95%
471.omnetpp (max) 81.000 -87.22%
471.omnetpp (max) (percentage) 3.000 -52.61%
471.omnetpp (unexplained diff vs previous) 43.000 -93.66%
471.omnetpp (unexplained diff vs previous) (percentage) 1.580 -76.63%
473.astar 83.000 84.44%
473.astar (avg) 83.000 84.44%
473.astar (avg) (percentage) 1.610 403.12%
473.astar (max) 219.000 742.31%
473.astar (max) (percentage) 4.250 2136.84%
473.astar (unexplained diff vs previous) 45.000 -
473.astar (unexplained diff vs previous) (percentage) 0.870 -
481.wrf (avg) (percentage) 1.800 -24.69%
481.wrf (max) (percentage) 0.830 -72.33%
482.sphinx3 14.000 -98.13%
482.sphinx3 (avg) 14.000 -98.13%
482.sphinx3 (avg) (percentage) 1.780 -31.54%
482.sphinx3 (max) 6.000 -99.44%
482.sphinx3 (max) (percentage) 0.760 -79.62%
482.sphinx3 (unexplained diff vs previous) 5.000 -99.55%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.600 -84.50%
483.xalancbmk (avg) (percentage) 2.960 1309.52%
483.xalancbmk (max) (percentage) 1.770 453.12%
Geometric Mean 5.956 -68.20%