Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=14e461bed44258fb872731a79ca5e82b70781d09 (+ 75 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/af7fa3135b6b046fe3ba869993221042a65301eb (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=06bbe63e367abd5f7b225cbae30d1e487f237c85 (+ 8 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=288a2f83ed42ce91cf92c2b08dad298a3da0cd9a (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/149/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0149 (2-changed-components) 2023-11-23T22:42:48 0:00:00
Previous 0145 (2-changed-components) 2023-11-19T05:38:12 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 37 73
Performance Improvements 87 141
Added Tests 50 0
Unchanged Tests 1062 1022
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 7.27% 55.000 59.000 - -99.35% -
401.bzip2 4.68% 2733.000 2861.000 - -81.16% -
473.astar 1.61% 5578.000 5668.000 - -62.04% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (unexplained diff vs previous) (percentage) 1000.00% 0.170 1.870 - 679.17% -
471.omnetpp (unexplained diff vs previous) 980.00% 5.000 54.000 - 107.69% -
401.bzip2 (unexplained diff vs previous) 966.67% 12.000 128.000 - -3.03% -
401.bzip2 (unexplained diff vs previous) (percentage) 915.91% 0.440 4.470 - 413.79% -
473.astar (unexplained diff vs previous) 718.18% 11.000 90.000 - 2900.00% -
473.astar (unexplained diff vs previous) (percentage) 695.00% 0.200 1.590 - 7850.00% -
453.povray (unexplained diff vs previous) (percentage) 458.54% 0.410 2.290 - 11350.00% -
453.povray (unexplained diff vs previous) 450.00% 2.000 11.000 - 1000.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) 400.00% 0.110 0.550 - -11.29% -
482.sphinx3 (unexplained diff vs previous) 400.00% 1.000 5.000 - -97.31% -
429.mcf (unexplained diff vs previous) (percentage) 117.95% 0.780 1.700 - 214.81% -
429.mcf (unexplained diff vs previous) 113.33% 15.000 32.000 - -65.59% -
464.h264ref (unexplained diff vs previous) 100.00% 5.000 10.000 - -33.33% -
450.soplex (unexplained diff vs previous) (percentage) 90.91% 2.090 3.990 - 98.51% -
464.h264ref (unexplained diff vs previous) (percentage) 88.89% 0.090 0.170 - 70.00% -
450.soplex (unexplained diff vs previous) 83.33% 12.000 22.000 - -92.93% -
447.dealII (unexplained diff vs previous) (percentage) 60.00% 0.250 0.400 - -48.72% -
447.dealII (unexplained diff vs previous) 60.00% 5.000 8.000 - -89.47% -
445.gobmk (unexplained diff vs previous) 36.84% 19.000 26.000 - -86.02% -
445.gobmk (unexplained diff vs previous) (percentage) 35.14% 0.370 0.500 - -67.74% -
433.milc (max) 33.33% 9.000 12.000 - -97.08% -
433.milc (max) (percentage) 31.40% 0.860 1.130 - -64.35% -
433.milc (unexplained diff vs previous) (percentage) 31.03% 0.870 1.140 - -65.24% -
433.milc (unexplained diff vs previous) 30.00% 10.000 13.000 - -96.94% -
450.soplex (max) (percentage) 20.00% 3.200 3.840 - 94.92% -
450.soplex (max) 16.67% 18.000 21.000 - -93.09% -
471.omnetpp (max) (percentage) 15.82% 1.580 1.830 - -70.34% -
471.omnetpp (max) 13.04% 46.000 52.000 - -92.15% -
401.bzip2 7.55% 53.000 57.000 - -13.64% -
401.bzip2 (avg) 7.55% 53.000 57.000 - -13.64% -
401.bzip2 (avg) (percentage) 3.08% 1.950 2.010 - 356.82% -
403.gcc (avg) (percentage) 1.19% 5.870 5.940 - 873.77% -
473.astar 1.03% 97.000 98.000 - 250.00% -
473.astar (avg) 1.03% 97.000 98.000 - 250.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -3.84% 573.000 551.000 - -96.44% -
453.povray -2.24% 492.000 481.000 - -92.65% -
471.omnetpp -1.83% 2945.000 2891.000 - -73.08% -
429.mcf -1.67% 1911.000 1879.000 - -89.05% -
400.perlbench -1.18% 1527.000 1509.000 - -85.83% -
433.milc -1.13% 1155.000 1142.000 - -91.20% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (unexplained diff vs previous) -77.27% 44.000 10.000 - 11.11% -
458.sjeng (unexplained diff vs previous) (percentage) -77.19% 0.570 0.130 - 116.67% -
462.libquantum (unexplained diff vs previous) (percentage) -66.96% 2.240 0.740 - 7300.00% -
462.libquantum (unexplained diff vs previous) -66.67% 3.000 1.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -58.33% 0.240 0.100 - -86.67% -
444.namd (max) -57.14% 7.000 3.000 - -99.54% -
456.hmmer (unexplained diff vs previous) -57.14% 7.000 3.000 - -96.39% -
456.hmmer (max) -50.00% 10.000 5.000 - -96.43% -
456.hmmer (max) (percentage) -50.00% 0.360 0.180 - -85.83% -
444.namd (max) (percentage) -46.08% 1.020 0.550 - -92.34% -
462.libquantum (max) (percentage) -25.11% 2.190 1.640 - 156.25% -
458.sjeng (max) (percentage) -24.56% 0.570 0.430 - 95.45% -
458.sjeng (max) -23.26% 43.000 33.000 - 6.45% -
444.namd (unexplained diff vs previous) -20.00% 5.000 4.000 - -99.43% -
444.namd (unexplained diff vs previous) (percentage) -18.84% 0.690 0.560 - -92.76% -
447.dealII (max) -11.11% 9.000 8.000 - -92.52% -
400.perlbench (max) -8.70% 23.000 21.000 - -93.91% -
445.gobmk (max) -8.33% 84.000 77.000 - -57.92% -
447.dealII (max) (percentage) -8.33% 0.480 0.440 - -60.00% -
429.mcf (max) -8.31% 349.000 320.000 - 247.83% -
482.sphinx3 (max) (percentage) -8.00% 0.750 0.690 - -53.06% -
453.povray (max) -7.69% 13.000 12.000 - -92.73% -
464.h264ref (max) (percentage) -7.32% 0.820 0.760 - 76.74% -
445.gobmk (max) (percentage) -6.79% 1.620 1.510 - -1.31% -
403.gcc (unexplained diff vs previous) (percentage) -6.74% 7.270 6.780 - 841.67% -
401.bzip2 (max) (percentage) -6.71% 6.860 6.400 - 627.27% -
429.mcf (max) (percentage) -6.67% 18.300 17.080 - 3062.96% -
444.namd -6.67% 15.000 14.000 - -96.06% -
444.namd (avg) -6.67% 15.000 14.000 - -96.06% -
473.astar (max) (percentage) -6.59% 4.100 3.830 - 2635.71% -
400.perlbench (max) (percentage) -6.49% 1.540 1.440 - -55.56% -
464.h264ref (max) -6.38% 47.000 44.000 - -29.03% -
470.lbm (max) (percentage) -6.38% 0.940 0.880 - -82.75% -
470.lbm (max) -5.88% 34.000 32.000 - -95.10% -
450.soplex -5.56% 18.000 17.000 - -91.10% -
450.soplex (avg) -5.56% 18.000 17.000 - -91.10% -
453.povray (max) (percentage) -5.04% 2.780 2.640 - 4.35% -
473.astar (max) -4.82% 228.000 217.000 - 985.00% -
456.hmmer -3.33% 30.000 29.000 - -61.33% -
456.hmmer (avg) -3.33% 30.000 29.000 - -61.33% -
434.zeusmp (max) (percentage) -3.13% 1.600 1.550 - -32.02% -
447.dealII -3.12% 32.000 31.000 - -70.19% -
447.dealII (avg) -3.12% 32.000 31.000 - -70.19% -
436.cactusADM (max) (percentage) -3.08% 0.650 0.630 - -80.00% -
429.mcf -2.78% 108.000 105.000 - 16.67% -
429.mcf (avg) -2.78% 108.000 105.000 - 16.67% -
410.bwaves (max) (percentage) -2.73% 1.100 1.070 - 409.52% -
400.perlbench -2.70% 37.000 36.000 - -76.92% -
400.perlbench (avg) -2.70% 37.000 36.000 - -76.92% -
459.GemsFDTD (max) (percentage) -2.53% 1.580 1.540 - 156.67% -
435.gromacs (max) (percentage) -2.47% 0.810 0.790 - -60.89% -
403.gcc (max) (percentage) -2.21% 7.690 7.520 - 944.44% -
481.wrf (max) (percentage) -2.17% 0.920 0.900 - -71.43% -
465.tonto (max) (percentage) -2.14% 1.870 1.830 - -48.88% -
401.bzip2 (max) -2.14% 187.000 183.000 - 37.59% -
456.hmmer (avg) (percentage) -1.92% 1.040 1.020 - 50.00% -
437.leslie3d (max) (percentage) -1.90% 5.800 5.690 - 143.16% -
470.lbm -1.85% 54.000 53.000 - -82.51% -
470.lbm (avg) -1.85% 54.000 53.000 - -82.51% -
447.dealII (avg) (percentage) -1.83% 1.640 1.610 - 50.47% -
464.h264ref (avg) (percentage) -1.80% 1.110 1.090 - 101.85% -
454.calculix (max) (percentage) -1.80% 5.000 4.910 - 4.47% -
400.perlbench (avg) (percentage) -1.63% 2.460 2.420 - 64.63% -
464.h264ref -1.56% 64.000 63.000 - -19.23% -
464.h264ref (avg) -1.56% 64.000 63.000 - -19.23% -
445.gobmk (avg) (percentage) -1.53% 1.310 1.290 - 67.53% -
445.gobmk -1.49% 67.000 66.000 - -28.26% -
445.gobmk (avg) -1.49% 67.000 66.000 - -28.26% -
458.sjeng -1.47% 68.000 67.000 - 42.55% -
458.sjeng (avg) -1.47% 68.000 67.000 - 42.55% -
444.namd (avg) (percentage) -1.43% 2.100 2.070 - -47.33% -
471.omnetpp -1.39% 72.000 71.000 - -58.24% -
471.omnetpp (avg) -1.39% 72.000 71.000 - -58.24% -
470.lbm (avg) (percentage) -1.36% 1.470 1.450 - -38.82% -
482.sphinx3 (avg) (percentage) -1.34% 2.240 2.210 - 51.37% -
429.mcf (avg) (percentage) -1.23% 5.690 5.620 - 960.38% -
458.sjeng (avg) (percentage) -1.12% 0.890 0.880 - 166.67% -
462.libquantum (avg) (percentage) -1.04% 1.930 1.910 - 193.85% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_compile_successful 55.56% 18.000 28.000 - -3.45% -
nb_execution_successful 55.56% 18.000 28.000 - -3.45% -
nb_total_tests 50.00% 20.000 30.000 - 3.45% -

Added Tests - Code Size
410.bwaves
416.gamess
434.zeusmp
435.gromacs
436.cactusADM
437.leslie3d
454.calculix
459.GemsFDTD
465.tonto
481.wrf

Added Tests - Execution
410.bwaves
416.gamess
434.zeusmp
435.gromacs
436.cactusADM
437.leslie3d
454.calculix
459.GemsFDTD
465.tonto
481.wrf

Added Tests - sample variability
410.bwaves
410.bwaves (avg)
410.bwaves (max)
416.gamess
416.gamess (avg)
416.gamess (max)
434.zeusmp
434.zeusmp (avg)
434.zeusmp (max)
435.gromacs
435.gromacs (avg)
435.gromacs (max)
436.cactusADM
436.cactusADM (avg)
436.cactusADM (max)
437.leslie3d
437.leslie3d (avg)
437.leslie3d (max)
454.calculix
454.calculix (avg)
454.calculix (max)
459.GemsFDTD
459.GemsFDTD (avg)
459.GemsFDTD (max)
465.tonto
465.tonto (avg)
465.tonto (max)
481.wrf
481.wrf (avg)
481.wrf (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
400.perlbench 1.16% 930652.000 941436.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 31.53% 8205.000 10792.000 - 0.00% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
453.povray (unexplained diff vs previous) (percentage) 11350.00% 0.020 2.290 - 458.54% -
473.astar (unexplained diff vs previous) (percentage) 7850.00% 0.020 1.590 - 695.00% -
462.libquantum (unexplained diff vs previous) (percentage) 7300.00% 0.010 0.740 - -66.96% -
429.mcf (max) (percentage) 3062.96% 0.540 17.080 - -6.67% -
473.astar (unexplained diff vs previous) 2900.00% 3.000 90.000 - 718.18% -
473.astar (max) (percentage) 2635.71% 0.140 3.830 - -6.59% -
483.xalancbmk (max) (percentage) 1321.95% 0.410 5.830 - 0.00% -
453.povray (unexplained diff vs previous) 1000.00% 1.000 11.000 - 450.00% -
473.astar (max) 985.00% 20.000 217.000 - -4.82% -
429.mcf (avg) (percentage) 960.38% 0.530 5.620 - -1.23% -
403.gcc (max) (percentage) 944.44% 0.720 7.520 - -2.21% -
403.gcc (avg) (percentage) 873.77% 0.610 5.940 - 1.19% -
403.gcc (unexplained diff vs previous) (percentage) 841.67% 0.720 6.780 - -6.74% -
473.astar (avg) (percentage) 815.79% 0.190 1.740 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 679.17% 0.240 1.870 - 1000.00% -
401.bzip2 (max) (percentage) 627.27% 0.880 6.400 - -6.71% -
401.bzip2 (unexplained diff vs previous) (percentage) 413.79% 0.870 4.470 - 915.91% -
410.bwaves (max) (percentage) 409.52% 0.210 1.070 - -2.73% -
401.bzip2 (avg) (percentage) 356.82% 0.440 2.010 - 3.08% -
473.astar 250.00% 28.000 98.000 - 1.03% -
473.astar (avg) 250.00% 28.000 98.000 - 1.03% -
429.mcf (max) 247.83% 92.000 320.000 - -8.31% -
429.mcf (unexplained diff vs previous) (percentage) 214.81% 0.540 1.700 - 117.95% -
462.libquantum (avg) (percentage) 193.85% 0.650 1.910 - -1.04% -
458.sjeng (avg) (percentage) 166.67% 0.330 0.880 - -1.12% -
450.soplex (avg) (percentage) 157.26% 1.240 3.190 - 0.63% -
459.GemsFDTD (max) (percentage) 156.67% 0.600 1.540 - -2.53% -
462.libquantum (max) (percentage) 156.25% 0.640 1.640 - -25.11% -
437.leslie3d (max) (percentage) 143.16% 2.340 5.690 - -1.90% -
458.sjeng (unexplained diff vs previous) (percentage) 116.67% 0.060 0.130 - -77.19% -
410.bwaves (max) 110.00% 30.000 63.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) 107.69% 26.000 54.000 - 980.00% -
464.h264ref (avg) (percentage) 101.85% 0.540 1.090 - -1.80% -
437.leslie3d (avg) (percentage) 98.74% 2.380 4.730 - 0.00% -
450.soplex (unexplained diff vs previous) (percentage) 98.51% 2.010 3.990 - 90.91% -
458.sjeng (max) (percentage) 95.45% 0.220 0.430 - -24.56% -
450.soplex (max) (percentage) 94.92% 1.970 3.840 - 20.00% -
459.GemsFDTD (avg) (percentage) 87.78% 0.900 1.690 - 0.00% -
464.h264ref (max) (percentage) 76.74% 0.430 0.760 - -7.32% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
454.calculix (avg) (percentage) 73.63% 2.010 3.490 - 0.00% -
453.povray (avg) (percentage) 72.80% 1.250 2.160 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) 72.73% 0.110 0.190 - 0.00% -
464.h264ref (unexplained diff vs previous) (percentage) 70.00% 0.100 0.170 - 88.89% -
437.leslie3d (max) 68.66% 351.000 592.000 - 0.00% -
445.gobmk (avg) (percentage) 67.53% 0.770 1.290 - -1.53% -
400.perlbench (avg) (percentage) 64.63% 1.470 2.420 - -1.63% -
471.omnetpp (avg) (percentage) 54.72% 1.590 2.460 - -0.40% -
482.sphinx3 (avg) (percentage) 51.37% 1.460 2.210 - -1.34% -
447.dealII (avg) (percentage) 50.47% 1.070 1.610 - -1.83% -
456.hmmer (avg) (percentage) 50.00% 0.680 1.020 - -1.92% -
458.sjeng 42.55% 47.000 67.000 - -1.47% -
458.sjeng (avg) 42.55% 47.000 67.000 - -1.47% -
437.leslie3d 37.82% 357.000 492.000 - 0.00% -
437.leslie3d (avg) 37.82% 357.000 492.000 - 0.00% -
401.bzip2 (max) 37.59% 133.000 183.000 - -2.14% -
410.bwaves (avg) (percentage) 25.69% 1.090 1.370 - 0.00% -
434.zeusmp (avg) (percentage) 22.30% 1.480 1.810 - 0.00% -
435.gromacs (avg) (percentage) 21.39% 1.730 2.100 - 0.00% -
429.mcf 16.67% 90.000 105.000 - -2.78% -
429.mcf (avg) 16.67% 90.000 105.000 - -2.78% -
465.tonto 12.41% 137.000 154.000 - 0.00% -
465.tonto (avg) 12.41% 137.000 154.000 - 0.00% -
458.sjeng (unexplained diff vs previous) 11.11% 9.000 10.000 - -77.27% -
433.milc (avg) (percentage) 8.24% 1.700 1.840 - -0.54% -
458.sjeng (max) 6.45% 31.000 33.000 - -23.26% -
454.calculix (max) (percentage) 4.47% 4.700 4.910 - -1.80% -
453.povray (max) (percentage) 4.35% 2.530 2.640 - -5.04% -
481.wrf (avg) (percentage) 1.59% 1.890 1.920 - -0.52% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 55.56% -
nb_execution_successful -3.45% 29.000 28.000 - 55.56% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -3.57% 120221.000 115929.000 - 0.00% -
464.h264ref -2.37% 428364.000 418228.000 - 0.00% -
434.zeusmp -1.62% 165699.000 163021.000 - 0.00% -
459.GemsFDTD -1.57% 253716.000 249721.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.73% 22860.000 61.000 - 0.00% -
403.gcc -99.35% 9010.000 59.000 - 7.27% -
462.libquantum -98.84% 11614.000 135.000 - 0.75% -
482.sphinx3 -96.95% 29928.000 912.000 - -0.55% -
450.soplex -96.44% 15466.000 551.000 - -3.84% -
453.povray -92.65% 6544.000 481.000 - -2.24% -
444.namd -92.06% 9050.000 719.000 - -0.55% -
433.milc -91.20% 12976.000 1142.000 - -1.13% -
429.mcf -89.05% 17158.000 1879.000 - -1.67% -
436.cactusADM -87.81% 17544.000 2138.000 - 0.00% -
400.perlbench -85.83% 10650.000 1509.000 - -1.18% -
401.bzip2 -81.16% 15185.000 2861.000 - 4.68% -
434.zeusmp -80.83% 13211.000 2532.000 - 0.00% -
447.dealII -79.71% 9751.000 1978.000 - -0.40% -
459.GemsFDTD -74.22% 23356.000 6022.000 - 0.00% -
456.hmmer -73.63% 11046.000 2913.000 - -0.10% -
471.omnetpp -73.08% 10739.000 2891.000 - -1.83% -
470.lbm -71.14% 12819.000 3700.000 - -0.19% -
481.wrf -67.54% 16834.000 5464.000 - 0.00% -
473.astar -62.04% 14932.000 5668.000 - 1.61% -
464.h264ref -60.00% 14556.000 5823.000 - 0.17% -
410.bwaves -59.72% 14755.000 5943.000 - 0.00% -
445.gobmk -56.97% 11995.000 5161.000 - -0.50% -
435.gromacs -55.17% 15419.000 6912.000 - 0.00% -
458.sjeng -45.88% 14275.000 7725.000 - 0.13% -
416.gamess -44.82% 22704.000 12529.000 - 0.00% -
437.leslie3d -30.58% 15005.000 10417.000 - 0.00% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.81% 1074.000 2.000 - 0.00% -
454.calculix -99.56% 459.000 2.000 - 0.00% -
454.calculix (avg) -99.56% 459.000 2.000 - 0.00% -
444.namd (max) -99.54% 649.000 3.000 - -57.14% -
444.namd (unexplained diff vs previous) -99.43% 700.000 4.000 - -20.00% -
482.sphinx3 (max) -98.63% 439.000 6.000 - 0.00% -
436.cactusADM (max) -97.64% 552.000 13.000 - 0.00% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -97.31% 186.000 5.000 - 400.00% -
462.libquantum (max) -97.30% 74.000 2.000 - 0.00% -
433.milc (max) -97.08% 411.000 12.000 - 33.33% -
433.milc (unexplained diff vs previous) -96.94% 425.000 13.000 - 30.00% -
456.hmmer (max) -96.43% 140.000 5.000 - -50.00% -
456.hmmer (unexplained diff vs previous) -96.39% 83.000 3.000 - -57.14% -
444.namd -96.06% 355.000 14.000 - -6.67% -
444.namd (avg) -96.06% 355.000 14.000 - -6.67% -
482.sphinx3 -95.41% 436.000 20.000 - 0.00% -
482.sphinx3 (avg) -95.41% 436.000 20.000 - 0.00% -
470.lbm (max) -95.10% 653.000 32.000 - -5.88% -
400.perlbench (unexplained diff vs previous) -94.96% 357.000 18.000 - 0.00% -
416.gamess (max) -94.66% 656.000 35.000 - 0.00% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
400.perlbench (max) -93.91% 345.000 21.000 - -8.70% -
403.gcc (unexplained diff vs previous) -93.85% 65.000 4.000 - 0.00% -
403.gcc (max) -93.75% 64.000 4.000 - 0.00% -
450.soplex (max) -93.09% 304.000 21.000 - 16.67% -
450.soplex (unexplained diff vs previous) -92.93% 311.000 22.000 - 83.33% -
444.namd (unexplained diff vs previous) (percentage) -92.76% 7.730 0.560 - -18.84% -
453.povray (max) -92.73% 165.000 12.000 - -7.69% -
447.dealII (max) -92.52% 107.000 8.000 - -11.11% -
444.namd (max) (percentage) -92.34% 7.180 0.550 - -46.08% -
471.omnetpp (max) -92.15% 662.000 52.000 - 13.04% -
450.soplex -91.10% 191.000 17.000 - -5.56% -
450.soplex (avg) -91.10% 191.000 17.000 - -5.56% -
481.wrf (max) -90.75% 530.000 49.000 - 0.00% -
433.milc -90.45% 220.000 21.000 - 0.00% -
433.milc (avg) -90.45% 220.000 21.000 - 0.00% -
416.gamess (max) (percentage) -90.31% 2.890 0.280 - 0.00% -
447.dealII (unexplained diff vs previous) -89.47% 76.000 8.000 - 60.00% -
436.cactusADM -88.66% 291.000 33.000 - 0.00% -
436.cactusADM (avg) -88.66% 291.000 33.000 - 0.00% -
453.povray -87.65% 81.000 10.000 - 0.00% -
453.povray (avg) -87.65% 81.000 10.000 - 0.00% -
434.zeusmp (max) -87.04% 301.000 39.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -86.67% 0.750 0.100 - -58.33% -
445.gobmk (unexplained diff vs previous) -86.02% 186.000 26.000 - 36.84% -
456.hmmer (max) (percentage) -85.83% 1.270 0.180 - -50.00% -
470.lbm (max) (percentage) -82.75% 5.100 0.880 - -6.38% -
435.gromacs (max) -82.64% 311.000 54.000 - 0.00% -
470.lbm -82.51% 303.000 53.000 - -1.85% -
470.lbm (avg) -82.51% 303.000 53.000 - -1.85% -
436.cactusADM (max) (percentage) -80.00% 3.150 0.630 - -3.08% -
400.perlbench -76.92% 156.000 36.000 - -2.70% -
400.perlbench (avg) -76.92% 156.000 36.000 - -2.70% -
434.zeusmp -76.92% 195.000 45.000 - 0.00% -
434.zeusmp (avg) -76.92% 195.000 45.000 - 0.00% -
416.gamess -72.56% 492.000 135.000 - 0.00% -
416.gamess (avg) -72.56% 492.000 135.000 - 0.00% -
481.wrf (max) (percentage) -71.43% 3.150 0.900 - -2.17% -
471.omnetpp (max) (percentage) -70.34% 6.170 1.830 - 15.82% -
447.dealII -70.19% 104.000 31.000 - -3.12% -
447.dealII (avg) -70.19% 104.000 31.000 - -3.12% -
445.gobmk (unexplained diff vs previous) (percentage) -67.74% 1.550 0.500 - 35.14% -
481.wrf -67.30% 318.000 104.000 - 0.00% -
481.wrf (avg) -67.30% 318.000 104.000 - 0.00% -
429.mcf (unexplained diff vs previous) -65.59% 93.000 32.000 - 113.33% -
433.milc (unexplained diff vs previous) (percentage) -65.24% 3.280 1.140 - 31.03% -
400.perlbench (unexplained diff vs previous) (percentage) -64.48% 3.350 1.190 - 0.00% -
433.milc (max) (percentage) -64.35% 3.170 1.130 - 31.40% -
456.hmmer -61.33% 75.000 29.000 - -3.33% -
456.hmmer (avg) -61.33% 75.000 29.000 - -3.33% -
435.gromacs (max) (percentage) -60.89% 2.020 0.790 - -2.47% -
447.dealII (max) (percentage) -60.00% 1.100 0.440 - -8.33% -
471.omnetpp -58.24% 170.000 71.000 - -1.39% -
471.omnetpp (avg) -58.24% 170.000 71.000 - -1.39% -
445.gobmk (max) -57.92% 183.000 77.000 - -8.33% -
400.perlbench (max) (percentage) -55.56% 3.240 1.440 - -6.49% -
482.sphinx3 (max) (percentage) -53.06% 1.470 0.690 - -8.00% -
459.GemsFDTD -51.90% 210.000 101.000 - 0.00% -
459.GemsFDTD (avg) -51.90% 210.000 101.000 - 0.00% -
416.gamess (avg) (percentage) -50.23% 2.170 1.080 - 0.00% -
470.lbm (unexplained diff vs previous) -50.00% 14.000 7.000 - 0.00% -
410.bwaves -49.38% 160.000 81.000 - 0.00% -
410.bwaves (avg) -49.38% 160.000 81.000 - 0.00% -
465.tonto (max) (percentage) -48.88% 3.580 1.830 - -2.14% -
447.dealII (unexplained diff vs previous) (percentage) -48.72% 0.780 0.400 - 60.00% -
444.namd (avg) (percentage) -47.33% 3.930 2.070 - -1.43% -
435.gromacs -45.49% 266.000 145.000 - 0.00% -
435.gromacs (avg) -45.49% 266.000 145.000 - 0.00% -
470.lbm (avg) (percentage) -38.82% 2.370 1.450 - -1.36% -
459.GemsFDTD (max) -34.29% 140.000 92.000 - 0.00% -
464.h264ref (unexplained diff vs previous) -33.33% 15.000 10.000 - 100.00% -
465.tonto (max) -32.76% 293.000 197.000 - 0.00% -
434.zeusmp (max) (percentage) -32.02% 2.280 1.550 - -3.13% -
464.h264ref (max) -29.03% 62.000 44.000 - -6.38% -
445.gobmk -28.26% 92.000 66.000 - -1.49% -
445.gobmk (avg) -28.26% 92.000 66.000 - -1.49% -
464.h264ref -19.23% 78.000 63.000 - -1.56% -
464.h264ref (avg) -19.23% 78.000 63.000 - -1.56% -
465.tonto (avg) (percentage) -14.37% 1.670 1.430 - 0.00% -
401.bzip2 -13.64% 66.000 57.000 - 7.55% -
401.bzip2 (avg) -13.64% 66.000 57.000 - 7.55% -
482.sphinx3 (unexplained diff vs previous) (percentage) -11.29% 0.620 0.550 - 400.00% -
436.cactusADM (avg) (percentage) -4.22% 1.660 1.590 - 0.00% -
401.bzip2 (unexplained diff vs previous) -3.03% 132.000 128.000 - 966.67% -
445.gobmk (max) (percentage) -1.31% 1.530 1.510 - -6.79% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 3.45% 29.000 30.000 - 50.00% -

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 1509.000 -1.18%
401.bzip2 2861.000 4.68%
403.gcc 59.000 7.27%
410.bwaves 5943.000 -
416.gamess 12529.000 -
429.mcf 1879.000 -1.67%
433.milc 1142.000 -1.13%
434.zeusmp 2532.000 -
435.gromacs 6912.000 -
436.cactusADM 2138.000 -
437.leslie3d 10417.000 -
444.namd 719.000 -
445.gobmk 5161.000 -
447.dealII 1978.000 -
450.soplex 551.000 -3.84%
453.povray 481.000 -2.24%
454.calculix 61.000 -
456.hmmer 2913.000 -
458.sjeng 7725.000 -
459.GemsFDTD 6022.000 -
462.libquantum 135.000 -
464.h264ref 5823.000 -
465.tonto 10792.000 -
470.lbm 3700.000 -
471.omnetpp 2891.000 -1.83%
473.astar 5668.000 1.61%
481.wrf 5464.000 -
482.sphinx3 912.000 -
Geometric Mean 2064.609 40.35%
Code Size (bytes)
Name Current %
400.perlbench 941436.000 -
401.bzip2 49698.000 -
403.gcc 2671145.000 -
410.bwaves 17640.000 -
416.gamess 6229331.000 -
429.mcf 10979.000 -
433.milc 95589.000 -
434.zeusmp 163021.000 -
435.gromacs 677981.000 -
436.cactusADM 551244.000 -
437.leslie3d 64764.000 -
444.namd 190746.000 -
445.gobmk 1399537.000 -
447.dealII 2123042.000 -
450.soplex 259980.000 -
453.povray 702162.000 -
454.calculix 1268827.000 -
456.hmmer 230496.000 -
458.sjeng 115929.000 -
459.GemsFDTD 249721.000 -
462.libquantum 31321.000 -
464.h264ref 418228.000 -
465.tonto 2868529.000 -
470.lbm 10916.000 -
471.omnetpp 566294.000 -
473.astar 32056.000 -
481.wrf 2617296.000 -
482.sphinx3 152548.000 -
Geometric Mean 272058.565 40.66%
Score
Name Current %
nb_compile_successful 28.000 55.56%
nb_execution_successful 28.000 55.56%
nb_total_tests 30.000 50.00%
Geometric Mean 28.651 53.68%
sample variability
Name Current %
400.perlbench 36.000 -2.70%
400.perlbench (avg) 36.000 -2.70%
400.perlbench (avg) (percentage) 2.420 -1.63%
400.perlbench (max) 21.000 -8.70%
400.perlbench (max) (percentage) 1.440 -6.49%
400.perlbench (unexplained diff vs previous) 18.000 -
400.perlbench (unexplained diff vs previous) (percentage) 1.190 -
401.bzip2 57.000 7.55%
401.bzip2 (avg) 57.000 7.55%
401.bzip2 (avg) (percentage) 2.010 3.08%
401.bzip2 (max) 183.000 -2.14%
401.bzip2 (max) (percentage) 6.400 -6.71%
401.bzip2 (unexplained diff vs previous) 128.000 966.67%
401.bzip2 (unexplained diff vs previous) (percentage) 4.470 915.91%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 5.940 1.19%
403.gcc (max) 4.000 -
403.gcc (max) (percentage) 7.520 -2.21%
403.gcc (unexplained diff vs previous) 4.000 -
403.gcc (unexplained diff vs previous) (percentage) 6.780 -6.74%
410.bwaves 81.000 -
410.bwaves (avg) 81.000 -
410.bwaves (avg) (percentage) 1.370 -
410.bwaves (max) 63.000 -
410.bwaves (max) (percentage) 1.070 -2.73%
416.gamess 135.000 -
416.gamess (avg) 135.000 -
416.gamess (avg) (percentage) 1.080 -
416.gamess (max) 35.000 -
416.gamess (max) (percentage) 0.280 -
429.mcf 105.000 -2.78%
429.mcf (avg) 105.000 -2.78%
429.mcf (avg) (percentage) 5.620 -1.23%
429.mcf (max) 320.000 -8.31%
429.mcf (max) (percentage) 17.080 -6.67%
429.mcf (unexplained diff vs previous) 32.000 113.33%
429.mcf (unexplained diff vs previous) (percentage) 1.700 117.95%
433.milc 21.000 -
433.milc (avg) 21.000 -
433.milc (avg) (percentage) 1.840 -
433.milc (max) 12.000 33.33%
433.milc (max) (percentage) 1.130 31.40%
433.milc (unexplained diff vs previous) 13.000 30.00%
433.milc (unexplained diff vs previous) (percentage) 1.140 31.03%
434.zeusmp 45.000 -
434.zeusmp (avg) 45.000 -
434.zeusmp (avg) (percentage) 1.810 -
434.zeusmp (max) 39.000 -
434.zeusmp (max) (percentage) 1.550 -3.13%
435.gromacs 145.000 -
435.gromacs (avg) 145.000 -
435.gromacs (avg) (percentage) 2.100 -
435.gromacs (max) 54.000 -
435.gromacs (max) (percentage) 0.790 -2.47%
436.cactusADM 33.000 -
436.cactusADM (avg) 33.000 -
436.cactusADM (avg) (percentage) 1.590 -
436.cactusADM (max) 13.000 -
436.cactusADM (max) (percentage) 0.630 -3.08%
437.leslie3d 492.000 -
437.leslie3d (avg) 492.000 -
437.leslie3d (avg) (percentage) 4.730 -
437.leslie3d (max) 592.000 -
437.leslie3d (max) (percentage) 5.690 -1.90%
444.namd 14.000 -6.67%
444.namd (avg) 14.000 -6.67%
444.namd (avg) (percentage) 2.070 -1.43%
444.namd (max) 3.000 -57.14%
444.namd (max) (percentage) 0.550 -46.08%
444.namd (unexplained diff vs previous) 4.000 -20.00%
444.namd (unexplained diff vs previous) (percentage) 0.560 -18.84%
445.gobmk 66.000 -1.49%
445.gobmk (avg) 66.000 -1.49%
445.gobmk (avg) (percentage) 1.290 -1.53%
445.gobmk (max) 77.000 -8.33%
445.gobmk (max) (percentage) 1.510 -6.79%
445.gobmk (unexplained diff vs previous) 26.000 36.84%
445.gobmk (unexplained diff vs previous) (percentage) 0.500 35.14%
447.dealII 31.000 -3.12%
447.dealII (avg) 31.000 -3.12%
447.dealII (avg) (percentage) 1.610 -1.83%
447.dealII (max) 8.000 -11.11%
447.dealII (max) (percentage) 0.440 -8.33%
447.dealII (unexplained diff vs previous) 8.000 60.00%
447.dealII (unexplained diff vs previous) (percentage) 0.400 60.00%
450.soplex 17.000 -5.56%
450.soplex (avg) 17.000 -5.56%
450.soplex (avg) (percentage) 3.190 -
450.soplex (max) 21.000 16.67%
450.soplex (max) (percentage) 3.840 20.00%
450.soplex (unexplained diff vs previous) 22.000 83.33%
450.soplex (unexplained diff vs previous) (percentage) 3.990 90.91%
453.povray 10.000 -
453.povray (avg) 10.000 -
453.povray (avg) (percentage) 2.160 -
453.povray (max) 12.000 -7.69%
453.povray (max) (percentage) 2.640 -5.04%
453.povray (unexplained diff vs previous) 11.000 450.00%
453.povray (unexplained diff vs previous) (percentage) 2.290 458.54%
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 3.490 -
454.calculix (max) 2.000 -
454.calculix (max) (percentage) 4.910 -1.80%
456.hmmer 29.000 -3.33%
456.hmmer (avg) 29.000 -3.33%
456.hmmer (avg) (percentage) 1.020 -1.92%
456.hmmer (max) 5.000 -50.00%
456.hmmer (max) (percentage) 0.180 -50.00%
456.hmmer (unexplained diff vs previous) 3.000 -57.14%
456.hmmer (unexplained diff vs previous) (percentage) 0.100 -58.33%
458.sjeng 67.000 -1.47%
458.sjeng (avg) 67.000 -1.47%
458.sjeng (avg) (percentage) 0.880 -1.12%
458.sjeng (max) 33.000 -23.26%
458.sjeng (max) (percentage) 0.430 -24.56%
458.sjeng (unexplained diff vs previous) 10.000 -77.27%
458.sjeng (unexplained diff vs previous) (percentage) 0.130 -77.19%
459.GemsFDTD 101.000 -
459.GemsFDTD (avg) 101.000 -
459.GemsFDTD (avg) (percentage) 1.690 -
459.GemsFDTD (max) 92.000 -
459.GemsFDTD (max) (percentage) 1.540 -2.53%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.910 -1.04%
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.640 -25.11%
462.libquantum (unexplained diff vs previous) 1.000 -66.67%
462.libquantum (unexplained diff vs previous) (percentage) 0.740 -66.96%
464.h264ref 63.000 -1.56%
464.h264ref (avg) 63.000 -1.56%
464.h264ref (avg) (percentage) 1.090 -1.80%
464.h264ref (max) 44.000 -6.38%
464.h264ref (max) (percentage) 0.760 -7.32%
464.h264ref (unexplained diff vs previous) 10.000 100.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.170 88.89%
465.tonto 154.000 -
465.tonto (avg) 154.000 -
465.tonto (avg) (percentage) 1.430 -
465.tonto (max) 197.000 -
465.tonto (max) (percentage) 1.830 -2.14%
470.lbm 53.000 -1.85%
470.lbm (avg) 53.000 -1.85%
470.lbm (avg) (percentage) 1.450 -1.36%
470.lbm (max) 32.000 -5.88%
470.lbm (max) (percentage) 0.880 -6.38%
470.lbm (unexplained diff vs previous) 7.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.190 -
471.omnetpp 71.000 -1.39%
471.omnetpp (avg) 71.000 -1.39%
471.omnetpp (avg) (percentage) 2.460 -
471.omnetpp (max) 52.000 13.04%
471.omnetpp (max) (percentage) 1.830 15.82%
471.omnetpp (unexplained diff vs previous) 54.000 980.00%
471.omnetpp (unexplained diff vs previous) (percentage) 1.870 1000.00%
473.astar 98.000 1.03%
473.astar (avg) 98.000 1.03%
473.astar (avg) (percentage) 1.740 -
473.astar (max) 217.000 -4.82%
473.astar (max) (percentage) 3.830 -6.59%
473.astar (unexplained diff vs previous) 90.000 718.18%
473.astar (unexplained diff vs previous) (percentage) 1.590 695.00%
481.wrf 104.000 -
481.wrf (avg) 104.000 -
481.wrf (avg) (percentage) 1.920 -
481.wrf (max) 49.000 -
481.wrf (max) (percentage) 0.900 -2.17%
482.sphinx3 20.000 -
482.sphinx3 (avg) 20.000 -
482.sphinx3 (avg) (percentage) 2.210 -1.34%
482.sphinx3 (max) 6.000 -
482.sphinx3 (max) (percentage) 0.690 -8.00%
482.sphinx3 (unexplained diff vs previous) 5.000 400.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.550 400.00%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.830 -
Geometric Mean 8.104 118.97%