Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=b6532accdd8e24329cc69bb58bc2883796008776 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/877fb9bdb06d18df51c6043f74dde66fe6f46b78 (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=e67f8e6dbd5ec98578a775b2e09b254f071e0f57 (unchanged)
git_linux https://gitlab.com/Linaro/lkft/mirrors/stable/linux-stable/commit/3b232d34011a152832ba40289e9d476dfcb846d2 (+ 698 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/434/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0434 (v6.10.13-71353-g3b232d34011a1) 2024-09-29T11:22:18 0:00:00
Previous 0433 (glibc-2.40.9000-00225-ge67f8e6dbd5) 2024-09-24T08:10:46 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 115 85
Performance Improvements 47 128
Added Tests 6 6
Unchanged Tests 1068 1017
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
429.mcf 10.50% 1876.000 2073.000 - -87.92% -
437.leslie3d 8.34% 10596.000 11480.000 - -23.49% -
471.omnetpp 5.42% 2802.000 2954.000 - -72.49% -
450.soplex 5.27% 531.000 559.000 - -96.39% -
473.astar 4.88% 5572.000 5844.000 - -60.86% -
433.milc 3.78% 1110.000 1152.000 - -91.12% -
459.GemsFDTD 3.43% 5008.000 5180.000 - -77.82% -
482.sphinx3 3.08% 909.000 937.000 - -96.87% -
435.gromacs 2.92% 6976.000 7180.000 - -53.43% -
401.bzip2 2.75% 2906.000 2986.000 - -80.34% -
434.zeusmp 2.06% 2475.000 2526.000 - -80.88% -
453.povray 2.04% 490.000 500.000 - -92.36% -
410.bwaves 2.01% 6024.000 6145.000 - -58.35% -
444.namd 1.73% 750.000 763.000 - -91.57% -
470.lbm 1.54% 3690.000 3747.000 - -70.77% -
445.gobmk 1.53% 5089.000 5167.000 - -56.92% -
465.tonto 1.44% 11037.000 11196.000 - 36.45% -
464.h264ref 1.25% 5860.000 5933.000 - -59.24% -
458.sjeng 1.06% 7572.000 7652.000 - -46.40% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
437.leslie3d (unexplained diff vs previous) 396.63% 178.000 884.000 - 157.73% -
437.leslie3d (unexplained diff vs previous) (percentage) 358.33% 1.680 7.700 - 236.24% -
470.lbm (unexplained diff vs previous) 137.50% 24.000 57.000 - 307.14% -
470.lbm (unexplained diff vs previous) (percentage) 133.85% 0.650 1.520 - 1281.82% -
437.leslie3d (max) 117.50% 440.000 957.000 - 172.65% -
437.leslie3d (max) (percentage) 100.48% 4.160 8.340 - 256.41% -
410.bwaves (max) 64.00% 75.000 123.000 - 310.00% -
410.bwaves (max) (percentage) 60.80% 1.250 2.010 - 857.14% -
410.bwaves (unexplained diff vs previous) 59.21% 76.000 121.000 - 706.67% -
410.bwaves (unexplained diff vs previous) (percentage) 56.35% 1.260 1.970 - 1870.00% -
433.milc (max) 43.33% 30.000 43.000 - -89.54% -
447.dealII (unexplained diff vs previous) 40.00% 5.000 7.000 - -90.79% -
433.milc (max) (percentage) 38.97% 2.720 3.780 - 19.24% -
447.dealII (unexplained diff vs previous) (percentage) 38.46% 0.260 0.360 - -53.85% -
433.milc (unexplained diff vs previous) 35.48% 31.000 42.000 - -90.12% -
465.tonto (unexplained diff vs previous) 31.40% 121.000 159.000 - 152.38% -
433.milc (unexplained diff vs previous) (percentage) 30.82% 2.790 3.650 - 11.28% -
465.tonto (unexplained diff vs previous) (percentage) 29.09% 1.100 1.420 - 84.42% -
458.sjeng (max) 20.90% 67.000 81.000 - 161.29% -
458.sjeng (max) (percentage) 19.10% 0.890 1.060 - 381.82% -
444.namd (max) 18.18% 11.000 13.000 - -98.00% -
458.sjeng (unexplained diff vs previous) 17.65% 68.000 80.000 - 788.89% -
464.h264ref (max) 17.46% 63.000 74.000 - 19.35% -
471.omnetpp (max) 16.79% 137.000 160.000 - -75.83% -
458.sjeng (unexplained diff vs previous) (percentage) 16.67% 0.900 1.050 - 1650.00% -
435.gromacs (unexplained diff vs previous) 16.57% 175.000 204.000 - -35.85% -
459.GemsFDTD (max) 16.45% 152.000 177.000 - 26.43% -
464.h264ref (max) (percentage) 15.74% 1.080 1.250 - 190.70% -
464.h264ref (unexplained diff vs previous) 14.06% 64.000 73.000 - 386.67% -
435.gromacs (unexplained diff vs previous) (percentage) 13.15% 2.510 2.840 - 37.86% -
464.h264ref (unexplained diff vs previous) (percentage) 12.84% 1.090 1.230 - 1130.00% -
459.GemsFDTD (max) (percentage) 12.83% 3.040 3.430 - 471.67% -
453.povray 12.50% 8.000 9.000 - -88.89% -
453.povray (avg) 12.50% 8.000 9.000 - -88.89% -
429.mcf 12.26% 106.000 119.000 - 32.22% -
429.mcf (avg) 12.26% 106.000 119.000 - 32.22% -
436.cactusADM (unexplained diff vs previous) (percentage) 11.85% 1.350 1.510 - -53.54% -
465.tonto (max) 11.81% 144.000 161.000 - -45.05% -
437.leslie3d 10.61% 443.000 490.000 - 37.25% -
437.leslie3d (avg) 10.61% 443.000 490.000 - 37.25% -
436.cactusADM (unexplained diff vs previous) 10.34% 29.000 32.000 - -94.39% -
444.namd (max) (percentage) 10.19% 1.570 1.730 - -75.91% -
471.omnetpp (max) (percentage) 10.16% 4.920 5.420 - -12.16% -
465.tonto (max) (percentage) 9.92% 1.310 1.440 - -59.78% -
459.GemsFDTD (unexplained diff vs previous) 9.55% 157.000 172.000 - 258.33% -
436.cactusADM (max) (percentage) 8.76% 1.370 1.490 - -52.70% -
434.zeusmp (max) 8.33% 48.000 52.000 - -82.72% -
444.namd (unexplained diff vs previous) 8.33% 12.000 13.000 - -98.14% -
473.astar 7.07% 99.000 106.000 - 278.57% -
473.astar (avg) 7.07% 99.000 106.000 - 278.57% -
436.cactusADM (max) 6.90% 29.000 31.000 - -94.38% -
450.soplex 6.67% 15.000 16.000 - -91.62% -
450.soplex (avg) 6.67% 15.000 16.000 - -91.62% -
444.namd (unexplained diff vs previous) (percentage) 6.25% 1.600 1.700 - -78.01% -
434.zeusmp (max) (percentage) 6.19% 1.940 2.060 - -9.65% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 6.07% 3.130 3.320 - 1480.95% -
433.milc 5.88% 17.000 18.000 - -91.82% -
433.milc (avg) 5.88% 17.000 18.000 - -91.82% -
471.omnetpp 5.88% 68.000 72.000 - -57.65% -
471.omnetpp (avg) 5.88% 68.000 72.000 - -57.65% -
459.GemsFDTD 5.41% 74.000 78.000 - -62.86% -
459.GemsFDTD (avg) 5.41% 74.000 78.000 - -62.86% -
471.omnetpp (unexplained diff vs previous) 4.83% 145.000 152.000 - 484.62% -
434.zeusmp (unexplained diff vs previous) 4.08% 49.000 51.000 - -83.44% -
464.h264ref 4.08% 49.000 51.000 - -34.62% -
464.h264ref (avg) 4.08% 49.000 51.000 - -34.62% -
435.gromacs 3.70% 135.000 140.000 - -47.37% -
435.gromacs (avg) 3.70% 135.000 140.000 - -47.37% -
401.bzip2 3.39% 59.000 61.000 - -7.58% -
401.bzip2 (avg) 3.39% 59.000 61.000 - -7.58% -
429.mcf (max) 3.33% 210.000 217.000 - 135.87% -
410.bwaves 2.90% 69.000 71.000 - -55.62% -
410.bwaves (avg) 2.90% 69.000 71.000 - -55.62% -
434.zeusmp 2.86% 35.000 36.000 - -81.54% -
434.zeusmp (avg) 2.86% 35.000 36.000 - -81.54% -
459.GemsFDTD (avg) (percentage) 2.70% 1.480 1.520 - 68.89% -
401.bzip2 (max) 2.50% 80.000 82.000 - -38.35% -
473.astar (avg) (percentage) 2.23% 1.790 1.830 - 863.16% -
434.zeusmp (unexplained diff vs previous) (percentage) 2.02% 1.980 2.020 - -13.30% -
433.milc (avg) (percentage) 1.94% 1.550 1.580 - -7.06% -
437.leslie3d (avg) (percentage) 1.91% 4.190 4.270 - 79.41% -
445.gobmk 1.79% 56.000 57.000 - -38.04% -
445.gobmk (avg) 1.79% 56.000 57.000 - -38.04% -
458.sjeng 1.75% 57.000 58.000 - 23.40% -
458.sjeng (avg) 1.75% 57.000 58.000 - 23.40% -
410.bwaves (avg) (percentage) 1.74% 1.150 1.170 - 7.34% -
471.omnetpp (avg) (percentage) 1.65% 2.430 2.470 - 55.35% -
465.tonto 1.42% 141.000 143.000 - 4.38% -
465.tonto (avg) 1.42% 141.000 143.000 - 4.38% -
434.zeusmp (avg) (percentage) 1.40% 1.430 1.450 - -2.03% -
458.sjeng (avg) (percentage) 1.32% 0.760 0.770 - 133.33% -
429.mcf (avg) (percentage) 1.23% 5.700 5.770 - 988.68% -
464.h264ref (avg) (percentage) 1.18% 0.850 0.860 - 59.26% -
482.sphinx3 (avg) (percentage) 1.05% 1.900 1.920 - 31.51% -
435.gromacs (avg) (percentage) 1.03% 1.940 1.960 - 13.29% -
450.soplex (avg) (percentage) 1.01% 2.960 2.990 - 141.13% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
436.cactusADM -1.49% 2150.000 2118.000 - -87.93% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
462.libquantum (max) (percentage) -62.06% 4.270 1.620 - 153.13% -
462.libquantum (max) -60.00% 5.000 2.000 - -97.30% -
453.povray (unexplained diff vs previous) (percentage) -48.45% 3.880 2.000 - 9900.00% -
453.povray (unexplained diff vs previous) -47.37% 19.000 10.000 - 900.00% -
445.gobmk (unexplained diff vs previous) (percentage) -29.44% 2.140 1.510 - -2.58% -
445.gobmk (unexplained diff vs previous) -28.44% 109.000 78.000 - -58.06% -
429.mcf (unexplained diff vs previous) (percentage) -24.78% 12.630 9.500 - 1659.26% -
445.gobmk (max) (percentage) -24.76% 2.100 1.580 - 3.27% -
445.gobmk (max) -23.58% 106.000 81.000 - -55.74% -
473.astar (unexplained diff vs previous) (percentage) -20.78% 5.870 4.650 - 23150.00% -
450.soplex (unexplained diff vs previous) (percentage) -19.32% 6.210 5.010 - 149.25% -
429.mcf (unexplained diff vs previous) -16.88% 237.000 197.000 - 111.83% -
473.astar (unexplained diff vs previous) -16.82% 327.000 272.000 - 8966.67% -
450.soplex (unexplained diff vs previous) -15.15% 33.000 28.000 - -91.00% -
482.sphinx3 (unexplained diff vs previous) (percentage) -15.06% 3.520 2.990 - 382.26% -
453.povray (max) (percentage) -12.60% 3.730 3.260 - 28.85% -
482.sphinx3 (unexplained diff vs previous) -12.50% 32.000 28.000 - -84.95% -
473.astar (max) (percentage) -11.91% 5.540 4.880 - 3385.71% -
453.povray (max) -11.11% 18.000 16.000 - -90.30% -
450.soplex (max) (percentage) -9.91% 5.850 5.270 - 167.51% -
482.sphinx3 (max) (percentage) -9.41% 3.400 3.080 - 109.52% -
473.astar (max) -7.47% 308.000 285.000 - 1325.00% -
447.dealII (max) -7.14% 14.000 13.000 - -87.85% -
482.sphinx3 (max) -6.67% 30.000 28.000 - -93.62% -
450.soplex (max) -6.45% 31.000 29.000 - -90.46% -
429.mcf (max) (percentage) -6.42% 11.220 10.500 - 1844.44% -
401.bzip2 (unexplained diff vs previous) (percentage) -6.29% 2.860 2.680 - 208.05% -
400.perlbench (max) (percentage) -6.13% 3.260 3.060 - -5.56% -
400.perlbench (max) -6.00% 50.000 47.000 - -86.38% -
447.dealII (max) (percentage) -5.41% 0.740 0.700 - -36.36% -
435.gromacs (max) (percentage) -4.89% 3.070 2.920 - 44.55% -
456.hmmer (max) (percentage) -4.35% 0.230 0.220 - -82.68% -
401.bzip2 (unexplained diff vs previous) -3.61% 83.000 80.000 - -39.39% -
436.cactusADM -3.57% 28.000 27.000 - -90.72% -
436.cactusADM (avg) -3.57% 28.000 27.000 - -90.72% -
400.perlbench -3.23% 31.000 30.000 - -80.77% -
400.perlbench (avg) -3.23% 31.000 30.000 - -80.77% -
435.gromacs (max) -2.34% 214.000 209.000 - -32.80% -
400.perlbench (avg) (percentage) -1.95% 2.050 2.010 - 36.73% -
454.calculix (max) (percentage) -1.63% 1.230 1.210 - -74.26% -
403.gcc (max) (percentage) -1.59% 6.300 6.200 - 761.11% -
447.dealII (avg) (percentage) -1.50% 1.330 1.310 - 22.43% -
456.hmmer (avg) (percentage) -1.32% 0.760 0.750 - 10.29% -
481.wrf (max) (percentage) -1.21% 1.650 1.630 - -48.25% -
462.libquantum (avg) (percentage) -1.17% 1.710 1.690 - 160.00% -
401.bzip2 (max) (percentage) -1.08% 2.780 2.750 - 212.50% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 36.45% 8205.000 11196.000 - 1.44% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
473.astar (unexplained diff vs previous) (percentage) 23150.00% 0.020 4.650 - -20.78% -
453.povray (unexplained diff vs previous) (percentage) 9900.00% 0.020 2.000 - -48.45% -
473.astar (unexplained diff vs previous) 8966.67% 3.000 272.000 - -16.82% -
473.astar (max) (percentage) 3385.71% 0.140 4.880 - -11.91% -
471.omnetpp (unexplained diff vs previous) (percentage) 2045.83% 0.240 5.150 - -0.39% -
410.bwaves (unexplained diff vs previous) (percentage) 1870.00% 0.100 1.970 - 56.35% -
429.mcf (max) (percentage) 1844.44% 0.540 10.500 - -6.42% -
429.mcf (unexplained diff vs previous) (percentage) 1659.26% 0.540 9.500 - -24.78% -
458.sjeng (unexplained diff vs previous) (percentage) 1650.00% 0.060 1.050 - 16.67% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 1480.95% 0.210 3.320 - 6.07% -
473.astar (max) 1325.00% 20.000 285.000 - -7.47% -
483.xalancbmk (max) (percentage) 1290.24% 0.410 5.700 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) 1281.82% 0.110 1.520 - 133.85% -
464.h264ref (unexplained diff vs previous) (percentage) 1130.00% 0.100 1.230 - 12.84% -
429.mcf (avg) (percentage) 988.68% 0.530 5.770 - 1.23% -
403.gcc (avg) (percentage) 945.90% 0.610 6.380 - 0.00% -
453.povray (unexplained diff vs previous) 900.00% 1.000 10.000 - -47.37% -
473.astar (avg) (percentage) 863.16% 0.190 1.830 - 2.23% -
410.bwaves (max) (percentage) 857.14% 0.210 2.010 - 60.80% -
458.sjeng (unexplained diff vs previous) 788.89% 9.000 80.000 - 17.65% -
403.gcc (max) (percentage) 761.11% 0.720 6.200 - -1.59% -
410.bwaves (unexplained diff vs previous) 706.67% 15.000 121.000 - 59.21% -
471.omnetpp (unexplained diff vs previous) 484.62% 26.000 152.000 - 4.83% -
459.GemsFDTD (max) (percentage) 471.67% 0.600 3.430 - 12.83% -
464.h264ref (unexplained diff vs previous) 386.67% 15.000 73.000 - 14.06% -
482.sphinx3 (unexplained diff vs previous) (percentage) 382.26% 0.620 2.990 - -15.06% -
458.sjeng (max) (percentage) 381.82% 0.220 1.060 - 19.10% -
401.bzip2 (avg) (percentage) 368.18% 0.440 2.060 - 0.49% -
410.bwaves (max) 310.00% 30.000 123.000 - 64.00% -
470.lbm (unexplained diff vs previous) 307.14% 14.000 57.000 - 137.50% -
473.astar 278.57% 28.000 106.000 - 7.07% -
473.astar (avg) 278.57% 28.000 106.000 - 7.07% -
459.GemsFDTD (unexplained diff vs previous) 258.33% 48.000 172.000 - 9.55% -
437.leslie3d (max) (percentage) 256.41% 2.340 8.340 - 100.48% -
437.leslie3d (unexplained diff vs previous) (percentage) 236.24% 2.290 7.700 - 358.33% -
401.bzip2 (max) (percentage) 212.50% 0.880 2.750 - -1.08% -
401.bzip2 (unexplained diff vs previous) (percentage) 208.05% 0.870 2.680 - -6.29% -
464.h264ref (max) (percentage) 190.70% 0.430 1.250 - 15.74% -
437.leslie3d (max) 172.65% 351.000 957.000 - 117.50% -
450.soplex (max) (percentage) 167.51% 1.970 5.270 - -9.91% -
458.sjeng (max) 161.29% 31.000 81.000 - 20.90% -
462.libquantum (avg) (percentage) 160.00% 0.650 1.690 - -1.17% -
437.leslie3d (unexplained diff vs previous) 157.73% 343.000 884.000 - 396.63% -
462.libquantum (max) (percentage) 153.13% 0.640 1.620 - -62.06% -
465.tonto (unexplained diff vs previous) 152.38% 63.000 159.000 - 31.40% -
450.soplex (unexplained diff vs previous) (percentage) 149.25% 2.010 5.010 - -19.32% -
450.soplex (avg) (percentage) 141.13% 1.240 2.990 - 1.01% -
429.mcf (max) 135.87% 92.000 217.000 - 3.33% -
458.sjeng (avg) (percentage) 133.33% 0.330 0.770 - 1.32% -
429.mcf (unexplained diff vs previous) 111.83% 93.000 197.000 - -16.88% -
482.sphinx3 (max) (percentage) 109.52% 1.470 3.080 - -9.41% -
465.tonto (unexplained diff vs previous) (percentage) 84.42% 0.770 1.420 - 29.09% -
437.leslie3d (avg) (percentage) 79.41% 2.380 4.270 - 1.91% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
459.GemsFDTD (avg) (percentage) 68.89% 0.900 1.520 - 2.70% -
464.h264ref (avg) (percentage) 59.26% 0.540 0.860 - 1.18% -
471.omnetpp (avg) (percentage) 55.35% 1.590 2.470 - 1.65% -
454.calculix (avg) (percentage) 52.24% 2.010 3.060 - 0.00% -
445.gobmk (avg) (percentage) 45.45% 0.770 1.120 - 0.90% -
435.gromacs (max) (percentage) 44.55% 2.020 2.920 - -4.89% -
453.povray (avg) (percentage) 44.00% 1.250 1.800 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) 37.86% 2.060 2.840 - 13.15% -
437.leslie3d 37.25% 357.000 490.000 - 10.61% -
437.leslie3d (avg) 37.25% 357.000 490.000 - 10.61% -
400.perlbench (avg) (percentage) 36.73% 1.470 2.010 - -1.95% -
429.mcf 32.22% 90.000 119.000 - 12.26% -
429.mcf (avg) 32.22% 90.000 119.000 - 12.26% -
482.sphinx3 (avg) (percentage) 31.51% 1.460 1.920 - 1.05% -
453.povray (max) (percentage) 28.85% 2.530 3.260 - -12.60% -
459.GemsFDTD (max) 26.43% 140.000 177.000 - 16.45% -
458.sjeng 23.40% 47.000 58.000 - 1.75% -
458.sjeng (avg) 23.40% 47.000 58.000 - 1.75% -
447.dealII (avg) (percentage) 22.43% 1.070 1.310 - -1.50% -
464.h264ref (max) 19.35% 62.000 74.000 - 17.46% -
433.milc (max) (percentage) 19.24% 3.170 3.780 - 38.97% -
435.gromacs (avg) (percentage) 13.29% 1.730 1.960 - 1.03% -
433.milc (unexplained diff vs previous) (percentage) 11.28% 3.280 3.650 - 30.82% -
456.hmmer (avg) (percentage) 10.29% 0.680 0.750 - -1.32% -
410.bwaves (avg) (percentage) 7.34% 1.090 1.170 - 1.74% -
465.tonto 4.38% 137.000 143.000 - 1.42% -
465.tonto (avg) 4.38% 137.000 143.000 - 1.42% -
445.gobmk (max) (percentage) 3.27% 1.530 1.580 - -24.76% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -17.24% 29.000 24.000 - 0.00% -
nb_execution_successful -17.24% 29.000 24.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -8.10% 120221.000 110489.000 - 0.00% -
437.leslie3d -7.59% 65177.000 60228.000 - 0.00% -
459.GemsFDTD -4.56% 253716.000 242157.000 - 0.00% -
434.zeusmp -3.91% 165699.000 159221.000 - 0.00% -
464.h264ref -3.09% 428364.000 415120.000 - 0.00% -
410.bwaves -2.91% 17880.000 17360.000 - 0.00% -
453.povray -1.83% 709140.000 696190.000 - 0.00% -
435.gromacs -1.78% 679165.000 667109.000 - 0.00% -
433.milc -1.69% 95873.000 94253.000 - 0.00% -
444.namd -1.52% 191059.000 188162.000 - 0.00% -
400.perlbench -1.43% 930652.000 917322.000 - 0.00% -
470.lbm -1.43% 10908.000 10752.000 - 0.00% -
436.cactusADM -1.34% 554386.000 546946.000 - 0.00% -
450.soplex -1.20% 260647.000 257521.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.83% 11614.000 136.000 - 0.00% -
482.sphinx3 -96.87% 29928.000 937.000 - 3.08% -
450.soplex -96.39% 15466.000 559.000 - 5.27% -
453.povray -92.36% 6544.000 500.000 - 2.04% -
444.namd -91.57% 9050.000 763.000 - 1.73% -
433.milc -91.12% 12976.000 1152.000 - 3.78% -
436.cactusADM -87.93% 17544.000 2118.000 - -1.49% -
429.mcf -87.92% 17158.000 2073.000 - 10.50% -
400.perlbench -85.57% 10650.000 1537.000 - 0.00% -
434.zeusmp -80.88% 13211.000 2526.000 - 2.06% -
401.bzip2 -80.34% 15185.000 2986.000 - 2.75% -
447.dealII -80.30% 9751.000 1921.000 - -0.36% -
459.GemsFDTD -77.82% 23356.000 5180.000 - 3.43% -
456.hmmer -73.91% 11046.000 2882.000 - 0.00% -
471.omnetpp -72.49% 10739.000 2954.000 - 5.42% -
470.lbm -70.77% 12819.000 3747.000 - 1.54% -
473.astar -60.86% 14932.000 5844.000 - 4.88% -
464.h264ref -59.24% 14556.000 5933.000 - 1.25% -
410.bwaves -58.35% 14755.000 6145.000 - 2.01% -
445.gobmk -56.92% 11995.000 5167.000 - 1.53% -
435.gromacs -53.43% 15419.000 7180.000 - 2.92% -
458.sjeng -46.40% 14275.000 7652.000 - 1.06% -
437.leslie3d -23.49% 15005.000 11480.000 - 8.34% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (unexplained diff vs previous) -98.14% 700.000 13.000 - 8.33% -
444.namd (max) -98.00% 649.000 13.000 - 18.18% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
462.libquantum (max) -97.30% 74.000 2.000 - -60.00% -
444.namd -96.90% 355.000 11.000 - 0.00% -
444.namd (avg) -96.90% 355.000 11.000 - 0.00% -
482.sphinx3 -96.10% 436.000 17.000 - 0.00% -
482.sphinx3 (avg) -96.10% 436.000 17.000 - 0.00% -
416.gamess (max) (percentage) -95.85% 2.890 0.120 - 0.00% -
456.hmmer (max) -95.71% 140.000 6.000 - 0.00% -
436.cactusADM (unexplained diff vs previous) -94.39% 570.000 32.000 - 10.34% -
436.cactusADM (max) -94.38% 552.000 31.000 - 6.90% -
482.sphinx3 (max) -93.62% 439.000 28.000 - -6.67% -
433.milc -91.82% 220.000 18.000 - 5.88% -
433.milc (avg) -91.82% 220.000 18.000 - 5.88% -
450.soplex -91.62% 191.000 16.000 - 6.67% -
450.soplex (avg) -91.62% 191.000 16.000 - 6.67% -
470.lbm (max) -91.27% 653.000 57.000 - 0.00% -
450.soplex (unexplained diff vs previous) -91.00% 311.000 28.000 - -15.15% -
447.dealII (unexplained diff vs previous) -90.79% 76.000 7.000 - 40.00% -
436.cactusADM -90.72% 291.000 27.000 - -3.57% -
436.cactusADM (avg) -90.72% 291.000 27.000 - -3.57% -
450.soplex (max) -90.46% 304.000 29.000 - -6.45% -
453.povray (max) -90.30% 165.000 16.000 - -11.11% -
433.milc (unexplained diff vs previous) -90.12% 425.000 42.000 - 35.48% -
433.milc (max) -89.54% 411.000 43.000 - 43.33% -
453.povray -88.89% 81.000 9.000 - 12.50% -
453.povray (avg) -88.89% 81.000 9.000 - 12.50% -
447.dealII (max) -87.85% 107.000 13.000 - -7.14% -
400.perlbench (max) -86.38% 345.000 47.000 - -6.00% -
470.lbm -86.14% 303.000 42.000 - 0.00% -
470.lbm (avg) -86.14% 303.000 42.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -84.95% 186.000 28.000 - -12.50% -
434.zeusmp (unexplained diff vs previous) -83.44% 308.000 51.000 - 4.08% -
434.zeusmp (max) -82.72% 301.000 52.000 - 8.33% -
456.hmmer (max) (percentage) -82.68% 1.270 0.220 - -4.35% -
434.zeusmp -81.54% 195.000 36.000 - 2.86% -
434.zeusmp (avg) -81.54% 195.000 36.000 - 2.86% -
400.perlbench -80.77% 156.000 30.000 - -3.23% -
400.perlbench (avg) -80.77% 156.000 30.000 - -3.23% -
444.namd (unexplained diff vs previous) (percentage) -78.01% 7.730 1.700 - 6.25% -
447.dealII -75.96% 104.000 25.000 - 0.00% -
447.dealII (avg) -75.96% 104.000 25.000 - 0.00% -
444.namd (max) (percentage) -75.91% 7.180 1.730 - 10.19% -
471.omnetpp (max) -75.83% 662.000 160.000 - 16.79% -
454.calculix (max) (percentage) -74.26% 4.700 1.210 - -1.63% -
456.hmmer -72.00% 75.000 21.000 - 0.00% -
456.hmmer (avg) -72.00% 75.000 21.000 - 0.00% -
470.lbm (max) (percentage) -69.80% 5.100 1.540 - -0.65% -
459.GemsFDTD -62.86% 210.000 78.000 - 5.41% -
459.GemsFDTD (avg) -62.86% 210.000 78.000 - 5.41% -
416.gamess (avg) (percentage) -61.29% 2.170 0.840 - 0.00% -
444.namd (avg) (percentage) -60.05% 3.930 1.570 - 0.00% -
465.tonto (max) (percentage) -59.78% 3.580 1.440 - 9.92% -
445.gobmk (unexplained diff vs previous) -58.06% 186.000 78.000 - -28.44% -
471.omnetpp -57.65% 170.000 72.000 - 5.88% -
471.omnetpp (avg) -57.65% 170.000 72.000 - 5.88% -
445.gobmk (max) -55.74% 183.000 81.000 - -23.58% -
410.bwaves -55.62% 160.000 71.000 - 2.90% -
410.bwaves (avg) -55.62% 160.000 71.000 - 2.90% -
447.dealII (unexplained diff vs previous) (percentage) -53.85% 0.780 0.360 - 38.46% -
436.cactusADM (unexplained diff vs previous) (percentage) -53.54% 3.250 1.510 - 11.85% -
436.cactusADM (max) (percentage) -52.70% 3.150 1.490 - 8.76% -
470.lbm (avg) (percentage) -51.90% 2.370 1.140 - 0.00% -
481.wrf (max) (percentage) -48.25% 3.150 1.630 - -1.21% -
435.gromacs -47.37% 266.000 140.000 - 3.70% -
435.gromacs (avg) -47.37% 266.000 140.000 - 3.70% -
465.tonto (max) -45.05% 293.000 161.000 - 11.81% -
401.bzip2 (unexplained diff vs previous) -39.39% 132.000 80.000 - -3.61% -
401.bzip2 (max) -38.35% 133.000 82.000 - 2.50% -
445.gobmk -38.04% 92.000 57.000 - 1.79% -
445.gobmk (avg) -38.04% 92.000 57.000 - 1.79% -
447.dealII (max) (percentage) -36.36% 1.100 0.700 - -5.41% -
435.gromacs (unexplained diff vs previous) -35.85% 318.000 204.000 - 16.57% -
464.h264ref -34.62% 78.000 51.000 - 4.08% -
464.h264ref (avg) -34.62% 78.000 51.000 - 4.08% -
435.gromacs (max) -32.80% 311.000 209.000 - -2.34% -
465.tonto (avg) (percentage) -23.35% 1.670 1.280 - 0.00% -
436.cactusADM (avg) (percentage) -21.08% 1.660 1.310 - 0.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -13.30% 2.330 2.020 - 2.02% -
481.wrf (avg) (percentage) -12.17% 1.890 1.660 - 0.00% -
471.omnetpp (max) (percentage) -12.16% 6.170 5.420 - 10.16% -
434.zeusmp (max) (percentage) -9.65% 2.280 2.060 - 6.19% -
401.bzip2 -7.58% 66.000 61.000 - 3.39% -
401.bzip2 (avg) -7.58% 66.000 61.000 - 3.39% -
433.milc (avg) (percentage) -7.06% 1.700 1.580 - 1.94% -
400.perlbench (max) (percentage) -5.56% 3.240 3.060 - -6.13% -
445.gobmk (unexplained diff vs previous) (percentage) -2.58% 1.550 1.510 - -29.44% -
434.zeusmp (avg) (percentage) -2.03% 1.480 1.450 - 1.40% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 17.24% 29.000 34.000 - 0.00% -

Added Tests - sample variability
400.perlbench (unexplained diff vs previous)
400.perlbench (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.41s
Execution (seconds)
Name Current %
400.perlbench 1537.000 -
401.bzip2 2986.000 2.75%
410.bwaves 6145.000 2.01%
429.mcf 2073.000 10.50%
433.milc 1152.000 3.78%
434.zeusmp 2526.000 2.06%
435.gromacs 7180.000 2.92%
436.cactusADM 2118.000 -1.49%
437.leslie3d 11480.000 8.34%
444.namd 763.000 1.73%
445.gobmk 5167.000 1.53%
447.dealII 1921.000 -
450.soplex 559.000 5.27%
453.povray 500.000 2.04%
456.hmmer 2882.000 -
458.sjeng 7652.000 1.06%
459.GemsFDTD 5180.000 3.43%
462.libquantum 136.000 -
464.h264ref 5933.000 1.25%
465.tonto 11196.000 1.44%
470.lbm 3747.000 1.54%
471.omnetpp 2954.000 5.42%
473.astar 5844.000 4.88%
482.sphinx3 937.000 3.08%
Geometric Mean 2510.599 2.60%
Code Size (bytes)
Name Current %
400.perlbench 917322.000 -
401.bzip2 49414.000 -
410.bwaves 17360.000 -
429.mcf 10911.000 -
433.milc 94253.000 -
434.zeusmp 159221.000 -
435.gromacs 667109.000 -
436.cactusADM 546946.000 -
437.leslie3d 60228.000 -
444.namd 188162.000 -
445.gobmk 1397629.000 -
447.dealII 2099712.000 -
450.soplex 257521.000 -
453.povray 696190.000 -
456.hmmer 229084.000 -
458.sjeng 110489.000 -
459.GemsFDTD 242157.000 -
462.libquantum 31313.000 -
464.h264ref 415120.000 -
465.tonto 2874773.000 -
470.lbm 10752.000 -
471.omnetpp 565455.000 -
473.astar 31888.000 -
482.sphinx3 151844.000 -
Geometric Mean 182607.913 -
sample variability
Name Current %
400.perlbench 30.000 -3.23%
400.perlbench (avg) 30.000 -3.23%
400.perlbench (avg) (percentage) 2.010 -1.95%
400.perlbench (max) 47.000 -6.00%
400.perlbench (max) (percentage) 3.060 -6.13%
400.perlbench (unexplained diff vs previous) 0.000 -
400.perlbench (unexplained diff vs previous) (percentage) 0.000 -
401.bzip2 61.000 3.39%
401.bzip2 (avg) 61.000 3.39%
401.bzip2 (avg) (percentage) 2.060 -
401.bzip2 (max) 82.000 2.50%
401.bzip2 (max) (percentage) 2.750 -1.08%
401.bzip2 (unexplained diff vs previous) 80.000 -3.61%
401.bzip2 (unexplained diff vs previous) (percentage) 2.680 -6.29%
403.gcc (avg) (percentage) 6.380 -
403.gcc (max) (percentage) 6.200 -1.59%
410.bwaves 71.000 2.90%
410.bwaves (avg) 71.000 2.90%
410.bwaves (avg) (percentage) 1.170 1.74%
410.bwaves (max) 123.000 64.00%
410.bwaves (max) (percentage) 2.010 60.80%
410.bwaves (unexplained diff vs previous) 121.000 59.21%
410.bwaves (unexplained diff vs previous) (percentage) 1.970 56.35%
416.gamess (avg) (percentage) 0.840 -
416.gamess (max) (percentage) 0.120 -
429.mcf 119.000 12.26%
429.mcf (avg) 119.000 12.26%
429.mcf (avg) (percentage) 5.770 1.23%
429.mcf (max) 217.000 3.33%
429.mcf (max) (percentage) 10.500 -6.42%
429.mcf (unexplained diff vs previous) 197.000 -16.88%
429.mcf (unexplained diff vs previous) (percentage) 9.500 -24.78%
433.milc 18.000 5.88%
433.milc (avg) 18.000 5.88%
433.milc (avg) (percentage) 1.580 1.94%
433.milc (max) 43.000 43.33%
433.milc (max) (percentage) 3.780 38.97%
433.milc (unexplained diff vs previous) 42.000 35.48%
433.milc (unexplained diff vs previous) (percentage) 3.650 30.82%
434.zeusmp 36.000 2.86%
434.zeusmp (avg) 36.000 2.86%
434.zeusmp (avg) (percentage) 1.450 1.40%
434.zeusmp (max) 52.000 8.33%
434.zeusmp (max) (percentage) 2.060 6.19%
434.zeusmp (unexplained diff vs previous) 51.000 4.08%
434.zeusmp (unexplained diff vs previous) (percentage) 2.020 2.02%
435.gromacs 140.000 3.70%
435.gromacs (avg) 140.000 3.70%
435.gromacs (avg) (percentage) 1.960 1.03%
435.gromacs (max) 209.000 -2.34%
435.gromacs (max) (percentage) 2.920 -4.89%
435.gromacs (unexplained diff vs previous) 204.000 16.57%
435.gromacs (unexplained diff vs previous) (percentage) 2.840 13.15%
436.cactusADM 27.000 -3.57%
436.cactusADM (avg) 27.000 -3.57%
436.cactusADM (avg) (percentage) 1.310 -
436.cactusADM (max) 31.000 6.90%
436.cactusADM (max) (percentage) 1.490 8.76%
436.cactusADM (unexplained diff vs previous) 32.000 10.34%
436.cactusADM (unexplained diff vs previous) (percentage) 1.510 11.85%
437.leslie3d 490.000 10.61%
437.leslie3d (avg) 490.000 10.61%
437.leslie3d (avg) (percentage) 4.270 1.91%
437.leslie3d (max) 957.000 117.50%
437.leslie3d (max) (percentage) 8.340 100.48%
437.leslie3d (unexplained diff vs previous) 884.000 396.63%
437.leslie3d (unexplained diff vs previous) (percentage) 7.700 358.33%
444.namd 11.000 -
444.namd (avg) 11.000 -
444.namd (avg) (percentage) 1.570 -
444.namd (max) 13.000 18.18%
444.namd (max) (percentage) 1.730 10.19%
444.namd (unexplained diff vs previous) 13.000 8.33%
444.namd (unexplained diff vs previous) (percentage) 1.700 6.25%
445.gobmk 57.000 1.79%
445.gobmk (avg) 57.000 1.79%
445.gobmk (avg) (percentage) 1.120 -
445.gobmk (max) 81.000 -23.58%
445.gobmk (max) (percentage) 1.580 -24.76%
445.gobmk (unexplained diff vs previous) 78.000 -28.44%
445.gobmk (unexplained diff vs previous) (percentage) 1.510 -29.44%
447.dealII 25.000 -
447.dealII (avg) 25.000 -
447.dealII (avg) (percentage) 1.310 -1.50%
447.dealII (max) 13.000 -7.14%
447.dealII (max) (percentage) 0.700 -5.41%
447.dealII (unexplained diff vs previous) 7.000 40.00%
447.dealII (unexplained diff vs previous) (percentage) 0.360 38.46%
450.soplex 16.000 6.67%
450.soplex (avg) 16.000 6.67%
450.soplex (avg) (percentage) 2.990 1.01%
450.soplex (max) 29.000 -6.45%
450.soplex (max) (percentage) 5.270 -9.91%
450.soplex (unexplained diff vs previous) 28.000 -15.15%
450.soplex (unexplained diff vs previous) (percentage) 5.010 -19.32%
453.povray 9.000 12.50%
453.povray (avg) 9.000 12.50%
453.povray (avg) (percentage) 1.800 -
453.povray (max) 16.000 -11.11%
453.povray (max) (percentage) 3.260 -12.60%
453.povray (unexplained diff vs previous) 10.000 -47.37%
453.povray (unexplained diff vs previous) (percentage) 2.000 -48.45%
454.calculix (avg) (percentage) 3.060 -
454.calculix (max) (percentage) 1.210 -1.63%
456.hmmer 21.000 -
456.hmmer (avg) 21.000 -
456.hmmer (avg) (percentage) 0.750 -1.32%
456.hmmer (max) 6.000 -
456.hmmer (max) (percentage) 0.220 -4.35%
456.hmmer (unexplained diff vs previous) 0.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.000 -
458.sjeng 58.000 1.75%
458.sjeng (avg) 58.000 1.75%
458.sjeng (avg) (percentage) 0.770 1.32%
458.sjeng (max) 81.000 20.90%
458.sjeng (max) (percentage) 1.060 19.10%
458.sjeng (unexplained diff vs previous) 80.000 17.65%
458.sjeng (unexplained diff vs previous) (percentage) 1.050 16.67%
459.GemsFDTD 78.000 5.41%
459.GemsFDTD (avg) 78.000 5.41%
459.GemsFDTD (avg) (percentage) 1.520 2.70%
459.GemsFDTD (max) 177.000 16.45%
459.GemsFDTD (max) (percentage) 3.430 12.83%
459.GemsFDTD (unexplained diff vs previous) 172.000 9.55%
459.GemsFDTD (unexplained diff vs previous) (percentage) 3.320 6.07%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.690 -1.17%
462.libquantum (max) 2.000 -60.00%
462.libquantum (max) (percentage) 1.620 -62.06%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 51.000 4.08%
464.h264ref (avg) 51.000 4.08%
464.h264ref (avg) (percentage) 0.860 1.18%
464.h264ref (max) 74.000 17.46%
464.h264ref (max) (percentage) 1.250 15.74%
464.h264ref (unexplained diff vs previous) 73.000 14.06%
464.h264ref (unexplained diff vs previous) (percentage) 1.230 12.84%
465.tonto 143.000 1.42%
465.tonto (avg) 143.000 1.42%
465.tonto (avg) (percentage) 1.280 -
465.tonto (max) 161.000 11.81%
465.tonto (max) (percentage) 1.440 9.92%
465.tonto (unexplained diff vs previous) 159.000 31.40%
465.tonto (unexplained diff vs previous) (percentage) 1.420 29.09%
470.lbm 42.000 -
470.lbm (avg) 42.000 -
470.lbm (avg) (percentage) 1.140 -
470.lbm (max) 57.000 -
470.lbm (max) (percentage) 1.540 -
470.lbm (unexplained diff vs previous) 57.000 137.50%
470.lbm (unexplained diff vs previous) (percentage) 1.520 133.85%
471.omnetpp 72.000 5.88%
471.omnetpp (avg) 72.000 5.88%
471.omnetpp (avg) (percentage) 2.470 1.65%
471.omnetpp (max) 160.000 16.79%
471.omnetpp (max) (percentage) 5.420 10.16%
471.omnetpp (unexplained diff vs previous) 152.000 4.83%
471.omnetpp (unexplained diff vs previous) (percentage) 5.150 -
473.astar 106.000 7.07%
473.astar (avg) 106.000 7.07%
473.astar (avg) (percentage) 1.830 2.23%
473.astar (max) 285.000 -7.47%
473.astar (max) (percentage) 4.880 -11.91%
473.astar (unexplained diff vs previous) 272.000 -16.82%
473.astar (unexplained diff vs previous) (percentage) 4.650 -20.78%
481.wrf (avg) (percentage) 1.660 -
481.wrf (max) (percentage) 1.630 -1.21%
482.sphinx3 17.000 -
482.sphinx3 (avg) 17.000 -
482.sphinx3 (avg) (percentage) 1.920 1.05%
482.sphinx3 (max) 28.000 -6.67%
482.sphinx3 (max) (percentage) 3.080 -9.41%
482.sphinx3 (unexplained diff vs previous) 28.000 -12.50%
482.sphinx3 (unexplained diff vs previous) (percentage) 2.990 -15.06%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.700 -
Geometric Mean 7.717 -29.07%