Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e2a26339456487e80344a8c3bd0f10711ff28567 (+ 189 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/061a82fa2b751b42d0d8ddfcd45367c848d3ee64 (unchanged)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=4514f82929819b933f1dd90fea36c346e6de3bb8 (+ 72 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=68eca432f6e41c1de6f4a7af6d60d9daa9f5eba1 (+ 210 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/176/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0176 (3-changed-components) 2023-12-29T16:56:19 0:00:00
Previous 0170 (3-changed-components) 2023-12-18T21:18:38 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 22 70
Performance Improvements 114 149
Added Tests 22 6
Unchanged Tests 1078 1011
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
471.omnetpp 3.96% 2906.000 3021.000 - -71.87% -
450.soplex 1.95% 565.000 576.000 - -96.28% -
400.perlbench 1.91% 1572.000 1602.000 - -84.96% -
447.dealII 1.48% 1953.000 1982.000 - -79.67% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (unexplained diff vs previous) 266.67% 3.000 11.000 - -96.46% -
450.soplex (unexplained diff vs previous) (percentage) 260.38% 0.530 1.910 - -4.98% -
435.gromacs (max) (percentage) 119.18% 0.730 1.600 - -20.79% -
435.gromacs (max) 117.31% 52.000 113.000 - -63.67% -
453.povray (unexplained diff vs previous) 100.00% 1.000 2.000 - 100.00% -
453.povray (unexplained diff vs previous) (percentage) 100.00% 0.200 0.400 - 1900.00% -
444.namd (unexplained diff vs previous) 66.67% 3.000 5.000 - -99.29% -
444.namd (unexplained diff vs previous) (percentage) 65.85% 0.410 0.680 - -91.20% -
471.omnetpp (unexplained diff vs previous) 43.75% 80.000 115.000 - 342.31% -
471.omnetpp (unexplained diff vs previous) (percentage) 38.55% 2.750 3.810 - 1487.50% -
447.dealII (unexplained diff vs previous) 38.10% 21.000 29.000 - -61.84% -
447.dealII (unexplained diff vs previous) (percentage) 35.19% 1.080 1.460 - 87.18% -
471.omnetpp (max) 32.22% 90.000 119.000 - -82.02% -
471.omnetpp (max) (percentage) 26.92% 3.120 3.960 - -35.82% -
444.namd (max) 25.00% 4.000 5.000 - -99.23% -
471.omnetpp 5.88% 68.000 72.000 - -57.65% -
471.omnetpp (avg) 5.88% 68.000 72.000 - -57.65% -
471.omnetpp (avg) (percentage) 1.27% 2.370 2.400 - 50.94% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -2.16% 139.000 136.000 - -98.83% -
403.gcc -1.79% 56.000 55.000 - -99.39% -
435.gromacs -1.60% 7209.000 7094.000 - -53.99% -
401.bzip2 -1.38% 3050.000 3008.000 - -80.19% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
473.astar (unexplained diff vs previous) -98.51% 269.000 4.000 - 33.33% -
473.astar (unexplained diff vs previous) (percentage) -98.47% 4.570 0.070 - 250.00% -
401.bzip2 (unexplained diff vs previous) -84.33% 268.000 42.000 - -68.18% -
401.bzip2 (unexplained diff vs previous) (percentage) -84.07% 8.790 1.400 - 60.92% -
445.gobmk (unexplained diff vs previous) -82.35% 68.000 12.000 - -93.55% -
445.gobmk (unexplained diff vs previous) (percentage) -82.31% 1.300 0.230 - -85.16% -
458.sjeng (unexplained diff vs previous) (percentage) -76.77% 0.990 0.230 - 283.33% -
458.sjeng (unexplained diff vs previous) -76.32% 76.000 18.000 - 100.00% -
433.milc (unexplained diff vs previous) -70.00% 10.000 3.000 - -99.29% -
433.milc (unexplained diff vs previous) (percentage) -69.77% 0.860 0.260 - -92.07% -
470.lbm (unexplained diff vs previous) -69.70% 33.000 10.000 - -28.57% -
470.lbm (unexplained diff vs previous) (percentage) -69.32% 0.880 0.270 - 145.45% -
433.milc (max) -57.89% 19.000 8.000 - -98.05% -
433.milc (max) (percentage) -56.63% 1.660 0.720 - -77.29% -
482.sphinx3 (unexplained diff vs previous) (percentage) -50.39% 1.290 0.640 - 3.23% -
454.calculix (max) -50.00% 2.000 1.000 - -99.91% -
482.sphinx3 (unexplained diff vs previous) -50.00% 12.000 6.000 - -96.77% -
416.gamess (max) -45.16% 31.000 17.000 - -97.41% -
416.gamess (max) (percentage) -44.00% 0.250 0.140 - -95.16% -
462.libquantum (unexplained diff vs previous) -40.00% 5.000 3.000 - 200.00% -
462.libquantum (unexplained diff vs previous) (percentage) -38.61% 3.600 2.210 - 22000.00% -
454.calculix (max) (percentage) -29.89% 4.450 3.120 - -33.62% -
462.libquantum (max) -20.00% 5.000 4.000 - -94.59% -
482.sphinx3 (max) -16.67% 12.000 10.000 - -97.72% -
462.libquantum (max) (percentage) -16.62% 3.730 3.110 - 385.94% -
482.sphinx3 (max) (percentage) -12.98% 1.310 1.140 - -22.45% -
456.hmmer (max) -11.11% 18.000 16.000 - -88.57% -
456.hmmer (unexplained diff vs previous) -11.11% 18.000 16.000 - -80.72% -
456.hmmer (unexplained diff vs previous) (percentage) -9.84% 0.610 0.550 - -26.67% -
453.povray (max) -9.09% 11.000 10.000 - -93.94% -
470.lbm (max) -9.09% 33.000 30.000 - -95.41% -
447.dealII (max) (percentage) -8.67% 1.730 1.580 - 43.64% -
473.astar (max) -8.54% 281.000 257.000 - 1185.00% -
473.astar (max) (percentage) -8.37% 4.780 4.380 - 3028.57% -
470.lbm (max) (percentage) -7.87% 0.890 0.820 - -83.92% -
410.bwaves (max) -7.69% 13.000 12.000 - -60.00% -
401.bzip2 (max) -7.51% 293.000 271.000 - 103.76% -
436.cactusADM (max) -6.67% 15.000 14.000 - -97.46% -
456.hmmer (max) (percentage) -6.45% 0.620 0.580 - -54.33% -
458.sjeng (max) (percentage) -6.25% 1.280 1.200 - 445.45% -
401.bzip2 (max) (percentage) -6.23% 9.630 9.030 - 926.14% -
458.sjeng (max) -6.12% 98.000 92.000 - 196.77% -
400.perlbench (max) (percentage) -6.08% 1.480 1.390 - -57.10% -
447.dealII (max) -6.06% 33.000 31.000 - -71.03% -
429.mcf (max) -5.92% 355.000 334.000 - 263.04% -
445.gobmk (max) (percentage) -5.84% 1.370 1.290 - -15.69% -
429.mcf (max) (percentage) -5.79% 23.330 21.980 - 3970.37% -
464.h264ref (max) (percentage) -5.77% 1.040 0.980 - 127.91% -
445.gobmk (max) -5.63% 71.000 67.000 - -63.39% -
433.milc -5.00% 20.000 19.000 - -91.36% -
433.milc (avg) -5.00% 20.000 19.000 - -91.36% -
464.h264ref (max) -4.92% 61.000 58.000 - -6.45% -
453.povray (max) (percentage) -4.89% 2.250 2.140 - -15.42% -
450.soplex (max) -4.76% 21.000 20.000 - -93.42% -
410.bwaves (max) (percentage) -4.55% 0.220 0.210 - 0.00% -
450.soplex (max) (percentage) -4.49% 3.790 3.620 - 83.76% -
400.perlbench (max) -4.35% 23.000 22.000 - -93.62% -
416.gamess -3.12% 128.000 124.000 - -74.80% -
416.gamess (avg) -3.12% 128.000 124.000 - -74.80% -
416.gamess (avg) (percentage) -2.94% 1.020 0.990 - -54.38% -
436.cactusADM (max) (percentage) -2.78% 0.720 0.700 - -77.78% -
481.wrf (max) -2.78% 36.000 35.000 - -93.40% -
454.calculix (avg) (percentage) -2.62% 3.430 3.340 - 66.17% -
459.GemsFDTD (max) (percentage) -2.56% 0.780 0.760 - 26.67% -
465.tonto -2.55% 157.000 153.000 - 11.68% -
465.tonto (avg) -2.55% 157.000 153.000 - 11.68% -
434.zeusmp (avg) (percentage) -2.34% 1.710 1.670 - 12.84% -
434.zeusmp -2.33% 43.000 42.000 - -78.46% -
434.zeusmp (avg) -2.33% 43.000 42.000 - -78.46% -
465.tonto (avg) (percentage) -2.19% 1.370 1.340 - -19.76% -
481.wrf (avg) (percentage) -2.19% 1.830 1.790 - -5.29% -
429.mcf -2.15% 93.000 91.000 - 1.11% -
429.mcf (avg) -2.15% 93.000 91.000 - 1.11% -
459.GemsFDTD (max) -2.13% 47.000 46.000 - -67.14% -
435.gromacs -2.04% 147.000 144.000 - -45.86% -
435.gromacs (avg) -2.04% 147.000 144.000 - -45.86% -
459.GemsFDTD -2.04% 98.000 96.000 - -54.29% -
459.GemsFDTD (avg) -2.04% 98.000 96.000 - -54.29% -
470.lbm -2.04% 49.000 48.000 - -84.16% -
470.lbm (avg) -2.04% 49.000 48.000 - -84.16% -
403.gcc (max) (percentage) -1.99% 5.030 4.930 - 584.72% -
481.wrf -1.96% 102.000 100.000 - -68.55% -
481.wrf (avg) -1.96% 102.000 100.000 - -68.55% -
453.povray (avg) (percentage) -1.94% 2.060 2.020 - 61.60% -
473.astar -1.85% 108.000 106.000 - 278.57% -
473.astar (avg) -1.85% 108.000 106.000 - 278.57% -
437.leslie3d (avg) (percentage) -1.75% 4.580 4.500 - 89.08% -
429.mcf (avg) (percentage) -1.63% 6.130 6.030 - 1037.74% -
473.astar (avg) (percentage) -1.63% 1.840 1.810 - 852.63% -
445.gobmk (avg) (percentage) -1.61% 1.240 1.220 - 58.44% -
410.bwaves (avg) (percentage) -1.56% 1.280 1.260 - 15.60% -
445.gobmk -1.56% 64.000 63.000 - -31.52% -
445.gobmk (avg) -1.56% 64.000 63.000 - -31.52% -
481.wrf (max) (percentage) -1.54% 0.650 0.640 - -79.68% -
470.lbm (avg) (percentage) -1.52% 1.320 1.300 - -45.15% -
458.sjeng -1.52% 66.000 65.000 - 38.30% -
458.sjeng (avg) -1.52% 66.000 65.000 - 38.30% -
401.bzip2 -1.47% 68.000 67.000 - 1.52% -
401.bzip2 (avg) -1.47% 68.000 67.000 - 1.52% -
482.sphinx3 (avg) (percentage) -1.42% 2.110 2.080 - 42.47% -
436.cactusADM (avg) (percentage) -1.30% 1.540 1.520 - -8.43% -
410.bwaves -1.30% 77.000 76.000 - -52.50% -
410.bwaves (avg) -1.30% 77.000 76.000 - -52.50% -
459.GemsFDTD (avg) (percentage) -1.24% 1.610 1.590 - 76.67% -
433.milc (avg) (percentage) -1.16% 1.720 1.700 - 0.00% -
458.sjeng (avg) (percentage) -1.16% 0.860 0.850 - 157.58% -
444.namd (avg) (percentage) -1.06% 1.890 1.870 - -52.42% -
437.leslie3d -1.02% 491.000 486.000 - 36.13% -
437.leslie3d (avg) -1.02% 491.000 486.000 - 36.13% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 3.45% 29.000 30.000 - 3.45% -

Added Tests - sample variability
410.bwaves (unexplained diff vs previous)
410.bwaves (unexplained diff vs previous) (percentage)
416.gamess (unexplained diff vs previous)
416.gamess (unexplained diff vs previous) (percentage)
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
434.zeusmp (unexplained diff vs previous)
434.zeusmp (unexplained diff vs previous) (percentage)
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
436.cactusADM (unexplained diff vs previous)
436.cactusADM (unexplained diff vs previous) (percentage)
437.leslie3d (unexplained diff vs previous)
437.leslie3d (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)
459.GemsFDTD (unexplained diff vs previous)
459.GemsFDTD (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)
481.wrf (unexplained diff vs previous)
481.wrf (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
456.hmmer 1.42% 231209.000 234488.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 39.83% 8205.000 11473.000 - -0.10% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 22000.00% 0.010 2.210 - -38.61% -
429.mcf (max) (percentage) 3970.37% 0.540 21.980 - -5.79% -
473.astar (max) (percentage) 3028.57% 0.140 4.380 - -8.37% -
453.povray (unexplained diff vs previous) (percentage) 1900.00% 0.020 0.400 - 100.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 1487.50% 0.240 3.810 - 38.55% -
483.xalancbmk (max) (percentage) 1309.76% 0.410 5.780 - 0.00% -
473.astar (max) 1185.00% 20.000 257.000 - -8.54% -
429.mcf (avg) (percentage) 1037.74% 0.530 6.030 - -1.63% -
401.bzip2 (max) (percentage) 926.14% 0.880 9.030 - -6.23% -
403.gcc (avg) (percentage) 880.33% 0.610 5.980 - 0.00% -
473.astar (avg) (percentage) 852.63% 0.190 1.810 - -1.63% -
403.gcc (max) (percentage) 584.72% 0.720 4.930 - -1.99% -
458.sjeng (max) (percentage) 445.45% 0.220 1.200 - -6.25% -
401.bzip2 (avg) (percentage) 411.36% 0.440 2.250 - -0.44% -
462.libquantum (max) (percentage) 385.94% 0.640 3.110 - -16.62% -
471.omnetpp (unexplained diff vs previous) 342.31% 26.000 115.000 - 43.75% -
458.sjeng (unexplained diff vs previous) (percentage) 283.33% 0.060 0.230 - -76.77% -
473.astar 278.57% 28.000 106.000 - -1.85% -
473.astar (avg) 278.57% 28.000 106.000 - -1.85% -
429.mcf (max) 263.04% 92.000 334.000 - -5.92% -
473.astar (unexplained diff vs previous) (percentage) 250.00% 0.020 0.070 - -98.47% -
462.libquantum (unexplained diff vs previous) 200.00% 1.000 3.000 - -40.00% -
458.sjeng (max) 196.77% 31.000 92.000 - -6.12% -
462.libquantum (avg) (percentage) 184.62% 0.650 1.850 - 0.00% -
458.sjeng (avg) (percentage) 157.58% 0.330 0.850 - -1.16% -
450.soplex (avg) (percentage) 148.39% 1.240 3.080 - -0.65% -
470.lbm (unexplained diff vs previous) (percentage) 145.45% 0.110 0.270 - -69.32% -
464.h264ref (max) (percentage) 127.91% 0.430 0.980 - -5.77% -
437.leslie3d (max) (percentage) 126.07% 2.340 5.290 - -0.38% -
401.bzip2 (max) 103.76% 133.000 271.000 - -7.51% -
453.povray (unexplained diff vs previous) 100.00% 1.000 2.000 - 100.00% -
458.sjeng (unexplained diff vs previous) 100.00% 9.000 18.000 - -76.32% -
437.leslie3d (avg) (percentage) 89.08% 2.380 4.500 - -1.75% -
464.h264ref (avg) (percentage) 88.89% 0.540 1.020 - 0.00% -
447.dealII (unexplained diff vs previous) (percentage) 87.18% 0.780 1.460 - 35.19% -
450.soplex (max) (percentage) 83.76% 1.970 3.620 - -4.49% -
459.GemsFDTD (avg) (percentage) 76.67% 0.900 1.590 - -1.24% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 71.43% 0.210 0.360 - 0.00% -
454.calculix (avg) (percentage) 66.17% 2.010 3.340 - -2.62% -
437.leslie3d (max) 62.96% 351.000 572.000 - 0.53% -
453.povray (avg) (percentage) 61.60% 1.250 2.020 - -1.94% -
401.bzip2 (unexplained diff vs previous) (percentage) 60.92% 0.870 1.400 - -84.07% -
445.gobmk (avg) (percentage) 58.44% 0.770 1.220 - -1.61% -
400.perlbench (avg) (percentage) 52.38% 1.470 2.240 - 0.00% -
471.omnetpp (avg) (percentage) 50.94% 1.590 2.400 - 1.27% -
447.dealII (avg) (percentage) 43.93% 1.070 1.540 - 0.00% -
447.dealII (max) (percentage) 43.64% 1.100 1.580 - -8.67% -
482.sphinx3 (avg) (percentage) 42.47% 1.460 2.080 - -1.42% -
458.sjeng 38.30% 47.000 65.000 - -1.52% -
458.sjeng (avg) 38.30% 47.000 65.000 - -1.52% -
456.hmmer (avg) (percentage) 36.76% 0.680 0.930 - -1.06% -
437.leslie3d 36.13% 357.000 486.000 - -1.02% -
437.leslie3d (avg) 36.13% 357.000 486.000 - -1.02% -
473.astar (unexplained diff vs previous) 33.33% 3.000 4.000 - -98.51% -
459.GemsFDTD (max) (percentage) 26.67% 0.600 0.760 - -2.56% -
435.gromacs (avg) (percentage) 17.34% 1.730 2.030 - -0.49% -
410.bwaves (avg) (percentage) 15.60% 1.090 1.260 - -1.56% -
434.zeusmp (avg) (percentage) 12.84% 1.480 1.670 - -2.34% -
465.tonto 11.68% 137.000 153.000 - -2.55% -
465.tonto (avg) 11.68% 137.000 153.000 - -2.55% -
482.sphinx3 (unexplained diff vs previous) (percentage) 3.23% 0.620 0.640 - -50.39% -
401.bzip2 1.52% 66.000 67.000 - -1.47% -
401.bzip2 (avg) 1.52% 66.000 67.000 - -1.47% -
429.mcf 1.11% 90.000 91.000 - -2.15% -
429.mcf (avg) 1.11% 90.000 91.000 - -2.15% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
464.h264ref -2.43% 428364.000 417940.000 - 0.00% -
434.zeusmp -1.73% 165699.000 162829.000 - 0.00% -
410.bwaves -1.34% 17880.000 17640.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.72% 22860.000 63.000 - 0.00% -
403.gcc -99.39% 9010.000 55.000 - -1.79% -
462.libquantum -98.83% 11614.000 136.000 - -2.16% -
482.sphinx3 -96.87% 29928.000 936.000 - 0.65% -
450.soplex -96.28% 15466.000 576.000 - 1.95% -
453.povray -92.36% 6544.000 500.000 - 0.40% -
444.namd -91.90% 9050.000 733.000 - 0.69% -
429.mcf -91.12% 17158.000 1523.000 - 0.00% -
433.milc -91.05% 12976.000 1161.000 - -0.26% -
436.cactusADM -87.98% 17544.000 2108.000 - 0.48% -
400.perlbench -84.96% 10650.000 1602.000 - 1.91% -
434.zeusmp -80.52% 13211.000 2574.000 - 0.12% -
401.bzip2 -80.19% 15185.000 3008.000 - -1.38% -
447.dealII -79.67% 9751.000 1982.000 - 1.48% -
459.GemsFDTD -73.89% 23356.000 6099.000 - -0.36% -
456.hmmer -73.60% 11046.000 2916.000 - -0.55% -
471.omnetpp -71.87% 10739.000 3021.000 - 3.96% -
470.lbm -70.92% 12819.000 3728.000 - -0.27% -
481.wrf -66.78% 16834.000 5593.000 - -0.25% -
473.astar -60.57% 14932.000 5887.000 - -0.07% -
464.h264ref -59.27% 14556.000 5929.000 - 0.10% -
410.bwaves -59.12% 14755.000 6032.000 - -0.05% -
445.gobmk -56.43% 11995.000 5226.000 - 0.23% -
435.gromacs -53.99% 15419.000 7094.000 - -1.60% -
458.sjeng -45.98% 14275.000 7711.000 - 0.23% -
416.gamess -44.72% 22704.000 12550.000 - 0.00% -
437.leslie3d -27.92% 15005.000 10816.000 - 0.84% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.91% 1074.000 1.000 - -50.00% -
454.calculix -99.56% 459.000 2.000 - 0.00% -
454.calculix (avg) -99.56% 459.000 2.000 - 0.00% -
433.milc (unexplained diff vs previous) -99.29% 425.000 3.000 - -70.00% -
444.namd (unexplained diff vs previous) -99.29% 700.000 5.000 - 66.67% -
444.namd (max) -99.23% 649.000 5.000 - 25.00% -
434.zeusmp (unexplained diff vs previous) -99.03% 308.000 3.000 - 0.00% -
436.cactusADM (unexplained diff vs previous) -98.25% 570.000 10.000 - 0.00% -
433.milc (max) -98.05% 411.000 8.000 - -57.89% -
482.sphinx3 (max) -97.72% 439.000 10.000 - -16.67% -
436.cactusADM (max) -97.46% 552.000 14.000 - -6.67% -
416.gamess (max) -97.41% 656.000 17.000 - -45.16% -
462.libquantum -97.33% 75.000 2.000 - 0.00% -
462.libquantum (avg) -97.33% 75.000 2.000 - 0.00% -
403.gcc (max) -96.88% 64.000 2.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -96.77% 186.000 6.000 - -50.00% -
450.soplex (unexplained diff vs previous) -96.46% 311.000 11.000 - 266.67% -
444.namd -96.34% 355.000 13.000 - 0.00% -
444.namd (avg) -96.34% 355.000 13.000 - 0.00% -
482.sphinx3 -95.64% 436.000 19.000 - 0.00% -
482.sphinx3 (avg) -95.64% 436.000 19.000 - 0.00% -
470.lbm (max) -95.41% 653.000 30.000 - -9.09% -
416.gamess (max) (percentage) -95.16% 2.890 0.140 - -44.00% -
434.zeusmp (unexplained diff vs previous) (percentage) -94.85% 2.330 0.120 - 0.00% -
462.libquantum (max) -94.59% 74.000 4.000 - -20.00% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
453.povray (max) -93.94% 165.000 10.000 - -9.09% -
400.perlbench (max) -93.62% 345.000 22.000 - -4.35% -
445.gobmk (unexplained diff vs previous) -93.55% 186.000 12.000 - -82.35% -
450.soplex (max) -93.42% 304.000 20.000 - -4.76% -
481.wrf (max) -93.40% 530.000 35.000 - -2.78% -
433.milc (unexplained diff vs previous) (percentage) -92.07% 3.280 0.260 - -69.77% -
433.milc -91.36% 220.000 19.000 - -5.00% -
433.milc (avg) -91.36% 220.000 19.000 - -5.00% -
444.namd (unexplained diff vs previous) (percentage) -91.20% 7.730 0.680 - 65.85% -
450.soplex -91.10% 191.000 17.000 - 0.00% -
450.soplex (avg) -91.10% 191.000 17.000 - 0.00% -
444.namd (max) (percentage) -90.39% 7.180 0.690 - 1.47% -
434.zeusmp (max) -90.37% 301.000 29.000 - 0.00% -
436.cactusADM -89.00% 291.000 32.000 - 0.00% -
436.cactusADM (avg) -89.00% 291.000 32.000 - 0.00% -
456.hmmer (max) -88.57% 140.000 16.000 - -11.11% -
453.povray -87.65% 81.000 10.000 - 0.00% -
453.povray (avg) -87.65% 81.000 10.000 - 0.00% -
481.wrf (unexplained diff vs previous) -87.50% 112.000 14.000 - 0.00% -
465.tonto (unexplained diff vs previous) (percentage) -87.01% 0.770 0.100 - 0.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -85.54% 3.250 0.470 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) -85.16% 1.550 0.230 - -82.31% -
470.lbm -84.16% 303.000 48.000 - -2.04% -
470.lbm (avg) -84.16% 303.000 48.000 - -2.04% -
470.lbm (max) (percentage) -83.92% 5.100 0.820 - -7.87% -
465.tonto (unexplained diff vs previous) -82.54% 63.000 11.000 - 0.00% -
471.omnetpp (max) -82.02% 662.000 119.000 - 32.22% -
456.hmmer (unexplained diff vs previous) -80.72% 83.000 16.000 - -11.11% -
410.bwaves (unexplained diff vs previous) -80.00% 15.000 3.000 - 0.00% -
481.wrf (max) (percentage) -79.68% 3.150 0.640 - -1.54% -
434.zeusmp -78.46% 195.000 42.000 - -2.33% -
434.zeusmp (avg) -78.46% 195.000 42.000 - -2.33% -
436.cactusADM (max) (percentage) -77.78% 3.150 0.700 - -2.78% -
400.perlbench -77.56% 156.000 35.000 - 0.00% -
400.perlbench (avg) -77.56% 156.000 35.000 - 0.00% -
433.milc (max) (percentage) -77.29% 3.170 0.720 - -56.63% -
416.gamess -74.80% 492.000 124.000 - -3.12% -
416.gamess (avg) -74.80% 492.000 124.000 - -3.12% -
437.leslie3d (unexplained diff vs previous) -73.76% 343.000 90.000 - 0.00% -
447.dealII -71.15% 104.000 30.000 - 0.00% -
447.dealII (avg) -71.15% 104.000 30.000 - 0.00% -
447.dealII (max) -71.03% 107.000 31.000 - -6.06% -
481.wrf -68.55% 318.000 100.000 - -1.96% -
481.wrf (avg) -68.55% 318.000 100.000 - -1.96% -
401.bzip2 (unexplained diff vs previous) -68.18% 132.000 42.000 - -84.33% -
459.GemsFDTD (max) -67.14% 140.000 46.000 - -2.13% -
456.hmmer -64.00% 75.000 27.000 - 0.00% -
456.hmmer (avg) -64.00% 75.000 27.000 - 0.00% -
435.gromacs (unexplained diff vs previous) -63.84% 318.000 115.000 - 0.00% -
437.leslie3d (unexplained diff vs previous) (percentage) -63.76% 2.290 0.830 - 0.00% -
435.gromacs (max) -63.67% 311.000 113.000 - 117.31% -
445.gobmk (max) -63.39% 183.000 67.000 - -5.63% -
481.wrf (unexplained diff vs previous) (percentage) -62.69% 0.670 0.250 - 0.00% -
447.dealII (unexplained diff vs previous) -61.84% 76.000 29.000 - 38.10% -
410.bwaves (max) -60.00% 30.000 12.000 - -7.69% -
471.omnetpp -57.65% 170.000 72.000 - 5.88% -
471.omnetpp (avg) -57.65% 170.000 72.000 - 5.88% -
400.perlbench (max) (percentage) -57.10% 3.240 1.390 - -6.08% -
416.gamess (avg) (percentage) -54.38% 2.170 0.990 - -2.94% -
456.hmmer (max) (percentage) -54.33% 1.270 0.580 - -6.45% -
459.GemsFDTD -54.29% 210.000 96.000 - -2.04% -
459.GemsFDTD (avg) -54.29% 210.000 96.000 - -2.04% -
459.GemsFDTD (unexplained diff vs previous) -54.17% 48.000 22.000 - 0.00% -
465.tonto (max) (percentage) -52.51% 3.580 1.700 - -0.58% -
410.bwaves -52.50% 160.000 76.000 - -1.30% -
410.bwaves (avg) -52.50% 160.000 76.000 - -1.30% -
444.namd (avg) (percentage) -52.42% 3.930 1.870 - -1.06% -
434.zeusmp (max) (percentage) -50.44% 2.280 1.130 - -0.88% -
410.bwaves (unexplained diff vs previous) (percentage) -50.00% 0.100 0.050 - 0.00% -
435.gromacs -45.86% 266.000 144.000 - -2.04% -
435.gromacs (avg) -45.86% 266.000 144.000 - -2.04% -
470.lbm (avg) (percentage) -45.15% 2.370 1.300 - -1.52% -
471.omnetpp (max) (percentage) -35.82% 6.170 3.960 - 26.92% -
454.calculix (max) (percentage) -33.62% 4.700 3.120 - -29.89% -
465.tonto (max) -33.45% 293.000 195.000 - -0.51% -
445.gobmk -31.52% 92.000 63.000 - -1.56% -
445.gobmk (avg) -31.52% 92.000 63.000 - -1.56% -
470.lbm (unexplained diff vs previous) -28.57% 14.000 10.000 - -69.70% -
456.hmmer (unexplained diff vs previous) (percentage) -26.67% 0.750 0.550 - -9.84% -
464.h264ref -23.08% 78.000 60.000 - 0.00% -
464.h264ref (avg) -23.08% 78.000 60.000 - 0.00% -
482.sphinx3 (max) (percentage) -22.45% 1.470 1.140 - -12.98% -
435.gromacs (unexplained diff vs previous) (percentage) -21.36% 2.060 1.620 - 0.00% -
435.gromacs (max) (percentage) -20.79% 2.020 1.600 - 119.18% -
465.tonto (avg) (percentage) -19.76% 1.670 1.340 - -2.19% -
445.gobmk (max) (percentage) -15.69% 1.530 1.290 - -5.84% -
453.povray (max) (percentage) -15.42% 2.530 2.140 - -4.89% -
436.cactusADM (avg) (percentage) -8.43% 1.660 1.520 - -1.30% -
464.h264ref (max) -6.45% 62.000 58.000 - -4.92% -
481.wrf (avg) (percentage) -5.29% 1.890 1.790 - -2.19% -
450.soplex (unexplained diff vs previous) (percentage) -4.98% 2.010 1.910 - 260.38% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 3.45% 29.000 30.000 - 3.45% -

Added Tests - sample variability
416.gamess (unexplained diff vs previous)
416.gamess (unexplained diff vs previous) (percentage)
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)

Report Time: 0.43s
Execution (seconds)
Name Current %
400.perlbench 1602.000 1.91%
401.bzip2 3008.000 -1.38%
403.gcc 55.000 -1.79%
410.bwaves 6032.000 -
416.gamess 12550.000 -
429.mcf 1523.000 -
433.milc 1161.000 -
434.zeusmp 2574.000 -
435.gromacs 7094.000 -1.60%
436.cactusADM 2108.000 -
437.leslie3d 10816.000 -
444.namd 733.000 -
445.gobmk 5226.000 -
447.dealII 1982.000 1.48%
450.soplex 576.000 1.95%
453.povray 500.000 -
454.calculix 63.000 -
456.hmmer 2916.000 -
458.sjeng 7711.000 -
459.GemsFDTD 6099.000 -
462.libquantum 136.000 -2.16%
464.h264ref 5929.000 -
465.tonto 11473.000 -
470.lbm 3728.000 -
471.omnetpp 3021.000 3.96%
473.astar 5887.000 -
481.wrf 5593.000 -
482.sphinx3 936.000 -
Geometric Mean 2087.971 -
Code Size (bytes)
Name Current %
400.perlbench 929250.000 -
401.bzip2 49690.000 -
403.gcc 2683621.000 -
410.bwaves 17640.000 -
416.gamess 6251075.000 -
429.mcf 10979.000 -
433.milc 95617.000 -
434.zeusmp 162829.000 -
435.gromacs 677869.000 -
436.cactusADM 555604.000 -
437.leslie3d 64776.000 -
444.namd 190766.000 -
445.gobmk 1399885.000 -
447.dealII 2118491.000 -
450.soplex 259941.000 -
453.povray 706478.000 -
454.calculix 1270147.000 -
456.hmmer 234488.000 -
458.sjeng 120025.000 -
459.GemsFDTD 254373.000 -
462.libquantum 31353.000 -
464.h264ref 417940.000 -
465.tonto 2873397.000 -
470.lbm 10916.000 -
471.omnetpp 566460.000 -
473.astar 32228.000 -
481.wrf 2623960.000 -
482.sphinx3 152612.000 -
Geometric Mean 272916.684 -
Score
Name Current %
nb_compile_successful 28.000 -
nb_execution_successful 28.000 -
nb_total_tests 30.000 3.45%
Geometric Mean 28.651 1.14%
sample variability
Name Current %
400.perlbench 35.000 -
400.perlbench (avg) 35.000 -
400.perlbench (avg) (percentage) 2.240 -
400.perlbench (max) 22.000 -4.35%
400.perlbench (max) (percentage) 1.390 -6.08%
401.bzip2 67.000 -1.47%
401.bzip2 (avg) 67.000 -1.47%
401.bzip2 (avg) (percentage) 2.250 -
401.bzip2 (max) 271.000 -7.51%
401.bzip2 (max) (percentage) 9.030 -6.23%
401.bzip2 (unexplained diff vs previous) 42.000 -84.33%
401.bzip2 (unexplained diff vs previous) (percentage) 1.400 -84.07%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 5.980 -
403.gcc (max) 2.000 -
403.gcc (max) (percentage) 4.930 -1.99%
410.bwaves 76.000 -1.30%
410.bwaves (avg) 76.000 -1.30%
410.bwaves (avg) (percentage) 1.260 -1.56%
410.bwaves (max) 12.000 -7.69%
410.bwaves (max) (percentage) 0.210 -4.55%
410.bwaves (unexplained diff vs previous) 3.000 -
410.bwaves (unexplained diff vs previous) (percentage) 0.050 -
416.gamess 124.000 -3.12%
416.gamess (avg) 124.000 -3.12%
416.gamess (avg) (percentage) 0.990 -2.94%
416.gamess (max) 17.000 -45.16%
416.gamess (max) (percentage) 0.140 -44.00%
416.gamess (unexplained diff vs previous) 0.000 -
416.gamess (unexplained diff vs previous) (percentage) 0.000 -
429.mcf 91.000 -2.15%
429.mcf (avg) 91.000 -2.15%
429.mcf (avg) (percentage) 6.030 -1.63%
429.mcf (max) 334.000 -5.92%
429.mcf (max) (percentage) 21.980 -5.79%
429.mcf (unexplained diff vs previous) 0.000 -
429.mcf (unexplained diff vs previous) (percentage) 0.000 -
433.milc 19.000 -5.00%
433.milc (avg) 19.000 -5.00%
433.milc (avg) (percentage) 1.700 -1.16%
433.milc (max) 8.000 -57.89%
433.milc (max) (percentage) 0.720 -56.63%
433.milc (unexplained diff vs previous) 3.000 -70.00%
433.milc (unexplained diff vs previous) (percentage) 0.260 -69.77%
434.zeusmp 42.000 -2.33%
434.zeusmp (avg) 42.000 -2.33%
434.zeusmp (avg) (percentage) 1.670 -2.34%
434.zeusmp (max) 29.000 -
434.zeusmp (max) (percentage) 1.130 -
434.zeusmp (unexplained diff vs previous) 3.000 -
434.zeusmp (unexplained diff vs previous) (percentage) 0.120 -
435.gromacs 144.000 -2.04%
435.gromacs (avg) 144.000 -2.04%
435.gromacs (avg) (percentage) 2.030 -
435.gromacs (max) 113.000 117.31%
435.gromacs (max) (percentage) 1.600 119.18%
435.gromacs (unexplained diff vs previous) 115.000 -
435.gromacs (unexplained diff vs previous) (percentage) 1.620 -
436.cactusADM 32.000 -
436.cactusADM (avg) 32.000 -
436.cactusADM (avg) (percentage) 1.520 -1.30%
436.cactusADM (max) 14.000 -6.67%
436.cactusADM (max) (percentage) 0.700 -2.78%
436.cactusADM (unexplained diff vs previous) 10.000 -
436.cactusADM (unexplained diff vs previous) (percentage) 0.470 -
437.leslie3d 486.000 -1.02%
437.leslie3d (avg) 486.000 -1.02%
437.leslie3d (avg) (percentage) 4.500 -1.75%
437.leslie3d (max) 572.000 -
437.leslie3d (max) (percentage) 5.290 -
437.leslie3d (unexplained diff vs previous) 90.000 -
437.leslie3d (unexplained diff vs previous) (percentage) 0.830 -
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.870 -1.06%
444.namd (max) 5.000 25.00%
444.namd (max) (percentage) 0.690 -
444.namd (unexplained diff vs previous) 5.000 66.67%
444.namd (unexplained diff vs previous) (percentage) 0.680 65.85%
445.gobmk 63.000 -1.56%
445.gobmk (avg) 63.000 -1.56%
445.gobmk (avg) (percentage) 1.220 -1.61%
445.gobmk (max) 67.000 -5.63%
445.gobmk (max) (percentage) 1.290 -5.84%
445.gobmk (unexplained diff vs previous) 12.000 -82.35%
445.gobmk (unexplained diff vs previous) (percentage) 0.230 -82.31%
447.dealII 30.000 -
447.dealII (avg) 30.000 -
447.dealII (avg) (percentage) 1.540 -
447.dealII (max) 31.000 -6.06%
447.dealII (max) (percentage) 1.580 -8.67%
447.dealII (unexplained diff vs previous) 29.000 38.10%
447.dealII (unexplained diff vs previous) (percentage) 1.460 35.19%
450.soplex 17.000 -
450.soplex (avg) 17.000 -
450.soplex (avg) (percentage) 3.080 -
450.soplex (max) 20.000 -4.76%
450.soplex (max) (percentage) 3.620 -4.49%
450.soplex (unexplained diff vs previous) 11.000 266.67%
450.soplex (unexplained diff vs previous) (percentage) 1.910 260.38%
453.povray 10.000 -
453.povray (avg) 10.000 -
453.povray (avg) (percentage) 2.020 -1.94%
453.povray (max) 10.000 -9.09%
453.povray (max) (percentage) 2.140 -4.89%
453.povray (unexplained diff vs previous) 2.000 100.00%
453.povray (unexplained diff vs previous) (percentage) 0.400 100.00%
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 3.340 -2.62%
454.calculix (max) 1.000 -50.00%
454.calculix (max) (percentage) 3.120 -29.89%
454.calculix (unexplained diff vs previous) 0.000 -
454.calculix (unexplained diff vs previous) (percentage) 0.000 -
456.hmmer 27.000 -
456.hmmer (avg) 27.000 -
456.hmmer (avg) (percentage) 0.930 -
456.hmmer (max) 16.000 -11.11%
456.hmmer (max) (percentage) 0.580 -6.45%
456.hmmer (unexplained diff vs previous) 16.000 -11.11%
456.hmmer (unexplained diff vs previous) (percentage) 0.550 -9.84%
458.sjeng 65.000 -1.52%
458.sjeng (avg) 65.000 -1.52%
458.sjeng (avg) (percentage) 0.850 -1.16%
458.sjeng (max) 92.000 -6.12%
458.sjeng (max) (percentage) 1.200 -6.25%
458.sjeng (unexplained diff vs previous) 18.000 -76.32%
458.sjeng (unexplained diff vs previous) (percentage) 0.230 -76.77%
459.GemsFDTD 96.000 -2.04%
459.GemsFDTD (avg) 96.000 -2.04%
459.GemsFDTD (avg) (percentage) 1.590 -1.24%
459.GemsFDTD (max) 46.000 -2.13%
459.GemsFDTD (max) (percentage) 0.760 -2.56%
459.GemsFDTD (unexplained diff vs previous) 22.000 -
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.360 -
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.850 -
462.libquantum (max) 4.000 -20.00%
462.libquantum (max) (percentage) 3.110 -16.62%
462.libquantum (unexplained diff vs previous) 3.000 -40.00%
462.libquantum (unexplained diff vs previous) (percentage) 2.210 -38.61%
464.h264ref 60.000 -
464.h264ref (avg) 60.000 -
464.h264ref (avg) (percentage) 1.020 -
464.h264ref (max) 58.000 -4.92%
464.h264ref (max) (percentage) 0.980 -5.77%
465.tonto 153.000 -2.55%
465.tonto (avg) 153.000 -2.55%
465.tonto (avg) (percentage) 1.340 -2.19%
465.tonto (max) 195.000 -
465.tonto (max) (percentage) 1.700 -
465.tonto (unexplained diff vs previous) 11.000 -
465.tonto (unexplained diff vs previous) (percentage) 0.100 -
470.lbm 48.000 -2.04%
470.lbm (avg) 48.000 -2.04%
470.lbm (avg) (percentage) 1.300 -1.52%
470.lbm (max) 30.000 -9.09%
470.lbm (max) (percentage) 0.820 -7.87%
470.lbm (unexplained diff vs previous) 10.000 -69.70%
470.lbm (unexplained diff vs previous) (percentage) 0.270 -69.32%
471.omnetpp 72.000 5.88%
471.omnetpp (avg) 72.000 5.88%
471.omnetpp (avg) (percentage) 2.400 1.27%
471.omnetpp (max) 119.000 32.22%
471.omnetpp (max) (percentage) 3.960 26.92%
471.omnetpp (unexplained diff vs previous) 115.000 43.75%
471.omnetpp (unexplained diff vs previous) (percentage) 3.810 38.55%
473.astar 106.000 -1.85%
473.astar (avg) 106.000 -1.85%
473.astar (avg) (percentage) 1.810 -1.63%
473.astar (max) 257.000 -8.54%
473.astar (max) (percentage) 4.380 -8.37%
473.astar (unexplained diff vs previous) 4.000 -98.51%
473.astar (unexplained diff vs previous) (percentage) 0.070 -98.47%
481.wrf 100.000 -1.96%
481.wrf (avg) 100.000 -1.96%
481.wrf (avg) (percentage) 1.790 -2.19%
481.wrf (max) 35.000 -2.78%
481.wrf (max) (percentage) 0.640 -1.54%
481.wrf (unexplained diff vs previous) 14.000 -
481.wrf (unexplained diff vs previous) (percentage) 0.250 -
482.sphinx3 19.000 -
482.sphinx3 (avg) 19.000 -
482.sphinx3 (avg) (percentage) 2.080 -1.42%
482.sphinx3 (max) 10.000 -16.67%
482.sphinx3 (max) (percentage) 1.140 -12.98%
482.sphinx3 (unexplained diff vs previous) 6.000 -50.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.640 -50.39%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) (percentage) 5.780 -
Geometric Mean 5.101 -45.85%