Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=63224e96d0cd9e726bcd1d6d512255d92b7c5317 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/d0cf0c6c8449009697ad29dd7cb60e7f655628f2 (+ 106 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=1d44530a5be2442e064baa48139adc9fdfb1fc6b (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=428d764c681607c504c3274cdf38b863f34e9793 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os-build/61/
View Options
Report
Filter

gnu-aarch64-master-Os test results

Run Order Start Time Duration
Current 0061 (basepoints/gcc-14-01899-gd0cf0c6c844) 2023-06-16T01:13:15 0:00:00
Previous 0060 (basepoints/gcc-14-01793-g99dec205a12f) 2023-06-15T02:49:37 0:00:00
Baseline 0001 (3-changed-components) 2023-01-01T08:38:47 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 50 74
Performance Improvements 107 138
Added Tests 7 2
Unchanged Tests 1072 1022
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
483.xalancbmk 3.16% 4149.000 4280.000 - -61.69% -
471.omnetpp 3.04% 2957.000 3047.000 - -71.63% -
435.gromacs 2.12% 7214.000 7367.000 - -52.22% -
429.mcf 1.73% 1503.000 1529.000 - -91.09% -
482.sphinx3 1.49% 939.000 953.000 - -96.82% -
400.perlbench 1.44% 1602.000 1625.000 - -84.74% -
444.namd 1.10% 724.000 732.000 - -91.91% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) (percentage) 1207.69% 0.130 1.700 - 214.81% -
429.mcf (unexplained diff vs previous) 1200.00% 2.000 26.000 - -72.04% -
445.gobmk (unexplained diff vs previous) (percentage) 744.44% 0.090 0.760 - -50.97% -
444.namd (unexplained diff vs previous) 700.00% 1.000 8.000 - -98.86% -
445.gobmk (unexplained diff vs previous) 700.00% 5.000 40.000 - -78.49% -
444.namd (unexplained diff vs previous) (percentage) 678.57% 0.140 1.090 - -85.90% -
482.sphinx3 (unexplained diff vs previous) 366.67% 3.000 14.000 - -92.47% -
482.sphinx3 (unexplained diff vs previous) (percentage) 359.37% 0.320 1.470 - 137.10% -
471.omnetpp (unexplained diff vs previous) 328.57% 21.000 90.000 - 246.15% -
471.omnetpp (unexplained diff vs previous) (percentage) 315.49% 0.710 2.950 - 1129.17% -
435.gromacs (unexplained diff vs previous) 218.75% 48.000 153.000 - -51.89% -
435.gromacs (unexplained diff vs previous) (percentage) 210.45% 0.670 2.080 - 0.97% -
429.mcf (max) 188.89% 9.000 26.000 - -71.74% -
429.mcf (max) (percentage) 170.31% 0.640 1.730 - 220.37% -
471.omnetpp (max) 119.05% 42.000 92.000 - -86.10% -
471.omnetpp (max) (percentage) 109.66% 1.450 3.040 - -50.73% -
437.leslie3d (unexplained diff vs previous) (percentage) 95.45% 4.620 9.030 - 294.32% -
464.h264ref (unexplained diff vs previous) (percentage) 80.00% 0.150 0.270 - 170.00% -
437.leslie3d (unexplained diff vs previous) 79.17% 528.000 946.000 - 175.80% -
464.h264ref (unexplained diff vs previous) 77.78% 9.000 16.000 - 6.67% -
482.sphinx3 (max) 55.56% 9.000 14.000 - -96.81% -
482.sphinx3 (max) (percentage) 49.00% 1.000 1.490 - 1.36% -
444.namd (max) 33.33% 6.000 8.000 - -98.77% -
458.sjeng (unexplained diff vs previous) (percentage) 18.18% 0.330 0.390 - 550.00% -
481.wrf (unexplained diff vs previous) 16.67% 42.000 49.000 - -56.25% -
435.gromacs (max) 16.42% 134.000 156.000 - -49.84% -
444.namd (max) (percentage) 15.79% 0.950 1.100 - -84.68% -
481.wrf (unexplained diff vs previous) (percentage) 15.79% 0.760 0.880 - 31.34% -
458.sjeng (max) 15.38% 26.000 30.000 - -3.23% -
458.sjeng (unexplained diff vs previous) 15.38% 26.000 30.000 - 233.33% -
458.sjeng (max) (percentage) 14.71% 0.340 0.390 - 77.27% -
435.gromacs (max) (percentage) 13.37% 1.870 2.120 - 4.95% -
483.xalancbmk 4.17% 48.000 50.000 - -32.43% -
483.xalancbmk (avg) 4.17% 48.000 50.000 - -32.43% -
483.xalancbmk (max) 3.17% 252.000 260.000 - 477.78% -
437.leslie3d (avg) (percentage) 2.17% 4.600 4.700 - 97.48% -
471.omnetpp 2.15% 93.000 95.000 - -44.12% -
471.omnetpp (avg) 2.15% 93.000 95.000 - -44.12% -
400.perlbench 1.82% 55.000 56.000 - -64.10% -
400.perlbench (avg) 1.82% 55.000 56.000 - -64.10% -
435.gromacs 1.51% 199.000 202.000 - -24.06% -
435.gromacs (avg) 1.51% 199.000 202.000 - -24.06% -
462.libquantum (unexplained diff vs previous) (percentage) 1.41% 0.710 0.720 - 7100.00% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d -8.29% 11418.000 10472.000 - -30.21% -
403.gcc -4.84% 62.000 59.000 - -99.35% -
450.soplex -4.39% 592.000 566.000 - -96.34% -
447.dealII -1.05% 1991.000 1970.000 - -79.80% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
470.lbm (unexplained diff vs previous) -93.75% 16.000 1.000 - -92.86% -
470.lbm (unexplained diff vs previous) (percentage) -93.02% 0.430 0.030 - -72.73% -
416.gamess (unexplained diff vs previous) -83.33% 30.000 5.000 - -99.26% -
416.gamess (unexplained diff vs previous) (percentage) -83.33% 0.240 0.040 - -98.66% -
434.zeusmp (unexplained diff vs previous) -76.19% 21.000 5.000 - -98.38% -
434.zeusmp (unexplained diff vs previous) (percentage) -76.19% 0.840 0.200 - -91.42% -
436.cactusADM (unexplained diff vs previous) (percentage) -70.21% 0.470 0.140 - -95.69% -
436.cactusADM (unexplained diff vs previous) -70.00% 10.000 3.000 - -99.47% -
433.milc (unexplained diff vs previous) -66.67% 3.000 1.000 - -99.76% -
433.milc (unexplained diff vs previous) (percentage) -65.38% 0.260 0.090 - -97.26% -
456.hmmer (unexplained diff vs previous) -60.00% 5.000 2.000 - -97.59% -
456.hmmer (unexplained diff vs previous) (percentage) -58.82% 0.170 0.070 - -90.67% -
470.lbm (max) -54.17% 24.000 11.000 - -98.32% -
470.lbm (max) (percentage) -51.52% 0.660 0.320 - -93.73% -
459.GemsFDTD (max) -50.00% 64.000 32.000 - -77.14% -
459.GemsFDTD (max) (percentage) -50.00% 1.060 0.530 - -11.67% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -47.46% 0.590 0.310 - 47.62% -
459.GemsFDTD (unexplained diff vs previous) -47.22% 36.000 19.000 - -60.42% -
456.hmmer (max) -42.86% 7.000 4.000 - -97.14% -
456.hmmer (max) (percentage) -41.67% 0.240 0.140 - -88.98% -
436.cactusADM (max) (percentage) -40.91% 0.660 0.390 - -87.62% -
436.cactusADM (max) -38.46% 13.000 8.000 - -98.55% -
453.povray (unexplained diff vs previous) (percentage) -38.26% 1.490 0.920 - 4500.00% -
453.povray (unexplained diff vs previous) -37.50% 8.000 5.000 - 400.00% -
433.milc (max) -33.33% 3.000 2.000 - -99.51% -
410.bwaves (unexplained diff vs previous) -28.00% 25.000 18.000 - 20.00% -
410.bwaves (unexplained diff vs previous) (percentage) -26.83% 0.410 0.300 - 200.00% -
453.povray (max) -25.00% 8.000 6.000 - -96.36% -
433.milc (max) (percentage) -23.08% 0.260 0.200 - -93.69% -
453.povray (max) (percentage) -20.26% 1.530 1.220 - -51.78% -
410.bwaves (max) (percentage) -17.07% 0.410 0.340 - 61.90% -
410.bwaves (max) -16.67% 24.000 20.000 - -33.33% -
437.leslie3d (max) -11.76% 1582.000 1396.000 - 297.72% -
445.gobmk (max) -10.87% 46.000 41.000 - -77.60% -
445.gobmk (max) (percentage) -10.23% 0.880 0.790 - -48.37% -
434.zeusmp (max) -10.00% 20.000 18.000 - -94.02% -
401.bzip2 (max) -9.68% 31.000 28.000 - -78.95% -
434.zeusmp (max) (percentage) -9.64% 0.830 0.750 - -67.11% -
447.dealII (max) -9.09% 11.000 10.000 - -90.65% -
450.soplex (max) -9.09% 22.000 20.000 - -93.42% -
401.bzip2 (max) (percentage) -8.49% 1.060 0.970 - 10.23% -
416.gamess (max) (percentage) -8.33% 0.240 0.220 - -92.39% -
454.calculix (max) (percentage) -7.56% 2.910 2.690 - -42.77% -
450.soplex (max) (percentage) -7.25% 3.860 3.580 - 81.73% -
447.dealII (max) (percentage) -7.14% 0.560 0.520 - -52.73% -
416.gamess (max) -6.90% 29.000 27.000 - -95.88% -
437.leslie3d -6.29% 525.000 492.000 - 37.82% -
437.leslie3d (avg) -6.29% 525.000 492.000 - 37.82% -
473.astar (max) (percentage) -6.12% 0.980 0.920 - 557.14% -
481.wrf (max) (percentage) -6.03% 1.160 1.090 - -65.40% -
481.wrf (max) -4.76% 63.000 60.000 - -88.68% -
465.tonto (max) (percentage) -4.30% 0.930 0.890 - -75.14% -
450.soplex -4.17% 24.000 23.000 - -87.96% -
450.soplex (avg) -4.17% 24.000 23.000 - -87.96% -
465.tonto (max) -3.96% 101.000 97.000 - -66.89% -
416.gamess (avg) (percentage) -3.77% 1.590 1.530 - -29.49% -
437.leslie3d (max) (percentage) -3.75% 13.860 13.340 - 470.09% -
416.gamess -3.55% 197.000 190.000 - -61.38% -
416.gamess (avg) -3.55% 197.000 190.000 - -61.38% -
454.calculix (avg) (percentage) -3.16% 3.800 3.680 - 83.08% -
433.milc -3.12% 32.000 31.000 - -85.91% -
433.milc (avg) -3.12% 32.000 31.000 - -85.91% -
458.sjeng -3.03% 99.000 96.000 - 104.26% -
458.sjeng (avg) -3.03% 99.000 96.000 - 104.26% -
456.hmmer (avg) (percentage) -2.63% 1.520 1.480 - 117.65% -
410.bwaves -2.61% 115.000 112.000 - -30.00% -
410.bwaves (avg) -2.61% 115.000 112.000 - -30.00% -
470.lbm -2.50% 80.000 78.000 - -74.26% -
470.lbm (avg) -2.50% 80.000 78.000 - -74.26% -
464.h264ref (avg) (percentage) -2.45% 1.630 1.590 - 194.44% -
400.perlbench (max) (percentage) -2.43% 5.760 5.620 - 73.46% -
458.sjeng (avg) (percentage) -2.34% 1.280 1.250 - 278.79% -
447.dealII -2.33% 43.000 42.000 - -59.62% -
447.dealII (avg) -2.33% 43.000 42.000 - -59.62% -
470.lbm (avg) (percentage) -2.31% 2.160 2.110 - -10.97% -
456.hmmer -2.27% 44.000 43.000 - -42.67% -
456.hmmer (avg) -2.27% 44.000 43.000 - -42.67% -
436.cactusADM (avg) (percentage) -2.20% 2.270 2.220 - 33.73% -
459.GemsFDTD (avg) (percentage) -2.16% 2.320 2.270 - 152.22% -
445.gobmk -2.13% 94.000 92.000 - 0.00% -
445.gobmk (avg) -2.13% 94.000 92.000 - 0.00% -
464.h264ref -2.11% 95.000 93.000 - 19.23% -
464.h264ref (avg) -2.11% 95.000 93.000 - 19.23% -
410.bwaves (avg) (percentage) -2.11% 1.900 1.860 - 70.64% -
434.zeusmp (avg) (percentage) -2.09% 2.390 2.340 - 58.11% -
436.cactusADM -2.08% 48.000 47.000 - -83.85% -
436.cactusADM (avg) -2.08% 48.000 47.000 - -83.85% -
453.povray (avg) (percentage) -2.08% 2.890 2.830 - 126.40% -
481.wrf (avg) (percentage) -1.93% 2.590 2.540 - 34.39% -
433.milc (avg) (percentage) -1.81% 2.770 2.720 - 60.00% -
462.libquantum (max) (percentage) -1.72% 1.740 1.710 - 167.19% -
429.mcf (avg) (percentage) -1.71% 7.000 6.880 - 1198.11% -
434.zeusmp -1.69% 59.000 58.000 - -70.26% -
434.zeusmp (avg) -1.69% 59.000 58.000 - -70.26% -
445.gobmk (avg) (percentage) -1.69% 1.780 1.750 - 127.27% -
462.libquantum (avg) (percentage) -1.58% 2.530 2.490 - 283.08% -
459.GemsFDTD -1.42% 141.000 139.000 - -33.81% -
459.GemsFDTD (avg) -1.42% 141.000 139.000 - -33.81% -
403.gcc (max) (percentage) -1.41% 12.730 12.550 - 1643.06% -
444.namd (avg) (percentage) -1.37% 2.920 2.880 - -26.72% -
400.perlbench (max) -1.09% 92.000 91.000 - -73.62% -
482.sphinx3 (avg) (percentage) -1.01% 2.960 2.930 - 100.68% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful 3.57% 28.000 29.000 - 0.00% -

Added Tests - Code Size
473.astar

Added Tests - Execution
473.astar

Added Tests - sample variability
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)
473.astar
473.astar (avg)
473.astar (max)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 33.82% 8205.000 10980.000 - 0.26% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 7100.00% 0.010 0.720 - 1.41% -
453.povray (unexplained diff vs previous) (percentage) 4500.00% 0.020 0.920 - -38.26% -
403.gcc (max) (percentage) 1643.06% 0.720 12.550 - -1.41% -
483.xalancbmk (max) (percentage) 1382.93% 0.410 6.080 - -0.16% -
429.mcf (avg) (percentage) 1198.11% 0.530 6.880 - -1.71% -
471.omnetpp (unexplained diff vs previous) (percentage) 1129.17% 0.240 2.950 - 315.49% -
473.astar (avg) (percentage) 1110.53% 0.190 2.300 - 0.00% -
403.gcc (avg) (percentage) 857.38% 0.610 5.840 - 0.00% -
473.astar (max) (percentage) 557.14% 0.140 0.920 - -6.12% -
458.sjeng (unexplained diff vs previous) (percentage) 550.00% 0.060 0.390 - 18.18% -
483.xalancbmk (max) 477.78% 45.000 260.000 - 3.17% -
437.leslie3d (max) (percentage) 470.09% 2.340 13.340 - -3.75% -
401.bzip2 (avg) (percentage) 452.27% 0.440 2.430 - 0.00% -
453.povray (unexplained diff vs previous) 400.00% 1.000 5.000 - -37.50% -
473.astar 385.71% 28.000 136.000 - 0.00% -
473.astar (avg) 385.71% 28.000 136.000 - 0.00% -
437.leslie3d (max) 297.72% 351.000 1396.000 - -11.76% -
437.leslie3d (unexplained diff vs previous) (percentage) 294.32% 2.290 9.030 - 95.45% -
462.libquantum (avg) (percentage) 283.08% 0.650 2.490 - -1.58% -
458.sjeng (avg) (percentage) 278.79% 0.330 1.250 - -2.34% -
471.omnetpp (unexplained diff vs previous) 246.15% 26.000 90.000 - 328.57% -
450.soplex (avg) (percentage) 235.48% 1.240 4.160 - 0.00% -
458.sjeng (unexplained diff vs previous) 233.33% 9.000 30.000 - 15.38% -
429.mcf (max) (percentage) 220.37% 0.540 1.730 - 170.31% -
429.mcf (unexplained diff vs previous) (percentage) 214.81% 0.540 1.700 - 1207.69% -
410.bwaves (unexplained diff vs previous) (percentage) 200.00% 0.100 0.300 - -26.83% -
464.h264ref (avg) (percentage) 194.44% 0.540 1.590 - -2.45% -
437.leslie3d (unexplained diff vs previous) 175.80% 343.000 946.000 - 79.17% -
464.h264ref (unexplained diff vs previous) (percentage) 170.00% 0.100 0.270 - 80.00% -
473.astar (max) 170.00% 20.000 54.000 - 0.00% -
462.libquantum (max) (percentage) 167.19% 0.640 1.710 - -1.72% -
459.GemsFDTD (avg) (percentage) 152.22% 0.900 2.270 - -2.16% -
482.sphinx3 (unexplained diff vs previous) (percentage) 137.10% 0.620 1.470 - 359.37% -
400.perlbench (avg) (percentage) 136.73% 1.470 3.480 - 0.00% -
445.gobmk (avg) (percentage) 127.27% 0.770 1.750 - -1.69% -
453.povray (avg) (percentage) 126.40% 1.250 2.830 - -2.08% -
456.hmmer (avg) (percentage) 117.65% 0.680 1.480 - -2.63% -
458.sjeng 104.26% 47.000 96.000 - -3.03% -
458.sjeng (avg) 104.26% 47.000 96.000 - -3.03% -
447.dealII (avg) (percentage) 102.80% 1.070 2.170 - 0.00% -
482.sphinx3 (avg) (percentage) 100.68% 1.460 2.930 - -1.01% -
471.omnetpp (avg) (percentage) 98.11% 1.590 3.150 - -0.32% -
437.leslie3d (avg) (percentage) 97.48% 2.380 4.700 - 2.17% -
454.calculix (avg) (percentage) 83.08% 2.010 3.680 - -3.16% -
450.soplex (max) (percentage) 81.73% 1.970 3.580 - -7.25% -
458.sjeng (max) (percentage) 77.27% 0.220 0.390 - 14.71% -
483.xalancbmk (avg) (percentage) 76.12% 0.670 1.180 - 0.00% -
400.perlbench (max) (percentage) 73.46% 3.240 5.620 - -2.43% -
410.bwaves (avg) (percentage) 70.64% 1.090 1.860 - -2.11% -
410.bwaves (max) (percentage) 61.90% 0.210 0.340 - -17.07% -
433.milc (avg) (percentage) 60.00% 1.700 2.720 - -1.81% -
435.gromacs (avg) (percentage) 58.96% 1.730 2.750 - -0.72% -
434.zeusmp (avg) (percentage) 58.11% 1.480 2.340 - -2.09% -
465.tonto 51.82% 137.000 208.000 - 0.00% -
465.tonto (avg) 51.82% 137.000 208.000 - 0.00% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 47.62% 0.210 0.310 - -47.46% -
437.leslie3d 37.82% 357.000 492.000 - -6.29% -
437.leslie3d (avg) 37.82% 357.000 492.000 - -6.29% -
481.wrf (avg) (percentage) 34.39% 1.890 2.540 - -1.93% -
436.cactusADM (avg) (percentage) 33.73% 1.660 2.220 - -2.20% -
481.wrf (unexplained diff vs previous) (percentage) 31.34% 0.670 0.880 - 15.79% -
410.bwaves (unexplained diff vs previous) 20.00% 15.000 18.000 - -28.00% -
464.h264ref 19.23% 78.000 93.000 - -2.11% -
464.h264ref (avg) 19.23% 78.000 93.000 - -2.11% -
429.mcf 16.67% 90.000 105.000 - 0.00% -
429.mcf (avg) 16.67% 90.000 105.000 - 0.00% -
465.tonto (avg) (percentage) 13.77% 1.670 1.900 - 0.00% -
401.bzip2 (max) (percentage) 10.23% 0.880 0.970 - -8.49% -
401.bzip2 9.09% 66.000 72.000 - 0.00% -
401.bzip2 (avg) 9.09% 66.000 72.000 - 0.00% -
464.h264ref (unexplained diff vs previous) 6.67% 15.000 16.000 - 77.78% -
435.gromacs (max) (percentage) 4.95% 2.020 2.120 - 13.37% -
482.sphinx3 (max) (percentage) 1.36% 1.470 1.490 - 49.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
458.sjeng -6.99% 120221.000 111821.000 - 0.00% -
464.h264ref -1.06% 428364.000 423816.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.72% 22860.000 64.000 - 0.00% -
403.gcc -99.35% 9010.000 59.000 - -4.84% -
462.libquantum -98.80% 11614.000 139.000 - -0.71% -
482.sphinx3 -96.82% 29928.000 953.000 - 1.49% -
450.soplex -96.34% 15466.000 566.000 - -4.39% -
444.namd -91.91% 9050.000 732.000 - 1.10% -
453.povray -91.70% 6544.000 543.000 - 0.93% -
429.mcf -91.09% 17158.000 1529.000 - 1.73% -
433.milc -90.98% 12976.000 1171.000 - -0.09% -
436.cactusADM -87.90% 17544.000 2122.000 - 0.14% -
400.perlbench -84.74% 10650.000 1625.000 - 1.44% -
434.zeusmp -81.00% 13211.000 2510.000 - 0.20% -
401.bzip2 -80.38% 15185.000 2979.000 - -0.10% -
447.dealII -79.80% 9751.000 1970.000 - -1.05% -
459.GemsFDTD -73.77% 23356.000 6126.000 - 0.31% -
456.hmmer -73.47% 11046.000 2931.000 - 0.07% -
471.omnetpp -71.63% 10739.000 3047.000 - 3.04% -
470.lbm -70.86% 12819.000 3735.000 - 0.03% -
481.wrf -66.97% 16834.000 5561.000 - 0.89% -
483.xalancbmk -61.69% 11172.000 4280.000 - 3.16% -
473.astar -60.38% 14932.000 5916.000 - 0.00% -
464.h264ref -59.74% 14556.000 5860.000 - 0.27% -
410.bwaves -58.88% 14755.000 6067.000 - -0.30% -
445.gobmk -56.17% 11995.000 5258.000 - -0.76% -
435.gromacs -52.22% 15419.000 7367.000 - 2.12% -
458.sjeng -45.73% 14275.000 7747.000 - -0.39% -
416.gamess -45.24% 22704.000 12433.000 - 0.04% -
437.leslie3d -30.21% 15005.000 10472.000 - -8.29% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (max) -99.91% 1074.000 1.000 - 0.00% -
433.milc (unexplained diff vs previous) -99.76% 425.000 1.000 - -66.67% -
454.calculix -99.56% 459.000 2.000 - 0.00% -
454.calculix (avg) -99.56% 459.000 2.000 - 0.00% -
433.milc (max) -99.51% 411.000 2.000 - -33.33% -
436.cactusADM (unexplained diff vs previous) -99.47% 570.000 3.000 - -70.00% -
416.gamess (unexplained diff vs previous) -99.26% 676.000 5.000 - -83.33% -
444.namd (unexplained diff vs previous) -98.86% 700.000 8.000 - 700.00% -
444.namd (max) -98.77% 649.000 8.000 - 33.33% -
416.gamess (unexplained diff vs previous) (percentage) -98.66% 2.980 0.040 - -83.33% -
436.cactusADM (max) -98.55% 552.000 8.000 - -38.46% -
434.zeusmp (unexplained diff vs previous) -98.38% 308.000 5.000 - -76.19% -
470.lbm (max) -98.32% 653.000 11.000 - -54.17% -
456.hmmer (unexplained diff vs previous) -97.59% 83.000 2.000 - -60.00% -
462.libquantum (max) -97.30% 74.000 2.000 - 0.00% -
433.milc (unexplained diff vs previous) (percentage) -97.26% 3.280 0.090 - -65.38% -
456.hmmer (max) -97.14% 140.000 4.000 - -42.86% -
482.sphinx3 (max) -96.81% 439.000 14.000 - 55.56% -
453.povray (max) -96.36% 165.000 6.000 - -25.00% -
462.libquantum -96.00% 75.000 3.000 - 0.00% -
462.libquantum (avg) -96.00% 75.000 3.000 - 0.00% -
416.gamess (max) -95.88% 656.000 27.000 - -6.90% -
436.cactusADM (unexplained diff vs previous) (percentage) -95.69% 3.250 0.140 - -70.21% -
403.gcc -94.44% 54.000 3.000 - 0.00% -
403.gcc (avg) -94.44% 54.000 3.000 - 0.00% -
444.namd -94.08% 355.000 21.000 - 0.00% -
444.namd (avg) -94.08% 355.000 21.000 - 0.00% -
434.zeusmp (max) -94.02% 301.000 18.000 - -10.00% -
482.sphinx3 -93.81% 436.000 27.000 - 0.00% -
482.sphinx3 (avg) -93.81% 436.000 27.000 - 0.00% -
470.lbm (max) (percentage) -93.73% 5.100 0.320 - -51.52% -
433.milc (max) (percentage) -93.69% 3.170 0.200 - -23.08% -
450.soplex (max) -93.42% 304.000 20.000 - -9.09% -
470.lbm (unexplained diff vs previous) -92.86% 14.000 1.000 - -93.75% -
482.sphinx3 (unexplained diff vs previous) -92.47% 186.000 14.000 - 366.67% -
416.gamess (max) (percentage) -92.39% 2.890 0.220 - -8.33% -
434.zeusmp (unexplained diff vs previous) (percentage) -91.42% 2.330 0.200 - -76.19% -
456.hmmer (unexplained diff vs previous) (percentage) -90.67% 0.750 0.070 - -58.82% -
447.dealII (max) -90.65% 107.000 10.000 - -9.09% -
403.gcc (max) -89.06% 64.000 7.000 - 0.00% -
456.hmmer (max) (percentage) -88.98% 1.270 0.140 - -41.67% -
481.wrf (max) -88.68% 530.000 60.000 - -4.76% -
450.soplex -87.96% 191.000 23.000 - -4.17% -
450.soplex (avg) -87.96% 191.000 23.000 - -4.17% -
436.cactusADM (max) (percentage) -87.62% 3.150 0.390 - -40.91% -
471.omnetpp (max) -86.10% 662.000 92.000 - 119.05% -
433.milc -85.91% 220.000 31.000 - -3.12% -
433.milc (avg) -85.91% 220.000 31.000 - -3.12% -
444.namd (unexplained diff vs previous) (percentage) -85.90% 7.730 1.090 - 678.57% -
444.namd (max) (percentage) -84.68% 7.180 1.100 - 15.79% -
436.cactusADM -83.85% 291.000 47.000 - -2.08% -
436.cactusADM (avg) -83.85% 291.000 47.000 - -2.08% -
453.povray -81.48% 81.000 15.000 - 0.00% -
453.povray (avg) -81.48% 81.000 15.000 - 0.00% -
401.bzip2 (max) -78.95% 133.000 28.000 - -9.68% -
445.gobmk (unexplained diff vs previous) -78.49% 186.000 40.000 - 700.00% -
445.gobmk (max) -77.60% 183.000 41.000 - -10.87% -
459.GemsFDTD (max) -77.14% 140.000 32.000 - -50.00% -
464.h264ref (max) -75.81% 62.000 15.000 - 0.00% -
465.tonto (max) (percentage) -75.14% 3.580 0.890 - -4.30% -
470.lbm -74.26% 303.000 78.000 - -2.50% -
470.lbm (avg) -74.26% 303.000 78.000 - -2.50% -
400.perlbench (max) -73.62% 345.000 91.000 - -1.09% -
470.lbm (unexplained diff vs previous) (percentage) -72.73% 0.110 0.030 - -93.02% -
429.mcf (unexplained diff vs previous) -72.04% 93.000 26.000 - 1200.00% -
429.mcf (max) -71.74% 92.000 26.000 - 188.89% -
434.zeusmp -70.26% 195.000 58.000 - -1.69% -
434.zeusmp (avg) -70.26% 195.000 58.000 - -1.69% -
434.zeusmp (max) (percentage) -67.11% 2.280 0.750 - -9.64% -
465.tonto (max) -66.89% 293.000 97.000 - -3.96% -
481.wrf (max) (percentage) -65.40% 3.150 1.090 - -6.03% -
400.perlbench -64.10% 156.000 56.000 - 1.82% -
400.perlbench (avg) -64.10% 156.000 56.000 - 1.82% -
416.gamess -61.38% 492.000 190.000 - -3.55% -
416.gamess (avg) -61.38% 492.000 190.000 - -3.55% -
459.GemsFDTD (unexplained diff vs previous) -60.42% 48.000 19.000 - -47.22% -
447.dealII -59.62% 104.000 42.000 - -2.33% -
447.dealII (avg) -59.62% 104.000 42.000 - -2.33% -
481.wrf (unexplained diff vs previous) -56.25% 112.000 49.000 - 16.67% -
481.wrf -55.66% 318.000 141.000 - -0.70% -
481.wrf (avg) -55.66% 318.000 141.000 - -0.70% -
447.dealII (max) (percentage) -52.73% 1.100 0.520 - -7.14% -
435.gromacs (unexplained diff vs previous) -51.89% 318.000 153.000 - 218.75% -
453.povray (max) (percentage) -51.78% 2.530 1.220 - -20.26% -
445.gobmk (unexplained diff vs previous) (percentage) -50.97% 1.550 0.760 - 744.44% -
471.omnetpp (max) (percentage) -50.73% 6.170 3.040 - 109.66% -
435.gromacs (max) -49.84% 311.000 156.000 - 16.42% -
445.gobmk (max) (percentage) -48.37% 1.530 0.790 - -10.23% -
471.omnetpp -44.12% 170.000 95.000 - 2.15% -
471.omnetpp (avg) -44.12% 170.000 95.000 - 2.15% -
454.calculix (max) (percentage) -42.77% 4.700 2.690 - -7.56% -
456.hmmer -42.67% 75.000 43.000 - -2.27% -
456.hmmer (avg) -42.67% 75.000 43.000 - -2.27% -
464.h264ref (max) (percentage) -37.21% 0.430 0.270 - 0.00% -
459.GemsFDTD -33.81% 210.000 139.000 - -1.42% -
459.GemsFDTD (avg) -33.81% 210.000 139.000 - -1.42% -
410.bwaves (max) -33.33% 30.000 20.000 - -16.67% -
483.xalancbmk -32.43% 74.000 50.000 - 4.17% -
483.xalancbmk (avg) -32.43% 74.000 50.000 - 4.17% -
410.bwaves -30.00% 160.000 112.000 - -2.61% -
410.bwaves (avg) -30.00% 160.000 112.000 - -2.61% -
416.gamess (avg) (percentage) -29.49% 2.170 1.530 - -3.77% -
444.namd (avg) (percentage) -26.72% 3.930 2.880 - -1.37% -
435.gromacs -24.06% 266.000 202.000 - 1.51% -
435.gromacs (avg) -24.06% 266.000 202.000 - 1.51% -
459.GemsFDTD (max) (percentage) -11.67% 0.600 0.530 - -50.00% -
470.lbm (avg) (percentage) -10.97% 2.370 2.110 - -2.31% -
458.sjeng (max) -3.23% 31.000 30.000 - 15.38% -

Added Tests - sample variability
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)

Report Time: 0.46s
Execution (seconds)
Name Current %
400.perlbench 1625.000 1.44%
401.bzip2 2979.000 -
403.gcc 59.000 -4.84%
410.bwaves 6067.000 -
416.gamess 12433.000 -
429.mcf 1529.000 1.73%
433.milc 1171.000 -
434.zeusmp 2510.000 -
435.gromacs 7367.000 2.12%
436.cactusADM 2122.000 -
437.leslie3d 10472.000 -8.29%
444.namd 732.000 1.10%
445.gobmk 5258.000 -
447.dealII 1970.000 -1.05%
450.soplex 566.000 -4.39%
453.povray 543.000 -
454.calculix 64.000 -
456.hmmer 2931.000 -
458.sjeng 7747.000 -
459.GemsFDTD 6126.000 -
462.libquantum 139.000 -
464.h264ref 5860.000 -
465.tonto 10980.000 -
470.lbm 3735.000 -
471.omnetpp 3047.000 3.04%
473.astar 5916.000 -
481.wrf 5561.000 -
482.sphinx3 953.000 1.49%
483.xalancbmk 4280.000 3.16%
Geometric Mean 2151.987 3.51%
Code Size (bytes)
Name Current %
400.perlbench 929964.000 -
401.bzip2 49800.000 -
403.gcc 2665737.000 -
410.bwaves 17732.000 -
416.gamess 6246395.000 -
429.mcf 10951.000 -
433.milc 95822.000 -
434.zeusmp 165643.000 -
435.gromacs 678393.000 -
436.cactusADM 553198.000 -
437.leslie3d 65185.000 -
444.namd 190898.000 -
445.gobmk 1399583.000 -
447.dealII 2122649.000 -
450.soplex 260242.000 -
453.povray 712529.000 -
454.calculix 1272103.000 -
456.hmmer 231108.000 -
458.sjeng 111821.000 -
459.GemsFDTD 253069.000 -
462.libquantum 31329.000 -
464.h264ref 423816.000 -
465.tonto 2869501.000 -
470.lbm 10908.000 -
471.omnetpp 566702.000 -
473.astar 32056.000 -
481.wrf 2634840.000 -
482.sphinx3 152636.000 -
483.xalancbmk 3287204.000 -
Geometric Mean 296898.610 -7.65%
Score
Name Current %
nb_compile_successful 29.000 -
nb_execution_successful 29.000 3.57%
nb_total_tests 29.000 -
Geometric Mean 29.000 1.18%
sample variability
Name Current %
400.perlbench 56.000 1.82%
400.perlbench (avg) 56.000 1.82%
400.perlbench (avg) (percentage) 3.480 -
400.perlbench (max) 91.000 -1.09%
400.perlbench (max) (percentage) 5.620 -2.43%
401.bzip2 72.000 -
401.bzip2 (avg) 72.000 -
401.bzip2 (avg) (percentage) 2.430 -
401.bzip2 (max) 28.000 -9.68%
401.bzip2 (max) (percentage) 0.970 -8.49%
403.gcc 3.000 -
403.gcc (avg) 3.000 -
403.gcc (avg) (percentage) 5.840 -
403.gcc (max) 7.000 -
403.gcc (max) (percentage) 12.550 -1.41%
410.bwaves 112.000 -2.61%
410.bwaves (avg) 112.000 -2.61%
410.bwaves (avg) (percentage) 1.860 -2.11%
410.bwaves (max) 20.000 -16.67%
410.bwaves (max) (percentage) 0.340 -17.07%
410.bwaves (unexplained diff vs previous) 18.000 -28.00%
410.bwaves (unexplained diff vs previous) (percentage) 0.300 -26.83%
416.gamess 190.000 -3.55%
416.gamess (avg) 190.000 -3.55%
416.gamess (avg) (percentage) 1.530 -3.77%
416.gamess (max) 27.000 -6.90%
416.gamess (max) (percentage) 0.220 -8.33%
416.gamess (unexplained diff vs previous) 5.000 -83.33%
416.gamess (unexplained diff vs previous) (percentage) 0.040 -83.33%
429.mcf 105.000 -
429.mcf (avg) 105.000 -
429.mcf (avg) (percentage) 6.880 -1.71%
429.mcf (max) 26.000 188.89%
429.mcf (max) (percentage) 1.730 170.31%
429.mcf (unexplained diff vs previous) 26.000 1200.00%
429.mcf (unexplained diff vs previous) (percentage) 1.700 1207.69%
433.milc 31.000 -3.12%
433.milc (avg) 31.000 -3.12%
433.milc (avg) (percentage) 2.720 -1.81%
433.milc (max) 2.000 -33.33%
433.milc (max) (percentage) 0.200 -23.08%
433.milc (unexplained diff vs previous) 1.000 -66.67%
433.milc (unexplained diff vs previous) (percentage) 0.090 -65.38%
434.zeusmp 58.000 -1.69%
434.zeusmp (avg) 58.000 -1.69%
434.zeusmp (avg) (percentage) 2.340 -2.09%
434.zeusmp (max) 18.000 -10.00%
434.zeusmp (max) (percentage) 0.750 -9.64%
434.zeusmp (unexplained diff vs previous) 5.000 -76.19%
434.zeusmp (unexplained diff vs previous) (percentage) 0.200 -76.19%
435.gromacs 202.000 1.51%
435.gromacs (avg) 202.000 1.51%
435.gromacs (avg) (percentage) 2.750 -
435.gromacs (max) 156.000 16.42%
435.gromacs (max) (percentage) 2.120 13.37%
435.gromacs (unexplained diff vs previous) 153.000 218.75%
435.gromacs (unexplained diff vs previous) (percentage) 2.080 210.45%
436.cactusADM 47.000 -2.08%
436.cactusADM (avg) 47.000 -2.08%
436.cactusADM (avg) (percentage) 2.220 -2.20%
436.cactusADM (max) 8.000 -38.46%
436.cactusADM (max) (percentage) 0.390 -40.91%
436.cactusADM (unexplained diff vs previous) 3.000 -70.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.140 -70.21%
437.leslie3d 492.000 -6.29%
437.leslie3d (avg) 492.000 -6.29%
437.leslie3d (avg) (percentage) 4.700 2.17%
437.leslie3d (max) 1396.000 -11.76%
437.leslie3d (max) (percentage) 13.340 -3.75%
437.leslie3d (unexplained diff vs previous) 946.000 79.17%
437.leslie3d (unexplained diff vs previous) (percentage) 9.030 95.45%
444.namd 21.000 -
444.namd (avg) 21.000 -
444.namd (avg) (percentage) 2.880 -1.37%
444.namd (max) 8.000 33.33%
444.namd (max) (percentage) 1.100 15.79%
444.namd (unexplained diff vs previous) 8.000 700.00%
444.namd (unexplained diff vs previous) (percentage) 1.090 678.57%
445.gobmk 92.000 -2.13%
445.gobmk (avg) 92.000 -2.13%
445.gobmk (avg) (percentage) 1.750 -1.69%
445.gobmk (max) 41.000 -10.87%
445.gobmk (max) (percentage) 0.790 -10.23%
445.gobmk (unexplained diff vs previous) 40.000 700.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.760 744.44%
447.dealII 42.000 -2.33%
447.dealII (avg) 42.000 -2.33%
447.dealII (avg) (percentage) 2.170 -
447.dealII (max) 10.000 -9.09%
447.dealII (max) (percentage) 0.520 -7.14%
450.soplex 23.000 -4.17%
450.soplex (avg) 23.000 -4.17%
450.soplex (avg) (percentage) 4.160 -
450.soplex (max) 20.000 -9.09%
450.soplex (max) (percentage) 3.580 -7.25%
453.povray 15.000 -
453.povray (avg) 15.000 -
453.povray (avg) (percentage) 2.830 -2.08%
453.povray (max) 6.000 -25.00%
453.povray (max) (percentage) 1.220 -20.26%
453.povray (unexplained diff vs previous) 5.000 -37.50%
453.povray (unexplained diff vs previous) (percentage) 0.920 -38.26%
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 3.680 -3.16%
454.calculix (max) 1.000 -
454.calculix (max) (percentage) 2.690 -7.56%
454.calculix (unexplained diff vs previous) 0.000 -
454.calculix (unexplained diff vs previous) (percentage) 0.000 -
456.hmmer 43.000 -2.27%
456.hmmer (avg) 43.000 -2.27%
456.hmmer (avg) (percentage) 1.480 -2.63%
456.hmmer (max) 4.000 -42.86%
456.hmmer (max) (percentage) 0.140 -41.67%
456.hmmer (unexplained diff vs previous) 2.000 -60.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.070 -58.82%
458.sjeng 96.000 -3.03%
458.sjeng (avg) 96.000 -3.03%
458.sjeng (avg) (percentage) 1.250 -2.34%
458.sjeng (max) 30.000 15.38%
458.sjeng (max) (percentage) 0.390 14.71%
458.sjeng (unexplained diff vs previous) 30.000 15.38%
458.sjeng (unexplained diff vs previous) (percentage) 0.390 18.18%
459.GemsFDTD 139.000 -1.42%
459.GemsFDTD (avg) 139.000 -1.42%
459.GemsFDTD (avg) (percentage) 2.270 -2.16%
459.GemsFDTD (max) 32.000 -50.00%
459.GemsFDTD (max) (percentage) 0.530 -50.00%
459.GemsFDTD (unexplained diff vs previous) 19.000 -47.22%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.310 -47.46%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 2.490 -1.58%
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.710 -1.72%
462.libquantum (unexplained diff vs previous) 1.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.720 1.41%
464.h264ref 93.000 -2.11%
464.h264ref (avg) 93.000 -2.11%
464.h264ref (avg) (percentage) 1.590 -2.45%
464.h264ref (max) 15.000 -
464.h264ref (max) (percentage) 0.270 -
464.h264ref (unexplained diff vs previous) 16.000 77.78%
464.h264ref (unexplained diff vs previous) (percentage) 0.270 80.00%
465.tonto 208.000 -
465.tonto (avg) 208.000 -
465.tonto (avg) (percentage) 1.900 -
465.tonto (max) 97.000 -3.96%
465.tonto (max) (percentage) 0.890 -4.30%
470.lbm 78.000 -2.50%
470.lbm (avg) 78.000 -2.50%
470.lbm (avg) (percentage) 2.110 -2.31%
470.lbm (max) 11.000 -54.17%
470.lbm (max) (percentage) 0.320 -51.52%
470.lbm (unexplained diff vs previous) 1.000 -93.75%
470.lbm (unexplained diff vs previous) (percentage) 0.030 -93.02%
471.omnetpp 95.000 2.15%
471.omnetpp (avg) 95.000 2.15%
471.omnetpp (avg) (percentage) 3.150 -
471.omnetpp (max) 92.000 119.05%
471.omnetpp (max) (percentage) 3.040 109.66%
471.omnetpp (unexplained diff vs previous) 90.000 328.57%
471.omnetpp (unexplained diff vs previous) (percentage) 2.950 315.49%
473.astar 136.000 -
473.astar (avg) 136.000 -
473.astar (avg) (percentage) 2.300 -
473.astar (max) 54.000 -
473.astar (max) (percentage) 0.920 -6.12%
481.wrf 141.000 -
481.wrf (avg) 141.000 -
481.wrf (avg) (percentage) 2.540 -1.93%
481.wrf (max) 60.000 -4.76%
481.wrf (max) (percentage) 1.090 -6.03%
481.wrf (unexplained diff vs previous) 49.000 16.67%
481.wrf (unexplained diff vs previous) (percentage) 0.880 15.79%
482.sphinx3 27.000 -
482.sphinx3 (avg) 27.000 -
482.sphinx3 (avg) (percentage) 2.930 -1.01%
482.sphinx3 (max) 14.000 55.56%
482.sphinx3 (max) (percentage) 1.490 49.00%
482.sphinx3 (unexplained diff vs previous) 14.000 366.67%
482.sphinx3 (unexplained diff vs previous) (percentage) 1.470 359.37%
483.xalancbmk 50.000 4.17%
483.xalancbmk (avg) 50.000 4.17%
483.xalancbmk (avg) (percentage) 1.180 -
483.xalancbmk (max) 260.000 3.17%
483.xalancbmk (max) (percentage) 6.080 -
Geometric Mean 7.024 -7.67%