Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=de2efa143e3652d69c278dd1eb10a856593917c0 (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/7ab79a40b546a1470abaf76bec74c63e9990fe47 (+ 1 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=f957f47df75b9fab995754011491edebc6feb147 (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=288a2f83ed42ce91cf92c2b08dad298a3da0cd9a (unchanged)
regression GNU-995
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/146/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0146 (basepoints/gcc-14-05110-g7ab79a40b54) 2023-11-02T17:50:56 0:00:00
Previous 0145 (basepoints/gcc-14-05109-ga291237b628) 2023-11-01T11:41:43 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 28 14
Performance Improvements 105 193
Added Tests 16 2
Unchanged Tests 1087 1027
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 7.50% 11283.000 12129.000 - 4.20% -
403.gcc 1.69% 59.000 60.000 - 7.14% -
454.calculix 1.64% 61.000 62.000 - 0.00% -
400.perlbench 1.52% 1510.000 1533.000 - -7.82% -
462.libquantum 1.45% 138.000 140.000 - 0.00% -
429.mcf 1.41% 1420.000 1440.000 - 1.12% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
434.zeusmp (unexplained diff vs previous) 600.00% 1.000 7.000 - -99.94% -
434.zeusmp (unexplained diff vs previous) (percentage) 575.00% 0.040 0.270 - -99.94% -
445.gobmk (unexplained diff vs previous) 457.14% 7.000 39.000 - -99.30% -
445.gobmk (unexplained diff vs previous) (percentage) 450.00% 0.140 0.770 - -99.31% -
437.leslie3d (unexplained diff vs previous) 179.21% 303.000 846.000 - -63.44% -
437.leslie3d (unexplained diff vs previous) (percentage) 159.48% 2.690 6.980 - -64.89% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - -99.99% -
462.libquantum (unexplained diff vs previous) (percentage) 98.61% 0.720 1.430 - -99.99% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 67.65% 0.340 0.570 - -99.77% -
459.GemsFDTD (unexplained diff vs previous) 66.67% 21.000 35.000 - -99.77% -
401.bzip2 (unexplained diff vs previous) (percentage) 55.22% 0.670 1.040 - -99.72% -
401.bzip2 (unexplained diff vs previous) 55.00% 20.000 31.000 - -99.71% -
450.soplex (unexplained diff vs previous) (percentage) 53.89% 1.670 2.570 - -99.90% -
403.gcc 50.00% 2.000 3.000 - -25.00% -
403.gcc (avg) 50.00% 2.000 3.000 - -25.00% -
450.soplex (unexplained diff vs previous) 50.00% 10.000 15.000 - -99.90% -
473.astar (unexplained diff vs previous) (percentage) 19.17% 1.930 2.300 - -98.84% -
473.astar (unexplained diff vs previous) 16.67% 102.000 119.000 - -98.83% -
437.leslie3d 8.29% 591.000 640.000 - 80.28% -
437.leslie3d (avg) 8.29% 591.000 640.000 - 80.28% -
437.leslie3d (max) 2.43% 1402.000 1436.000 - -25.56% -
429.mcf (max) 1.38% 145.000 147.000 - -88.88% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
464.h264ref -1.06% 385948.000 381840.000 - -1.52% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
450.soplex -2.50% 599.000 584.000 - 3.00% -
473.astar -2.25% 5288.000 5169.000 - 0.80% -
401.bzip2 -1.03% 3003.000 2972.000 - 1.05% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
458.sjeng (unexplained diff vs previous) -81.82% 33.000 6.000 - -99.91% -
458.sjeng (unexplained diff vs previous) (percentage) -81.82% 0.440 0.080 - -99.91% -
454.calculix (max) -75.00% 4.000 1.000 - -98.36% -
454.calculix (max) (percentage) -64.91% 8.180 2.870 - -97.12% -
444.namd (unexplained diff vs previous) -62.50% 16.000 6.000 - -99.93% -
444.namd (unexplained diff vs previous) (percentage) -62.33% 2.150 0.810 - -99.93% -
456.hmmer (max) -55.56% 9.000 4.000 - -99.81% -
471.omnetpp (unexplained diff vs previous) (percentage) -53.23% 1.240 0.580 - -99.85% -
471.omnetpp (unexplained diff vs previous) -52.94% 34.000 16.000 - -99.85% -
456.hmmer (max) (percentage) -50.00% 0.340 0.170 - -99.77% -
410.bwaves (max) -47.92% 48.000 25.000 - -99.24% -
410.bwaves (max) (percentage) -46.51% 0.860 0.460 - -99.21% -
433.milc (unexplained diff vs previous) -33.33% 6.000 4.000 - -99.96% -
433.milc (unexplained diff vs previous) (percentage) -32.69% 0.520 0.350 - -99.96% -
458.sjeng (max) -28.07% 57.000 41.000 - -98.86% -
458.sjeng (max) (percentage) -27.27% 0.770 0.560 - -98.82% -
436.cactusADM (unexplained diff vs previous) (percentage) -23.26% 0.430 0.330 - -99.96% -
436.cactusADM (unexplained diff vs previous) -22.22% 9.000 7.000 - -99.96% -
410.bwaves (unexplained diff vs previous) (percentage) -22.03% 0.590 0.460 - -99.67% -
410.bwaves (unexplained diff vs previous) -21.21% 33.000 26.000 - -99.67% -
453.povray (max) -20.00% 5.000 4.000 - -99.20% -
444.namd (max) (percentage) -8.53% 2.110 1.930 - -97.90% -
453.povray (max) (percentage) -8.25% 0.970 0.890 - -99.02% -
471.omnetpp (max) -8.11% 37.000 34.000 - -98.43% -
471.omnetpp (max) (percentage) -8.09% 1.360 1.250 - -98.43% -
444.namd (max) -6.67% 15.000 14.000 - -97.96% -
436.cactusADM (max) (percentage) -6.52% 0.460 0.430 - -99.52% -
444.namd -5.88% 17.000 16.000 - -76.47% -
444.namd (avg) -5.88% 17.000 16.000 - -76.47% -
450.soplex (max) -5.26% 19.000 18.000 - -96.70% -
473.astar (max) -4.96% 141.000 134.000 - -96.07% -
416.gamess (avg) (percentage) -4.84% 1.240 1.180 - -73.30% -
437.leslie3d (max) (percentage) -4.75% 12.430 11.840 - -28.59% -
462.libquantum (max) (percentage) -4.60% 1.740 1.660 - -98.32% -
416.gamess -4.55% 154.000 147.000 - -73.27% -
416.gamess (avg) -4.55% 154.000 147.000 - -73.27% -
435.gromacs (max) -4.37% 229.000 219.000 - -93.87% -
482.sphinx3 -4.00% 25.000 24.000 - -67.57% -
482.sphinx3 (avg) -4.00% 25.000 24.000 - -67.57% -
401.bzip2 (max) -3.23% 93.000 90.000 - -96.11% -
435.gromacs (max) (percentage) -3.18% 3.140 3.040 - -93.82% -
456.hmmer -3.12% 32.000 31.000 - -75.59% -
456.hmmer (avg) -3.12% 32.000 31.000 - -75.59% -
470.lbm (max) (percentage) -2.94% 2.720 2.640 - -96.36% -
447.dealII (max) -2.94% 34.000 33.000 - -97.62% -
470.lbm (max) -2.91% 103.000 100.000 - -96.36% -
435.gromacs -2.91% 172.000 167.000 - -54.50% -
435.gromacs (avg) -2.91% 172.000 167.000 - -54.50% -
400.perlbench (max) (percentage) -2.79% 4.300 4.180 - -95.03% -
445.gobmk (max) (percentage) -2.78% 1.080 1.050 - -98.00% -
473.astar (max) (percentage) -2.62% 2.670 2.600 - -96.09% -
453.povray (avg) (percentage) -2.62% 2.290 2.230 - -71.45% -
401.bzip2 (max) (percentage) -2.56% 3.120 3.040 - -96.13% -
434.zeusmp (max) (percentage) -2.55% 1.570 1.530 - -98.13% -
473.astar -2.54% 118.000 115.000 - -46.01% -
473.astar (avg) -2.54% 118.000 115.000 - -46.01% -
458.sjeng -2.47% 81.000 79.000 - -67.22% -
458.sjeng (avg) -2.47% 81.000 79.000 - -67.22% -
464.h264ref (max) -2.41% 83.000 81.000 - -97.41% -
465.tonto (avg) (percentage) -2.33% 1.720 1.680 - -49.70% -
434.zeusmp -2.33% 43.000 42.000 - -71.43% -
434.zeusmp (avg) -2.33% 43.000 42.000 - -71.43% -
481.wrf (avg) (percentage) -2.26% 2.210 2.160 - -65.38% -
482.sphinx3 (max) (percentage) -2.26% 7.090 6.930 - -92.82% -
465.tonto (max) (percentage) -2.25% 3.550 3.470 - -82.03% -
465.tonto -2.09% 191.000 187.000 - -48.77% -
465.tonto (avg) -2.09% 191.000 187.000 - -48.77% -
464.h264ref (max) (percentage) -2.05% 1.460 1.430 - -97.37% -
403.gcc (max) (percentage) -2.01% 4.970 4.870 - -95.10% -
447.dealII (max) (percentage) -2.00% 2.000 1.960 - -97.56% -
481.wrf (max) (percentage) -2.00% 2.500 2.450 - -96.43% -
482.sphinx3 (avg) (percentage) -1.86% 2.690 2.640 - -66.71% -
445.gobmk (max) -1.85% 54.000 53.000 - -97.99% -
433.milc (avg) (percentage) -1.83% 2.180 2.140 - -64.63% -
458.sjeng (avg) (percentage) -1.83% 1.090 1.070 - -66.46% -
456.hmmer (avg) (percentage) -1.80% 1.110 1.090 - -75.28% -
465.tonto (max) -1.78% 394.000 387.000 - -81.68% -
434.zeusmp (avg) (percentage) -1.76% 1.700 1.670 - -71.45% -
435.gromacs (avg) (percentage) -1.69% 2.360 2.320 - -54.15% -
470.lbm (avg) (percentage) -1.69% 1.770 1.740 - -72.51% -
454.calculix (avg) (percentage) -1.66% 4.210 4.140 - -48.38% -
429.mcf (avg) (percentage) -1.65% 7.270 7.150 - -6.78% -
481.wrf -1.64% 122.000 120.000 - -65.22% -
481.wrf (avg) -1.64% 122.000 120.000 - -65.22% -
401.bzip2 -1.59% 63.000 62.000 - -61.49% -
401.bzip2 (avg) -1.59% 63.000 62.000 - -61.49% -
459.GemsFDTD (avg) (percentage) -1.55% 1.940 1.910 - -67.29% -
482.sphinx3 (max) -1.52% 66.000 65.000 - -92.86% -
436.cactusADM (avg) (percentage) -1.51% 1.990 1.960 - -67.22% -
470.lbm -1.49% 67.000 66.000 - -72.38% -
470.lbm (avg) -1.49% 67.000 66.000 - -72.38% -
481.wrf (max) -1.45% 138.000 136.000 - -96.42% -
445.gobmk (avg) (percentage) -1.42% 1.410 1.390 - -64.45% -
401.bzip2 (avg) (percentage) -1.42% 2.120 2.090 - -61.86% -
471.omnetpp (avg) (percentage) -1.38% 2.890 2.850 - -65.24% -
410.bwaves (avg) (percentage) -1.37% 1.460 1.440 - -67.35% -
444.namd (avg) (percentage) -1.29% 2.320 2.290 - -75.16% -
471.omnetpp -1.27% 79.000 78.000 - -65.02% -
471.omnetpp (avg) -1.27% 79.000 78.000 - -65.02% -
410.bwaves -1.22% 82.000 81.000 - -67.47% -
410.bwaves (avg) -1.22% 82.000 81.000 - -67.47% -

Added Tests - sample variability
416.gamess (unexplained diff vs previous)
416.gamess (unexplained diff vs previous) (percentage)
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)
481.wrf (unexplained diff vs previous)
481.wrf (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
400.perlbench 1.62% 893848.000 908307.000 - 0.45% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc 7.14% 56.000 60.000 - 1.69% -
437.leslie3d 4.20% 11640.000 12129.000 - 7.50% -
450.soplex 3.00% 567.000 584.000 - -2.50% -
465.tonto 2.02% 10941.000 11162.000 - 0.43% -
434.zeusmp 1.59% 2514.000 2554.000 - 0.27% -
471.omnetpp 1.32% 2724.000 2760.000 - 0.58% -
429.mcf 1.12% 1424.000 1440.000 - 1.41% -
401.bzip2 1.05% 2941.000 2972.000 - -1.03% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d 80.28% 355.000 640.000 - 8.29% -
437.leslie3d (avg) 80.28% 355.000 640.000 - 8.29% -
437.leslie3d (avg) (percentage) 73.11% 3.050 5.280 - 0.76% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
434.zeusmp -1.82% 152037.000 149275.000 - 0.00% -
464.h264ref -1.52% 387748.000 381840.000 - -1.06% -
459.GemsFDTD -1.27% 244728.000 241628.000 - 0.00% -
437.leslie3d -1.04% 59479.000 58862.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
400.perlbench -7.82% 1663.000 1533.000 - 1.52% -
444.namd -1.34% 747.000 737.000 - -0.81% -
464.h264ref -1.32% 5761.000 5685.000 - -0.59% -
458.sjeng -1.22% 7567.000 7475.000 - -0.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
454.calculix (unexplained diff vs previous) (percentage) -100.00% 37020.970 1.610 - 0.00% -
454.calculix (unexplained diff vs previous) -100.00% 22953.000 1.000 - 0.00% -
462.libquantum (unexplained diff vs previous) -99.99% 13518.000 2.000 - 100.00% -
462.libquantum (unexplained diff vs previous) (percentage) -99.99% 9655.710 1.430 - 98.61% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.98% 2759.050 0.420 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -99.98% 26073.000 4.000 - 0.00% -
433.milc (unexplained diff vs previous) -99.96% 11289.000 4.000 - -33.33% -
433.milc (unexplained diff vs previous) (percentage) -99.96% 980.800 0.350 - -32.69% -
436.cactusADM (unexplained diff vs previous) (percentage) -99.96% 787.130 0.330 - -23.26% -
436.cactusADM (unexplained diff vs previous) -99.96% 16396.000 7.000 - -22.22% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.94% 448.690 0.270 - 575.00% -
456.hmmer (unexplained diff vs previous) (percentage) -99.94% 278.630 0.170 - 0.00% -
434.zeusmp (unexplained diff vs previous) -99.94% 11280.000 7.000 - 600.00% -
456.hmmer (unexplained diff vs previous) -99.94% 8030.000 5.000 - 0.00% -
416.gamess (unexplained diff vs previous) (percentage) -99.94% 62.730 0.040 - 0.00% -
416.gamess (unexplained diff vs previous) -99.94% 7819.000 5.000 - 0.00% -
444.namd (unexplained diff vs previous) -99.93% 8356.000 6.000 - -62.50% -
444.namd (unexplained diff vs previous) (percentage) -99.93% 1118.610 0.810 - -62.33% -
458.sjeng (unexplained diff vs previous) -99.91% 6876.000 6.000 - -81.82% -
458.sjeng (unexplained diff vs previous) (percentage) -99.91% 90.870 0.080 - -81.82% -
450.soplex (unexplained diff vs previous) (percentage) -99.90% 2557.320 2.570 - 53.89% -
450.soplex (unexplained diff vs previous) -99.90% 14500.000 15.000 - 50.00% -
429.mcf (unexplained diff vs previous) (percentage) -99.89% 1305.690 1.390 - 0.00% -
429.mcf (unexplained diff vs previous) -99.89% 18593.000 20.000 - 0.00% -
470.lbm (unexplained diff vs previous) -99.86% 9990.000 14.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -99.86% 263.730 0.370 - 0.00% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.85% 391.300 0.580 - -53.23% -
471.omnetpp (unexplained diff vs previous) -99.85% 10659.000 16.000 - -52.94% -
456.hmmer (max) -99.81% 2120.000 4.000 - -55.56% -
459.GemsFDTD (unexplained diff vs previous) -99.77% 15458.000 35.000 - 66.67% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -99.77% 250.410 0.570 - 67.65% -
416.gamess (max) -99.77% 4805.000 11.000 - 0.00% -
456.hmmer (max) (percentage) -99.77% 73.590 0.170 - -50.00% -
416.gamess (max) (percentage) -99.77% 38.550 0.090 - 0.00% -
481.wrf (unexplained diff vs previous) (percentage) -99.75% 219.530 0.540 - 0.00% -
481.wrf (unexplained diff vs previous) -99.75% 12149.000 30.000 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.72% 368.340 1.040 - 55.22% -
401.bzip2 (unexplained diff vs previous) -99.71% 10833.000 31.000 - 55.00% -
410.bwaves (unexplained diff vs previous) (percentage) -99.67% 137.900 0.460 - -22.03% -
410.bwaves (unexplained diff vs previous) -99.67% 7794.000 26.000 - -21.21% -
436.cactusADM (max) (percentage) -99.52% 88.730 0.430 - -6.52% -
436.cactusADM (max) -99.51% 1848.000 9.000 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) -99.31% 111.110 0.770 - 450.00% -
445.gobmk (unexplained diff vs previous) -99.30% 5581.000 39.000 - 457.14% -
410.bwaves (max) -99.24% 3276.000 25.000 - -47.92% -
410.bwaves (max) (percentage) -99.21% 57.970 0.460 - -46.51% -
453.povray (max) -99.20% 499.000 4.000 - -20.00% -
435.gromacs (unexplained diff vs previous) -99.08% 7028.000 65.000 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) -99.07% 96.680 0.900 - 0.00% -
453.povray (max) (percentage) -99.02% 91.250 0.890 - -8.25% -
458.sjeng (max) -98.86% 3602.000 41.000 - -28.07% -
473.astar (unexplained diff vs previous) (percentage) -98.84% 197.850 2.300 - 19.17% -
473.astar (unexplained diff vs previous) -98.83% 10146.000 119.000 - 16.67% -
458.sjeng (max) (percentage) -98.82% 47.610 0.560 - -27.27% -
462.libquantum (max) -98.55% 138.000 2.000 - 0.00% -
471.omnetpp (max) -98.43% 2169.000 34.000 - -8.11% -
471.omnetpp (max) (percentage) -98.43% 79.650 1.250 - -8.09% -
454.calculix (max) -98.36% 61.000 1.000 - -75.00% -
462.libquantum (max) (percentage) -98.32% 98.970 1.660 - -4.60% -
434.zeusmp (max) (percentage) -98.13% 81.770 1.530 - -2.55% -
434.zeusmp (max) -98.10% 2055.000 39.000 - 0.00% -
445.gobmk (max) (percentage) -98.00% 52.630 1.050 - -2.78% -
445.gobmk (max) -97.99% 2643.000 53.000 - -1.85% -
444.namd (max) -97.96% 685.000 14.000 - -6.67% -
444.namd (max) (percentage) -97.90% 91.790 1.930 - -8.53% -
447.dealII (max) -97.62% 1384.000 33.000 - -2.94% -
447.dealII (max) (percentage) -97.56% 80.280 1.960 - -2.00% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
464.h264ref (max) -97.41% 3127.000 81.000 - -2.41% -
464.h264ref (max) (percentage) -97.37% 54.290 1.430 - -2.05% -
465.tonto (unexplained diff vs previous) (percentage) -97.34% 16.190 0.430 - 0.00% -
465.tonto (unexplained diff vs previous) -97.29% 1771.000 48.000 - 0.00% -
454.calculix (max) (percentage) -97.12% 99.730 2.870 - -64.91% -
459.GemsFDTD (max) (percentage) -97.10% 71.460 2.070 - 0.00% -
459.GemsFDTD (max) -97.10% 4411.000 128.000 - 0.79% -
450.soplex (max) -96.70% 545.000 18.000 - -5.26% -
450.soplex (max) (percentage) -96.69% 96.240 3.190 - 0.00% -
481.wrf (max) (percentage) -96.43% 68.700 2.450 - -2.00% -
481.wrf (max) -96.42% 3801.000 136.000 - -1.45% -
403.gcc (max) -96.36% 55.000 2.000 - 0.00% -
470.lbm (max) (percentage) -96.36% 72.510 2.640 - -2.94% -
470.lbm (max) -96.36% 2746.000 100.000 - -2.91% -
401.bzip2 (max) (percentage) -96.13% 78.650 3.040 - -2.56% -
401.bzip2 (max) -96.11% 2313.000 90.000 - -3.23% -
473.astar (max) (percentage) -96.09% 66.430 2.600 - -2.62% -
433.milc (max) -96.07% 1044.000 41.000 - 0.00% -
473.astar (max) -96.07% 3406.000 134.000 - -4.96% -
433.milc (max) (percentage) -96.03% 90.750 3.600 - 0.00% -
400.perlbench (max) -95.42% 1397.000 64.000 - 0.00% -
403.gcc (max) (percentage) -95.10% 99.360 4.870 - -2.01% -
400.perlbench (max) (percentage) -95.03% 84.060 4.180 - -2.79% -
435.gromacs (max) -93.87% 3573.000 219.000 - -4.37% -
435.gromacs (max) (percentage) -93.82% 49.160 3.040 - -3.18% -
482.sphinx3 (max) -92.86% 911.000 65.000 - -1.52% -
482.sphinx3 (max) (percentage) -92.82% 96.500 6.930 - -2.26% -
429.mcf (max) (percentage) -88.95% 92.890 10.260 - 0.00% -
429.mcf (max) -88.88% 1322.000 147.000 - 1.38% -
465.tonto (max) (percentage) -82.03% 19.310 3.470 - -2.25% -
465.tonto (max) -81.68% 2112.000 387.000 - -1.78% -
444.namd -76.47% 68.000 16.000 - -5.88% -
444.namd (avg) -76.47% 68.000 16.000 - -5.88% -
456.hmmer -75.59% 127.000 31.000 - -3.12% -
456.hmmer (avg) -75.59% 127.000 31.000 - -3.12% -
456.hmmer (avg) (percentage) -75.28% 4.410 1.090 - -1.80% -
444.namd (avg) (percentage) -75.16% 9.220 2.290 - -1.29% -
416.gamess (avg) (percentage) -73.30% 4.420 1.180 - -4.84% -
416.gamess -73.27% 550.000 147.000 - -4.55% -
416.gamess (avg) -73.27% 550.000 147.000 - -4.55% -
470.lbm (avg) (percentage) -72.51% 6.330 1.740 - -1.69% -
470.lbm -72.38% 239.000 66.000 - -1.49% -
470.lbm (avg) -72.38% 239.000 66.000 - -1.49% -
434.zeusmp (avg) (percentage) -71.45% 5.850 1.670 - -1.76% -
453.povray (avg) (percentage) -71.45% 7.810 2.230 - -2.62% -
434.zeusmp -71.43% 147.000 42.000 - -2.33% -
434.zeusmp (avg) -71.43% 147.000 42.000 - -2.33% -
453.povray -71.43% 42.000 12.000 - 0.00% -
453.povray (avg) -71.43% 42.000 12.000 - 0.00% -
462.libquantum (avg) (percentage) -71.02% 7.660 2.220 - -0.89% -
462.libquantum -70.00% 10.000 3.000 - 0.00% -
462.libquantum (avg) -70.00% 10.000 3.000 - 0.00% -
464.h264ref -69.62% 293.000 89.000 - 0.00% -
464.h264ref (avg) -69.62% 293.000 89.000 - 0.00% -
464.h264ref (avg) (percentage) -69.16% 5.090 1.570 - 0.00% -
447.dealII -67.68% 99.000 32.000 - 0.00% -
447.dealII (avg) -67.68% 99.000 32.000 - 0.00% -
482.sphinx3 -67.57% 74.000 24.000 - -4.00% -
482.sphinx3 (avg) -67.57% 74.000 24.000 - -4.00% -
410.bwaves -67.47% 249.000 81.000 - -1.22% -
410.bwaves (avg) -67.47% 249.000 81.000 - -1.22% -
410.bwaves (avg) (percentage) -67.35% 4.410 1.440 - -1.37% -
459.GemsFDTD (avg) (percentage) -67.29% 5.840 1.910 - -1.55% -
436.cactusADM (avg) (percentage) -67.22% 5.980 1.960 - -1.51% -
459.GemsFDTD -67.22% 360.000 118.000 - -0.84% -
459.GemsFDTD (avg) -67.22% 360.000 118.000 - -0.84% -
458.sjeng -67.22% 241.000 79.000 - -2.47% -
458.sjeng (avg) -67.22% 241.000 79.000 - -2.47% -
436.cactusADM -66.94% 124.000 41.000 - 0.00% -
436.cactusADM (avg) -66.94% 124.000 41.000 - 0.00% -
447.dealII (avg) (percentage) -66.72% 5.770 1.920 - 0.00% -
482.sphinx3 (avg) (percentage) -66.71% 7.930 2.640 - -1.86% -
458.sjeng (avg) (percentage) -66.46% 3.190 1.070 - -1.83% -
481.wrf (avg) (percentage) -65.38% 6.240 2.160 - -2.26% -
471.omnetpp (avg) (percentage) -65.24% 8.200 2.850 - -1.38% -
433.milc -65.22% 69.000 24.000 - 0.00% -
433.milc (avg) -65.22% 69.000 24.000 - 0.00% -
481.wrf -65.22% 345.000 120.000 - -1.64% -
481.wrf (avg) -65.22% 345.000 120.000 - -1.64% -
471.omnetpp -65.02% 223.000 78.000 - -1.27% -
471.omnetpp (avg) -65.02% 223.000 78.000 - -1.27% -
437.leslie3d (unexplained diff vs previous) (percentage) -64.89% 19.880 6.980 - 159.48% -
433.milc (avg) (percentage) -64.63% 6.050 2.140 - -1.83% -
445.gobmk (avg) (percentage) -64.45% 3.910 1.390 - -1.42% -
445.gobmk -64.29% 196.000 70.000 - 0.00% -
445.gobmk (avg) -64.29% 196.000 70.000 - 0.00% -
437.leslie3d (unexplained diff vs previous) -63.44% 2314.000 846.000 - 179.21% -
401.bzip2 (avg) (percentage) -61.86% 5.480 2.090 - -1.42% -
401.bzip2 -61.49% 161.000 62.000 - -1.59% -
401.bzip2 (avg) -61.49% 161.000 62.000 - -1.59% -
400.perlbench -60.95% 105.000 41.000 - 0.00% -
400.perlbench (avg) -60.95% 105.000 41.000 - 0.00% -
400.perlbench (avg) (percentage) -57.01% 6.350 2.730 - 0.00% -
435.gromacs -54.50% 367.000 167.000 - -2.91% -
435.gromacs (avg) -54.50% 367.000 167.000 - -2.91% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
435.gromacs (avg) (percentage) -54.15% 5.060 2.320 - -1.69% -
450.soplex (avg) (percentage) -50.86% 6.430 3.160 - -0.32% -
450.soplex -50.00% 36.000 18.000 - 0.00% -
450.soplex (avg) -50.00% 36.000 18.000 - 0.00% -
454.calculix -50.00% 4.000 2.000 - 0.00% -
454.calculix (avg) -50.00% 4.000 2.000 - 0.00% -
465.tonto (avg) (percentage) -49.70% 3.340 1.680 - -2.33% -
465.tonto -48.77% 365.000 187.000 - -2.09% -
465.tonto (avg) -48.77% 365.000 187.000 - -2.09% -
454.calculix (avg) (percentage) -48.38% 8.020 4.140 - -1.66% -
473.astar (avg) (percentage) -46.15% 4.160 2.240 - 0.00% -
473.astar -46.01% 213.000 115.000 - -2.54% -
473.astar (avg) -46.01% 213.000 115.000 - -2.54% -
403.gcc (avg) (percentage) -31.42% 7.320 5.020 - 0.00% -
437.leslie3d (max) (percentage) -28.59% 16.580 11.840 - -4.75% -
437.leslie3d (max) -25.56% 1929.000 1436.000 - 2.43% -
403.gcc -25.00% 4.000 3.000 - 50.00% -
403.gcc (avg) -25.00% 4.000 3.000 - 50.00% -
429.mcf (avg) (percentage) -6.78% 7.670 7.150 - -1.65% -
429.mcf -6.42% 109.000 102.000 - -0.97% -
429.mcf (avg) -6.42% 109.000 102.000 - -0.97% -

Added Tests - sample variability
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)

Report Time: 0.52s
Execution (seconds)
Name Current %
400.perlbench 1533.000 1.52%
401.bzip2 2972.000 -1.03%
403.gcc 60.000 1.69%
410.bwaves 5652.000 -
416.gamess 12480.000 -
429.mcf 1440.000 1.41%
433.milc 1142.000 -
434.zeusmp 2554.000 -
435.gromacs 7231.000 -
436.cactusADM 2102.000 -
437.leslie3d 12129.000 7.50%
444.namd 737.000 -
445.gobmk 5055.000 -
447.dealII 1716.000 -
450.soplex 584.000 -2.50%
453.povray 545.000 -
454.calculix 62.000 1.64%
456.hmmer 2902.000 -
458.sjeng 7475.000 -
459.GemsFDTD 6191.000 -
462.libquantum 140.000 1.45%
464.h264ref 5685.000 -
465.tonto 11162.000 -
470.lbm 3819.000 -
471.omnetpp 2760.000 -
473.astar 5169.000 -2.25%
481.wrf 5572.000 -
482.sphinx3 946.000 -
Geometric Mean 2063.650 -
Code Size (bytes)
Name Current %
400.perlbench 908307.000 -
401.bzip2 44031.000 -
403.gcc 2452575.000 -
410.bwaves 16604.000 -
416.gamess 5722065.000 -
429.mcf 9119.000 -
433.milc 61125.000 -
434.zeusmp 149275.000 -
435.gromacs 502429.000 -
436.cactusADM 350969.000 -
437.leslie3d 58862.000 -
444.namd 188128.000 -
445.gobmk 1378310.000 -
447.dealII 311118.000 -
450.soplex 204302.000 -
453.povray 657825.000 -
454.calculix 840334.000 -
456.hmmer 108287.000 -
458.sjeng 103652.000 -
459.GemsFDTD 241628.000 -
462.libquantum 16053.000 -
464.h264ref 381840.000 -1.06%
465.tonto 1952158.000 -
470.lbm 9707.000 -
471.omnetpp 480712.000 -
473.astar 22920.000 -
481.wrf 1945622.000 -
482.sphinx3 112775.000 -
Geometric Mean 205301.524 -
sample variability
Name Current %
400.perlbench 41.000 -
400.perlbench (avg) 41.000 -
400.perlbench (avg) (percentage) 2.730 -
400.perlbench (max) 64.000 -
400.perlbench (max) (percentage) 4.180 -2.79%
401.bzip2 62.000 -1.59%
401.bzip2 (avg) 62.000 -1.59%
401.bzip2 (avg) (percentage) 2.090 -1.42%
401.bzip2 (max) 90.000 -3.23%
401.bzip2 (max) (percentage) 3.040 -2.56%
401.bzip2 (unexplained diff vs previous) 31.000 55.00%
401.bzip2 (unexplained diff vs previous) (percentage) 1.040 55.22%
403.gcc 3.000 50.00%
403.gcc (avg) 3.000 50.00%
403.gcc (avg) (percentage) 5.020 -
403.gcc (max) 2.000 -
403.gcc (max) (percentage) 4.870 -2.01%
410.bwaves 81.000 -1.22%
410.bwaves (avg) 81.000 -1.22%
410.bwaves (avg) (percentage) 1.440 -1.37%
410.bwaves (max) 25.000 -47.92%
410.bwaves (max) (percentage) 0.460 -46.51%
410.bwaves (unexplained diff vs previous) 26.000 -21.21%
410.bwaves (unexplained diff vs previous) (percentage) 0.460 -22.03%
416.gamess 147.000 -4.55%
416.gamess (avg) 147.000 -4.55%
416.gamess (avg) (percentage) 1.180 -4.84%
416.gamess (max) 11.000 -
416.gamess (max) (percentage) 0.090 -
416.gamess (unexplained diff vs previous) 5.000 -
416.gamess (unexplained diff vs previous) (percentage) 0.040 -
429.mcf 102.000 -
429.mcf (avg) 102.000 -
429.mcf (avg) (percentage) 7.150 -1.65%
429.mcf (max) 147.000 1.38%
429.mcf (max) (percentage) 10.260 -
429.mcf (unexplained diff vs previous) 20.000 -
429.mcf (unexplained diff vs previous) (percentage) 1.390 -
433.milc 24.000 -
433.milc (avg) 24.000 -
433.milc (avg) (percentage) 2.140 -1.83%
433.milc (max) 41.000 -
433.milc (max) (percentage) 3.600 -
433.milc (unexplained diff vs previous) 4.000 -33.33%
433.milc (unexplained diff vs previous) (percentage) 0.350 -32.69%
434.zeusmp 42.000 -2.33%
434.zeusmp (avg) 42.000 -2.33%
434.zeusmp (avg) (percentage) 1.670 -1.76%
434.zeusmp (max) 39.000 -
434.zeusmp (max) (percentage) 1.530 -2.55%
434.zeusmp (unexplained diff vs previous) 7.000 600.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.270 575.00%
435.gromacs 167.000 -2.91%
435.gromacs (avg) 167.000 -2.91%
435.gromacs (avg) (percentage) 2.320 -1.69%
435.gromacs (max) 219.000 -4.37%
435.gromacs (max) (percentage) 3.040 -3.18%
435.gromacs (unexplained diff vs previous) 65.000 -
435.gromacs (unexplained diff vs previous) (percentage) 0.900 -
436.cactusADM 41.000 -
436.cactusADM (avg) 41.000 -
436.cactusADM (avg) (percentage) 1.960 -1.51%
436.cactusADM (max) 9.000 -
436.cactusADM (max) (percentage) 0.430 -6.52%
436.cactusADM (unexplained diff vs previous) 7.000 -22.22%
436.cactusADM (unexplained diff vs previous) (percentage) 0.330 -23.26%
437.leslie3d 640.000 8.29%
437.leslie3d (avg) 640.000 8.29%
437.leslie3d (avg) (percentage) 5.280 -
437.leslie3d (max) 1436.000 2.43%
437.leslie3d (max) (percentage) 11.840 -4.75%
437.leslie3d (unexplained diff vs previous) 846.000 179.21%
437.leslie3d (unexplained diff vs previous) (percentage) 6.980 159.48%
444.namd 16.000 -5.88%
444.namd (avg) 16.000 -5.88%
444.namd (avg) (percentage) 2.290 -1.29%
444.namd (max) 14.000 -6.67%
444.namd (max) (percentage) 1.930 -8.53%
444.namd (unexplained diff vs previous) 6.000 -62.50%
444.namd (unexplained diff vs previous) (percentage) 0.810 -62.33%
445.gobmk 70.000 -
445.gobmk (avg) 70.000 -
445.gobmk (avg) (percentage) 1.390 -1.42%
445.gobmk (max) 53.000 -1.85%
445.gobmk (max) (percentage) 1.050 -2.78%
445.gobmk (unexplained diff vs previous) 39.000 457.14%
445.gobmk (unexplained diff vs previous) (percentage) 0.770 450.00%
447.dealII 32.000 -
447.dealII (avg) 32.000 -
447.dealII (avg) (percentage) 1.920 -
447.dealII (max) 33.000 -2.94%
447.dealII (max) (percentage) 1.960 -2.00%
450.soplex 18.000 -
450.soplex (avg) 18.000 -
450.soplex (avg) (percentage) 3.160 -
450.soplex (max) 18.000 -5.26%
450.soplex (max) (percentage) 3.190 -
450.soplex (unexplained diff vs previous) 15.000 50.00%
450.soplex (unexplained diff vs previous) (percentage) 2.570 53.89%
453.povray 12.000 -
453.povray (avg) 12.000 -
453.povray (avg) (percentage) 2.230 -2.62%
453.povray (max) 4.000 -20.00%
453.povray (max) (percentage) 0.890 -8.25%
453.povray (unexplained diff vs previous) 0.000 -
453.povray (unexplained diff vs previous) (percentage) 0.000 -
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 4.140 -1.66%
454.calculix (max) 1.000 -75.00%
454.calculix (max) (percentage) 2.870 -64.91%
454.calculix (unexplained diff vs previous) 1.000 -
454.calculix (unexplained diff vs previous) (percentage) 1.610 -
456.hmmer 31.000 -3.12%
456.hmmer (avg) 31.000 -3.12%
456.hmmer (avg) (percentage) 1.090 -1.80%
456.hmmer (max) 4.000 -55.56%
456.hmmer (max) (percentage) 0.170 -50.00%
456.hmmer (unexplained diff vs previous) 5.000 -
456.hmmer (unexplained diff vs previous) (percentage) 0.170 -
458.sjeng 79.000 -2.47%
458.sjeng (avg) 79.000 -2.47%
458.sjeng (avg) (percentage) 1.070 -1.83%
458.sjeng (max) 41.000 -28.07%
458.sjeng (max) (percentage) 0.560 -27.27%
458.sjeng (unexplained diff vs previous) 6.000 -81.82%
458.sjeng (unexplained diff vs previous) (percentage) 0.080 -81.82%
459.GemsFDTD 118.000 -
459.GemsFDTD (avg) 118.000 -
459.GemsFDTD (avg) (percentage) 1.910 -1.55%
459.GemsFDTD (max) 128.000 -
459.GemsFDTD (max) (percentage) 2.070 -
459.GemsFDTD (unexplained diff vs previous) 35.000 66.67%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.570 67.65%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 2.220 -
462.libquantum (max) 2.000 -
462.libquantum (max) (percentage) 1.660 -4.60%
462.libquantum (unexplained diff vs previous) 2.000 100.00%
462.libquantum (unexplained diff vs previous) (percentage) 1.430 98.61%
464.h264ref 89.000 -
464.h264ref (avg) 89.000 -
464.h264ref (avg) (percentage) 1.570 -
464.h264ref (max) 81.000 -2.41%
464.h264ref (max) (percentage) 1.430 -2.05%
465.tonto 187.000 -2.09%
465.tonto (avg) 187.000 -2.09%
465.tonto (avg) (percentage) 1.680 -2.33%
465.tonto (max) 387.000 -1.78%
465.tonto (max) (percentage) 3.470 -2.25%
465.tonto (unexplained diff vs previous) 48.000 -
465.tonto (unexplained diff vs previous) (percentage) 0.430 -
470.lbm 66.000 -1.49%
470.lbm (avg) 66.000 -1.49%
470.lbm (avg) (percentage) 1.740 -1.69%
470.lbm (max) 100.000 -2.91%
470.lbm (max) (percentage) 2.640 -2.94%
470.lbm (unexplained diff vs previous) 14.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.370 -
471.omnetpp 78.000 -1.27%
471.omnetpp (avg) 78.000 -1.27%
471.omnetpp (avg) (percentage) 2.850 -1.38%
471.omnetpp (max) 34.000 -8.11%
471.omnetpp (max) (percentage) 1.250 -8.09%
471.omnetpp (unexplained diff vs previous) 16.000 -52.94%
471.omnetpp (unexplained diff vs previous) (percentage) 0.580 -53.23%
473.astar 115.000 -2.54%
473.astar (avg) 115.000 -2.54%
473.astar (avg) (percentage) 2.240 -
473.astar (max) 134.000 -4.96%
473.astar (max) (percentage) 2.600 -2.62%
473.astar (unexplained diff vs previous) 119.000 16.67%
473.astar (unexplained diff vs previous) (percentage) 2.300 19.17%
481.wrf 120.000 -1.64%
481.wrf (avg) 120.000 -1.64%
481.wrf (avg) (percentage) 2.160 -2.26%
481.wrf (max) 136.000 -1.45%
481.wrf (max) (percentage) 2.450 -2.00%
481.wrf (unexplained diff vs previous) 30.000 -
481.wrf (unexplained diff vs previous) (percentage) 0.540 -
482.sphinx3 24.000 -4.00%
482.sphinx3 (avg) 24.000 -4.00%
482.sphinx3 (avg) (percentage) 2.640 -1.86%
482.sphinx3 (max) 65.000 -1.52%
482.sphinx3 (max) (percentage) 6.930 -2.26%
482.sphinx3 (unexplained diff vs previous) 4.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.420 -
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 7.579 -21.16%