Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=cd09b5ddefe662dcaa793cfa348730cbdee1e01b (+ 42 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/77faa3e198a6b6f9a55a8010bef1c394d2e3cf8e (+ 160 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=69239bd7a216007692470aa9d5f3658024638742 (+ 2 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=c476bbdaa3d7d176526c35c3353242443b07a27e (+ 158 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/100/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0100 (4-changed-components) 2023-10-10T13:14:42 0:00:00
Previous 0098 (3-changed-components) 2023-10-05T21:36:58 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 47 68
Performance Improvements 103 158
Added Tests 12 2
Unchanged Tests 1074 1008
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
454.calculix 6.35% 63.000 67.000 - -99.79% -
450.soplex 2.06% 583.000 595.000 - -96.21% -
429.mcf 1.94% 926.000 944.000 - -91.95% -
403.gcc 1.75% 57.000 58.000 - -99.35% -
473.astar 1.13% 4858.000 4913.000 - -62.16% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (unexplained diff vs previous) 1700.00% 1.000 18.000 - 80.00% -
429.mcf (unexplained diff vs previous) (percentage) 1636.36% 0.110 1.910 - 2022.22% -
450.soplex (unexplained diff vs previous) 1100.00% 1.000 12.000 - -82.61% -
450.soplex (unexplained diff vs previous) (percentage) 1088.24% 0.170 2.020 - 359.09% -
436.cactusADM (unexplained diff vs previous) 600.00% 1.000 7.000 - -99.40% -
436.cactusADM (unexplained diff vs previous) (percentage) 575.00% 0.040 0.270 - -94.39% -
445.gobmk (max) (percentage) 248.57% 0.350 1.220 - 351.85% -
445.gobmk (max) 244.44% 18.000 62.000 - 87.88% -
410.bwaves (unexplained diff vs previous) 227.27% 11.000 36.000 - -85.99% -
410.bwaves (unexplained diff vs previous) (percentage) 215.79% 0.190 0.600 - -65.52% -
447.dealII (unexplained diff vs previous) (percentage) 151.16% 0.430 1.080 - 44.00% -
447.dealII (unexplained diff vs previous) 150.00% 8.000 20.000 - -70.59% -
416.gamess (unexplained diff vs previous) 120.00% 20.000 44.000 - -95.09% -
416.gamess (unexplained diff vs previous) (percentage) 114.29% 0.140 0.300 - -90.16% -
403.gcc (max) 100.00% 1.000 2.000 - -96.15% -
434.zeusmp (unexplained diff vs previous) (percentage) 52.00% 0.500 0.760 - -66.22% -
429.mcf (max) 50.00% 12.000 18.000 - -51.35% -
434.zeusmp (unexplained diff vs previous) 50.00% 14.000 21.000 - -95.53% -
434.zeusmp (max) (percentage) 47.06% 0.510 0.750 - -67.39% -
429.mcf (max) (percentage) 44.78% 1.340 1.940 - 506.25% -
434.zeusmp (max) 42.86% 14.000 20.000 - -95.84% -
473.astar (unexplained diff vs previous) 37.50% 40.000 55.000 - -9.84% -
473.astar (unexplained diff vs previous) (percentage) 36.59% 0.820 1.120 - 138.30% -
454.calculix (max) 33.33% 3.000 4.000 - -98.53% -
456.hmmer (unexplained diff vs previous) (percentage) 24.14% 0.290 0.360 - -30.77% -
456.hmmer (unexplained diff vs previous) 23.08% 13.000 16.000 - -83.67% -
400.perlbench (unexplained diff vs previous) 22.22% 9.000 11.000 - -68.57% -
400.perlbench (unexplained diff vs previous) (percentage) 22.03% 0.590 0.720 - 132.26% -
459.GemsFDTD (unexplained diff vs previous) 20.00% 20.000 24.000 - -95.89% -
470.lbm (max) 20.00% 20.000 24.000 - -97.31% -
459.GemsFDTD (unexplained diff vs previous) (percentage) 17.86% 0.280 0.330 - -85.65% -
470.lbm (unexplained diff vs previous) 14.29% 21.000 24.000 - -97.08% -
470.lbm (max) (percentage) 14.29% 0.560 0.640 - -88.32% -
470.lbm (unexplained diff vs previous) (percentage) 14.29% 0.560 0.640 - -87.30% -
459.GemsFDTD (max) (percentage) 9.68% 0.310 0.340 - -85.09% -
459.GemsFDTD (max) 9.09% 22.000 24.000 - -95.85% -
454.calculix (max) (percentage) 5.83% 6.000 6.350 - 655.95% -
450.soplex 5.00% 20.000 21.000 - -43.24% -
450.soplex (avg) 5.00% 20.000 21.000 - -43.24% -
410.bwaves (max) 2.86% 35.000 36.000 - -86.15% -
454.calculix (avg) (percentage) 1.72% 4.060 4.130 - 103.45% -
410.bwaves (max) (percentage) 1.67% 0.600 0.610 - -65.54% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
445.gobmk -1.22% 5150.000 5087.000 - -58.96% -
447.dealII -1.07% 1873.000 1853.000 - -79.64% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
471.omnetpp (unexplained diff vs previous) -83.33% 18.000 3.000 - -95.16% -
471.omnetpp (unexplained diff vs previous) (percentage) -83.08% 0.650 0.110 - -83.58% -
437.leslie3d (unexplained diff vs previous) (percentage) -80.32% 4.930 0.970 - -5.83% -
437.leslie3d (unexplained diff vs previous) -80.03% 576.000 115.000 - -38.50% -
401.bzip2 (unexplained diff vs previous) -80.00% 30.000 6.000 - -91.30% -
401.bzip2 (unexplained diff vs previous) (percentage) -80.00% 1.000 0.200 - -51.22% -
444.namd (unexplained diff vs previous) (percentage) -75.28% 0.890 0.220 - -95.79% -
444.namd (unexplained diff vs previous) -75.00% 8.000 2.000 - -99.76% -
481.wrf (unexplained diff vs previous) -71.43% 56.000 16.000 - -98.21% -
481.wrf (unexplained diff vs previous) (percentage) -71.26% 0.870 0.250 - -93.86% -
464.h264ref (unexplained diff vs previous) (percentage) -65.62% 0.320 0.110 - -71.05% -
464.h264ref (unexplained diff vs previous) -65.00% 20.000 7.000 - -87.93% -
433.milc (unexplained diff vs previous) -50.00% 2.000 1.000 - -98.96% -
433.milc (unexplained diff vs previous) (percentage) -50.00% 0.180 0.090 - -87.14% -
458.sjeng (unexplained diff vs previous) -44.83% 29.000 16.000 - -66.67% -
458.sjeng (unexplained diff vs previous) (percentage) -44.74% 0.380 0.210 - -30.00% -
481.wrf (max) (percentage) -34.22% 1.870 1.230 - -70.43% -
481.wrf (max) -33.61% 119.000 79.000 - -91.36% -
462.libquantum (max) -33.33% 3.000 2.000 - -84.62% -
482.sphinx3 (unexplained diff vs previous) (percentage) -27.87% 1.220 0.880 - -71.24% -
482.sphinx3 (unexplained diff vs previous) -27.27% 11.000 8.000 - -99.28% -
471.omnetpp (max) (percentage) -24.62% 0.650 0.490 - -73.66% -
471.omnetpp (max) -23.53% 17.000 13.000 - -92.44% -
433.milc (max) -14.29% 7.000 6.000 - -93.81% -
444.namd (max) -12.50% 8.000 7.000 - -99.20% -
453.povray (max) (percentage) -10.47% 0.860 0.770 - -70.27% -
444.namd (max) (percentage) -10.11% 0.890 0.800 - -85.48% -
482.sphinx3 (max) (percentage) -8.33% 1.200 1.100 - -65.08% -
401.bzip2 (max) -8.11% 37.000 34.000 - -49.25% -
433.milc (max) (percentage) -7.58% 0.660 0.610 - -14.08% -
456.hmmer (max) (percentage) -7.50% 0.400 0.370 - -28.85% -
401.bzip2 (max) (percentage) -7.32% 1.230 1.140 - 185.00% -
458.sjeng (max) -6.90% 29.000 27.000 - -43.75% -
400.perlbench (max) -6.67% 30.000 28.000 - -68.54% -
464.h264ref (max) (percentage) -6.25% 0.320 0.300 - -23.08% -
400.perlbench (max) (percentage) -5.97% 2.010 1.890 - 136.25% -
456.hmmer (max) -5.88% 17.000 16.000 - -83.67% -
437.leslie3d (max) (percentage) -5.82% 7.220 6.800 - 119.35% -
458.sjeng (max) (percentage) -5.26% 0.380 0.360 - 20.00% -
464.h264ref (max) -5.26% 19.000 18.000 - -69.49% -
437.leslie3d (max) -4.86% 844.000 803.000 - 43.39% -
465.tonto (max) (percentage) -4.44% 1.350 1.290 - -24.56% -
447.dealII (max) (percentage) -3.90% 1.540 1.480 - -10.30% -
462.libquantum (max) (percentage) -3.68% 1.900 1.830 - 2187.50% -
447.dealII (max) -3.57% 28.000 27.000 - -82.00% -
465.tonto (max) -3.41% 176.000 170.000 - -23.77% -
464.h264ref -3.23% 93.000 90.000 - 52.54% -
464.h264ref (avg) -3.23% 93.000 90.000 - 52.54% -
436.cactusADM (max) (percentage) -3.13% 0.320 0.310 - -93.86% -
481.wrf (avg) (percentage) -3.07% 2.610 2.530 - -1.56% -
481.wrf -2.99% 167.000 162.000 - -71.33% -
481.wrf (avg) -2.99% 167.000 162.000 - -71.33% -
400.perlbench -2.94% 34.000 33.000 - -23.26% -
400.perlbench (avg) -2.94% 34.000 33.000 - -23.26% -
401.bzip2 (avg) (percentage) -2.74% 2.920 2.840 - 951.85% -
462.libquantum (avg) (percentage) -2.69% 2.230 2.170 - 1042.11% -
464.h264ref (avg) (percentage) -2.67% 1.500 1.460 - 274.36% -
473.astar (max) (percentage) -2.65% 1.890 1.840 - 291.49% -
450.soplex (max) (percentage) -2.63% 3.040 2.960 - 572.73% -
435.gromacs (avg) (percentage) -2.56% 2.730 2.660 - 180.00% -
435.gromacs -2.44% 205.000 200.000 - 25.00% -
435.gromacs (avg) -2.44% 205.000 200.000 - 25.00% -
444.namd (avg) (percentage) -2.39% 2.090 2.040 - -21.54% -
459.GemsFDTD (avg) (percentage) -2.31% 2.160 2.110 - 51.80% -
401.bzip2 -2.30% 87.000 85.000 - 88.89% -
401.bzip2 (avg) -2.30% 87.000 85.000 - 88.89% -
447.dealII -2.27% 44.000 43.000 - -45.57% -
447.dealII (avg) -2.27% 44.000 43.000 - -45.57% -
433.milc (avg) (percentage) -2.15% 2.330 2.280 - 230.43% -
436.cactusADM -2.04% 49.000 48.000 - -91.79% -
436.cactusADM (avg) -2.04% 49.000 48.000 - -91.79% -
471.omnetpp (avg) (percentage) -2.03% 2.460 2.410 - 104.24% -
459.GemsFDTD -1.95% 154.000 151.000 - -57.10% -
459.GemsFDTD (avg) -1.95% 154.000 151.000 - -57.10% -
437.leslie3d (avg) (percentage) -1.92% 3.640 3.570 - 113.77% -
434.zeusmp -1.85% 54.000 53.000 - -79.22% -
434.zeusmp (avg) -1.85% 54.000 53.000 - -79.22% -
429.mcf (avg) (percentage) -1.81% 7.180 7.050 - 1368.75% -
453.povray (avg) (percentage) -1.79% 2.230 2.190 - 43.14% -
400.perlbench (avg) (percentage) -1.77% 2.260 2.220 - 469.23% -
470.lbm (avg) (percentage) -1.77% 2.260 2.220 - -40.48% -
456.hmmer (avg) (percentage) -1.75% 1.140 1.120 - 273.33% -
410.bwaves (avg) (percentage) -1.70% 1.760 1.730 - -11.73% -
458.sjeng (avg) (percentage) -1.69% 1.180 1.160 - 404.35% -
447.dealII (avg) (percentage) -1.67% 2.400 2.360 - 171.26% -
416.gamess (avg) (percentage) -1.61% 1.240 1.220 - -26.06% -
482.sphinx3 (avg) (percentage) -1.59% 2.510 2.470 - 5.11% -
436.cactusADM (avg) (percentage) -1.58% 1.900 1.870 - -22.08% -
434.zeusmp (avg) (percentage) -1.55% 1.940 1.910 - 56.56% -
471.omnetpp -1.49% 67.000 66.000 - -39.45% -
471.omnetpp (avg) -1.49% 67.000 66.000 - -39.45% -
445.gobmk -1.33% 75.000 74.000 - 174.07% -
445.gobmk (avg) -1.33% 75.000 74.000 - 174.07% -
470.lbm -1.19% 84.000 83.000 - -86.35% -
470.lbm (avg) -1.19% 84.000 83.000 - -86.35% -
450.soplex (avg) (percentage) -1.12% 3.570 3.530 - 1370.83% -
458.sjeng -1.11% 90.000 89.000 - 147.22% -
458.sjeng (avg) -1.11% 90.000 89.000 - 147.22% -
416.gamess -1.10% 182.000 180.000 - -62.89% -
416.gamess (avg) -1.10% 182.000 180.000 - -62.89% -
473.astar (max) -1.10% 91.000 90.000 - 47.54% -

Added Tests - sample variability
435.gromacs (unexplained diff vs previous)
435.gromacs (unexplained diff vs previous) (percentage)
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
454.calculix (unexplained diff vs previous)
454.calculix (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
465.tonto (unexplained diff vs previous)
465.tonto (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d 2.26% 48001.000 49088.000 - 0.00% -
471.omnetpp 1.00% 295652.000 298623.000 - 0.00% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 1.17% 13044.000 13197.000 - 0.79% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (max) (percentage) 2187.50% 0.080 1.830 - -3.68% -
429.mcf (unexplained diff vs previous) (percentage) 2022.22% 0.090 1.910 - 1636.36% -
450.soplex (avg) (percentage) 1370.83% 0.240 3.530 - -1.12% -
429.mcf (avg) (percentage) 1368.75% 0.480 7.050 - -1.81% -
462.libquantum (avg) (percentage) 1042.11% 0.190 2.170 - -2.69% -
401.bzip2 (avg) (percentage) 951.85% 0.270 2.840 - -2.74% -
445.gobmk (unexplained diff vs previous) (percentage) 726.67% 0.150 1.240 - 0.00% -
454.calculix (unexplained diff vs previous) (percentage) 696.00% 0.750 5.970 - 0.00% -
454.calculix (max) (percentage) 655.95% 0.840 6.350 - 5.83% -
450.soplex (max) (percentage) 572.73% 0.440 2.960 - -2.63% -
445.gobmk (avg) (percentage) 563.64% 0.220 1.460 - -0.68% -
429.mcf (max) (percentage) 506.25% 0.320 1.940 - 44.78% -
403.gcc (max) (percentage) 486.44% 0.590 3.460 - -0.57% -
400.perlbench (avg) (percentage) 469.23% 0.390 2.220 - -1.77% -
458.sjeng (avg) (percentage) 404.35% 0.230 1.160 - -1.69% -
473.astar (avg) (percentage) 402.50% 0.400 2.010 - -0.99% -
450.soplex (unexplained diff vs previous) (percentage) 359.09% 0.440 2.020 - 1088.24% -
403.gcc (avg) (percentage) 353.49% 0.430 1.950 - 0.00% -
445.gobmk (max) (percentage) 351.85% 0.270 1.220 - 248.57% -
473.astar (max) (percentage) 291.49% 0.470 1.840 - -2.65% -
464.h264ref (avg) (percentage) 274.36% 0.390 1.460 - -2.67% -
456.hmmer (avg) (percentage) 273.33% 0.300 1.120 - -1.75% -
445.gobmk (unexplained diff vs previous) 250.00% 18.000 63.000 - 0.00% -
433.milc (avg) (percentage) 230.43% 0.690 2.280 - -2.15% -
401.bzip2 (max) (percentage) 185.00% 0.400 1.140 - -7.32% -
435.gromacs (avg) (percentage) 180.00% 0.950 2.660 - -2.56% -
445.gobmk 174.07% 27.000 74.000 - -1.33% -
445.gobmk (avg) 174.07% 27.000 74.000 - -1.33% -
447.dealII (avg) (percentage) 171.26% 0.870 2.360 - -1.67% -
458.sjeng 147.22% 36.000 89.000 - -1.11% -
458.sjeng (avg) 147.22% 36.000 89.000 - -1.11% -
473.astar (unexplained diff vs previous) (percentage) 138.30% 0.470 1.120 - 36.59% -
400.perlbench (max) (percentage) 136.25% 0.800 1.890 - -5.97% -
400.perlbench (unexplained diff vs previous) (percentage) 132.26% 0.310 0.720 - 22.03% -
437.leslie3d (max) (percentage) 119.35% 3.100 6.800 - -5.82% -
437.leslie3d (avg) (percentage) 113.77% 1.670 3.570 - -1.92% -
471.omnetpp (avg) (percentage) 104.24% 1.180 2.410 - -2.03% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 0.00% -
454.calculix (avg) (percentage) 103.45% 2.030 4.130 - 1.72% -
473.astar 92.16% 51.000 98.000 - 0.00% -
473.astar (avg) 92.16% 51.000 98.000 - 0.00% -
401.bzip2 88.89% 45.000 85.000 - -2.30% -
401.bzip2 (avg) 88.89% 45.000 85.000 - -2.30% -
445.gobmk (max) 87.88% 33.000 62.000 - 244.44% -
429.mcf (unexplained diff vs previous) 80.00% 10.000 18.000 - 1700.00% -
483.xalancbmk (max) (percentage) 72.83% 0.920 1.590 - 0.00% -
434.zeusmp (avg) (percentage) 56.56% 1.220 1.910 - -1.55% -
464.h264ref 52.54% 59.000 90.000 - -3.23% -
464.h264ref (avg) 52.54% 59.000 90.000 - -3.23% -
459.GemsFDTD (avg) (percentage) 51.80% 1.390 2.110 - -2.31% -
473.astar (max) 47.54% 61.000 90.000 - -1.10% -
447.dealII (unexplained diff vs previous) (percentage) 44.00% 0.750 1.080 - 151.16% -
437.leslie3d (max) 43.39% 560.000 803.000 - -4.86% -
453.povray (avg) (percentage) 43.14% 1.530 2.190 - -1.79% -
437.leslie3d 39.87% 301.000 421.000 - -0.94% -
437.leslie3d (avg) 39.87% 301.000 421.000 - -0.94% -
435.gromacs (max) (percentage) 32.95% 1.760 2.340 - 0.00% -
435.gromacs 25.00% 160.000 200.000 - -2.44% -
435.gromacs (avg) 25.00% 160.000 200.000 - -2.44% -
458.sjeng (max) (percentage) 20.00% 0.300 0.360 - -5.26% -
429.mcf 17.86% 56.000 66.000 - 0.00% -
429.mcf (avg) 17.86% 56.000 66.000 - 0.00% -
482.sphinx3 (avg) (percentage) 5.11% 2.350 2.470 - -1.59% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -3.45% 29.000 28.000 - 0.00% -
nb_execution_successful -3.45% 29.000 28.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
454.calculix -99.79% 32491.000 67.000 - 6.35% -
403.gcc -99.35% 8890.000 58.000 - 1.75% -
462.libquantum -99.05% 17005.000 161.000 - 0.00% -
482.sphinx3 -97.51% 36586.000 912.000 - 0.88% -
450.soplex -96.21% 15709.000 595.000 - 2.06% -
444.namd -94.36% 15957.000 900.000 - -0.22% -
453.povray -92.56% 7395.000 550.000 - -0.72% -
429.mcf -91.95% 11727.000 944.000 - 1.94% -
433.milc -91.93% 13668.000 1103.000 - -0.09% -
436.cactusADM -89.41% 24396.000 2584.000 - -0.27% -
434.zeusmp -86.72% 20932.000 2780.000 - -0.75% -
400.perlbench -86.46% 11215.000 1519.000 - -0.72% -
401.bzip2 -82.29% 16971.000 3006.000 - -0.20% -
447.dealII -79.64% 9101.000 1853.000 - -1.07% -
470.lbm -77.00% 16312.000 3751.000 - 0.64% -
456.hmmer -76.44% 18995.000 4476.000 - 0.36% -
459.GemsFDTD -71.70% 25364.000 7178.000 - 0.34% -
481.wrf -70.77% 21991.000 6429.000 - 0.25% -
471.omnetpp -70.23% 9265.000 2758.000 - -0.11% -
473.astar -62.16% 12983.000 4913.000 - 1.13% -
410.bwaves -59.48% 14745.000 5974.000 - 0.61% -
464.h264ref -59.11% 15208.000 6218.000 - 0.11% -
445.gobmk -58.96% 12396.000 5087.000 - -1.22% -
435.gromacs -55.20% 16853.000 7550.000 - 0.49% -
458.sjeng -52.09% 16018.000 7675.000 - -0.21% -
416.gamess -49.72% 29409.000 14788.000 - 0.30% -
437.leslie3d -34.65% 18070.000 11809.000 - 0.98% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (unexplained diff vs previous) -99.76% 834.000 2.000 - -75.00% -
454.calculix -99.70% 659.000 2.000 - 0.00% -
454.calculix (avg) -99.70% 659.000 2.000 - 0.00% -
436.cactusADM (unexplained diff vs previous) -99.40% 1173.000 7.000 - 600.00% -
436.cactusADM (max) -99.35% 1231.000 8.000 - 0.00% -
482.sphinx3 (unexplained diff vs previous) -99.28% 1118.000 8.000 - -27.27% -
444.namd (max) -99.20% 879.000 7.000 - -12.50% -
482.sphinx3 (max) -99.13% 1152.000 10.000 - 0.00% -
433.milc (unexplained diff vs previous) -98.96% 96.000 1.000 - -50.00% -
454.calculix (max) -98.53% 272.000 4.000 - 33.33% -
454.calculix (unexplained diff vs previous) -98.36% 244.000 4.000 - 0.00% -
481.wrf (unexplained diff vs previous) -98.21% 894.000 16.000 - -71.43% -
453.povray (max) -97.91% 191.000 4.000 - 0.00% -
453.povray (unexplained diff vs previous) -97.63% 169.000 4.000 - 0.00% -
482.sphinx3 -97.44% 859.000 22.000 - 0.00% -
482.sphinx3 (avg) -97.44% 859.000 22.000 - 0.00% -
403.gcc -97.37% 38.000 1.000 - 0.00% -
403.gcc (avg) -97.37% 38.000 1.000 - 0.00% -
470.lbm (max) -97.31% 893.000 24.000 - 20.00% -
470.lbm (unexplained diff vs previous) -97.08% 822.000 24.000 - 14.29% -
403.gcc (max) -96.15% 52.000 2.000 - 100.00% -
459.GemsFDTD (unexplained diff vs previous) -95.89% 584.000 24.000 - 20.00% -
459.GemsFDTD (max) -95.85% 578.000 24.000 - 9.09% -
434.zeusmp (max) -95.84% 481.000 20.000 - 42.86% -
444.namd (unexplained diff vs previous) (percentage) -95.79% 5.230 0.220 - -75.28% -
444.namd -95.65% 414.000 18.000 - 0.00% -
444.namd (avg) -95.65% 414.000 18.000 - 0.00% -
434.zeusmp (unexplained diff vs previous) -95.53% 470.000 21.000 - 50.00% -
416.gamess (max) -95.23% 923.000 44.000 - 0.00% -
471.omnetpp (unexplained diff vs previous) -95.16% 62.000 3.000 - -83.33% -
416.gamess (unexplained diff vs previous) -95.09% 896.000 44.000 - 120.00% -
436.cactusADM (unexplained diff vs previous) (percentage) -94.39% 4.810 0.270 - 575.00% -
436.cactusADM (max) (percentage) -93.86% 5.050 0.310 - -3.13% -
481.wrf (unexplained diff vs previous) (percentage) -93.86% 4.070 0.250 - -71.26% -
433.milc (max) -93.81% 97.000 6.000 - -14.29% -
471.omnetpp (max) -92.44% 172.000 13.000 - -23.53% -
436.cactusADM -91.79% 585.000 48.000 - -2.04% -
436.cactusADM (avg) -91.79% 585.000 48.000 - -2.04% -
481.wrf (max) -91.36% 914.000 79.000 - -33.61% -
401.bzip2 (unexplained diff vs previous) -91.30% 69.000 6.000 - -80.00% -
462.libquantum -90.62% 32.000 3.000 - 0.00% -
462.libquantum (avg) -90.62% 32.000 3.000 - 0.00% -
416.gamess (max) (percentage) -90.45% 3.140 0.300 - 0.00% -
416.gamess (unexplained diff vs previous) (percentage) -90.16% 3.050 0.300 - 114.29% -
453.povray -89.38% 113.000 12.000 - 0.00% -
453.povray (avg) -89.38% 113.000 12.000 - 0.00% -
470.lbm (max) (percentage) -88.32% 5.480 0.640 - 14.29% -
464.h264ref (unexplained diff vs previous) -87.93% 58.000 7.000 - -65.00% -
435.gromacs (unexplained diff vs previous) -87.33% 292.000 37.000 - 0.00% -
470.lbm (unexplained diff vs previous) (percentage) -87.30% 5.040 0.640 - 14.29% -
433.milc (unexplained diff vs previous) (percentage) -87.14% 0.700 0.090 - -50.00% -
470.lbm -86.35% 608.000 83.000 - -1.19% -
470.lbm (avg) -86.35% 608.000 83.000 - -1.19% -
410.bwaves (max) -86.15% 260.000 36.000 - 2.86% -
410.bwaves (unexplained diff vs previous) -85.99% 257.000 36.000 - 227.27% -
459.GemsFDTD (unexplained diff vs previous) (percentage) -85.65% 2.300 0.330 - 17.86% -
444.namd (max) (percentage) -85.48% 5.510 0.800 - -10.11% -
459.GemsFDTD (max) (percentage) -85.09% 2.280 0.340 - 9.68% -
462.libquantum (max) -84.62% 13.000 2.000 - -33.33% -
456.hmmer (max) -83.67% 98.000 16.000 - -5.88% -
456.hmmer (unexplained diff vs previous) -83.67% 98.000 16.000 - 23.08% -
471.omnetpp (unexplained diff vs previous) (percentage) -83.58% 0.670 0.110 - -83.08% -
450.soplex (unexplained diff vs previous) -82.61% 69.000 12.000 - 1100.00% -
447.dealII (max) -82.00% 150.000 27.000 - -3.57% -
434.zeusmp -79.22% 255.000 53.000 - -1.85% -
434.zeusmp (avg) -79.22% 255.000 53.000 - -1.85% -
450.soplex (max) -75.36% 69.000 17.000 - 0.00% -
471.omnetpp (max) (percentage) -73.66% 1.860 0.490 - -24.62% -
433.milc -73.40% 94.000 25.000 - 0.00% -
433.milc (avg) -73.40% 94.000 25.000 - 0.00% -
435.gromacs (unexplained diff vs previous) (percentage) -71.68% 1.730 0.490 - 0.00% -
481.wrf -71.33% 565.000 162.000 - -2.99% -
481.wrf (avg) -71.33% 565.000 162.000 - -2.99% -
482.sphinx3 (unexplained diff vs previous) (percentage) -71.24% 3.060 0.880 - -27.87% -
464.h264ref (unexplained diff vs previous) (percentage) -71.05% 0.380 0.110 - -65.62% -
447.dealII (unexplained diff vs previous) -70.59% 68.000 20.000 - 150.00% -
481.wrf (max) (percentage) -70.43% 4.160 1.230 - -34.22% -
453.povray (max) (percentage) -70.27% 2.590 0.770 - -10.47% -
464.h264ref (max) -69.49% 59.000 18.000 - -5.26% -
400.perlbench (unexplained diff vs previous) -68.57% 35.000 11.000 - 22.22% -
400.perlbench (max) -68.54% 89.000 28.000 - -6.67% -
453.povray (unexplained diff vs previous) (percentage) -68.12% 2.290 0.730 - 0.00% -
434.zeusmp (max) (percentage) -67.39% 2.300 0.750 - 47.06% -
458.sjeng (unexplained diff vs previous) -66.67% 48.000 16.000 - -44.83% -
434.zeusmp (unexplained diff vs previous) (percentage) -66.22% 2.250 0.760 - 52.00% -
410.bwaves (max) (percentage) -65.54% 1.770 0.610 - 1.67% -
410.bwaves (unexplained diff vs previous) (percentage) -65.52% 1.740 0.600 - 215.79% -
482.sphinx3 (max) (percentage) -65.08% 3.150 1.100 - -8.33% -
410.bwaves -64.36% 289.000 103.000 - -0.96% -
410.bwaves (avg) -64.36% 289.000 103.000 - -0.96% -
416.gamess -62.89% 485.000 180.000 - -1.10% -
416.gamess (avg) -62.89% 485.000 180.000 - -1.10% -
459.GemsFDTD -57.10% 352.000 151.000 - -1.95% -
459.GemsFDTD (avg) -57.10% 352.000 151.000 - -1.95% -
429.mcf (max) -51.35% 37.000 18.000 - 50.00% -
401.bzip2 (unexplained diff vs previous) (percentage) -51.22% 0.410 0.200 - -80.00% -
401.bzip2 (max) -49.25% 67.000 34.000 - -8.11% -
447.dealII -45.57% 79.000 43.000 - -2.27% -
447.dealII (avg) -45.57% 79.000 43.000 - -2.27% -
458.sjeng (max) -43.75% 48.000 27.000 - -6.90% -
450.soplex -43.24% 37.000 21.000 - 5.00% -
450.soplex (avg) -43.24% 37.000 21.000 - 5.00% -
435.gromacs (max) -40.54% 296.000 176.000 - 0.57% -
470.lbm (avg) (percentage) -40.48% 3.730 2.220 - -1.77% -
471.omnetpp -39.45% 109.000 66.000 - -1.49% -
471.omnetpp (avg) -39.45% 109.000 66.000 - -1.49% -
437.leslie3d (unexplained diff vs previous) -38.50% 187.000 115.000 - -80.03% -
465.tonto (unexplained diff vs previous) (percentage) -30.97% 1.130 0.780 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) -30.77% 0.520 0.360 - 24.14% -
465.tonto (unexplained diff vs previous) -30.41% 148.000 103.000 - 0.00% -
458.sjeng (unexplained diff vs previous) (percentage) -30.00% 0.300 0.210 - -44.74% -
456.hmmer (max) (percentage) -28.85% 0.520 0.370 - -7.50% -
416.gamess (avg) (percentage) -26.06% 1.650 1.220 - -1.61% -
465.tonto (max) (percentage) -24.56% 1.710 1.290 - -4.44% -
465.tonto (max) -23.77% 223.000 170.000 - -3.41% -
400.perlbench -23.26% 43.000 33.000 - -2.94% -
400.perlbench (avg) -23.26% 43.000 33.000 - -2.94% -
464.h264ref (max) (percentage) -23.08% 0.390 0.300 - -6.25% -
436.cactusADM (avg) (percentage) -22.08% 2.400 1.870 - -1.58% -
444.namd (avg) (percentage) -21.54% 2.600 2.040 - -2.39% -
465.tonto (avg) (percentage) -14.91% 1.140 0.970 - 0.00% -
433.milc (max) (percentage) -14.08% 0.710 0.610 - -7.58% -
465.tonto -13.51% 148.000 128.000 - 0.79% -
465.tonto (avg) -13.51% 148.000 128.000 - 0.79% -
410.bwaves (avg) (percentage) -11.73% 1.960 1.730 - -1.70% -
456.hmmer -10.71% 56.000 50.000 - 0.00% -
456.hmmer (avg) -10.71% 56.000 50.000 - 0.00% -
447.dealII (max) (percentage) -10.30% 1.650 1.480 - -3.90% -
473.astar (unexplained diff vs previous) -9.84% 61.000 55.000 - 37.50% -
437.leslie3d (unexplained diff vs previous) (percentage) -5.83% 1.030 0.970 - -80.32% -
481.wrf (avg) (percentage) -1.56% 2.570 2.530 - -3.07% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.38s
Execution (seconds)
Name Current %
400.perlbench 1519.000 -
401.bzip2 3006.000 -
403.gcc 58.000 1.75%
410.bwaves 5974.000 -
416.gamess 14788.000 -
429.mcf 944.000 1.94%
433.milc 1103.000 -
434.zeusmp 2780.000 -
435.gromacs 7550.000 -
436.cactusADM 2584.000 -
437.leslie3d 11809.000 -
444.namd 900.000 -
445.gobmk 5087.000 -1.22%
447.dealII 1853.000 -1.07%
450.soplex 595.000 2.06%
453.povray 550.000 -
454.calculix 67.000 6.35%
456.hmmer 4476.000 -
458.sjeng 7675.000 -
459.GemsFDTD 7178.000 -
462.libquantum 161.000 -
464.h264ref 6218.000 -
465.tonto 13197.000 -
470.lbm 3751.000 -
471.omnetpp 2758.000 -
473.astar 4913.000 1.13%
481.wrf 6429.000 -
482.sphinx3 912.000 -
Geometric Mean 2175.183 -
Code Size (bytes)
Name Current %
400.perlbench 563174.000 -
401.bzip2 33574.000 -
403.gcc 1587086.000 -
410.bwaves 14746.000 -
416.gamess 4574180.000 -
429.mcf 7709.000 -
433.milc 47813.000 -
434.zeusmp 141002.000 -
435.gromacs 394574.000 -
436.cactusADM 257262.000 -
437.leslie3d 49088.000 -
444.namd 162551.000 -
445.gobmk 1088248.000 -
447.dealII 191234.000 -
450.soplex 135716.000 -
453.povray 493409.000 -
454.calculix 679935.000 -
456.hmmer 74829.000 -
458.sjeng 81277.000 -
459.GemsFDTD 197014.000 -
462.libquantum 13158.000 -
464.h264ref 301522.000 -
465.tonto 1433800.000 -
470.lbm 10845.000 -
471.omnetpp 298623.000 -
473.astar 15609.000 -
481.wrf 1539636.000 -
482.sphinx3 80098.000 -
Geometric Mean 157649.209 -
sample variability
Name Current %
400.perlbench 33.000 -2.94%
400.perlbench (avg) 33.000 -2.94%
400.perlbench (avg) (percentage) 2.220 -1.77%
400.perlbench (max) 28.000 -6.67%
400.perlbench (max) (percentage) 1.890 -5.97%
400.perlbench (unexplained diff vs previous) 11.000 22.22%
400.perlbench (unexplained diff vs previous) (percentage) 0.720 22.03%
401.bzip2 85.000 -2.30%
401.bzip2 (avg) 85.000 -2.30%
401.bzip2 (avg) (percentage) 2.840 -2.74%
401.bzip2 (max) 34.000 -8.11%
401.bzip2 (max) (percentage) 1.140 -7.32%
401.bzip2 (unexplained diff vs previous) 6.000 -80.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.200 -80.00%
403.gcc 1.000 -
403.gcc (avg) 1.000 -
403.gcc (avg) (percentage) 1.950 -
403.gcc (max) 2.000 100.00%
403.gcc (max) (percentage) 3.460 -
410.bwaves 103.000 -
410.bwaves (avg) 103.000 -
410.bwaves (avg) (percentage) 1.730 -1.70%
410.bwaves (max) 36.000 2.86%
410.bwaves (max) (percentage) 0.610 1.67%
410.bwaves (unexplained diff vs previous) 36.000 227.27%
410.bwaves (unexplained diff vs previous) (percentage) 0.600 215.79%
416.gamess 180.000 -1.10%
416.gamess (avg) 180.000 -1.10%
416.gamess (avg) (percentage) 1.220 -1.61%
416.gamess (max) 44.000 -
416.gamess (max) (percentage) 0.300 -
416.gamess (unexplained diff vs previous) 44.000 120.00%
416.gamess (unexplained diff vs previous) (percentage) 0.300 114.29%
429.mcf 66.000 -
429.mcf (avg) 66.000 -
429.mcf (avg) (percentage) 7.050 -1.81%
429.mcf (max) 18.000 50.00%
429.mcf (max) (percentage) 1.940 44.78%
429.mcf (unexplained diff vs previous) 18.000 1700.00%
429.mcf (unexplained diff vs previous) (percentage) 1.910 1636.36%
433.milc 25.000 -
433.milc (avg) 25.000 -
433.milc (avg) (percentage) 2.280 -2.15%
433.milc (max) 6.000 -14.29%
433.milc (max) (percentage) 0.610 -7.58%
433.milc (unexplained diff vs previous) 1.000 -50.00%
433.milc (unexplained diff vs previous) (percentage) 0.090 -50.00%
434.zeusmp 53.000 -1.85%
434.zeusmp (avg) 53.000 -1.85%
434.zeusmp (avg) (percentage) 1.910 -1.55%
434.zeusmp (max) 20.000 42.86%
434.zeusmp (max) (percentage) 0.750 47.06%
434.zeusmp (unexplained diff vs previous) 21.000 50.00%
434.zeusmp (unexplained diff vs previous) (percentage) 0.760 52.00%
435.gromacs 200.000 -2.44%
435.gromacs (avg) 200.000 -2.44%
435.gromacs (avg) (percentage) 2.660 -2.56%
435.gromacs (max) 176.000 -
435.gromacs (max) (percentage) 2.340 -
435.gromacs (unexplained diff vs previous) 37.000 -
435.gromacs (unexplained diff vs previous) (percentage) 0.490 -
436.cactusADM 48.000 -2.04%
436.cactusADM (avg) 48.000 -2.04%
436.cactusADM (avg) (percentage) 1.870 -1.58%
436.cactusADM (max) 8.000 -
436.cactusADM (max) (percentage) 0.310 -3.13%
436.cactusADM (unexplained diff vs previous) 7.000 600.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.270 575.00%
437.leslie3d 421.000 -
437.leslie3d (avg) 421.000 -
437.leslie3d (avg) (percentage) 3.570 -1.92%
437.leslie3d (max) 803.000 -4.86%
437.leslie3d (max) (percentage) 6.800 -5.82%
437.leslie3d (unexplained diff vs previous) 115.000 -80.03%
437.leslie3d (unexplained diff vs previous) (percentage) 0.970 -80.32%
444.namd 18.000 -
444.namd (avg) 18.000 -
444.namd (avg) (percentage) 2.040 -2.39%
444.namd (max) 7.000 -12.50%
444.namd (max) (percentage) 0.800 -10.11%
444.namd (unexplained diff vs previous) 2.000 -75.00%
444.namd (unexplained diff vs previous) (percentage) 0.220 -75.28%
445.gobmk 74.000 -1.33%
445.gobmk (avg) 74.000 -1.33%
445.gobmk (avg) (percentage) 1.460 -
445.gobmk (max) 62.000 244.44%
445.gobmk (max) (percentage) 1.220 248.57%
445.gobmk (unexplained diff vs previous) 63.000 -
445.gobmk (unexplained diff vs previous) (percentage) 1.240 -
447.dealII 43.000 -2.27%
447.dealII (avg) 43.000 -2.27%
447.dealII (avg) (percentage) 2.360 -1.67%
447.dealII (max) 27.000 -3.57%
447.dealII (max) (percentage) 1.480 -3.90%
447.dealII (unexplained diff vs previous) 20.000 150.00%
447.dealII (unexplained diff vs previous) (percentage) 1.080 151.16%
450.soplex 21.000 5.00%
450.soplex (avg) 21.000 5.00%
450.soplex (avg) (percentage) 3.530 -1.12%
450.soplex (max) 17.000 -
450.soplex (max) (percentage) 2.960 -2.63%
450.soplex (unexplained diff vs previous) 12.000 1100.00%
450.soplex (unexplained diff vs previous) (percentage) 2.020 1088.24%
453.povray 12.000 -
453.povray (avg) 12.000 -
453.povray (avg) (percentage) 2.190 -1.79%
453.povray (max) 4.000 -
453.povray (max) (percentage) 0.770 -10.47%
453.povray (unexplained diff vs previous) 4.000 -
453.povray (unexplained diff vs previous) (percentage) 0.730 -
454.calculix 2.000 -
454.calculix (avg) 2.000 -
454.calculix (avg) (percentage) 4.130 1.72%
454.calculix (max) 4.000 33.33%
454.calculix (max) (percentage) 6.350 5.83%
454.calculix (unexplained diff vs previous) 4.000 -
454.calculix (unexplained diff vs previous) (percentage) 5.970 -
456.hmmer 50.000 -
456.hmmer (avg) 50.000 -
456.hmmer (avg) (percentage) 1.120 -1.75%
456.hmmer (max) 16.000 -5.88%
456.hmmer (max) (percentage) 0.370 -7.50%
456.hmmer (unexplained diff vs previous) 16.000 23.08%
456.hmmer (unexplained diff vs previous) (percentage) 0.360 24.14%
458.sjeng 89.000 -1.11%
458.sjeng (avg) 89.000 -1.11%
458.sjeng (avg) (percentage) 1.160 -1.69%
458.sjeng (max) 27.000 -6.90%
458.sjeng (max) (percentage) 0.360 -5.26%
458.sjeng (unexplained diff vs previous) 16.000 -44.83%
458.sjeng (unexplained diff vs previous) (percentage) 0.210 -44.74%
459.GemsFDTD 151.000 -1.95%
459.GemsFDTD (avg) 151.000 -1.95%
459.GemsFDTD (avg) (percentage) 2.110 -2.31%
459.GemsFDTD (max) 24.000 9.09%
459.GemsFDTD (max) (percentage) 0.340 9.68%
459.GemsFDTD (unexplained diff vs previous) 24.000 20.00%
459.GemsFDTD (unexplained diff vs previous) (percentage) 0.330 17.86%
462.libquantum 3.000 -
462.libquantum (avg) 3.000 -
462.libquantum (avg) (percentage) 2.170 -2.69%
462.libquantum (max) 2.000 -33.33%
462.libquantum (max) (percentage) 1.830 -3.68%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 90.000 -3.23%
464.h264ref (avg) 90.000 -3.23%
464.h264ref (avg) (percentage) 1.460 -2.67%
464.h264ref (max) 18.000 -5.26%
464.h264ref (max) (percentage) 0.300 -6.25%
464.h264ref (unexplained diff vs previous) 7.000 -65.00%
464.h264ref (unexplained diff vs previous) (percentage) 0.110 -65.62%
465.tonto 128.000 -
465.tonto (avg) 128.000 -
465.tonto (avg) (percentage) 0.970 -
465.tonto (max) 170.000 -3.41%
465.tonto (max) (percentage) 1.290 -4.44%
465.tonto (unexplained diff vs previous) 103.000 -
465.tonto (unexplained diff vs previous) (percentage) 0.780 -
470.lbm 83.000 -1.19%
470.lbm (avg) 83.000 -1.19%
470.lbm (avg) (percentage) 2.220 -1.77%
470.lbm (max) 24.000 20.00%
470.lbm (max) (percentage) 0.640 14.29%
470.lbm (unexplained diff vs previous) 24.000 14.29%
470.lbm (unexplained diff vs previous) (percentage) 0.640 14.29%
471.omnetpp 66.000 -1.49%
471.omnetpp (avg) 66.000 -1.49%
471.omnetpp (avg) (percentage) 2.410 -2.03%
471.omnetpp (max) 13.000 -23.53%
471.omnetpp (max) (percentage) 0.490 -24.62%
471.omnetpp (unexplained diff vs previous) 3.000 -83.33%
471.omnetpp (unexplained diff vs previous) (percentage) 0.110 -83.08%
473.astar 98.000 -
473.astar (avg) 98.000 -
473.astar (avg) (percentage) 2.010 -
473.astar (max) 90.000 -1.10%
473.astar (max) (percentage) 1.840 -2.65%
473.astar (unexplained diff vs previous) 55.000 37.50%
473.astar (unexplained diff vs previous) (percentage) 1.120 36.59%
481.wrf 162.000 -2.99%
481.wrf (avg) 162.000 -2.99%
481.wrf (avg) (percentage) 2.530 -3.07%
481.wrf (max) 79.000 -33.61%
481.wrf (max) (percentage) 1.230 -34.22%
481.wrf (unexplained diff vs previous) 16.000 -71.43%
481.wrf (unexplained diff vs previous) (percentage) 0.250 -71.26%
482.sphinx3 22.000 -
482.sphinx3 (avg) 22.000 -
482.sphinx3 (avg) (percentage) 2.470 -1.59%
482.sphinx3 (max) 10.000 -
482.sphinx3 (max) (percentage) 1.100 -8.33%
482.sphinx3 (unexplained diff vs previous) 8.000 -27.27%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.880 -27.87%
483.xalancbmk (avg) (percentage) 1.140 -
483.xalancbmk (max) (percentage) 1.590 -
Geometric Mean 6.178 -10.67%