Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=4810a2d92b9e1a13774c1286cd8a0f718f81abba (+ 31 commits)
git_gcc https://github.com/gcc-mirror/gcc/commit/74bca21db31e3f4ab6543b56c3f26b4dfe586fef (+ 31 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=3de2f8755c6c036dcd0b1f4acd6bcdefe0e775c0 (+ 7 commits)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=dd22391461d1c15b8296f13695365005f8be2e90 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-aarch64-master-Os_LTO-build/207/
View Options
Report
Filter

gnu-aarch64-master-Os_LTO test results

Run Order Start Time Duration
Current 0207 (3-changed-components) 2024-03-07T02:03:48 0:00:00
Previous 0206 (4-changed-components) 2024-02-29T13:31:56 0:00:00
Baseline 0001 (4-changed-components) 2023-02-20T22:48:24 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 42 14
Performance Improvements 81 188
Added Tests 2 2
Unchanged Tests 1111 1032
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 5.76% 10836.000 11460.000 - -1.55% -
453.povray 1.65% 545.000 554.000 - 1.28% -
471.omnetpp 1.65% 2730.000 2775.000 - 1.87% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
437.leslie3d (unexplained diff vs previous) 1256.52% 46.000 624.000 - -73.03% -
437.leslie3d (unexplained diff vs previous) (percentage) 1197.62% 0.420 5.450 - -72.59% -
429.mcf (unexplained diff vs previous) 333.33% 3.000 13.000 - -99.93% -
429.mcf (unexplained diff vs previous) (percentage) 318.18% 0.220 0.920 - -99.93% -
433.milc (unexplained diff vs previous) 300.00% 1.000 4.000 - -99.96% -
433.milc (unexplained diff vs previous) (percentage) 288.89% 0.090 0.350 - -99.96% -
410.bwaves (unexplained diff vs previous) 157.14% 7.000 18.000 - -99.77% -
410.bwaves (unexplained diff vs previous) (percentage) 150.00% 0.120 0.300 - -99.78% -
482.sphinx3 (max) 133.33% 3.000 7.000 - -99.23% -
482.sphinx3 (unexplained diff vs previous) 133.33% 3.000 7.000 - -99.97% -
482.sphinx3 (unexplained diff vs previous) (percentage) 129.73% 0.370 0.850 - -99.97% -
445.gobmk (unexplained diff vs previous) 117.65% 17.000 37.000 - -99.34% -
445.gobmk (unexplained diff vs previous) (percentage) 117.65% 0.340 0.740 - -99.33% -
482.sphinx3 (max) (percentage) 97.67% 0.430 0.850 - -99.12% -
437.leslie3d (max) 88.03% 351.000 660.000 - -65.79% -
437.leslie3d (max) (percentage) 77.78% 3.240 5.760 - -65.26% -
473.astar (unexplained diff vs previous) (percentage) 76.19% 0.210 0.370 - -99.81% -
473.astar (unexplained diff vs previous) 72.73% 11.000 19.000 - -99.81% -
401.bzip2 (unexplained diff vs previous) (percentage) 66.67% 0.300 0.500 - -99.86% -
401.bzip2 (unexplained diff vs previous) 66.67% 9.000 15.000 - -99.86% -
471.omnetpp (unexplained diff vs previous) 40.62% 32.000 45.000 - -99.58% -
471.omnetpp (unexplained diff vs previous) (percentage) 38.46% 1.170 1.620 - -99.59% -
453.povray (max) 28.57% 7.000 9.000 - -98.20% -
453.povray (unexplained diff vs previous) 28.57% 7.000 9.000 - -99.84% -
458.sjeng (unexplained diff vs previous) (percentage) 27.50% 0.400 0.510 - -99.44% -
453.povray (max) (percentage) 26.92% 1.300 1.650 - -98.19% -
458.sjeng (unexplained diff vs previous) 26.67% 30.000 38.000 - -99.45% -
453.povray (unexplained diff vs previous) (percentage) 26.56% 1.280 1.620 - -99.84% -
456.hmmer (max) 14.29% 7.000 8.000 - -99.62% -
444.namd (unexplained diff vs previous) (percentage) 13.59% 1.030 1.170 - -99.90% -
444.namd (unexplained diff vs previous) 12.50% 8.000 9.000 - -99.89% -
456.hmmer (unexplained diff vs previous) 12.50% 8.000 9.000 - -99.89% -
444.namd (max) (percentage) 11.54% 1.040 1.160 - -98.74% -
456.hmmer (max) (percentage) 11.11% 0.270 0.300 - -99.59% -
456.hmmer (unexplained diff vs previous) (percentage) 11.11% 0.270 0.300 - -99.89% -
437.leslie3d 6.23% 514.000 546.000 - 53.80% -
437.leslie3d (avg) 6.23% 514.000 546.000 - 53.80% -
471.omnetpp 1.56% 64.000 65.000 - -70.85% -
471.omnetpp (avg) 1.56% 64.000 65.000 - -70.85% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
444.namd -1.16% 778.000 769.000 - 2.95% -
447.dealII -1.02% 1662.000 1645.000 - -4.58% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
450.soplex (max) -83.33% 30.000 5.000 - -99.08% -
450.soplex (max) (percentage) -82.63% 5.180 0.900 - -99.06% -
436.cactusADM (max) -76.47% 17.000 4.000 - -99.78% -
464.h264ref (unexplained diff vs previous) (percentage) -75.00% 0.120 0.030 - -99.97% -
436.cactusADM (max) (percentage) -74.07% 0.810 0.210 - -99.76% -
464.h264ref (unexplained diff vs previous) -71.43% 7.000 2.000 - -99.97% -
434.zeusmp (unexplained diff vs previous) -62.50% 8.000 3.000 - -99.97% -
434.zeusmp (unexplained diff vs previous) (percentage) -62.50% 0.320 0.120 - -99.97% -
436.cactusADM (unexplained diff vs previous) (percentage) -62.50% 0.240 0.090 - -99.99% -
436.cactusADM (unexplained diff vs previous) -60.00% 5.000 2.000 - -99.99% -
435.gromacs (max) (percentage) -55.51% 2.270 1.010 - -97.95% -
435.gromacs (max) -55.21% 163.000 73.000 - -97.96% -
434.zeusmp (max) -35.71% 28.000 18.000 - -99.12% -
434.zeusmp (max) (percentage) -35.14% 1.110 0.720 - -99.12% -
450.soplex (unexplained diff vs previous) -33.33% 6.000 4.000 - -99.97% -
462.libquantum (max) -33.33% 3.000 2.000 - -98.55% -
450.soplex (unexplained diff vs previous) (percentage) -32.35% 1.020 0.690 - -99.97% -
435.gromacs (unexplained diff vs previous) (percentage) -23.68% 1.140 0.870 - -99.10% -
400.perlbench (unexplained diff vs previous) -23.53% 17.000 13.000 - -99.85% -
435.gromacs (unexplained diff vs previous) -23.17% 82.000 63.000 - -99.10% -
400.perlbench (unexplained diff vs previous) (percentage) -22.94% 1.090 0.840 - -99.84% -
462.libquantum (max) (percentage) -9.91% 2.220 2.000 - -97.98% -
459.GemsFDTD (max) (percentage) -8.11% 0.370 0.340 - -99.52% -
410.bwaves (max) (percentage) -8.00% 0.500 0.460 - -99.21% -
416.gamess (max) -7.69% 13.000 12.000 - -99.75% -
410.bwaves (max) -6.90% 29.000 27.000 - -99.18% -
400.perlbench (max) (percentage) -6.12% 1.470 1.380 - -98.36% -
470.lbm (max) -5.56% 36.000 34.000 - -98.76% -
459.GemsFDTD (max) -5.26% 19.000 18.000 - -99.59% -
447.dealII (max) -4.76% 21.000 20.000 - -98.55% -
447.dealII (max) (percentage) -4.69% 1.280 1.220 - -98.48% -
400.perlbench (max) -4.55% 22.000 21.000 - -98.50% -
465.tonto (max) -4.27% 281.000 269.000 - -87.26% -
458.sjeng (max) -4.23% 71.000 68.000 - -98.11% -
433.milc (max) -4.17% 24.000 23.000 - -97.80% -
458.sjeng (max) (percentage) -4.17% 0.960 0.920 - -98.07% -
465.tonto (max) (percentage) -3.80% 2.370 2.280 - -88.19% -
433.milc (max) (percentage) -3.72% 2.150 2.070 - -97.72% -
473.astar (max) (percentage) -3.71% 3.500 3.370 - -94.93% -
445.gobmk (max) -3.57% 84.000 81.000 - -96.94% -
471.omnetpp (max) (percentage) -3.44% 2.620 2.530 - -96.82% -
401.bzip2 (max) (percentage) -3.36% 2.380 2.300 - -97.08% -
473.astar (max) -3.35% 179.000 173.000 - -94.92% -
401.bzip2 -3.33% 60.000 58.000 - -63.98% -
401.bzip2 (avg) -3.33% 60.000 58.000 - -63.98% -
470.lbm (max) (percentage) -3.16% 0.950 0.920 - -98.73% -
445.gobmk (max) (percentage) -2.96% 1.690 1.640 - -96.88% -
436.cactusADM -2.86% 35.000 34.000 - -72.58% -
436.cactusADM (avg) -2.86% 35.000 34.000 - -72.58% -
454.calculix (max) (percentage) -2.83% 1.060 1.030 - -98.97% -
401.bzip2 (max) -2.82% 71.000 69.000 - -97.02% -
400.perlbench -2.78% 36.000 35.000 - -66.67% -
400.perlbench (avg) -2.78% 36.000 35.000 - -66.67% -
403.gcc (max) (percentage) -2.12% 5.180 5.070 - -94.90% -
434.zeusmp (avg) (percentage) -2.11% 1.420 1.390 - -76.24% -
470.lbm -2.04% 49.000 48.000 - -79.92% -
470.lbm (avg) -2.04% 49.000 48.000 - -79.92% -
464.h264ref (avg) (percentage) -1.82% 1.100 1.080 - -78.78% -
445.gobmk -1.82% 55.000 54.000 - -72.45% -
445.gobmk (avg) -1.82% 55.000 54.000 - -72.45% -
436.cactusADM (avg) (percentage) -1.81% 1.660 1.630 - -72.74% -
481.wrf (max) (percentage) -1.75% 1.140 1.120 - -98.37% -
410.bwaves (avg) (percentage) -1.63% 1.230 1.210 - -72.56% -
462.libquantum (avg) (percentage) -1.61% 1.860 1.830 - -76.11% -
464.h264ref -1.59% 63.000 62.000 - -78.84% -
464.h264ref (avg) -1.59% 63.000 62.000 - -78.84% -
401.bzip2 (avg) (percentage) -1.51% 1.990 1.960 - -64.23% -
435.gromacs (avg) (percentage) -1.46% 2.050 2.020 - -60.08% -
482.sphinx3 (avg) (percentage) -1.44% 2.090 2.060 - -74.02% -
471.omnetpp (max) -1.41% 71.000 70.000 - -96.77% -
410.bwaves -1.39% 72.000 71.000 - -71.49% -
410.bwaves (avg) -1.39% 72.000 71.000 - -71.49% -
450.soplex (avg) (percentage) -1.38% 2.890 2.850 - -55.68% -
429.mcf (avg) (percentage) -1.36% 6.600 6.510 - -15.12% -
400.perlbench (avg) (percentage) -1.27% 2.360 2.330 - -63.31% -
433.milc (avg) (percentage) -1.18% 1.700 1.680 - -72.23% -
429.mcf -1.09% 92.000 91.000 - -16.51% -
429.mcf (avg) -1.09% 92.000 91.000 - -16.51% -
473.astar (avg) (percentage) -1.04% 1.920 1.900 - -54.33% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
465.tonto 8.05% 10941.000 11822.000 - -0.45% -
410.bwaves 5.04% 5652.000 5937.000 - 0.30% -
444.namd 2.95% 747.000 769.000 - -1.16% -
450.soplex 2.82% 567.000 583.000 - -0.68% -
456.hmmer 2.81% 2882.000 2963.000 - 0.30% -
401.bzip2 2.18% 2941.000 3005.000 - -0.50% -
436.cactusADM 1.97% 2083.000 2124.000 - 0.09% -
471.omnetpp 1.87% 2724.000 2775.000 - 1.65% -
453.povray 1.28% 547.000 554.000 - 1.65% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d (avg) (percentage) 56.39% 3.050 4.770 - 0.42% -
437.leslie3d 53.80% 355.000 546.000 - 6.23% -
437.leslie3d (avg) 53.80% 355.000 546.000 - 6.23% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_compile_successful -13.79% 29.000 25.000 - 0.00% -
nb_execution_successful -13.79% 29.000 25.000 - 0.00% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
434.zeusmp -2.56% 152037.000 148143.000 - 0.00% -
459.GemsFDTD -2.38% 244728.000 238904.000 - 0.00% -
410.bwaves -1.67% 16760.000 16480.000 - 0.00% -
444.namd -1.61% 188385.000 185348.000 - 0.00% -
464.h264ref -1.46% 387748.000 382104.000 - 0.00% -
470.lbm -1.44% 9691.000 9551.000 - 0.00% -
435.gromacs -1.40% 499261.000 492285.000 - 0.00% -
437.leslie3d -1.33% 59479.000 58690.000 - 0.00% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
459.GemsFDTD -14.09% 6173.000 5303.000 - 0.21% -
482.sphinx3 -12.38% 945.000 828.000 - 0.85% -
400.perlbench -7.22% 1663.000 1543.000 - -0.84% -
447.dealII -4.58% 1724.000 1645.000 - -1.02% -
416.gamess -2.16% 12465.000 12196.000 - 0.02% -
437.leslie3d -1.55% 11640.000 11460.000 - 5.76% -
462.libquantum -1.43% 140.000 138.000 - 0.00% -
458.sjeng -1.24% 7567.000 7473.000 - 0.51% -
429.mcf -1.12% 1424.000 1408.000 - 0.93% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
436.cactusADM (unexplained diff vs previous) (percentage) -99.99% 787.130 0.090 - -62.50% -
436.cactusADM (unexplained diff vs previous) -99.99% 16396.000 2.000 - -60.00% -
464.h264ref (unexplained diff vs previous) (percentage) -99.97% 118.780 0.030 - -75.00% -
434.zeusmp (unexplained diff vs previous) -99.97% 11280.000 3.000 - -62.50% -
434.zeusmp (unexplained diff vs previous) (percentage) -99.97% 448.690 0.120 - -62.50% -
482.sphinx3 (unexplained diff vs previous) -99.97% 26073.000 7.000 - 133.33% -
450.soplex (unexplained diff vs previous) (percentage) -99.97% 2557.320 0.690 - -32.35% -
450.soplex (unexplained diff vs previous) -99.97% 14500.000 4.000 - -33.33% -
464.h264ref (unexplained diff vs previous) -99.97% 6843.000 2.000 - -71.43% -
482.sphinx3 (unexplained diff vs previous) (percentage) -99.97% 2759.050 0.850 - 129.73% -
433.milc (unexplained diff vs previous) -99.96% 11289.000 4.000 - 300.00% -
433.milc (unexplained diff vs previous) (percentage) -99.96% 980.800 0.350 - 288.89% -
429.mcf (unexplained diff vs previous) -99.93% 18593.000 13.000 - 333.33% -
429.mcf (unexplained diff vs previous) (percentage) -99.93% 1305.690 0.920 - 318.18% -
444.namd (unexplained diff vs previous) (percentage) -99.90% 1118.610 1.170 - 13.59% -
456.hmmer (unexplained diff vs previous) (percentage) -99.89% 278.630 0.300 - 11.11% -
444.namd (unexplained diff vs previous) -99.89% 8356.000 9.000 - 12.50% -
456.hmmer (unexplained diff vs previous) -99.89% 8030.000 9.000 - 12.50% -
401.bzip2 (unexplained diff vs previous) (percentage) -99.86% 368.340 0.500 - 66.67% -
401.bzip2 (unexplained diff vs previous) -99.86% 10833.000 15.000 - 66.67% -
400.perlbench (unexplained diff vs previous) -99.85% 8772.000 13.000 - -23.53% -
453.povray (unexplained diff vs previous) (percentage) -99.84% 1043.330 1.620 - 26.56% -
453.povray (unexplained diff vs previous) -99.84% 5707.000 9.000 - 28.57% -
400.perlbench (unexplained diff vs previous) (percentage) -99.84% 527.480 0.840 - -22.94% -
473.astar (unexplained diff vs previous) (percentage) -99.81% 197.850 0.370 - 76.19% -
473.astar (unexplained diff vs previous) -99.81% 10146.000 19.000 - 72.73% -
470.lbm (unexplained diff vs previous) (percentage) -99.81% 263.730 0.500 - 0.00% -
470.lbm (unexplained diff vs previous) -99.81% 9990.000 19.000 - 0.00% -
436.cactusADM (max) -99.78% 1848.000 4.000 - -76.47% -
410.bwaves (unexplained diff vs previous) (percentage) -99.78% 137.900 0.300 - 150.00% -
410.bwaves (unexplained diff vs previous) -99.77% 7794.000 18.000 - 157.14% -
436.cactusADM (max) (percentage) -99.76% 88.730 0.210 - -74.07% -
416.gamess (max) -99.75% 4805.000 12.000 - -7.69% -
416.gamess (max) (percentage) -99.74% 38.550 0.100 - -9.09% -
456.hmmer (max) -99.62% 2120.000 8.000 - 14.29% -
456.hmmer (max) (percentage) -99.59% 73.590 0.300 - 11.11% -
459.GemsFDTD (max) -99.59% 4411.000 18.000 - -5.26% -
471.omnetpp (unexplained diff vs previous) (percentage) -99.59% 391.300 1.620 - 38.46% -
471.omnetpp (unexplained diff vs previous) -99.58% 10659.000 45.000 - 40.62% -
459.GemsFDTD (max) (percentage) -99.52% 71.460 0.340 - -8.11% -
458.sjeng (unexplained diff vs previous) -99.45% 6876.000 38.000 - 26.67% -
458.sjeng (unexplained diff vs previous) (percentage) -99.44% 90.870 0.510 - 27.50% -
445.gobmk (unexplained diff vs previous) -99.34% 5581.000 37.000 - 117.65% -
445.gobmk (unexplained diff vs previous) (percentage) -99.33% 111.110 0.740 - 117.65% -
464.h264ref (max) -99.23% 3127.000 24.000 - 0.00% -
482.sphinx3 (max) -99.23% 911.000 7.000 - 133.33% -
464.h264ref (max) (percentage) -99.23% 54.290 0.420 - 0.00% -
410.bwaves (max) (percentage) -99.21% 57.970 0.460 - -8.00% -
410.bwaves (max) -99.18% 3276.000 27.000 - -6.90% -
434.zeusmp (max) -99.12% 2055.000 18.000 - -35.71% -
434.zeusmp (max) (percentage) -99.12% 81.770 0.720 - -35.14% -
482.sphinx3 (max) (percentage) -99.12% 96.500 0.850 - 97.67% -
435.gromacs (unexplained diff vs previous) -99.10% 7028.000 63.000 - -23.17% -
435.gromacs (unexplained diff vs previous) (percentage) -99.10% 96.680 0.870 - -23.68% -
450.soplex (max) -99.08% 545.000 5.000 - -83.33% -
450.soplex (max) (percentage) -99.06% 96.240 0.900 - -82.63% -
454.calculix (max) (percentage) -98.97% 99.730 1.030 - -2.83% -
444.namd (max) -98.83% 685.000 8.000 - 0.00% -
470.lbm (max) -98.76% 2746.000 34.000 - -5.56% -
444.namd (max) (percentage) -98.74% 91.790 1.160 - 11.54% -
470.lbm (max) (percentage) -98.73% 72.510 0.920 - -3.16% -
447.dealII (max) -98.55% 1384.000 20.000 - -4.76% -
462.libquantum (max) -98.55% 138.000 2.000 - -33.33% -
400.perlbench (max) -98.50% 1397.000 21.000 - -4.55% -
447.dealII (max) (percentage) -98.48% 80.280 1.220 - -4.69% -
481.wrf (max) (percentage) -98.37% 68.700 1.120 - -1.75% -
400.perlbench (max) (percentage) -98.36% 84.060 1.380 - -6.12% -
453.povray (max) -98.20% 499.000 9.000 - 28.57% -
453.povray (max) (percentage) -98.19% 91.250 1.650 - 26.92% -
458.sjeng (max) -98.11% 3602.000 68.000 - -4.23% -
458.sjeng (max) (percentage) -98.07% 47.610 0.920 - -4.17% -
462.libquantum (max) (percentage) -97.98% 98.970 2.000 - -9.91% -
435.gromacs (max) -97.96% 3573.000 73.000 - -55.21% -
435.gromacs (max) (percentage) -97.95% 49.160 1.010 - -55.51% -
433.milc (max) -97.80% 1044.000 23.000 - -4.17% -
433.milc (max) (percentage) -97.72% 90.750 2.070 - -3.72% -
483.xalancbmk (max) (percentage) -97.42% 68.670 1.770 - 0.00% -
401.bzip2 (max) (percentage) -97.08% 78.650 2.300 - -3.36% -
401.bzip2 (max) -97.02% 2313.000 69.000 - -2.82% -
445.gobmk (max) -96.94% 2643.000 81.000 - -3.57% -
445.gobmk (max) (percentage) -96.88% 52.630 1.640 - -2.96% -
471.omnetpp (max) (percentage) -96.82% 79.650 2.530 - -3.44% -
471.omnetpp (max) -96.77% 2169.000 70.000 - -1.41% -
473.astar (max) (percentage) -94.93% 66.430 3.370 - -3.71% -
473.astar (max) -94.92% 3406.000 173.000 - -3.35% -
403.gcc (max) (percentage) -94.90% 99.360 5.070 - -2.12% -
429.mcf (max) -89.33% 1322.000 141.000 - 0.71% -
429.mcf (max) (percentage) -89.19% 92.890 10.040 - 0.00% -
465.tonto (max) (percentage) -88.19% 19.310 2.280 - -3.80% -
465.tonto (max) -87.26% 2112.000 269.000 - -4.27% -
416.gamess -81.64% 550.000 101.000 - 0.00% -
416.gamess (avg) -81.64% 550.000 101.000 - 0.00% -
456.hmmer (avg) (percentage) -81.63% 4.410 0.810 - -1.22% -
416.gamess (avg) (percentage) -81.22% 4.420 0.830 - 0.00% -
456.hmmer -81.10% 127.000 24.000 - 0.00% -
456.hmmer (avg) -81.10% 127.000 24.000 - 0.00% -
444.namd (avg) (percentage) -80.91% 9.220 1.760 - -0.56% -
444.namd -80.88% 68.000 13.000 - 0.00% -
444.namd (avg) -80.88% 68.000 13.000 - 0.00% -
462.libquantum -80.00% 10.000 2.000 - 0.00% -
462.libquantum (avg) -80.00% 10.000 2.000 - 0.00% -
470.lbm -79.92% 239.000 48.000 - -2.04% -
470.lbm (avg) -79.92% 239.000 48.000 - -2.04% -
470.lbm (avg) (percentage) -79.62% 6.330 1.290 - -0.77% -
464.h264ref -78.84% 293.000 62.000 - -1.59% -
464.h264ref (avg) -78.84% 293.000 62.000 - -1.59% -
464.h264ref (avg) (percentage) -78.78% 5.090 1.080 - -1.82% -
453.povray -78.57% 42.000 9.000 - 0.00% -
453.povray (avg) -78.57% 42.000 9.000 - 0.00% -
453.povray (avg) (percentage) -77.21% 7.810 1.780 - -0.56% -
482.sphinx3 -77.03% 74.000 17.000 - 0.00% -
482.sphinx3 (avg) -77.03% 74.000 17.000 - 0.00% -
434.zeusmp (avg) (percentage) -76.24% 5.850 1.390 - -2.11% -
434.zeusmp -76.19% 147.000 35.000 - 0.00% -
434.zeusmp (avg) -76.19% 147.000 35.000 - 0.00% -
459.GemsFDTD -76.11% 360.000 86.000 - 0.00% -
459.GemsFDTD (avg) -76.11% 360.000 86.000 - 0.00% -
462.libquantum (avg) (percentage) -76.11% 7.660 1.830 - -1.61% -
447.dealII -75.76% 99.000 24.000 - 0.00% -
447.dealII (avg) -75.76% 99.000 24.000 - 0.00% -
458.sjeng -75.10% 241.000 60.000 - 0.00% -
458.sjeng (avg) -75.10% 241.000 60.000 - 0.00% -
458.sjeng (avg) (percentage) -74.61% 3.190 0.810 - 0.00% -
447.dealII (avg) (percentage) -74.18% 5.770 1.490 - 0.00% -
482.sphinx3 (avg) (percentage) -74.02% 7.930 2.060 - -1.44% -
437.leslie3d (unexplained diff vs previous) -73.03% 2314.000 624.000 - 1256.52% -
436.cactusADM (avg) (percentage) -72.74% 5.980 1.630 - -1.81% -
437.leslie3d (unexplained diff vs previous) (percentage) -72.59% 19.880 5.450 - 1197.62% -
436.cactusADM -72.58% 124.000 34.000 - -2.86% -
436.cactusADM (avg) -72.58% 124.000 34.000 - -2.86% -
410.bwaves (avg) (percentage) -72.56% 4.410 1.210 - -1.63% -
433.milc -72.46% 69.000 19.000 - 0.00% -
433.milc (avg) -72.46% 69.000 19.000 - 0.00% -
445.gobmk -72.45% 196.000 54.000 - -1.82% -
445.gobmk (avg) -72.45% 196.000 54.000 - -1.82% -
433.milc (avg) (percentage) -72.23% 6.050 1.680 - -1.18% -
459.GemsFDTD (avg) (percentage) -71.92% 5.840 1.640 - 0.00% -
445.gobmk (avg) (percentage) -71.87% 3.910 1.100 - 0.00% -
410.bwaves -71.49% 249.000 71.000 - -1.39% -
410.bwaves (avg) -71.49% 249.000 71.000 - -1.39% -
481.wrf (avg) (percentage) -71.15% 6.240 1.800 - 0.00% -
471.omnetpp (avg) (percentage) -71.10% 8.200 2.370 - -0.42% -
471.omnetpp -70.85% 223.000 65.000 - 1.56% -
471.omnetpp (avg) -70.85% 223.000 65.000 - 1.56% -
400.perlbench -66.67% 105.000 35.000 - -2.78% -
400.perlbench (avg) -66.67% 105.000 35.000 - -2.78% -
437.leslie3d (max) -65.79% 1929.000 660.000 - 88.03% -
437.leslie3d (max) (percentage) -65.26% 16.580 5.760 - 77.78% -
401.bzip2 (avg) (percentage) -64.23% 5.480 1.960 - -1.51% -
401.bzip2 -63.98% 161.000 58.000 - -3.33% -
401.bzip2 (avg) -63.98% 161.000 58.000 - -3.33% -
400.perlbench (avg) (percentage) -63.31% 6.350 2.330 - -1.27% -
435.gromacs -60.22% 367.000 146.000 - -0.68% -
435.gromacs (avg) -60.22% 367.000 146.000 - -0.68% -
435.gromacs (avg) (percentage) -60.08% 5.060 2.020 - -1.46% -
450.soplex (avg) (percentage) -55.68% 6.430 2.850 - -1.38% -
450.soplex -55.56% 36.000 16.000 - 0.00% -
450.soplex (avg) -55.56% 36.000 16.000 - 0.00% -
454.calculix (avg) (percentage) -55.11% 8.020 3.600 - 0.00% -
483.xalancbmk (avg) (percentage) -54.46% 6.500 2.960 - 0.00% -
473.astar (avg) (percentage) -54.33% 4.160 1.900 - -1.04% -
473.astar -53.99% 213.000 98.000 - 0.00% -
473.astar (avg) -53.99% 213.000 98.000 - 0.00% -
465.tonto (avg) (percentage) -53.89% 3.340 1.540 - 0.00% -
465.tonto -50.14% 365.000 182.000 - 0.00% -
465.tonto (avg) -50.14% 365.000 182.000 - 0.00% -
403.gcc (avg) (percentage) -30.46% 7.320 5.090 - 0.00% -
429.mcf -16.51% 109.000 91.000 - -1.09% -
429.mcf (avg) -16.51% 109.000 91.000 - -1.09% -
429.mcf (avg) (percentage) -15.12% 7.670 6.510 - -1.36% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 13.79% 29.000 33.000 - 0.00% -

Added Tests - sample variability
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)

Report Time: 0.40s
Execution (seconds)
Name Current %
400.perlbench 1543.000 -
401.bzip2 3005.000 -
410.bwaves 5937.000 -
416.gamess 12196.000 -
429.mcf 1408.000 -
433.milc 1155.000 -
434.zeusmp 2528.000 -
435.gromacs 7255.000 -
436.cactusADM 2124.000 -
437.leslie3d 11460.000 5.76%
444.namd 769.000 -1.16%
445.gobmk 4978.000 -
447.dealII 1645.000 -1.02%
450.soplex 583.000 -
453.povray 554.000 1.65%
456.hmmer 2963.000 -
458.sjeng 7473.000 -
459.GemsFDTD 5303.000 -
462.libquantum 138.000 -
464.h264ref 5782.000 -
465.tonto 11822.000 -
470.lbm 3791.000 -
471.omnetpp 2775.000 1.65%
473.astar 5160.000 -
482.sphinx3 828.000 -
Geometric Mean 2604.318 -
Code Size (bytes)
Name Current %
400.perlbench 890587.000 -
401.bzip2 43919.000 -
410.bwaves 16480.000 -
416.gamess 5680013.000 -
429.mcf 9055.000 -
433.milc 60853.000 -
434.zeusmp 148143.000 -
435.gromacs 492285.000 -
436.cactusADM 350629.000 -
437.leslie3d 58690.000 -
444.namd 185348.000 -
445.gobmk 1378998.000 -
447.dealII 310444.000 -
450.soplex 205303.000 -
453.povray 656269.000 -
456.hmmer 108215.000 -
458.sjeng 103292.000 -
459.GemsFDTD 238904.000 -
462.libquantum 16085.000 -
464.h264ref 382104.000 -
465.tonto 1942866.000 -
470.lbm 9551.000 -
471.omnetpp 479794.000 -
473.astar 22880.000 -
482.sphinx3 112243.000 -
Geometric Mean 159727.164 -
sample variability
Name Current %
400.perlbench 35.000 -2.78%
400.perlbench (avg) 35.000 -2.78%
400.perlbench (avg) (percentage) 2.330 -1.27%
400.perlbench (max) 21.000 -4.55%
400.perlbench (max) (percentage) 1.380 -6.12%
400.perlbench (unexplained diff vs previous) 13.000 -23.53%
400.perlbench (unexplained diff vs previous) (percentage) 0.840 -22.94%
401.bzip2 58.000 -3.33%
401.bzip2 (avg) 58.000 -3.33%
401.bzip2 (avg) (percentage) 1.960 -1.51%
401.bzip2 (max) 69.000 -2.82%
401.bzip2 (max) (percentage) 2.300 -3.36%
401.bzip2 (unexplained diff vs previous) 15.000 66.67%
401.bzip2 (unexplained diff vs previous) (percentage) 0.500 66.67%
403.gcc (avg) (percentage) 5.090 -
403.gcc (max) (percentage) 5.070 -2.12%
410.bwaves 71.000 -1.39%
410.bwaves (avg) 71.000 -1.39%
410.bwaves (avg) (percentage) 1.210 -1.63%
410.bwaves (max) 27.000 -6.90%
410.bwaves (max) (percentage) 0.460 -8.00%
410.bwaves (unexplained diff vs previous) 18.000 157.14%
410.bwaves (unexplained diff vs previous) (percentage) 0.300 150.00%
416.gamess 101.000 -
416.gamess (avg) 101.000 -
416.gamess (avg) (percentage) 0.830 -
416.gamess (max) 12.000 -7.69%
416.gamess (max) (percentage) 0.100 -
429.mcf 91.000 -1.09%
429.mcf (avg) 91.000 -1.09%
429.mcf (avg) (percentage) 6.510 -1.36%
429.mcf (max) 141.000 -
429.mcf (max) (percentage) 10.040 -
429.mcf (unexplained diff vs previous) 13.000 333.33%
429.mcf (unexplained diff vs previous) (percentage) 0.920 318.18%
433.milc 19.000 -
433.milc (avg) 19.000 -
433.milc (avg) (percentage) 1.680 -1.18%
433.milc (max) 23.000 -4.17%
433.milc (max) (percentage) 2.070 -3.72%
433.milc (unexplained diff vs previous) 4.000 300.00%
433.milc (unexplained diff vs previous) (percentage) 0.350 288.89%
434.zeusmp 35.000 -
434.zeusmp (avg) 35.000 -
434.zeusmp (avg) (percentage) 1.390 -2.11%
434.zeusmp (max) 18.000 -35.71%
434.zeusmp (max) (percentage) 0.720 -35.14%
434.zeusmp (unexplained diff vs previous) 3.000 -62.50%
434.zeusmp (unexplained diff vs previous) (percentage) 0.120 -62.50%
435.gromacs 146.000 -
435.gromacs (avg) 146.000 -
435.gromacs (avg) (percentage) 2.020 -1.46%
435.gromacs (max) 73.000 -55.21%
435.gromacs (max) (percentage) 1.010 -55.51%
435.gromacs (unexplained diff vs previous) 63.000 -23.17%
435.gromacs (unexplained diff vs previous) (percentage) 0.870 -23.68%
436.cactusADM 34.000 -2.86%
436.cactusADM (avg) 34.000 -2.86%
436.cactusADM (avg) (percentage) 1.630 -1.81%
436.cactusADM (max) 4.000 -76.47%
436.cactusADM (max) (percentage) 0.210 -74.07%
436.cactusADM (unexplained diff vs previous) 2.000 -60.00%
436.cactusADM (unexplained diff vs previous) (percentage) 0.090 -62.50%
437.leslie3d 546.000 6.23%
437.leslie3d (avg) 546.000 6.23%
437.leslie3d (avg) (percentage) 4.770 -
437.leslie3d (max) 660.000 88.03%
437.leslie3d (max) (percentage) 5.760 77.78%
437.leslie3d (unexplained diff vs previous) 624.000 1256.52%
437.leslie3d (unexplained diff vs previous) (percentage) 5.450 1197.62%
444.namd 13.000 -
444.namd (avg) 13.000 -
444.namd (avg) (percentage) 1.760 -
444.namd (max) 8.000 -
444.namd (max) (percentage) 1.160 11.54%
444.namd (unexplained diff vs previous) 9.000 12.50%
444.namd (unexplained diff vs previous) (percentage) 1.170 13.59%
445.gobmk 54.000 -1.82%
445.gobmk (avg) 54.000 -1.82%
445.gobmk (avg) (percentage) 1.100 -
445.gobmk (max) 81.000 -3.57%
445.gobmk (max) (percentage) 1.640 -2.96%
445.gobmk (unexplained diff vs previous) 37.000 117.65%
445.gobmk (unexplained diff vs previous) (percentage) 0.740 117.65%
447.dealII 24.000 -
447.dealII (avg) 24.000 -
447.dealII (avg) (percentage) 1.490 -
447.dealII (max) 20.000 -4.76%
447.dealII (max) (percentage) 1.220 -4.69%
450.soplex 16.000 -
450.soplex (avg) 16.000 -
450.soplex (avg) (percentage) 2.850 -1.38%
450.soplex (max) 5.000 -83.33%
450.soplex (max) (percentage) 0.900 -82.63%
450.soplex (unexplained diff vs previous) 4.000 -33.33%
450.soplex (unexplained diff vs previous) (percentage) 0.690 -32.35%
453.povray 9.000 -
453.povray (avg) 9.000 -
453.povray (avg) (percentage) 1.780 -
453.povray (max) 9.000 28.57%
453.povray (max) (percentage) 1.650 26.92%
453.povray (unexplained diff vs previous) 9.000 28.57%
453.povray (unexplained diff vs previous) (percentage) 1.620 26.56%
454.calculix (avg) (percentage) 3.600 -
454.calculix (max) (percentage) 1.030 -2.83%
456.hmmer 24.000 -
456.hmmer (avg) 24.000 -
456.hmmer (avg) (percentage) 0.810 -
456.hmmer (max) 8.000 14.29%
456.hmmer (max) (percentage) 0.300 11.11%
456.hmmer (unexplained diff vs previous) 9.000 12.50%
456.hmmer (unexplained diff vs previous) (percentage) 0.300 11.11%
458.sjeng 60.000 -
458.sjeng (avg) 60.000 -
458.sjeng (avg) (percentage) 0.810 -
458.sjeng (max) 68.000 -4.23%
458.sjeng (max) (percentage) 0.920 -4.17%
458.sjeng (unexplained diff vs previous) 38.000 26.67%
458.sjeng (unexplained diff vs previous) (percentage) 0.510 27.50%
459.GemsFDTD 86.000 -
459.GemsFDTD (avg) 86.000 -
459.GemsFDTD (avg) (percentage) 1.640 -
459.GemsFDTD (max) 18.000 -5.26%
459.GemsFDTD (max) (percentage) 0.340 -8.11%
462.libquantum 2.000 -
462.libquantum (avg) 2.000 -
462.libquantum (avg) (percentage) 1.830 -1.61%
462.libquantum (max) 2.000 -33.33%
462.libquantum (max) (percentage) 2.000 -9.91%
462.libquantum (unexplained diff vs previous) 0.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.000 -
464.h264ref 62.000 -1.59%
464.h264ref (avg) 62.000 -1.59%
464.h264ref (avg) (percentage) 1.080 -1.82%
464.h264ref (max) 24.000 -
464.h264ref (max) (percentage) 0.420 -
464.h264ref (unexplained diff vs previous) 2.000 -71.43%
464.h264ref (unexplained diff vs previous) (percentage) 0.030 -75.00%
465.tonto 182.000 -
465.tonto (avg) 182.000 -
465.tonto (avg) (percentage) 1.540 -
465.tonto (max) 269.000 -4.27%
465.tonto (max) (percentage) 2.280 -3.80%
470.lbm 48.000 -2.04%
470.lbm (avg) 48.000 -2.04%
470.lbm (avg) (percentage) 1.290 -
470.lbm (max) 34.000 -5.56%
470.lbm (max) (percentage) 0.920 -3.16%
470.lbm (unexplained diff vs previous) 19.000 -
470.lbm (unexplained diff vs previous) (percentage) 0.500 -
471.omnetpp 65.000 1.56%
471.omnetpp (avg) 65.000 1.56%
471.omnetpp (avg) (percentage) 2.370 -
471.omnetpp (max) 70.000 -1.41%
471.omnetpp (max) (percentage) 2.530 -3.44%
471.omnetpp (unexplained diff vs previous) 45.000 40.62%
471.omnetpp (unexplained diff vs previous) (percentage) 1.620 38.46%
473.astar 98.000 -
473.astar (avg) 98.000 -
473.astar (avg) (percentage) 1.900 -1.04%
473.astar (max) 173.000 -3.35%
473.astar (max) (percentage) 3.370 -3.71%
473.astar (unexplained diff vs previous) 19.000 72.73%
473.astar (unexplained diff vs previous) (percentage) 0.370 76.19%
481.wrf (avg) (percentage) 1.800 -
481.wrf (max) (percentage) 1.120 -1.75%
482.sphinx3 17.000 -
482.sphinx3 (avg) 17.000 -
482.sphinx3 (avg) (percentage) 2.060 -1.44%
482.sphinx3 (max) 7.000 133.33%
482.sphinx3 (max) (percentage) 0.850 97.67%
482.sphinx3 (unexplained diff vs previous) 7.000 133.33%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.850 129.73%
483.xalancbmk (avg) (percentage) 2.960 -
483.xalancbmk (max) (percentage) 1.770 -
Geometric Mean 6.043 -9.75%