Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_binutils https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=ee7af0e7107e918d37bd2686fea1db8f88d2242a (unchanged)
git_gcc https://github.com/gcc-mirror/gcc/commit/6a0a46c57999d1f805f6c604a8868ae588a104f2 (+ 110 commits)
git_glibc https://sourceware.org/git/?p=glibc.git;a=commit;h=8d7b6b4cb27d4dec1dd5f7960298c1699275f962 (unchanged)
git_linux https://git.linaro.org/kernel-org/linux-stable.git/commit/?id=30e92e8838a078df359a97e36d048f06bbd5bec1 (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--gnu-arm-master-Os_LTO-build/190/
View Options
Report
Filter

gnu-arm-master-Os_LTO test results

Run Order Start Time Duration
Current 0190 (basepoints/gcc-15-00625-g6a0a46c5799) 2024-05-14T07:24:24 0:00:00
Previous 0189 (v6.0.10-41678-g8e427351ca733) 2022-12-02T00:47:07 0:00:00
Baseline 0001 (3-changed-components) 2022-12-30T06:27:35 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 63 54
Performance Improvements 102 111
Unchanged Tests 1071 1071
Total Tests 1236

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
437.leslie3d 2.37% 48001.000 49140.000 - 2.37% -
453.povray 1.97% 489083.000 498741.000 - 1.99% -
473.astar 1.44% 15549.000 15773.000 - 1.44% -
447.dealII 1.41% 190250.000 192930.000 - 0.94% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
465.tonto 24.46% 10171.000 12659.000 - -2.95% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
429.mcf (max) (percentage) 2717.14% 0.350 9.860 - 2981.25% -
403.gcc (max) (percentage) 2675.00% 0.320 8.880 - 1405.08% -
450.soplex (avg) (percentage) 1333.33% 0.210 3.010 - 1154.17% -
429.mcf (avg) (percentage) 981.13% 0.530 5.730 - 1093.75% -
471.omnetpp (unexplained diff vs previous) (percentage) 700.00% 0.040 0.320 0.000 -52.24% 0.000
401.bzip2 (avg) (percentage) 700.00% 0.280 2.240 - 729.63% -
462.libquantum (avg) (percentage) 608.70% 0.230 1.630 - 757.89% -
401.bzip2 (max) (percentage) 588.24% 0.340 2.340 - 485.00% -
433.milc (unexplained diff vs previous) (percentage) 500.00% 0.030 0.180 0.000 -74.29% 0.000
454.calculix (max) (percentage) 472.46% 0.690 3.950 - 370.24% -
403.gcc (avg) (percentage) 464.29% 0.420 2.370 - 451.16% -
400.perlbench (avg) (percentage) 457.14% 0.350 1.950 - 400.00% -
450.soplex (max) (percentage) 452.38% 0.210 1.160 - 163.64% -
458.sjeng (max) (percentage) 450.00% 0.100 0.550 - 83.33% -
445.gobmk (avg) (percentage) 442.86% 0.210 1.140 - 418.18% -
473.astar (avg) (percentage) 342.50% 0.400 1.770 - 342.50% -
400.perlbench (max) (percentage) 326.42% 0.530 2.260 - 182.50% -
437.leslie3d (max) (percentage) 281.48% 2.430 9.270 - 199.03% -
458.sjeng (avg) (percentage) 275.00% 0.240 0.900 - 291.30% -
473.astar (max) (percentage) 230.77% 0.260 0.860 - 82.98% -
456.hmmer (avg) (percentage) 220.00% 0.250 0.800 - 166.67% -
429.mcf (max) 204.88% 41.000 125.000 - 237.84% -
437.leslie3d (avg) (percentage) 190.73% 1.510 4.390 - 162.87% -
462.libquantum (max) (percentage) 171.43% 0.280 0.760 - 850.00% -
458.sjeng (max) 168.75% 16.000 43.000 - -10.42% -
483.xalancbmk (max) (percentage) 161.02% 0.590 1.540 - 67.39% -
464.h264ref (avg) (percentage) 153.66% 0.410 1.040 - 166.67% -
437.leslie3d (max) 143.44% 442.000 1076.000 - 92.14% -
447.dealII (avg) (percentage) 125.88% 0.850 1.920 - 120.69% -
471.omnetpp (unexplained diff vs previous) 125.00% 4.000 9.000 0.000 -85.48% 0.000
435.gromacs (avg) (percentage) 122.09% 0.860 1.910 - 101.05% -
445.gobmk 119.23% 26.000 57.000 - 111.11% -
445.gobmk (avg) 119.23% 26.000 57.000 - 111.11% -
433.milc (avg) (percentage) 115.07% 0.730 1.570 - 127.54% -
483.xalancbmk (avg) (percentage) 107.27% 0.550 1.140 - 103.57% -
437.leslie3d 85.77% 274.000 509.000 - 69.10% -
437.leslie3d (avg) 85.77% 274.000 509.000 - 69.10% -
458.sjeng 84.21% 38.000 70.000 - 94.44% -
458.sjeng (avg) 84.21% 38.000 70.000 - 94.44% -
471.omnetpp (avg) (percentage) 68.14% 1.130 1.900 - 61.02% -
473.astar 66.67% 51.000 85.000 - 66.67% -
473.astar (avg) 66.67% 51.000 85.000 - 66.67% -
445.gobmk (max) (percentage) 41.18% 0.340 0.480 - 77.78% -
454.calculix (avg) (percentage) 38.15% 2.490 3.440 - 69.46% -
401.bzip2 36.17% 47.000 64.000 - 42.22% -
401.bzip2 (avg) 36.17% 47.000 64.000 - 42.22% -
473.astar (max) 24.24% 33.000 41.000 - -32.79% -
434.zeusmp (avg) (percentage) 24.11% 1.120 1.390 - 13.93% -
471.omnetpp (max) (percentage) 19.42% 1.390 1.660 - -10.75% -
453.povray (avg) (percentage) 18.62% 1.450 1.720 - 12.42% -
465.tonto (max) 18.45% 206.000 244.000 - 9.42% -
459.GemsFDTD (avg) (percentage) 17.56% 1.310 1.540 - 10.79% -
429.mcf 16.13% 62.000 72.000 - 28.57% -
429.mcf (avg) 16.13% 62.000 72.000 - 28.57% -
456.hmmer (max) (percentage) 15.62% 0.320 0.370 - -28.85% -
401.bzip2 (max) 15.52% 58.000 67.000 - 0.00% -
464.h264ref 4.92% 61.000 64.000 - 8.47% -
464.h264ref (avg) 4.92% 61.000 64.000 - 8.47% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
470.lbm -5.90% 10845.000 10205.000 - -5.90% -
434.zeusmp -2.01% 141896.000 139050.000 - -1.42% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
462.libquantum -98.73% 16968.000 215.000 - -98.74% -
482.sphinx3 -97.54% 37800.000 930.000 - -97.46% -
450.soplex -96.25% 15705.000 589.000 - -96.25% -
444.namd -94.31% 15937.000 907.000 - -94.32% -
453.povray -93.02% 7818.000 546.000 - -92.62% -
433.milc -91.97% 13701.000 1100.000 - -91.95% -
436.cactusADM -89.33% 24438.000 2607.000 - -89.31% -
429.mcf -89.22% 11784.000 1270.000 - -89.17% -
434.zeusmp -86.75% 21021.000 2785.000 - -86.70% -
400.perlbench -85.45% 11264.000 1639.000 - -85.39% -
401.bzip2 -83.06% 17084.000 2894.000 - -82.95% -
447.dealII -78.08% 9205.000 2018.000 - -77.83% -
470.lbm -78.02% 17142.000 3767.000 - -76.91% -
456.hmmer -76.45% 19765.000 4654.000 - -75.50% -
459.GemsFDTD -73.23% 26018.000 6965.000 - -72.54% -
471.omnetpp -69.23% 9170.000 2822.000 - -69.54% -
473.astar -62.66% 12916.000 4823.000 - -62.85% -
410.bwaves -61.30% 15368.000 5948.000 - -59.66% -
445.gobmk -59.29% 12452.000 5069.000 - -59.11% -
464.h264ref -58.71% 14977.000 6184.000 - -59.34% -
435.gromacs -55.11% 16825.000 7552.000 - -55.19% -
458.sjeng -51.34% 16117.000 7842.000 - -51.04% -
437.leslie3d -36.21% 18207.000 11614.000 - -35.73% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (max) -99.49% 1375.000 7.000 - -99.39% -
444.namd (max) -99.20% 627.000 5.000 - -99.43% -
436.cactusADM (max) -98.61% 794.000 11.000 - -99.11% -
434.zeusmp (max) -98.50% 401.000 6.000 - -98.75% -
482.sphinx3 -98.09% 892.000 17.000 - -98.02% -
482.sphinx3 (avg) -98.09% 892.000 17.000 - -98.02% -
470.lbm (max) -98.07% 570.000 11.000 - -98.77% -
462.libquantum (max) -97.87% 47.000 1.000 - -92.31% -
444.namd -96.48% 369.000 13.000 - -96.86% -
444.namd (avg) -96.48% 369.000 13.000 - -96.86% -
433.milc (max) -96.36% 110.000 4.000 - -95.88% -
453.povray (max) -95.60% 182.000 8.000 - -95.81% -
459.GemsFDTD (max) -95.47% 684.000 31.000 - -94.64% -
416.gamess (max) (percentage) -94.64% 2.800 0.150 - -95.22% -
436.cactusADM -93.99% 566.000 34.000 - -94.19% -
436.cactusADM (avg) -93.99% 566.000 34.000 - -94.19% -
481.wrf (max) (percentage) -92.69% 3.830 0.280 - -93.27% -
462.libquantum -92.31% 39.000 3.000 - -90.62% -
462.libquantum (avg) -92.31% 39.000 3.000 - -90.62% -
453.povray -92.04% 113.000 9.000 - -92.04% -
453.povray (avg) -92.04% 113.000 9.000 - -92.04% -
470.lbm (max) (percentage) -90.69% 3.330 0.310 - -94.34% -
410.bwaves (max) -89.88% 484.000 49.000 - -81.15% -
470.lbm -89.77% 577.000 59.000 - -90.30% -
470.lbm (avg) -89.77% 577.000 59.000 - -90.30% -
434.zeusmp (max) (percentage) -88.48% 1.910 0.220 - -90.43% -
436.cactusADM (max) (percentage) -85.85% 3.250 0.460 - -90.89% -
444.namd (max) (percentage) -85.28% 3.940 0.580 - -89.47% -
434.zeusmp -83.83% 235.000 38.000 - -85.10% -
434.zeusmp (avg) -83.83% 235.000 38.000 - -85.10% -
433.milc -83.00% 100.000 17.000 - -81.91% -
433.milc (avg) -83.00% 100.000 17.000 - -81.91% -
459.GemsFDTD (max) (percentage) -82.89% 2.630 0.450 - -80.26% -
450.soplex (max) -81.25% 32.000 6.000 - -91.30% -
464.h264ref (max) -81.08% 74.000 14.000 - -76.27% -
447.dealII (max) -78.31% 83.000 18.000 - -88.00% -
482.sphinx3 (max) (percentage) -76.37% 3.640 0.860 - -72.70% -
410.bwaves -76.00% 325.000 78.000 - -73.01% -
410.bwaves (avg) -76.00% 325.000 78.000 - -73.01% -
435.gromacs (max) -74.77% 218.000 55.000 - -81.42% -
410.bwaves (max) (percentage) -73.65% 3.150 0.830 - -53.11% -
456.hmmer (max) -73.02% 63.000 17.000 - -82.65% -
459.GemsFDTD -68.53% 340.000 107.000 - -69.60% -
459.GemsFDTD (avg) -68.53% 340.000 107.000 - -69.60% -
471.omnetpp (max) -63.78% 127.000 46.000 - -73.26% -
470.lbm (avg) (percentage) -52.82% 3.370 1.590 - -57.37% -
464.h264ref (max) (percentage) -52.00% 0.500 0.240 - -38.46% -
433.milc (max) (percentage) -51.85% 0.810 0.390 - -45.07% -
447.dealII -51.28% 78.000 38.000 - -51.90% -
447.dealII (avg) -51.28% 78.000 38.000 - -51.90% -
433.milc (unexplained diff vs previous) -50.00% 4.000 2.000 0.000 -97.92% 0.000
471.omnetpp -48.54% 103.000 53.000 - -51.38% -
471.omnetpp (avg) -48.54% 103.000 53.000 - -51.38% -
450.soplex -46.88% 32.000 17.000 - -54.05% -
450.soplex (avg) -46.88% 32.000 17.000 - -54.05% -
416.gamess (avg) (percentage) -43.33% 1.500 0.850 - -48.48% -
435.gromacs (max) (percentage) -43.08% 1.300 0.740 - -57.95% -
445.gobmk (max) -42.86% 42.000 24.000 - -27.27% -
436.cactusADM (avg) (percentage) -42.24% 2.320 1.340 - -44.17% -
410.bwaves (avg) (percentage) -37.74% 2.120 1.320 - -32.65% -
400.perlbench (max) -37.29% 59.000 37.000 - -58.43% -
453.povray (max) (percentage) -35.62% 2.330 1.500 - -42.08% -
444.namd (avg) (percentage) -34.91% 2.320 1.510 - -41.92% -
481.wrf (avg) (percentage) -25.10% 2.470 1.850 - -28.02% -
456.hmmer -24.49% 49.000 37.000 - -33.93% -
456.hmmer (avg) -24.49% 49.000 37.000 - -33.93% -
465.tonto (avg) (percentage) -24.19% 1.240 0.940 - -17.54% -
400.perlbench -20.51% 39.000 31.000 - -27.91% -
400.perlbench (avg) -20.51% 39.000 31.000 - -27.91% -
482.sphinx3 (avg) (percentage) -19.49% 2.360 1.900 - -19.15% -
465.tonto -6.35% 126.000 118.000 - -20.27% -
465.tonto (avg) -6.35% 126.000 118.000 - -20.27% -
465.tonto (max) (percentage) -4.93% 2.030 1.930 - 12.87% -
447.dealII (max) (percentage) -1.10% 0.910 0.900 - -45.45% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 134.48% 29.000 68.000 - 134.48% -
nb_compile_successful 65.52% 29.000 48.000 - 65.52% -
nb_execution_successful 65.52% 29.000 48.000 - 65.52% -

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
437.leslie3d 2.37% 48001.000 49140.000 - 2.37% -
453.povray 1.99% 489013.000 498741.000 - 1.97% -
473.astar 1.44% 15549.000 15773.000 - 1.44% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
429.mcf (max) (percentage) 2981.25% 0.320 9.860 - 2717.14% -
403.gcc (max) (percentage) 1405.08% 0.590 8.880 - 2675.00% -
450.soplex (avg) (percentage) 1154.17% 0.240 3.010 - 1333.33% -
429.mcf (avg) (percentage) 1093.75% 0.480 5.730 - 981.13% -
462.libquantum (max) (percentage) 850.00% 0.080 0.760 - 171.43% -
462.libquantum (avg) (percentage) 757.89% 0.190 1.630 - 608.70% -
401.bzip2 (avg) (percentage) 729.63% 0.270 2.240 - 700.00% -
401.bzip2 (max) (percentage) 485.00% 0.400 2.340 - 588.24% -
403.gcc (avg) (percentage) 451.16% 0.430 2.370 - 464.29% -
445.gobmk (avg) (percentage) 418.18% 0.220 1.140 - 442.86% -
400.perlbench (avg) (percentage) 400.00% 0.390 1.950 - 457.14% -
454.calculix (max) (percentage) 370.24% 0.840 3.950 - 472.46% -
473.astar (avg) (percentage) 342.50% 0.400 1.770 - 342.50% -
458.sjeng (avg) (percentage) 291.30% 0.230 0.900 - 275.00% -
429.mcf (max) 237.84% 37.000 125.000 - 204.88% -
437.leslie3d (max) (percentage) 199.03% 3.100 9.270 - 281.48% -
400.perlbench (max) (percentage) 182.50% 0.800 2.260 - 326.42% -
456.hmmer (avg) (percentage) 166.67% 0.300 0.800 - 220.00% -
464.h264ref (avg) (percentage) 166.67% 0.390 1.040 - 153.66% -
450.soplex (max) (percentage) 163.64% 0.440 1.160 - 452.38% -
437.leslie3d (avg) (percentage) 162.87% 1.670 4.390 - 190.73% -
433.milc (avg) (percentage) 127.54% 0.690 1.570 - 115.07% -
447.dealII (avg) (percentage) 120.69% 0.870 1.920 - 125.88% -
445.gobmk 111.11% 27.000 57.000 - 119.23% -
445.gobmk (avg) 111.11% 27.000 57.000 - 119.23% -
483.xalancbmk (avg) (percentage) 103.57% 0.560 1.140 - 107.27% -
435.gromacs (avg) (percentage) 101.05% 0.950 1.910 - 122.09% -
458.sjeng 94.44% 36.000 70.000 - 84.21% -
458.sjeng (avg) 94.44% 36.000 70.000 - 84.21% -
437.leslie3d (max) 92.14% 560.000 1076.000 - 143.44% -
458.sjeng (max) (percentage) 83.33% 0.300 0.550 - 450.00% -
473.astar (max) (percentage) 82.98% 0.470 0.860 - 230.77% -
445.gobmk (max) (percentage) 77.78% 0.270 0.480 - 41.18% -
454.calculix (avg) (percentage) 69.46% 2.030 3.440 - 38.15% -
437.leslie3d 69.10% 301.000 509.000 - 85.77% -
437.leslie3d (avg) 69.10% 301.000 509.000 - 85.77% -
483.xalancbmk (max) (percentage) 67.39% 0.920 1.540 - 161.02% -
473.astar 66.67% 51.000 85.000 - 66.67% -
473.astar (avg) 66.67% 51.000 85.000 - 66.67% -
471.omnetpp (avg) (percentage) 61.02% 1.180 1.900 - 68.14% -
401.bzip2 42.22% 45.000 64.000 - 36.17% -
401.bzip2 (avg) 42.22% 45.000 64.000 - 36.17% -
429.mcf 28.57% 56.000 72.000 - 16.13% -
429.mcf (avg) 28.57% 56.000 72.000 - 16.13% -
434.zeusmp (avg) (percentage) 13.93% 1.220 1.390 - 24.11% -
465.tonto (max) (percentage) 12.87% 1.710 1.930 - -4.93% -
453.povray (avg) (percentage) 12.42% 1.530 1.720 - 18.62% -
459.GemsFDTD (avg) (percentage) 10.79% 1.390 1.540 - 17.56% -
465.tonto (max) 9.42% 223.000 244.000 - 18.45% -
464.h264ref 8.47% 59.000 64.000 - 4.92% -
464.h264ref (avg) 8.47% 59.000 64.000 - 4.92% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
470.lbm -5.90% 10845.000 10205.000 - -5.90% -
434.zeusmp -1.42% 141060.000 139050.000 - -2.01% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -98.74% 17005.000 215.000 - -98.73% -
482.sphinx3 -97.46% 36586.000 930.000 - -97.54% -
450.soplex -96.25% 15709.000 589.000 - -96.25% -
444.namd -94.32% 15957.000 907.000 - -94.31% -
453.povray -92.62% 7395.000 546.000 - -93.02% -
433.milc -91.95% 13668.000 1100.000 - -91.97% -
436.cactusADM -89.31% 24396.000 2607.000 - -89.33% -
429.mcf -89.17% 11727.000 1270.000 - -89.22% -
434.zeusmp -86.70% 20932.000 2785.000 - -86.75% -
400.perlbench -85.39% 11215.000 1639.000 - -85.45% -
401.bzip2 -82.95% 16971.000 2894.000 - -83.06% -
447.dealII -77.83% 9101.000 2018.000 - -78.08% -
470.lbm -76.91% 16312.000 3767.000 - -78.02% -
456.hmmer -75.50% 18995.000 4654.000 - -76.45% -
459.GemsFDTD -72.54% 25364.000 6965.000 - -73.23% -
471.omnetpp -69.54% 9265.000 2822.000 - -69.23% -
473.astar -62.85% 12983.000 4823.000 - -62.66% -
410.bwaves -59.66% 14745.000 5948.000 - -61.30% -
464.h264ref -59.34% 15208.000 6184.000 - -58.71% -
445.gobmk -59.11% 12396.000 5069.000 - -59.29% -
435.gromacs -55.19% 16853.000 7552.000 - -55.11% -
458.sjeng -51.04% 16018.000 7842.000 - -51.34% -
437.leslie3d -35.73% 18070.000 11614.000 - -36.21% -
465.tonto -2.95% 13044.000 12659.000 - 24.46% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.43% 879.000 5.000 - -99.20% -
482.sphinx3 (max) -99.39% 1152.000 7.000 - -99.49% -
436.cactusADM (max) -99.11% 1231.000 11.000 - -98.61% -
470.lbm (max) -98.77% 893.000 11.000 - -98.07% -
434.zeusmp (max) -98.75% 481.000 6.000 - -98.50% -
482.sphinx3 -98.02% 859.000 17.000 - -98.09% -
482.sphinx3 (avg) -98.02% 859.000 17.000 - -98.09% -
433.milc (unexplained diff vs previous) -97.92% 96.000 2.000 0.000 -50.00% 0.000
444.namd -96.86% 414.000 13.000 - -96.48% -
444.namd (avg) -96.86% 414.000 13.000 - -96.48% -
433.milc (max) -95.88% 97.000 4.000 - -96.36% -
453.povray (max) -95.81% 191.000 8.000 - -95.60% -
416.gamess (max) (percentage) -95.22% 3.140 0.150 - -94.64% -
459.GemsFDTD (max) -94.64% 578.000 31.000 - -95.47% -
470.lbm (max) (percentage) -94.34% 5.480 0.310 - -90.69% -
436.cactusADM -94.19% 585.000 34.000 - -93.99% -
436.cactusADM (avg) -94.19% 585.000 34.000 - -93.99% -
481.wrf (max) (percentage) -93.27% 4.160 0.280 - -92.69% -
462.libquantum (max) -92.31% 13.000 1.000 - -97.87% -
453.povray -92.04% 113.000 9.000 - -92.04% -
453.povray (avg) -92.04% 113.000 9.000 - -92.04% -
450.soplex (max) -91.30% 69.000 6.000 - -81.25% -
436.cactusADM (max) (percentage) -90.89% 5.050 0.460 - -85.85% -
462.libquantum -90.62% 32.000 3.000 - -92.31% -
462.libquantum (avg) -90.62% 32.000 3.000 - -92.31% -
434.zeusmp (max) (percentage) -90.43% 2.300 0.220 - -88.48% -
470.lbm -90.30% 608.000 59.000 - -89.77% -
470.lbm (avg) -90.30% 608.000 59.000 - -89.77% -
444.namd (max) (percentage) -89.47% 5.510 0.580 - -85.28% -
447.dealII (max) -88.00% 150.000 18.000 - -78.31% -
471.omnetpp (unexplained diff vs previous) -85.48% 62.000 9.000 0.000 125.00% 0.000
434.zeusmp -85.10% 255.000 38.000 - -83.83% -
434.zeusmp (avg) -85.10% 255.000 38.000 - -83.83% -
456.hmmer (max) -82.65% 98.000 17.000 - -73.02% -
433.milc -81.91% 94.000 17.000 - -83.00% -
433.milc (avg) -81.91% 94.000 17.000 - -83.00% -
435.gromacs (max) -81.42% 296.000 55.000 - -74.77% -
410.bwaves (max) -81.15% 260.000 49.000 - -89.88% -
459.GemsFDTD (max) (percentage) -80.26% 2.280 0.450 - -82.89% -
464.h264ref (max) -76.27% 59.000 14.000 - -81.08% -
433.milc (unexplained diff vs previous) (percentage) -74.29% 0.700 0.180 0.000 500.00% 0.000
471.omnetpp (max) -73.26% 172.000 46.000 - -63.78% -
410.bwaves -73.01% 289.000 78.000 - -76.00% -
410.bwaves (avg) -73.01% 289.000 78.000 - -76.00% -
482.sphinx3 (max) (percentage) -72.70% 3.150 0.860 - -76.37% -
459.GemsFDTD -69.60% 352.000 107.000 - -68.53% -
459.GemsFDTD (avg) -69.60% 352.000 107.000 - -68.53% -
400.perlbench (max) -58.43% 89.000 37.000 - -37.29% -
435.gromacs (max) (percentage) -57.95% 1.760 0.740 - -43.08% -
470.lbm (avg) (percentage) -57.37% 3.730 1.590 - -52.82% -
450.soplex -54.05% 37.000 17.000 - -46.88% -
450.soplex (avg) -54.05% 37.000 17.000 - -46.88% -
410.bwaves (max) (percentage) -53.11% 1.770 0.830 - -73.65% -
471.omnetpp (unexplained diff vs previous) (percentage) -52.24% 0.670 0.320 0.000 700.00% 0.000
447.dealII -51.90% 79.000 38.000 - -51.28% -
447.dealII (avg) -51.90% 79.000 38.000 - -51.28% -
471.omnetpp -51.38% 109.000 53.000 - -48.54% -
471.omnetpp (avg) -51.38% 109.000 53.000 - -48.54% -
416.gamess (avg) (percentage) -48.48% 1.650 0.850 - -43.33% -
447.dealII (max) (percentage) -45.45% 1.650 0.900 - -1.10% -
433.milc (max) (percentage) -45.07% 0.710 0.390 - -51.85% -
436.cactusADM (avg) (percentage) -44.17% 2.400 1.340 - -42.24% -
453.povray (max) (percentage) -42.08% 2.590 1.500 - -35.62% -
444.namd (avg) (percentage) -41.92% 2.600 1.510 - -34.91% -
464.h264ref (max) (percentage) -38.46% 0.390 0.240 - -52.00% -
456.hmmer -33.93% 56.000 37.000 - -24.49% -
456.hmmer (avg) -33.93% 56.000 37.000 - -24.49% -
473.astar (max) -32.79% 61.000 41.000 - 24.24% -
410.bwaves (avg) (percentage) -32.65% 1.960 1.320 - -37.74% -
456.hmmer (max) (percentage) -28.85% 0.520 0.370 - 15.62% -
481.wrf (avg) (percentage) -28.02% 2.570 1.850 - -25.10% -
400.perlbench -27.91% 43.000 31.000 - -20.51% -
400.perlbench (avg) -27.91% 43.000 31.000 - -20.51% -
445.gobmk (max) -27.27% 33.000 24.000 - -42.86% -
465.tonto -20.27% 148.000 118.000 - -6.35% -
465.tonto (avg) -20.27% 148.000 118.000 - -6.35% -
482.sphinx3 (avg) (percentage) -19.15% 2.350 1.900 - -19.49% -
465.tonto (avg) (percentage) -17.54% 1.140 0.940 - -24.19% -
471.omnetpp (max) (percentage) -10.75% 1.860 1.660 - 19.42% -
458.sjeng (max) -10.42% 48.000 43.000 - 168.75% -
435.gromacs -10.00% 160.000 144.000 - 0.00% -
435.gromacs (avg) -10.00% 160.000 144.000 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 134.48% 29.000 68.000 - 134.48% -
nb_compile_successful 65.52% 29.000 48.000 - 65.52% -
nb_execution_successful 65.52% 29.000 48.000 - 65.52% -

Report Time: 0.36s
Execution (seconds)
Name Current %
400.perlbench 1639.000 -85.45%
401.bzip2 2894.000 -83.06%
410.bwaves 5948.000 -61.30%
429.mcf 1270.000 -89.22%
433.milc 1100.000 -91.97%
434.zeusmp 2785.000 -86.75%
435.gromacs 7552.000 -55.11%
436.cactusADM 2607.000 -89.33%
437.leslie3d 11614.000 -36.21%
444.namd 907.000 -94.31%
445.gobmk 5069.000 -59.29%
447.dealII 2018.000 -78.08%
450.soplex 589.000 -96.25%
453.povray 546.000 -93.02%
456.hmmer 4654.000 -76.45%
458.sjeng 7842.000 -51.34%
459.GemsFDTD 6965.000 -73.23%
462.libquantum 215.000 -98.73%
464.h264ref 6184.000 -58.71%
465.tonto 12659.000 24.46%
470.lbm 3767.000 -78.02%
471.omnetpp 2822.000 -69.23%
473.astar 4823.000 -62.66%
482.sphinx3 930.000 -97.54%
Geometric Mean 2656.656 -82.65%
Code Size (bytes)
Name Current %
400.perlbench 566288.000 -
401.bzip2 33502.000 -
410.bwaves 14683.000 -
429.mcf 7693.000 -
433.milc 47877.000 -
434.zeusmp 139050.000 -2.01%
435.gromacs 394278.000 -
436.cactusADM 257750.000 -
437.leslie3d 49140.000 2.37%
444.namd 163039.000 -
445.gobmk 1089512.000 -
447.dealII 192930.000 1.41%
450.soplex 136621.000 -
453.povray 498741.000 1.97%
456.hmmer 74837.000 -
458.sjeng 81621.000 -
459.GemsFDTD 197299.000 -
462.libquantum 13118.000 -
464.h264ref 301762.000 -
465.tonto 1448377.000 -
470.lbm 10205.000 -5.90%
471.omnetpp 297381.000 -
473.astar 15773.000 1.44%
482.sphinx3 79718.000 -
Geometric Mean 106354.314 -
Score
Name Current %
nb_compile_successful 48.000 65.52%
nb_execution_successful 48.000 65.52%
nb_total_tests 68.000 134.48%
Geometric Mean 53.909 85.89%
sample variability
Name Current %
400.perlbench 31.000 -20.51%
400.perlbench (avg) 31.000 -20.51%
400.perlbench (avg) (percentage) 1.950 457.14%
400.perlbench (max) 37.000 -37.29%
400.perlbench (max) (percentage) 2.260 326.42%
401.bzip2 64.000 36.17%
401.bzip2 (avg) 64.000 36.17%
401.bzip2 (avg) (percentage) 2.240 700.00%
401.bzip2 (max) 67.000 15.52%
401.bzip2 (max) (percentage) 2.340 588.24%
403.gcc (avg) (percentage) 2.370 464.29%
403.gcc (max) (percentage) 8.880 2675.00%
410.bwaves 78.000 -76.00%
410.bwaves (avg) 78.000 -76.00%
410.bwaves (avg) (percentage) 1.320 -37.74%
410.bwaves (max) 49.000 -89.88%
410.bwaves (max) (percentage) 0.830 -73.65%
416.gamess (avg) (percentage) 0.850 -43.33%
416.gamess (max) (percentage) 0.150 -94.64%
429.mcf 72.000 16.13%
429.mcf (avg) 72.000 16.13%
429.mcf (avg) (percentage) 5.730 981.13%
429.mcf (max) 125.000 204.88%
429.mcf (max) (percentage) 9.860 2717.14%
433.milc 17.000 -83.00%
433.milc (avg) 17.000 -83.00%
433.milc (avg) (percentage) 1.570 115.07%
433.milc (max) 4.000 -96.36%
433.milc (max) (percentage) 0.390 -51.85%
433.milc (unexplained diff vs previous) 2.000 -50.00%
433.milc (unexplained diff vs previous) (percentage) 0.180 500.00%
434.zeusmp 38.000 -83.83%
434.zeusmp (avg) 38.000 -83.83%
434.zeusmp (avg) (percentage) 1.390 24.11%
434.zeusmp (max) 6.000 -98.50%
434.zeusmp (max) (percentage) 0.220 -88.48%
435.gromacs 144.000 -
435.gromacs (avg) 144.000 -
435.gromacs (avg) (percentage) 1.910 122.09%
435.gromacs (max) 55.000 -74.77%
435.gromacs (max) (percentage) 0.740 -43.08%
436.cactusADM 34.000 -93.99%
436.cactusADM (avg) 34.000 -93.99%
436.cactusADM (avg) (percentage) 1.340 -42.24%
436.cactusADM (max) 11.000 -98.61%
436.cactusADM (max) (percentage) 0.460 -85.85%
437.leslie3d 509.000 85.77%
437.leslie3d (avg) 509.000 85.77%
437.leslie3d (avg) (percentage) 4.390 190.73%
437.leslie3d (max) 1076.000 143.44%
437.leslie3d (max) (percentage) 9.270 281.48%
444.namd 13.000 -96.48%
444.namd (avg) 13.000 -96.48%
444.namd (avg) (percentage) 1.510 -34.91%
444.namd (max) 5.000 -99.20%
444.namd (max) (percentage) 0.580 -85.28%
445.gobmk 57.000 119.23%
445.gobmk (avg) 57.000 119.23%
445.gobmk (avg) (percentage) 1.140 442.86%
445.gobmk (max) 24.000 -42.86%
445.gobmk (max) (percentage) 0.480 41.18%
447.dealII 38.000 -51.28%
447.dealII (avg) 38.000 -51.28%
447.dealII (avg) (percentage) 1.920 125.88%
447.dealII (max) 18.000 -78.31%
447.dealII (max) (percentage) 0.900 -1.10%
450.soplex 17.000 -46.88%
450.soplex (avg) 17.000 -46.88%
450.soplex (avg) (percentage) 3.010 1333.33%
450.soplex (max) 6.000 -81.25%
450.soplex (max) (percentage) 1.160 452.38%
453.povray 9.000 -92.04%
453.povray (avg) 9.000 -92.04%
453.povray (avg) (percentage) 1.720 18.62%
453.povray (max) 8.000 -95.60%
453.povray (max) (percentage) 1.500 -35.62%
454.calculix (avg) (percentage) 3.440 38.15%
454.calculix (max) (percentage) 3.950 472.46%
456.hmmer 37.000 -24.49%
456.hmmer (avg) 37.000 -24.49%
456.hmmer (avg) (percentage) 0.800 220.00%
456.hmmer (max) 17.000 -73.02%
456.hmmer (max) (percentage) 0.370 15.62%
458.sjeng 70.000 84.21%
458.sjeng (avg) 70.000 84.21%
458.sjeng (avg) (percentage) 0.900 275.00%
458.sjeng (max) 43.000 168.75%
458.sjeng (max) (percentage) 0.550 450.00%
459.GemsFDTD 107.000 -68.53%
459.GemsFDTD (avg) 107.000 -68.53%
459.GemsFDTD (avg) (percentage) 1.540 17.56%
459.GemsFDTD (max) 31.000 -95.47%
459.GemsFDTD (max) (percentage) 0.450 -82.89%
462.libquantum 3.000 -92.31%
462.libquantum (avg) 3.000 -92.31%
462.libquantum (avg) (percentage) 1.630 608.70%
462.libquantum (max) 1.000 -97.87%
462.libquantum (max) (percentage) 0.760 171.43%
464.h264ref 64.000 4.92%
464.h264ref (avg) 64.000 4.92%
464.h264ref (avg) (percentage) 1.040 153.66%
464.h264ref (max) 14.000 -81.08%
464.h264ref (max) (percentage) 0.240 -52.00%
465.tonto 118.000 -6.35%
465.tonto (avg) 118.000 -6.35%
465.tonto (avg) (percentage) 0.940 -24.19%
465.tonto (max) 244.000 18.45%
465.tonto (max) (percentage) 1.930 -4.93%
470.lbm 59.000 -89.77%
470.lbm (avg) 59.000 -89.77%
470.lbm (avg) (percentage) 1.590 -52.82%
470.lbm (max) 11.000 -98.07%
470.lbm (max) (percentage) 0.310 -90.69%
471.omnetpp 53.000 -48.54%
471.omnetpp (avg) 53.000 -48.54%
471.omnetpp (avg) (percentage) 1.900 68.14%
471.omnetpp (max) 46.000 -63.78%
471.omnetpp (max) (percentage) 1.660 19.42%
471.omnetpp (unexplained diff vs previous) 9.000 125.00%
471.omnetpp (unexplained diff vs previous) (percentage) 0.320 700.00%
473.astar 85.000 66.67%
473.astar (avg) 85.000 66.67%
473.astar (avg) (percentage) 1.770 342.50%
473.astar (max) 41.000 24.24%
473.astar (max) (percentage) 0.860 230.77%
481.wrf (avg) (percentage) 1.850 -25.10%
481.wrf (max) (percentage) 0.280 -92.69%
482.sphinx3 17.000 -98.09%
482.sphinx3 (avg) 17.000 -98.09%
482.sphinx3 (avg) (percentage) 1.900 -19.49%
482.sphinx3 (max) 7.000 -99.49%
482.sphinx3 (max) (percentage) 0.860 -76.37%
483.xalancbmk (avg) (percentage) 1.140 107.27%
483.xalancbmk (max) (percentage) 1.540 161.02%
Geometric Mean 7.448 -39.03%