Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/5d529c32cc2d5342a0d183881b6c3023435ed5d3 (+ 89 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Oz_LTO-build/217/
View Options
Report
Filter

llvm-arm-master-Oz_LTO test results

Run Order Start Time Duration
Current 0217 (llvmorg-20-init-16393-g5d529c32cc2d) 2024-12-23T19:33:20 0:00:00
Previous 0217 (no-change) 2022-11-08T18:14:53 0:00:00
Baseline 0001 (llvmorg-16-init-17036-g4f40923103d2) 2023-01-06T17:55:45 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 41 18
Performance Improvements 109 126
Added Tests 2 6
Unchanged Tests 628 630
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
450.soplex 2.15% 146877.000 150036.000 - 2.23% -
471.omnetpp 2.07% 316696.000 323250.000 - 2.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
470.lbm (unexplained diff vs previous) (percentage) 1800.00% 0.020 0.380 - -93.73% -
403.gcc (max) (percentage) 1622.00% 0.500 8.610 - 269.53% -
450.soplex (unexplained diff vs previous) (percentage) 1486.67% 0.150 2.380 - 417.39% -
462.libquantum (max) (percentage) 1233.33% 0.150 2.000 - -68.65% -
429.mcf (unexplained diff vs previous) (percentage) 1183.33% 0.180 2.310 - 1343.75% -
473.astar (max) (percentage) 1014.29% 0.070 0.780 - -88.36% -
403.gcc (avg) (percentage) 960.00% 0.300 3.180 - 478.18% -
482.sphinx3 (unexplained diff vs previous) (percentage) 920.00% 0.050 0.510 - -85.13% -
462.libquantum (avg) (percentage) 850.00% 0.160 1.520 - 97.40% -
462.libquantum (unexplained diff vs previous) (percentage) 646.67% 0.150 1.120 - 11100.00% -
483.xalancbmk (avg) (percentage) 433.33% 0.270 1.440 - -32.08% -
470.lbm (unexplained diff vs previous) 400.00% 3.000 15.000 - -98.13% -
429.mcf (max) (percentage) 380.85% 0.470 2.260 - -86.14% -
450.soplex (max) (percentage) 333.93% 0.560 2.430 - 47.27% -
447.dealII (unexplained diff vs previous) (percentage) 333.33% 0.060 0.260 - -55.17% -
473.astar (avg) (percentage) 300.00% 0.120 0.480 - -47.83% -
473.astar (max) 288.89% 9.000 35.000 - -95.74% -
450.soplex (avg) (percentage) 272.22% 0.360 1.340 - 168.00% -
433.milc (unexplained diff vs previous) (percentage) 218.75% 0.160 0.510 - 410.00% -
429.mcf (avg) (percentage) 207.50% 0.400 1.230 - -41.98% -
453.povray (unexplained diff vs previous) (percentage) 203.77% 0.530 1.610 - 0.00% -
401.bzip2 (avg) (percentage) 105.26% 0.190 0.390 - -40.00% -
456.hmmer (avg) (percentage) 100.00% 0.330 0.660 - -52.52% -
458.sjeng (unexplained diff vs previous) (percentage) 80.00% 0.050 0.090 - 0.00% -
473.astar (unexplained diff vs previous) (percentage) 75.00% 0.040 0.070 - -65.00% -
471.omnetpp (avg) (percentage) 74.68% 0.790 1.380 - -13.75% -
471.omnetpp (max) (percentage) 73.40% 2.030 3.520 - -62.07% -
471.omnetpp (unexplained diff vs previous) (percentage) 70.85% 1.990 3.400 - 246.94% -
401.bzip2 (max) (percentage) 58.33% 0.240 0.380 - -89.95% -
458.sjeng (max) (percentage) 45.00% 0.200 0.290 - -94.36% -
458.sjeng (avg) (percentage) 39.13% 0.230 0.320 - -60.00% -
473.astar 31.25% 16.000 21.000 - -81.25% -
473.astar (avg) 31.25% 16.000 21.000 - -81.25% -
453.povray (max) (percentage) 30.16% 1.260 1.640 - 355.56% -
429.mcf (unexplained diff vs previous) 23.81% 21.000 26.000 - 18.18% -
445.gobmk (avg) (percentage) 13.64% 0.220 0.250 - -62.69% -
456.hmmer (unexplained diff vs previous) (percentage) 12.50% 0.160 0.180 - -53.85% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
429.mcf -4.36% 7243.000 6927.000 - -4.31% -
462.libquantum -3.53% 12496.000 12055.000 - -3.59% -
401.bzip2 -2.52% 33998.000 33141.000 - -1.65% -
453.povray -2.48% 525021.000 512012.000 - -2.38% -
433.milc -2.45% 47105.000 45949.000 - -2.24% -
464.h264ref -2.16% 304295.000 297717.000 - -1.99% -
444.namd -1.93% 125071.000 122654.000 - -1.85% -
458.sjeng -1.93% 80589.000 79034.000 - -1.57% -
470.lbm -1.35% 9350.000 9224.000 - -1.22% -
456.hmmer -1.29% 80893.000 79849.000 - -1.25% -
447.dealII -1.08% 195856.000 193739.000 - -1.05% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.39% 9786.000 60.000 - -99.37% -
462.libquantum -98.85% 15502.000 178.000 - -98.76% -
482.sphinx3 -96.64% 29346.000 985.000 - -96.31% -
450.soplex -96.45% 15408.000 547.000 - -96.38% -
444.namd -94.12% 13714.000 807.000 - -93.49% -
453.povray -93.05% 7152.000 497.000 - -92.89% -
433.milc -91.59% 13953.000 1173.000 - -91.38% -
429.mcf -90.37% 11690.000 1126.000 - -91.95% -
401.bzip2 -83.30% 16915.000 2824.000 - -82.50% -
447.dealII -81.83% 10577.000 1922.000 - -79.51% -
471.omnetpp -75.46% 11746.000 2883.000 - -72.48% -
470.lbm -74.01% 15331.000 3985.000 - -69.94% -
456.hmmer -71.71% 13421.000 3797.000 - -67.97% -
473.astar -66.04% 13454.000 4569.000 - -62.77% -
464.h264ref -62.63% 15861.000 5927.000 - -59.12% -
445.gobmk -57.60% 11424.000 4844.000 - -56.33% -
458.sjeng -49.21% 16108.000 8182.000 - -52.03% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
444.namd (max) -99.67% 615.000 2.000 - -99.80% -
482.sphinx3 (max) -99.30% 716.000 5.000 - -99.43% -
444.namd -98.61% 360.000 5.000 - -98.43% -
444.namd (avg) -98.61% 360.000 5.000 - -98.43% -
482.sphinx3 -98.54% 478.000 7.000 - -98.49% -
482.sphinx3 (avg) -98.54% 478.000 7.000 - -98.49% -
470.lbm (max) -98.34% 906.000 15.000 - -98.02% -
433.milc (max) -97.73% 220.000 5.000 - -95.37% -
403.gcc -96.55% 29.000 1.000 - -98.08% -
403.gcc (avg) -96.55% 29.000 1.000 - -98.08% -
470.lbm -96.25% 533.000 20.000 - -95.46% -
470.lbm (avg) -96.25% 533.000 20.000 - -95.46% -
433.milc -94.96% 139.000 7.000 - -93.97% -
433.milc (avg) -94.96% 139.000 7.000 - -93.97% -
453.povray -94.87% 78.000 4.000 - -92.73% -
453.povray (avg) -94.87% 78.000 4.000 - -92.73% -
447.dealII (max) -94.09% 203.000 12.000 - -94.62% -
483.xalancbmk (max) (percentage) -93.90% 0.820 0.050 - -99.61% -
470.lbm (max) (percentage) -93.57% 5.910 0.380 - -93.36% -
444.namd (max) (percentage) -92.43% 4.490 0.340 - -95.80% -
462.libquantum -91.67% 24.000 2.000 - -98.18% -
462.libquantum (avg) -91.67% 24.000 2.000 - -98.18% -
462.libquantum (unexplained diff vs previous) -91.67% 24.000 2.000 - 100.00% -
453.povray (max) -91.11% 90.000 8.000 - -68.00% -
403.gcc (max) -89.58% 48.000 5.000 - -97.74% -
464.h264ref (max) -88.03% 117.000 14.000 - -76.67% -
401.bzip2 (unexplained diff vs previous) -87.50% 40.000 5.000 - -84.85% -
450.soplex -87.27% 55.000 7.000 - -90.67% -
450.soplex (avg) -87.27% 55.000 7.000 - -90.67% -
464.h264ref (unexplained diff vs previous) -87.07% 116.000 15.000 - 0.00% -
462.libquantum (max) -86.96% 23.000 3.000 - -99.67% -
447.dealII -86.79% 106.000 14.000 - -85.71% -
447.dealII (avg) -86.79% 106.000 14.000 - -85.71% -
470.lbm (avg) (percentage) -85.34% 3.480 0.510 - -84.68% -
450.soplex (max) -84.88% 86.000 13.000 - -94.78% -
456.hmmer (max) -84.21% 38.000 6.000 - -99.52% -
445.gobmk (max) -83.78% 37.000 6.000 - -98.55% -
453.povray (unexplained diff vs previous) -78.95% 38.000 8.000 - 0.00% -
482.sphinx3 (max) (percentage) -77.46% 2.440 0.550 - -83.38% -
401.bzip2 (max) -75.00% 40.000 10.000 - -98.36% -
433.milc (unexplained diff vs previous) -73.91% 23.000 6.000 - -53.85% -
464.h264ref -73.33% 45.000 12.000 - -71.43% -
464.h264ref (avg) -73.33% 45.000 12.000 - -71.43% -
444.namd (avg) (percentage) -72.24% 2.630 0.730 - -71.60% -
429.mcf -71.74% 46.000 13.000 - -95.61% -
429.mcf (avg) -71.74% 46.000 13.000 - -95.61% -
445.gobmk (unexplained diff vs previous) -70.59% 17.000 5.000 - -87.50% -
482.sphinx3 (unexplained diff vs previous) -68.75% 16.000 5.000 - -99.45% -
456.hmmer (unexplained diff vs previous) -68.18% 22.000 7.000 - -84.78% -
433.milc (max) (percentage) -67.72% 1.580 0.510 - -36.25% -
464.h264ref (max) (percentage) -66.22% 0.740 0.250 - -40.48% -
447.dealII (max) (percentage) -66.15% 1.920 0.650 - -72.69% -
464.h264ref (unexplained diff vs previous) (percentage) -65.75% 0.730 0.250 - 0.00% -
401.bzip2 -65.62% 32.000 11.000 - -89.42% -
401.bzip2 (avg) -65.62% 32.000 11.000 - -89.42% -
471.omnetpp (unexplained diff vs previous) -58.12% 234.000 98.000 - -4.85% -
471.omnetpp -57.61% 92.000 39.000 - -76.65% -
471.omnetpp (avg) -57.61% 92.000 39.000 - -76.65% -
445.gobmk (max) (percentage) -57.58% 0.330 0.140 - -96.25% -
471.omnetpp (max) -57.56% 238.000 101.000 - -89.61% -
482.sphinx3 (avg) (percentage) -55.21% 1.630 0.730 - -58.05% -
429.mcf (max) -53.70% 54.000 25.000 - -98.90% -
445.gobmk -52.00% 25.000 12.000 - -83.78% -
445.gobmk (avg) -52.00% 25.000 12.000 - -83.78% -
473.astar (unexplained diff vs previous) -50.00% 6.000 3.000 - -88.00% -
450.soplex (unexplained diff vs previous) -43.48% 23.000 13.000 - -81.16% -
456.hmmer -43.18% 44.000 25.000 - -84.76% -
456.hmmer (avg) -43.18% 44.000 25.000 - -84.76% -
433.milc (avg) (percentage) -38.00% 1.000 0.620 - -27.91% -
456.hmmer (max) (percentage) -37.93% 0.290 0.180 - -98.28% -
445.gobmk (unexplained diff vs previous) (percentage) -33.33% 0.150 0.100 - -72.22% -
458.sjeng -29.73% 37.000 26.000 - -80.88% -
458.sjeng (avg) -29.73% 37.000 26.000 - -80.88% -
458.sjeng (max) -28.12% 32.000 23.000 - -97.37% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - -27.59% -
447.dealII (avg) (percentage) -26.73% 1.010 0.740 - -29.52% -
401.bzip2 (unexplained diff vs previous) (percentage) -25.00% 0.240 0.180 - -10.00% -
447.dealII (unexplained diff vs previous) -16.67% 6.000 5.000 - -90.74% -
458.sjeng (unexplained diff vs previous) -12.50% 8.000 7.000 - 0.00% -
453.povray (avg) (percentage) -10.00% 1.100 0.990 - 23.75% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
450.soplex 2.23% 146765.000 150036.000 - 2.15% -
471.omnetpp 2.08% 316676.000 323250.000 - 2.07% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
462.libquantum (unexplained diff vs previous) (percentage) 11100.00% 0.010 1.120 - 646.67% -
429.mcf (unexplained diff vs previous) (percentage) 1343.75% 0.160 2.310 - 1183.33% -
403.gcc (avg) (percentage) 478.18% 0.550 3.180 - 960.00% -
450.soplex (unexplained diff vs previous) (percentage) 417.39% 0.460 2.380 - 1486.67% -
433.milc (unexplained diff vs previous) (percentage) 410.00% 0.100 0.510 - 218.75% -
453.povray (max) (percentage) 355.56% 0.360 1.640 - 30.16% -
403.gcc (max) (percentage) 269.53% 2.330 8.610 - 1622.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 246.94% 0.980 3.400 - 70.85% -
450.soplex (avg) (percentage) 168.00% 0.500 1.340 - 272.22% -
462.libquantum (unexplained diff vs previous) 100.00% 1.000 2.000 - -91.67% -
462.libquantum (avg) (percentage) 97.40% 0.770 1.520 - 850.00% -
450.soplex (max) (percentage) 47.27% 1.650 2.430 - 333.93% -
453.povray (avg) (percentage) 23.75% 0.800 0.990 - -10.00% -
429.mcf (unexplained diff vs previous) 18.18% 22.000 26.000 - 23.81% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
429.mcf -4.31% 7239.000 6927.000 - -4.36% -
462.libquantum -3.59% 12504.000 12055.000 - -3.53% -
453.povray -2.38% 524493.000 512012.000 - -2.48% -
433.milc -2.24% 47001.000 45949.000 - -2.45% -
464.h264ref -1.99% 303767.000 297717.000 - -2.16% -
444.namd -1.85% 124963.000 122654.000 - -1.93% -
401.bzip2 -1.65% 33698.000 33141.000 - -2.52% -
458.sjeng -1.57% 80297.000 79034.000 - -1.93% -
456.hmmer -1.25% 80861.000 79849.000 - -1.29% -
470.lbm -1.22% 9338.000 9224.000 - -1.35% -
447.dealII -1.05% 195792.000 193739.000 - -1.08% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.37% 9509.000 60.000 - -99.39% -
462.libquantum -98.76% 14409.000 178.000 - -98.85% -
450.soplex -96.38% 15112.000 547.000 - -96.45% -
482.sphinx3 -96.31% 26682.000 985.000 - -96.64% -
444.namd -93.49% 12400.000 807.000 - -94.12% -
453.povray -92.89% 6987.000 497.000 - -93.05% -
429.mcf -91.95% 13988.000 1126.000 - -90.37% -
433.milc -91.38% 13601.000 1173.000 - -91.59% -
401.bzip2 -82.50% 16137.000 2824.000 - -83.30% -
447.dealII -79.51% 9380.000 1922.000 - -81.83% -
471.omnetpp -72.48% 10477.000 2883.000 - -75.46% -
470.lbm -69.94% 13259.000 3985.000 - -74.01% -
456.hmmer -67.97% 11854.000 3797.000 - -71.71% -
473.astar -62.77% 12272.000 4569.000 - -66.04% -
464.h264ref -59.12% 14498.000 5927.000 - -62.63% -
445.gobmk -56.33% 11093.000 4844.000 - -57.60% -
458.sjeng -52.03% 17055.000 8182.000 - -49.21% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
444.namd (max) -99.80% 1003.000 2.000 - -99.67% -
462.libquantum (max) -99.67% 919.000 3.000 - -86.96% -
483.xalancbmk (max) (percentage) -99.61% 12.900 0.050 - -93.90% -
456.hmmer (max) -99.52% 1242.000 6.000 - -84.21% -
482.sphinx3 (unexplained diff vs previous) -99.45% 914.000 5.000 - -68.75% -
482.sphinx3 (max) -99.43% 883.000 5.000 - -99.30% -
429.mcf (max) -98.90% 2281.000 25.000 - -53.70% -
445.gobmk (max) -98.55% 413.000 6.000 - -83.78% -
482.sphinx3 -98.49% 464.000 7.000 - -98.54% -
482.sphinx3 (avg) -98.49% 464.000 7.000 - -98.54% -
444.namd -98.43% 318.000 5.000 - -98.61% -
444.namd (avg) -98.43% 318.000 5.000 - -98.61% -
401.bzip2 (max) -98.36% 609.000 10.000 - -75.00% -
456.hmmer (max) (percentage) -98.28% 10.480 0.180 - -37.93% -
462.libquantum -98.18% 110.000 2.000 - -91.67% -
462.libquantum (avg) -98.18% 110.000 2.000 - -91.67% -
470.lbm (unexplained diff vs previous) -98.13% 804.000 15.000 - 400.00% -
403.gcc -98.08% 52.000 1.000 - -96.55% -
403.gcc (avg) -98.08% 52.000 1.000 - -96.55% -
470.lbm (max) -98.02% 758.000 15.000 - -98.34% -
403.gcc (max) -97.74% 221.000 5.000 - -89.58% -
458.sjeng (max) -97.37% 876.000 23.000 - -28.12% -
445.gobmk (max) (percentage) -96.25% 3.730 0.140 - -57.58% -
444.namd (max) (percentage) -95.80% 8.090 0.340 - -92.43% -
473.astar (max) -95.74% 822.000 35.000 - 288.89% -
429.mcf -95.61% 296.000 13.000 - -71.74% -
429.mcf (avg) -95.61% 296.000 13.000 - -71.74% -
470.lbm -95.46% 441.000 20.000 - -96.25% -
470.lbm (avg) -95.46% 441.000 20.000 - -96.25% -
433.milc (max) -95.37% 108.000 5.000 - -97.73% -
450.soplex (max) -94.78% 249.000 13.000 - -84.88% -
447.dealII (max) -94.62% 223.000 12.000 - -94.09% -
458.sjeng (max) (percentage) -94.36% 5.140 0.290 - 45.00% -
433.milc -93.97% 116.000 7.000 - -94.96% -
433.milc (avg) -93.97% 116.000 7.000 - -94.96% -
470.lbm (unexplained diff vs previous) (percentage) -93.73% 6.060 0.380 - 1800.00% -
470.lbm (max) (percentage) -93.36% 5.720 0.380 - -93.57% -
453.povray -92.73% 55.000 4.000 - -94.87% -
453.povray (avg) -92.73% 55.000 4.000 - -94.87% -
447.dealII (unexplained diff vs previous) -90.74% 54.000 5.000 - -16.67% -
450.soplex -90.67% 75.000 7.000 - -87.27% -
450.soplex (avg) -90.67% 75.000 7.000 - -87.27% -
401.bzip2 (max) (percentage) -89.95% 3.780 0.380 - 58.33% -
471.omnetpp (max) -89.61% 972.000 101.000 - -57.56% -
401.bzip2 -89.42% 104.000 11.000 - -65.62% -
401.bzip2 (avg) -89.42% 104.000 11.000 - -65.62% -
473.astar (max) (percentage) -88.36% 6.700 0.780 - 1014.29% -
473.astar (unexplained diff vs previous) -88.00% 25.000 3.000 - -50.00% -
445.gobmk (unexplained diff vs previous) -87.50% 40.000 5.000 - -70.59% -
429.mcf (max) (percentage) -86.14% 16.310 2.260 - 380.85% -
447.dealII -85.71% 98.000 14.000 - -86.79% -
447.dealII (avg) -85.71% 98.000 14.000 - -86.79% -
482.sphinx3 (unexplained diff vs previous) (percentage) -85.13% 3.430 0.510 - 920.00% -
401.bzip2 (unexplained diff vs previous) -84.85% 33.000 5.000 - -87.50% -
456.hmmer (unexplained diff vs previous) -84.78% 46.000 7.000 - -68.18% -
456.hmmer -84.76% 164.000 25.000 - -43.18% -
456.hmmer (avg) -84.76% 164.000 25.000 - -43.18% -
470.lbm (avg) (percentage) -84.68% 3.330 0.510 - -85.34% -
445.gobmk -83.78% 74.000 12.000 - -52.00% -
445.gobmk (avg) -83.78% 74.000 12.000 - -52.00% -
482.sphinx3 (max) (percentage) -83.38% 3.310 0.550 - -77.46% -
473.astar -81.25% 112.000 21.000 - 31.25% -
473.astar (avg) -81.25% 112.000 21.000 - 31.25% -
450.soplex (unexplained diff vs previous) -81.16% 69.000 13.000 - -43.48% -
458.sjeng -80.88% 136.000 26.000 - -29.73% -
458.sjeng (avg) -80.88% 136.000 26.000 - -29.73% -
464.h264ref (max) -76.67% 60.000 14.000 - -88.03% -
471.omnetpp -76.65% 167.000 39.000 - -57.61% -
471.omnetpp (avg) -76.65% 167.000 39.000 - -57.61% -
447.dealII (max) (percentage) -72.69% 2.380 0.650 - -66.15% -
445.gobmk (unexplained diff vs previous) (percentage) -72.22% 0.360 0.100 - -33.33% -
444.namd (avg) (percentage) -71.60% 2.570 0.730 - -72.24% -
464.h264ref -71.43% 42.000 12.000 - -73.33% -
464.h264ref (avg) -71.43% 42.000 12.000 - -73.33% -
462.libquantum (max) (percentage) -68.65% 6.380 2.000 - 1233.33% -
453.povray (max) -68.00% 25.000 8.000 - -91.11% -
473.astar (unexplained diff vs previous) (percentage) -65.00% 0.200 0.070 - 75.00% -
445.gobmk (avg) (percentage) -62.69% 0.670 0.250 - 13.64% -
471.omnetpp (max) (percentage) -62.07% 9.280 3.520 - 73.40% -
458.sjeng (avg) (percentage) -60.00% 0.800 0.320 - 39.13% -
482.sphinx3 (avg) (percentage) -58.05% 1.740 0.730 - -55.21% -
447.dealII (unexplained diff vs previous) (percentage) -55.17% 0.580 0.260 - 333.33% -
456.hmmer (unexplained diff vs previous) (percentage) -53.85% 0.390 0.180 - 12.50% -
433.milc (unexplained diff vs previous) -53.85% 13.000 6.000 - -73.91% -
456.hmmer (avg) (percentage) -52.52% 1.390 0.660 - 100.00% -
473.astar (avg) (percentage) -47.83% 0.920 0.480 - 300.00% -
429.mcf (avg) (percentage) -41.98% 2.120 1.230 - 207.50% -
464.h264ref (max) (percentage) -40.48% 0.420 0.250 - -66.22% -
401.bzip2 (avg) (percentage) -40.00% 0.650 0.390 - 105.26% -
433.milc (max) (percentage) -36.25% 0.800 0.510 - -67.72% -
483.xalancbmk (avg) (percentage) -32.08% 2.120 1.440 - 433.33% -
447.dealII (avg) (percentage) -29.52% 1.050 0.740 - -26.73% -
433.milc (avg) (percentage) -27.91% 0.860 0.620 - -38.00% -
464.h264ref (avg) (percentage) -27.59% 0.290 0.210 - -27.59% -
471.omnetpp (avg) (percentage) -13.75% 1.600 1.380 - 74.68% -
401.bzip2 (unexplained diff vs previous) (percentage) -10.00% 0.200 0.180 - -25.00% -
471.omnetpp (unexplained diff vs previous) -4.85% 103.000 98.000 - -58.12% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)

Report Time: 0.41s
Execution (seconds)
Name Current %
401.bzip2 2824.000 -83.30%
403.gcc 60.000 -99.39%
429.mcf 1126.000 -90.37%
433.milc 1173.000 -91.59%
444.namd 807.000 -94.12%
445.gobmk 4844.000 -57.60%
447.dealII 1922.000 -81.83%
450.soplex 547.000 -96.45%
453.povray 497.000 -93.05%
456.hmmer 3797.000 -71.71%
458.sjeng 8182.000 -49.21%
462.libquantum 178.000 -98.85%
464.h264ref 5927.000 -62.63%
470.lbm 3985.000 -74.01%
471.omnetpp 2883.000 -75.46%
473.astar 4569.000 -66.04%
482.sphinx3 985.000 -96.64%
Geometric Mean 1478.931 -89.13%
Code Size (bytes)
Name Current %
401.bzip2 33141.000 -2.52%
403.gcc 1653391.000 -
429.mcf 6927.000 -4.36%
433.milc 45949.000 -2.45%
444.namd 122654.000 -1.93%
445.gobmk 1255485.000 -
447.dealII 193739.000 -1.08%
450.soplex 150036.000 2.15%
453.povray 512012.000 -2.48%
456.hmmer 79849.000 -1.29%
458.sjeng 79034.000 -1.93%
462.libquantum 12055.000 -3.53%
464.h264ref 297717.000 -2.16%
470.lbm 9224.000 -1.35%
471.omnetpp 323250.000 2.07%
473.astar 15607.000 -
482.sphinx3 87440.000 -
Geometric Mean 94294.780 -1.33%
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 11.000 -65.62%
401.bzip2 (avg) 11.000 -65.62%
401.bzip2 (avg) (percentage) 0.390 105.26%
401.bzip2 (max) 10.000 -75.00%
401.bzip2 (max) (percentage) 0.380 58.33%
401.bzip2 (unexplained diff vs previous) 5.000 -87.50%
401.bzip2 (unexplained diff vs previous) (percentage) 0.180 -25.00%
403.gcc 1.000 -96.55%
403.gcc (avg) 1.000 -96.55%
403.gcc (avg) (percentage) 3.180 960.00%
403.gcc (max) 5.000 -89.58%
403.gcc (max) (percentage) 8.610 1622.00%
429.mcf 13.000 -71.74%
429.mcf (avg) 13.000 -71.74%
429.mcf (avg) (percentage) 1.230 207.50%
429.mcf (max) 25.000 -53.70%
429.mcf (max) (percentage) 2.260 380.85%
429.mcf (unexplained diff vs previous) 26.000 23.81%
429.mcf (unexplained diff vs previous) (percentage) 2.310 1183.33%
433.milc 7.000 -94.96%
433.milc (avg) 7.000 -94.96%
433.milc (avg) (percentage) 0.620 -38.00%
433.milc (max) 5.000 -97.73%
433.milc (max) (percentage) 0.510 -67.72%
433.milc (unexplained diff vs previous) 6.000 -73.91%
433.milc (unexplained diff vs previous) (percentage) 0.510 218.75%
444.namd 5.000 -98.61%
444.namd (avg) 5.000 -98.61%
444.namd (avg) (percentage) 0.730 -72.24%
444.namd (max) 2.000 -99.67%
444.namd (max) (percentage) 0.340 -92.43%
444.namd (unexplained diff vs previous) 0.000 -
444.namd (unexplained diff vs previous) (percentage) 0.000 -
445.gobmk 12.000 -52.00%
445.gobmk (avg) 12.000 -52.00%
445.gobmk (avg) (percentage) 0.250 13.64%
445.gobmk (max) 6.000 -83.78%
445.gobmk (max) (percentage) 0.140 -57.58%
445.gobmk (unexplained diff vs previous) 5.000 -70.59%
445.gobmk (unexplained diff vs previous) (percentage) 0.100 -33.33%
447.dealII 14.000 -86.79%
447.dealII (avg) 14.000 -86.79%
447.dealII (avg) (percentage) 0.740 -26.73%
447.dealII (max) 12.000 -94.09%
447.dealII (max) (percentage) 0.650 -66.15%
447.dealII (unexplained diff vs previous) 5.000 -16.67%
447.dealII (unexplained diff vs previous) (percentage) 0.260 333.33%
450.soplex 7.000 -87.27%
450.soplex (avg) 7.000 -87.27%
450.soplex (avg) (percentage) 1.340 272.22%
450.soplex (max) 13.000 -84.88%
450.soplex (max) (percentage) 2.430 333.93%
450.soplex (unexplained diff vs previous) 13.000 -43.48%
450.soplex (unexplained diff vs previous) (percentage) 2.380 1486.67%
453.povray 4.000 -94.87%
453.povray (avg) 4.000 -94.87%
453.povray (avg) (percentage) 0.990 -10.00%
453.povray (max) 8.000 -91.11%
453.povray (max) (percentage) 1.640 30.16%
453.povray (unexplained diff vs previous) 8.000 -78.95%
453.povray (unexplained diff vs previous) (percentage) 1.610 203.77%
456.hmmer 25.000 -43.18%
456.hmmer (avg) 25.000 -43.18%
456.hmmer (avg) (percentage) 0.660 100.00%
456.hmmer (max) 6.000 -84.21%
456.hmmer (max) (percentage) 0.180 -37.93%
456.hmmer (unexplained diff vs previous) 7.000 -68.18%
456.hmmer (unexplained diff vs previous) (percentage) 0.180 12.50%
458.sjeng 26.000 -29.73%
458.sjeng (avg) 26.000 -29.73%
458.sjeng (avg) (percentage) 0.320 39.13%
458.sjeng (max) 23.000 -28.12%
458.sjeng (max) (percentage) 0.290 45.00%
458.sjeng (unexplained diff vs previous) 7.000 -12.50%
458.sjeng (unexplained diff vs previous) (percentage) 0.090 80.00%
462.libquantum 2.000 -91.67%
462.libquantum (avg) 2.000 -91.67%
462.libquantum (avg) (percentage) 1.520 850.00%
462.libquantum (max) 3.000 -86.96%
462.libquantum (max) (percentage) 2.000 1233.33%
462.libquantum (unexplained diff vs previous) 2.000 -91.67%
462.libquantum (unexplained diff vs previous) (percentage) 1.120 646.67%
464.h264ref 12.000 -73.33%
464.h264ref (avg) 12.000 -73.33%
464.h264ref (avg) (percentage) 0.210 -27.59%
464.h264ref (max) 14.000 -88.03%
464.h264ref (max) (percentage) 0.250 -66.22%
464.h264ref (unexplained diff vs previous) 15.000 -87.07%
464.h264ref (unexplained diff vs previous) (percentage) 0.250 -65.75%
470.lbm 20.000 -96.25%
470.lbm (avg) 20.000 -96.25%
470.lbm (avg) (percentage) 0.510 -85.34%
470.lbm (max) 15.000 -98.34%
470.lbm (max) (percentage) 0.380 -93.57%
470.lbm (unexplained diff vs previous) 15.000 400.00%
470.lbm (unexplained diff vs previous) (percentage) 0.380 1800.00%
471.omnetpp 39.000 -57.61%
471.omnetpp (avg) 39.000 -57.61%
471.omnetpp (avg) (percentage) 1.380 74.68%
471.omnetpp (max) 101.000 -57.56%
471.omnetpp (max) (percentage) 3.520 73.40%
471.omnetpp (unexplained diff vs previous) 98.000 -58.12%
471.omnetpp (unexplained diff vs previous) (percentage) 3.400 70.85%
473.astar 21.000 31.25%
473.astar (avg) 21.000 31.25%
473.astar (avg) (percentage) 0.480 300.00%
473.astar (max) 35.000 288.89%
473.astar (max) (percentage) 0.780 1014.29%
473.astar (unexplained diff vs previous) 3.000 -50.00%
473.astar (unexplained diff vs previous) (percentage) 0.070 75.00%
482.sphinx3 7.000 -98.54%
482.sphinx3 (avg) 7.000 -98.54%
482.sphinx3 (avg) (percentage) 0.730 -55.21%
482.sphinx3 (max) 5.000 -99.30%
482.sphinx3 (max) (percentage) 0.550 -77.46%
482.sphinx3 (unexplained diff vs previous) 5.000 -68.75%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.510 920.00%
483.xalancbmk (avg) (percentage) 1.440 433.33%
483.xalancbmk (max) (percentage) 0.050 -93.90%
Geometric Mean 2.390 -63.15%