Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/20bda93e438c63fb68a8130b7f88090c558e99b7 (+ 221 commits)
test_url https://ci.linaro.org/job/tcwg_bmk-code_size-spec2k6--llvm-arm-master-Os-build/222/
View Options
Report
Filter

llvm-arm-master-Os test results

Run Order Start Time Duration
Current 0222 (llvmorg-20-init-09615-g20bda93e438c) 2024-10-16T13:27:31 0:00:00
Previous 0222 (llvmorg-16-init-15661-g1deb6bce8931) 2022-12-14T06:10:43 0:00:00
Baseline 0001 (llvmorg-16-init-16061-ge7790fbed32b) 2022-12-29T19:41:07 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 31 31
Performance Improvements 109 111
Added Tests 2 2
Unchanged Tests 638 636
Total Tests 780

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
401.bzip2 9.84% 39458.000 43341.000 - 9.84% -
450.soplex 1.89% 198692.000 202447.000 - 1.88% -
471.omnetpp 1.10% 373579.000 377700.000 - 1.10% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
403.gcc (max) (percentage) 1409.80% 0.510 7.700 - 1926.32% -
462.libquantum (avg) (percentage) 980.00% 0.150 1.620 - 980.00% -
462.libquantum (max) (percentage) 848.00% 0.250 2.370 - 1085.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 816.67% 0.060 0.550 - 77.42% -
462.libquantum (unexplained diff vs previous) (percentage) 732.00% 0.250 2.080 - 1633.33% -
403.gcc (avg) (percentage) 702.04% 0.490 3.930 - 718.75% -
464.h264ref (unexplained diff vs previous) (percentage) 416.67% 0.060 0.310 - -20.51% -
401.bzip2 (max) (percentage) 366.67% 0.150 0.700 - 125.81% -
450.soplex (avg) (percentage) 231.25% 0.320 1.060 - 194.44% -
429.mcf (avg) (percentage) 202.27% 0.440 1.330 - 195.56% -
483.xalancbmk (avg) (percentage) 177.08% 0.480 1.330 - 166.00% -
456.hmmer (unexplained diff vs previous) (percentage) 175.00% 0.080 0.220 - 214.29% -
450.soplex (unexplained diff vs previous) (percentage) 156.67% 0.300 0.770 - 35.09% -
450.soplex (max) (percentage) 153.33% 0.300 0.760 - 33.33% -
429.mcf (max) (percentage) 152.00% 0.500 1.260 - 125.00% -
401.bzip2 (avg) (percentage) 128.57% 0.210 0.480 - 118.18% -
464.h264ref (unexplained diff vs previous) 88.89% 9.000 17.000 - -71.67% -
471.omnetpp (avg) (percentage) 84.72% 0.720 1.330 - 70.51% -
483.xalancbmk (max) (percentage) 76.32% 0.380 0.670 - 3.08% -
445.gobmk (avg) (percentage) 70.59% 0.170 0.290 - 61.11% -
401.bzip2 (unexplained diff vs previous) 66.67% 9.000 15.000 - -69.39% -
473.astar (avg) (percentage) 58.14% 0.430 0.680 - 36.00% -
473.astar (max) (percentage) 16.44% 0.730 0.850 - 6.25% -
456.hmmer (avg) (percentage) 15.00% 0.400 0.460 - 27.78% -
458.sjeng (max) (percentage) 6.00% 0.500 0.530 - 32.50% -
458.sjeng (unexplained diff vs previous) (percentage) 6.00% 0.500 0.530 - 112.00% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ Previous Current σ Δ (B) σ (B)
462.libquantum -9.90% 26716.000 24071.000 - -10.04% -
433.milc -2.74% 74971.000 72918.000 - -2.74% -
453.povray -2.33% 593161.000 579343.000 - -2.32% -
447.dealII -2.25% 1778560.000 1738527.000 - -2.22% -
444.namd -2.06% 175184.000 171581.000 - -2.04% -
464.h264ref -1.33% 354530.000 349828.000 - -1.35% -

Performance Improvements - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc -99.35% 9082.000 59.000 - -99.35% -
462.libquantum -99.27% 13149.000 96.000 - -99.27% -
482.sphinx3 -96.68% 26347.000 874.000 - -96.68% -
450.soplex -96.40% 14426.000 520.000 - -96.40% -
444.namd -94.76% 13488.000 707.000 - -94.76% -
453.povray -93.32% 6915.000 462.000 - -93.26% -
433.milc -91.52% 13471.000 1142.000 - -91.53% -
429.mcf -90.36% 11847.000 1142.000 - -90.48% -
401.bzip2 -82.73% 15665.000 2706.000 - -82.75% -
447.dealII -78.14% 8193.000 1791.000 - -78.15% -
470.lbm -76.63% 13546.000 3166.000 - -77.19% -
471.omnetpp -73.94% 10687.000 2785.000 - -74.08% -
456.hmmer -73.38% 13413.000 3570.000 - -73.29% -
473.astar -65.90% 14243.000 4857.000 - -65.94% -
464.h264ref -64.10% 15486.000 5559.000 - -64.12% -
445.gobmk -57.64% 12382.000 5245.000 - -57.70% -
458.sjeng -49.18% 16357.000 8312.000 - -49.34% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
433.milc (unexplained diff vs previous) -99.24% 132.000 1.000 - -99.45% -
482.sphinx3 (max) -99.19% 861.000 7.000 - -99.20% -
470.lbm (unexplained diff vs previous) -99.06% 745.000 7.000 - -98.59% -
444.namd (max) -98.97% 871.000 9.000 - -98.98% -
444.namd -98.73% 473.000 6.000 - -98.83% -
444.namd (avg) -98.73% 473.000 6.000 - -98.83% -
482.sphinx3 -98.45% 453.000 7.000 - -98.57% -
482.sphinx3 (avg) -98.45% 453.000 7.000 - -98.57% -
429.mcf (unexplained diff vs previous) -97.87% 47.000 1.000 - -98.51% -
433.milc (max) -97.74% 133.000 3.000 - -98.37% -
470.lbm (max) -96.88% 705.000 22.000 - -97.14% -
453.povray (max) -96.69% 121.000 4.000 - -96.97% -
453.povray (unexplained diff vs previous) -96.64% 119.000 4.000 - -96.92% -
470.lbm (unexplained diff vs previous) (percentage) -96.00% 5.500 0.220 - -93.85% -
453.povray -95.56% 90.000 4.000 - -95.70% -
453.povray (avg) -95.56% 90.000 4.000 - -95.70% -
403.gcc -95.45% 44.000 2.000 - -95.35% -
403.gcc (avg) -95.45% 44.000 2.000 - -95.35% -
433.milc -95.28% 127.000 6.000 - -95.59% -
433.milc (avg) -95.28% 127.000 6.000 - -95.59% -
462.libquantum -94.74% 19.000 1.000 - -94.74% -
462.libquantum (avg) -94.74% 19.000 1.000 - -94.74% -
462.libquantum (unexplained diff vs previous) -93.94% 33.000 2.000 - -87.50% -
462.libquantum (max) -93.75% 32.000 2.000 - -92.31% -
450.soplex (max) -93.02% 43.000 3.000 - -96.34% -
470.lbm -91.47% 258.000 22.000 - -93.17% -
470.lbm (avg) -91.47% 258.000 22.000 - -93.17% -
403.gcc (max) -91.30% 46.000 4.000 - -88.24% -
433.milc (unexplained diff vs previous) (percentage) -90.82% 0.980 0.090 - -93.33% -
450.soplex (unexplained diff vs previous) -90.70% 43.000 4.000 - -95.12% -
447.dealII -89.92% 119.000 12.000 - -90.00% -
447.dealII (avg) -89.92% 119.000 12.000 - -90.00% -
473.astar (unexplained diff vs previous) -89.52% 105.000 11.000 - -89.81% -
450.soplex -89.13% 46.000 5.000 - -90.20% -
450.soplex (avg) -89.13% 46.000 5.000 - -90.20% -
456.hmmer (max) -86.89% 61.000 8.000 - -80.00% -
470.lbm (max) (percentage) -86.37% 5.210 0.710 - -87.18% -
447.dealII (max) -82.56% 172.000 30.000 - -76.74% -
471.omnetpp (max) -80.82% 219.000 42.000 - -77.78% -
471.omnetpp (unexplained diff vs previous) -80.80% 224.000 43.000 - -53.26% -
444.namd (max) (percentage) -79.41% 6.460 1.330 - -79.66% -
429.mcf (unexplained diff vs previous) (percentage) -77.50% 0.400 0.090 - -83.93% -
429.mcf (max) -76.27% 59.000 14.000 - -79.10% -
445.gobmk (unexplained diff vs previous) -76.00% 25.000 6.000 - -80.00% -
444.namd (avg) (percentage) -72.93% 3.510 0.950 - -75.07% -
482.sphinx3 (max) (percentage) -72.78% 3.270 0.890 - -73.19% -
429.mcf -71.15% 52.000 15.000 - -71.70% -
429.mcf (avg) -71.15% 52.000 15.000 - -71.70% -
464.h264ref (max) -70.69% 58.000 17.000 - -71.67% -
456.hmmer -69.81% 53.000 16.000 - -66.67% -
456.hmmer (avg) -69.81% 53.000 16.000 - -66.67% -
458.sjeng -69.70% 99.000 30.000 - -68.42% -
458.sjeng (avg) -69.70% 99.000 30.000 - -68.42% -
473.astar (unexplained diff vs previous) (percentage) -68.92% 0.740 0.230 - -69.74% -
433.milc (max) (percentage) -68.69% 0.990 0.310 - -77.37% -
464.h264ref -65.31% 49.000 17.000 - -66.67% -
464.h264ref (avg) -65.31% 49.000 17.000 - -66.67% -
470.lbm (avg) (percentage) -62.83% 1.910 0.710 - -69.40% -
401.bzip2 -62.50% 32.000 12.000 - -64.71% -
401.bzip2 (avg) -62.50% 32.000 12.000 - -64.71% -
473.astar (max) -60.19% 103.000 41.000 - -64.04% -
445.gobmk (max) -58.33% 24.000 10.000 - -65.52% -
447.dealII (avg) (percentage) -53.42% 1.460 0.680 - -53.74% -
482.sphinx3 (avg) (percentage) -51.74% 1.720 0.830 - -55.38% -
471.omnetpp -51.32% 76.000 37.000 - -55.42% -
471.omnetpp (avg) -51.32% 76.000 37.000 - -55.42% -
453.povray (max) (percentage) -50.29% 1.750 0.870 - -54.92% -
456.hmmer (max) (percentage) -50.00% 0.460 0.230 - -23.33% -
453.povray (unexplained diff vs previous) (percentage) -49.42% 1.720 0.870 - -54.21% -
458.sjeng (unexplained diff vs previous) -46.34% 82.000 44.000 - 7.32% -
473.astar -45.90% 61.000 33.000 - -53.52% -
473.astar (avg) -45.90% 61.000 33.000 - -53.52% -
458.sjeng (max) -45.68% 81.000 44.000 - -32.31% -
445.gobmk (unexplained diff vs previous) (percentage) -45.00% 0.200 0.110 - -54.17% -
433.milc (avg) (percentage) -44.21% 0.950 0.530 - -47.52% -
458.sjeng (avg) (percentage) -39.34% 0.610 0.370 - -36.21% -
445.gobmk -28.57% 21.000 15.000 - -31.82% -
445.gobmk (avg) -28.57% 21.000 15.000 - -31.82% -
456.hmmer (unexplained diff vs previous) -27.27% 11.000 8.000 - -11.11% -
471.omnetpp (unexplained diff vs previous) (percentage) -26.67% 2.100 1.540 - 79.07% -
471.omnetpp (max) (percentage) -25.85% 2.050 1.520 - -13.64% -
453.povray (avg) (percentage) -22.14% 1.310 1.020 - -25.55% -
401.bzip2 (max) -21.74% 23.000 18.000 - -62.50% -
447.dealII (max) (percentage) -19.43% 2.110 1.700 - 7.59% -
464.h264ref (max) (percentage) -18.42% 0.380 0.310 - -20.51% -

Performance Improvements - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 9.84% 39458.000 43341.000 - 9.84% -
450.soplex 1.88% 198708.000 202447.000 - 1.89% -
471.omnetpp 1.10% 373579.000 377700.000 - 1.10% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
403.gcc (max) (percentage) 1926.32% 0.380 7.700 - 1409.80% -
462.libquantum (unexplained diff vs previous) (percentage) 1633.33% 0.120 2.080 - 732.00% -
462.libquantum (max) (percentage) 1085.00% 0.200 2.370 - 848.00% -
462.libquantum (avg) (percentage) 980.00% 0.150 1.620 - 980.00% -
403.gcc (avg) (percentage) 718.75% 0.480 3.930 - 702.04% -
456.hmmer (unexplained diff vs previous) (percentage) 214.29% 0.070 0.220 - 175.00% -
429.mcf (avg) (percentage) 195.56% 0.450 1.330 - 202.27% -
450.soplex (avg) (percentage) 194.44% 0.360 1.060 - 231.25% -
483.xalancbmk (avg) (percentage) 166.00% 0.500 1.330 - 177.08% -
401.bzip2 (max) (percentage) 125.81% 0.310 0.700 - 366.67% -
429.mcf (max) (percentage) 125.00% 0.560 1.260 - 152.00% -
401.bzip2 (avg) (percentage) 118.18% 0.220 0.480 - 128.57% -
458.sjeng (unexplained diff vs previous) (percentage) 112.00% 0.250 0.530 - 6.00% -
471.omnetpp (unexplained diff vs previous) (percentage) 79.07% 0.860 1.540 - -26.67% -
401.bzip2 (unexplained diff vs previous) (percentage) 77.42% 0.310 0.550 - 816.67% -
471.omnetpp (avg) (percentage) 70.51% 0.780 1.330 - 84.72% -
445.gobmk (avg) (percentage) 61.11% 0.180 0.290 - 70.59% -
473.astar (avg) (percentage) 36.00% 0.500 0.680 - 58.14% -
450.soplex (unexplained diff vs previous) (percentage) 35.09% 0.570 0.770 - 156.67% -
450.soplex (max) (percentage) 33.33% 0.570 0.760 - 153.33% -
458.sjeng (max) (percentage) 32.50% 0.400 0.530 - 6.00% -
456.hmmer (avg) (percentage) 27.78% 0.360 0.460 - 15.00% -
447.dealII (max) (percentage) 7.59% 1.580 1.700 - -19.43% -
458.sjeng (unexplained diff vs previous) 7.32% 41.000 44.000 - -46.34% -
473.astar (max) (percentage) 6.25% 0.800 0.850 - 16.44% -
483.xalancbmk (max) (percentage) 3.08% 0.650 0.670 - 76.32% -

Performance Regressions - Score Δ (B) Baseline Current σ (B) Δ σ
nb_execution_successful -5.56% 18.000 17.000 - -5.56% -
nb_compile_successful -5.26% 19.000 18.000 - -5.26% -

Performance Improvements - Code Size Δ (B) Baseline Current σ (B) Δ σ
462.libquantum -10.04% 26756.000 24071.000 - -9.90% -
433.milc -2.74% 74971.000 72918.000 - -2.74% -
453.povray -2.32% 593097.000 579343.000 - -2.33% -
447.dealII -2.22% 1777992.000 1738527.000 - -2.25% -
444.namd -2.04% 175160.000 171581.000 - -2.06% -
464.h264ref -1.35% 354618.000 349828.000 - -1.33% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
403.gcc -99.35% 9099.000 59.000 - -99.35% -
462.libquantum -99.27% 13126.000 96.000 - -99.27% -
482.sphinx3 -96.68% 26310.000 874.000 - -96.68% -
450.soplex -96.40% 14429.000 520.000 - -96.40% -
444.namd -94.76% 13495.000 707.000 - -94.76% -
453.povray -93.26% 6857.000 462.000 - -93.32% -
433.milc -91.53% 13482.000 1142.000 - -91.52% -
429.mcf -90.48% 11999.000 1142.000 - -90.36% -
401.bzip2 -82.75% 15683.000 2706.000 - -82.73% -
447.dealII -78.15% 8196.000 1791.000 - -78.14% -
470.lbm -77.19% 13882.000 3166.000 - -76.63% -
471.omnetpp -74.08% 10746.000 2785.000 - -73.94% -
456.hmmer -73.29% 13365.000 3570.000 - -73.38% -
473.astar -65.94% 14262.000 4857.000 - -65.90% -
464.h264ref -64.12% 15493.000 5559.000 - -64.10% -
445.gobmk -57.70% 12400.000 5245.000 - -57.64% -
458.sjeng -49.34% 16406.000 8312.000 - -49.18% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
433.milc (unexplained diff vs previous) -99.45% 182.000 1.000 - -99.24% -
482.sphinx3 (max) -99.20% 873.000 7.000 - -99.19% -
444.namd (max) -98.98% 882.000 9.000 - -98.97% -
444.namd -98.83% 514.000 6.000 - -98.73% -
444.namd (avg) -98.83% 514.000 6.000 - -98.73% -
470.lbm (unexplained diff vs previous) -98.59% 497.000 7.000 - -99.06% -
482.sphinx3 -98.57% 489.000 7.000 - -98.45% -
482.sphinx3 (avg) -98.57% 489.000 7.000 - -98.45% -
429.mcf (unexplained diff vs previous) -98.51% 67.000 1.000 - -97.87% -
433.milc (max) -98.37% 184.000 3.000 - -97.74% -
470.lbm (max) -97.14% 769.000 22.000 - -96.88% -
453.povray (max) -96.97% 132.000 4.000 - -96.69% -
453.povray (unexplained diff vs previous) -96.92% 130.000 4.000 - -96.64% -
450.soplex (max) -96.34% 82.000 3.000 - -93.02% -
453.povray -95.70% 93.000 4.000 - -95.56% -
453.povray (avg) -95.70% 93.000 4.000 - -95.56% -
433.milc -95.59% 136.000 6.000 - -95.28% -
433.milc (avg) -95.59% 136.000 6.000 - -95.28% -
403.gcc -95.35% 43.000 2.000 - -95.45% -
403.gcc (avg) -95.35% 43.000 2.000 - -95.45% -
450.soplex (unexplained diff vs previous) -95.12% 82.000 4.000 - -90.70% -
462.libquantum -94.74% 19.000 1.000 - -94.74% -
462.libquantum (avg) -94.74% 19.000 1.000 - -94.74% -
470.lbm (unexplained diff vs previous) (percentage) -93.85% 3.580 0.220 - -96.00% -
433.milc (unexplained diff vs previous) (percentage) -93.33% 1.350 0.090 - -90.82% -
470.lbm -93.17% 322.000 22.000 - -91.47% -
470.lbm (avg) -93.17% 322.000 22.000 - -91.47% -
462.libquantum (max) -92.31% 26.000 2.000 - -93.75% -
450.soplex -90.20% 51.000 5.000 - -89.13% -
450.soplex (avg) -90.20% 51.000 5.000 - -89.13% -
447.dealII -90.00% 120.000 12.000 - -89.92% -
447.dealII (avg) -90.00% 120.000 12.000 - -89.92% -
473.astar (unexplained diff vs previous) -89.81% 108.000 11.000 - -89.52% -
403.gcc (max) -88.24% 34.000 4.000 - -91.30% -
462.libquantum (unexplained diff vs previous) -87.50% 16.000 2.000 - -93.94% -
470.lbm (max) (percentage) -87.18% 5.540 0.710 - -86.37% -
429.mcf (unexplained diff vs previous) (percentage) -83.93% 0.560 0.090 - -77.50% -
445.gobmk (unexplained diff vs previous) -80.00% 30.000 6.000 - -76.00% -
456.hmmer (max) -80.00% 40.000 8.000 - -86.89% -
444.namd (max) (percentage) -79.66% 6.540 1.330 - -79.41% -
429.mcf (max) -79.10% 67.000 14.000 - -76.27% -
471.omnetpp (max) -77.78% 189.000 42.000 - -80.82% -
433.milc (max) (percentage) -77.37% 1.370 0.310 - -68.69% -
447.dealII (max) -76.74% 129.000 30.000 - -82.56% -
444.namd (avg) (percentage) -75.07% 3.810 0.950 - -72.93% -
482.sphinx3 (max) (percentage) -73.19% 3.320 0.890 - -72.78% -
429.mcf -71.70% 53.000 15.000 - -71.15% -
429.mcf (avg) -71.70% 53.000 15.000 - -71.15% -
464.h264ref (max) -71.67% 60.000 17.000 - -70.69% -
464.h264ref (unexplained diff vs previous) -71.67% 60.000 17.000 - 88.89% -
473.astar (unexplained diff vs previous) (percentage) -69.74% 0.760 0.230 - -68.92% -
470.lbm (avg) (percentage) -69.40% 2.320 0.710 - -62.83% -
401.bzip2 (unexplained diff vs previous) -69.39% 49.000 15.000 - 66.67% -
458.sjeng -68.42% 95.000 30.000 - -69.70% -
458.sjeng (avg) -68.42% 95.000 30.000 - -69.70% -
456.hmmer -66.67% 48.000 16.000 - -69.81% -
456.hmmer (avg) -66.67% 48.000 16.000 - -69.81% -
464.h264ref -66.67% 51.000 17.000 - -65.31% -
464.h264ref (avg) -66.67% 51.000 17.000 - -65.31% -
445.gobmk (max) -65.52% 29.000 10.000 - -58.33% -
401.bzip2 -64.71% 34.000 12.000 - -62.50% -
401.bzip2 (avg) -64.71% 34.000 12.000 - -62.50% -
473.astar (max) -64.04% 114.000 41.000 - -60.19% -
401.bzip2 (max) -62.50% 48.000 18.000 - -21.74% -
471.omnetpp -55.42% 83.000 37.000 - -51.32% -
471.omnetpp (avg) -55.42% 83.000 37.000 - -51.32% -
482.sphinx3 (avg) (percentage) -55.38% 1.860 0.830 - -51.74% -
453.povray (max) (percentage) -54.92% 1.930 0.870 - -50.29% -
453.povray (unexplained diff vs previous) (percentage) -54.21% 1.900 0.870 - -49.42% -
445.gobmk (unexplained diff vs previous) (percentage) -54.17% 0.240 0.110 - -45.00% -
447.dealII (avg) (percentage) -53.74% 1.470 0.680 - -53.42% -
473.astar -53.52% 71.000 33.000 - -45.90% -
473.astar (avg) -53.52% 71.000 33.000 - -45.90% -
471.omnetpp (unexplained diff vs previous) -53.26% 92.000 43.000 - -80.80% -
433.milc (avg) (percentage) -47.52% 1.010 0.530 - -44.21% -
458.sjeng (avg) (percentage) -36.21% 0.580 0.370 - -39.34% -
458.sjeng (max) -32.31% 65.000 44.000 - -45.68% -
445.gobmk -31.82% 22.000 15.000 - -28.57% -
445.gobmk (avg) -31.82% 22.000 15.000 - -28.57% -
453.povray (avg) (percentage) -25.55% 1.370 1.020 - -22.14% -
456.hmmer (max) (percentage) -23.33% 0.300 0.230 - -50.00% -
464.h264ref (max) (percentage) -20.51% 0.390 0.310 - -18.42% -
464.h264ref (unexplained diff vs previous) (percentage) -20.51% 0.390 0.310 - 416.67% -
445.gobmk (max) (percentage) -16.67% 0.240 0.200 - 0.00% -
471.omnetpp (max) (percentage) -13.64% 1.760 1.520 - -25.85% -
456.hmmer (unexplained diff vs previous) -11.11% 9.000 8.000 - -27.27% -
464.h264ref (avg) (percentage) -3.03% 0.330 0.320 - 0.00% -

Performance Improvements - Score Δ (B) Baseline Current σ (B) Δ σ
nb_total_tests 5.26% 19.000 20.000 - 5.26% -

Added Tests - sample variability
482.sphinx3 (unexplained diff vs previous)
482.sphinx3 (unexplained diff vs previous) (percentage)

Report Time: 0.49s
Execution (seconds)
Name Current %
401.bzip2 2706.000 -82.73%
403.gcc 59.000 -99.35%
429.mcf 1142.000 -90.36%
433.milc 1142.000 -91.52%
444.namd 707.000 -94.76%
445.gobmk 5245.000 -57.64%
447.dealII 1791.000 -78.14%
450.soplex 520.000 -96.40%
453.povray 462.000 -93.32%
456.hmmer 3570.000 -73.38%
458.sjeng 8312.000 -49.18%
462.libquantum 96.000 -99.27%
464.h264ref 5559.000 -64.10%
470.lbm 3166.000 -76.63%
471.omnetpp 2785.000 -73.94%
473.astar 4857.000 -65.90%
482.sphinx3 874.000 -96.68%
Geometric Mean 1364.449 -89.42%
Code Size (bytes)
Name Current %
401.bzip2 43341.000 9.84%
403.gcc 1955421.000 -
429.mcf 7974.000 -
433.milc 72918.000 -2.74%
444.namd 171581.000 -2.06%
445.gobmk 1308100.000 -
447.dealII 1738527.000 -2.25%
450.soplex 202447.000 1.89%
453.povray 579343.000 -2.33%
456.hmmer 166814.000 -
458.sjeng 96913.000 -
462.libquantum 24071.000 -9.90%
464.h264ref 349828.000 -1.33%
470.lbm 9802.000 -
471.omnetpp 377700.000 1.10%
473.astar 23287.000 -
482.sphinx3 114891.000 -
Geometric Mean 139977.944 -
Score
Name Current %
nb_compile_successful 18.000 -5.26%
nb_execution_successful 17.000 -5.56%
nb_total_tests 20.000 5.26%
Geometric Mean 18.292 -1.98%
sample variability
Name Current %
401.bzip2 12.000 -62.50%
401.bzip2 (avg) 12.000 -62.50%
401.bzip2 (avg) (percentage) 0.480 128.57%
401.bzip2 (max) 18.000 -21.74%
401.bzip2 (max) (percentage) 0.700 366.67%
401.bzip2 (unexplained diff vs previous) 15.000 66.67%
401.bzip2 (unexplained diff vs previous) (percentage) 0.550 816.67%
403.gcc 2.000 -95.45%
403.gcc (avg) 2.000 -95.45%
403.gcc (avg) (percentage) 3.930 702.04%
403.gcc (max) 4.000 -91.30%
403.gcc (max) (percentage) 7.700 1409.80%
429.mcf 15.000 -71.15%
429.mcf (avg) 15.000 -71.15%
429.mcf (avg) (percentage) 1.330 202.27%
429.mcf (max) 14.000 -76.27%
429.mcf (max) (percentage) 1.260 152.00%
429.mcf (unexplained diff vs previous) 1.000 -97.87%
429.mcf (unexplained diff vs previous) (percentage) 0.090 -77.50%
433.milc 6.000 -95.28%
433.milc (avg) 6.000 -95.28%
433.milc (avg) (percentage) 0.530 -44.21%
433.milc (max) 3.000 -97.74%
433.milc (max) (percentage) 0.310 -68.69%
433.milc (unexplained diff vs previous) 1.000 -99.24%
433.milc (unexplained diff vs previous) (percentage) 0.090 -90.82%
444.namd 6.000 -98.73%
444.namd (avg) 6.000 -98.73%
444.namd (avg) (percentage) 0.950 -72.93%
444.namd (max) 9.000 -98.97%
444.namd (max) (percentage) 1.330 -79.41%
445.gobmk 15.000 -28.57%
445.gobmk (avg) 15.000 -28.57%
445.gobmk (avg) (percentage) 0.290 70.59%
445.gobmk (max) 10.000 -58.33%
445.gobmk (max) (percentage) 0.200 -
445.gobmk (unexplained diff vs previous) 6.000 -76.00%
445.gobmk (unexplained diff vs previous) (percentage) 0.110 -45.00%
447.dealII 12.000 -89.92%
447.dealII (avg) 12.000 -89.92%
447.dealII (avg) (percentage) 0.680 -53.42%
447.dealII (max) 30.000 -82.56%
447.dealII (max) (percentage) 1.700 -19.43%
450.soplex 5.000 -89.13%
450.soplex (avg) 5.000 -89.13%
450.soplex (avg) (percentage) 1.060 231.25%
450.soplex (max) 3.000 -93.02%
450.soplex (max) (percentage) 0.760 153.33%
450.soplex (unexplained diff vs previous) 4.000 -90.70%
450.soplex (unexplained diff vs previous) (percentage) 0.770 156.67%
453.povray 4.000 -95.56%
453.povray (avg) 4.000 -95.56%
453.povray (avg) (percentage) 1.020 -22.14%
453.povray (max) 4.000 -96.69%
453.povray (max) (percentage) 0.870 -50.29%
453.povray (unexplained diff vs previous) 4.000 -96.64%
453.povray (unexplained diff vs previous) (percentage) 0.870 -49.42%
456.hmmer 16.000 -69.81%
456.hmmer (avg) 16.000 -69.81%
456.hmmer (avg) (percentage) 0.460 15.00%
456.hmmer (max) 8.000 -86.89%
456.hmmer (max) (percentage) 0.230 -50.00%
456.hmmer (unexplained diff vs previous) 8.000 -27.27%
456.hmmer (unexplained diff vs previous) (percentage) 0.220 175.00%
458.sjeng 30.000 -69.70%
458.sjeng (avg) 30.000 -69.70%
458.sjeng (avg) (percentage) 0.370 -39.34%
458.sjeng (max) 44.000 -45.68%
458.sjeng (max) (percentage) 0.530 6.00%
458.sjeng (unexplained diff vs previous) 44.000 -46.34%
458.sjeng (unexplained diff vs previous) (percentage) 0.530 6.00%
462.libquantum 1.000 -94.74%
462.libquantum (avg) 1.000 -94.74%
462.libquantum (avg) (percentage) 1.620 980.00%
462.libquantum (max) 2.000 -93.75%
462.libquantum (max) (percentage) 2.370 848.00%
462.libquantum (unexplained diff vs previous) 2.000 -93.94%
462.libquantum (unexplained diff vs previous) (percentage) 2.080 732.00%
464.h264ref 17.000 -65.31%
464.h264ref (avg) 17.000 -65.31%
464.h264ref (avg) (percentage) 0.320 -
464.h264ref (max) 17.000 -70.69%
464.h264ref (max) (percentage) 0.310 -18.42%
464.h264ref (unexplained diff vs previous) 17.000 88.89%
464.h264ref (unexplained diff vs previous) (percentage) 0.310 416.67%
470.lbm 22.000 -91.47%
470.lbm (avg) 22.000 -91.47%
470.lbm (avg) (percentage) 0.710 -62.83%
470.lbm (max) 22.000 -96.88%
470.lbm (max) (percentage) 0.710 -86.37%
470.lbm (unexplained diff vs previous) 7.000 -99.06%
470.lbm (unexplained diff vs previous) (percentage) 0.220 -96.00%
471.omnetpp 37.000 -51.32%
471.omnetpp (avg) 37.000 -51.32%
471.omnetpp (avg) (percentage) 1.330 84.72%
471.omnetpp (max) 42.000 -80.82%
471.omnetpp (max) (percentage) 1.520 -25.85%
471.omnetpp (unexplained diff vs previous) 43.000 -80.80%
471.omnetpp (unexplained diff vs previous) (percentage) 1.540 -26.67%
473.astar 33.000 -45.90%
473.astar (avg) 33.000 -45.90%
473.astar (avg) (percentage) 0.680 58.14%
473.astar (max) 41.000 -60.19%
473.astar (max) (percentage) 0.850 16.44%
473.astar (unexplained diff vs previous) 11.000 -89.52%
473.astar (unexplained diff vs previous) (percentage) 0.230 -68.92%
482.sphinx3 7.000 -98.45%
482.sphinx3 (avg) 7.000 -98.45%
482.sphinx3 (avg) (percentage) 0.830 -51.74%
482.sphinx3 (max) 7.000 -99.19%
482.sphinx3 (max) (percentage) 0.890 -72.78%
482.sphinx3 (unexplained diff vs previous) 0.000 -
482.sphinx3 (unexplained diff vs previous) (percentage) 0.000 -
483.xalancbmk (avg) (percentage) 1.330 177.08%
483.xalancbmk (max) (percentage) 0.670 76.32%
Geometric Mean 2.460 -73.90%