Runs:

Compare To:

Machine Info

Fields

hardware None
os None

Parameters

Run Info

Fields

NameValue

Parameters

NameValue
git_llvm https://github.com/llvm/llvm-project/commit/ea2ed80e6dac21bf19d87de8bef1ec01b00dbb8d (unchanged)
test_url https://ci.linaro.org/job/tcwg_bmk_ci_llvm-build-tcwg_bmk_apm-llvm-master-aarch64-spec2k6-Oz_LTO/321/
View Options
Report
Filter

llvm-aarch64-master-Oz_LTO test results

Run Order Start Time Duration
Current 0321 (no-change) 2022-10-06T01:48:31 0:00:00
Previous 0320 (llvmorg-21-init-08388-gca5b3a0f51ba) 2025-04-08T19:30:00 0:00:00
Baseline 0001 (llvmorg-16-init-17110-gb1256f2345f6) 2022-12-17T20:51:03 0:00:00

Tests Summary

Status Group # # (B)
Performance Regressions 96 93
Performance Improvements 52 24
Added Tests 19 30
Unchanged Tests 937 957
Total Tests 1104

Run-Over-Run Changes Detail

Performance Regressions - Code Size Δ Previous Current σ Δ (B) σ (B)
444.namd 6.77% 134715.000 143829.000 - 0.28% -
464.h264ref 3.95% 384911.000 400126.000 - 0.04% -
447.dealII 3.30% 315170.000 325579.000 - 0.11% -
403.gcc 3.15% 2413757.000 2489749.000 - 0.11% -
453.povray 3.02% 671895.000 692210.000 - 0.10% -
482.sphinx3 2.43% 117597.000 120454.000 - 0.17% -
462.libquantum 2.10% 16652.000 17001.000 - 0.07% -
400.perlbench 2.05% 883050.000 901163.000 - 0.10% -
456.hmmer 1.88% 111596.000 113689.000 - 0.07% -
473.astar 1.86% 23628.000 24068.000 - 1.17% -
458.sjeng 1.83% 105116.000 107040.000 - 0.16% -
471.omnetpp 1.71% 481972.000 490199.000 - 0.01% -
433.milc 1.69% 62106.000 63155.000 - 0.46% -
401.bzip2 1.66% 43672.000 44397.000 - 0.09% -
445.gobmk 1.63% 1821086.000 1850859.000 - 0.03% -
450.soplex 1.20% 221650.000 224319.000 - -0.04% -

Performance Regressions - Execution Δ Previous Current σ Δ (B) σ (B)
403.gcc 14610.14% 69.000 10150.000 - 0.00% -
462.libquantum 11723.23% 99.000 11705.000 - 21.18% -
482.sphinx3 3401.14% 792.000 27729.000 - 16.61% -
450.soplex 2621.72% 511.000 13908.000 - -0.57% -
453.povray 1297.08% 514.000 7181.000 - -1.64% -
444.namd 1291.92% 668.000 9298.000 - -0.84% -
433.milc 1087.74% 1223.000 14526.000 - -0.44% -
429.mcf 865.51% 1815.000 17524.000 - -7.30% -
400.perlbench 553.43% 1922.000 12559.000 - 13.30% -
447.dealII 402.03% 1917.000 9624.000 - 5.74% -
471.omnetpp 366.89% 2890.000 13493.000 - 13.22% -
401.bzip2 362.33% 2718.000 12566.000 - -10.00% -
470.lbm 322.01% 3148.000 13285.000 - 20.27% -
456.hmmer 276.99% 2890.000 10895.000 - 19.44% -
464.h264ref 209.59% 4412.000 13659.000 - 12.49% -
473.astar 202.09% 4933.000 14902.000 - 12.04% -
445.gobmk 134.31% 5430.000 12723.000 - 9.04% -
458.sjeng 93.98% 7478.000 14506.000 - -5.08% -

Performance Regressions - sample variability Δ Previous Current σ Δ (B) σ (B)
482.sphinx3 (unexplained diff vs previous) 7800.00% 3.000 237.000 - 46.30% -
444.namd (max) 5671.43% 7.000 404.000 - -16.70% -
444.namd 4600.00% 5.000 235.000 - 10.33% -
444.namd (avg) 4600.00% 5.000 235.000 - 10.33% -
403.gcc 3600.00% 1.000 37.000 - 19.35% -
403.gcc (avg) 3600.00% 1.000 37.000 - 19.35% -
429.mcf (unexplained diff vs previous) 2775.00% 4.000 115.000 - 0.00% -
482.sphinx3 2320.00% 15.000 363.000 - 68.06% -
482.sphinx3 (avg) 2320.00% 15.000 363.000 - 68.06% -
482.sphinx3 (max) 1765.85% 41.000 765.000 - 366.46% -
403.gcc (max) 1600.00% 2.000 34.000 - -15.00% -
433.milc 1175.00% 12.000 153.000 - 26.45% -
433.milc (avg) 1175.00% 12.000 153.000 - 26.45% -
401.bzip2 (unexplained diff vs previous) 1100.00% 1.000 12.000 - 0.00% -
453.povray 1000.00% 6.000 66.000 - 24.53% -
453.povray (avg) 1000.00% 6.000 66.000 - 24.53% -
447.dealII (max) 925.00% 12.000 123.000 - -2.38% -
456.hmmer (unexplained diff vs previous) 900.00% 2.000 20.000 - 0.00% -
462.libquantum 800.00% 2.000 18.000 - 80.00% -
462.libquantum (avg) 800.00% 2.000 18.000 - 80.00% -
456.hmmer (max) 713.33% 15.000 122.000 - 79.41% -
456.hmmer 690.91% 11.000 87.000 - 8.75% -
456.hmmer (avg) 690.91% 11.000 87.000 - 8.75% -
473.astar (unexplained diff vs previous) 650.00% 2.000 15.000 - 0.00% -
447.dealII 544.44% 18.000 116.000 - 41.46% -
447.dealII (avg) 544.44% 18.000 116.000 - 41.46% -
470.lbm (max) 518.87% 53.000 328.000 - 5366.67% -
450.soplex 500.00% 15.000 90.000 - 20.00% -
450.soplex (avg) 500.00% 15.000 90.000 - 20.00% -
453.povray (max) 468.42% 19.000 108.000 - 40.26% -
433.milc (max) 465.62% 32.000 181.000 - -32.71% -
450.soplex (max) 425.00% 36.000 189.000 - 23.53% -
470.lbm 324.24% 33.000 140.000 - 118.75% -
470.lbm (avg) 324.24% 33.000 140.000 - 118.75% -
458.sjeng (unexplained diff vs previous) 313.33% 15.000 62.000 - 0.00% -
444.namd (max) (percentage) 299.08% 1.090 4.350 - -16.02% -
462.libquantum (max) 280.00% 5.000 19.000 - 533.33% -
444.namd (avg) (percentage) 216.25% 0.800 2.530 - 10.96% -
429.mcf (unexplained diff vs previous) (percentage) 200.00% 0.220 0.660 - 0.00% -
444.namd (unexplained diff vs previous) 200.00% 4.000 12.000 - 0.00% -
456.hmmer (unexplained diff vs previous) (percentage) 157.14% 0.070 0.180 - 0.00% -
401.bzip2 (unexplained diff vs previous) (percentage) 150.00% 0.040 0.100 - 0.00% -
473.astar (unexplained diff vs previous) (percentage) 150.00% 0.040 0.100 - 0.00% -
464.h264ref (max) 138.71% 62.000 148.000 - 362.50% -
433.milc (unexplained diff vs previous) 133.33% 12.000 28.000 - 833.33% -
482.sphinx3 (unexplained diff vs previous) (percentage) 123.68% 0.380 0.850 - 25.00% -
458.sjeng (unexplained diff vs previous) (percentage) 115.00% 0.200 0.430 - 0.00% -
456.hmmer (max) (percentage) 103.64% 0.550 1.120 - 49.33% -
456.hmmer (avg) (percentage) 100.00% 0.400 0.800 - -9.09% -
447.dealII (max) (percentage) 93.94% 0.660 1.280 - -7.91% -
464.h264ref 90.00% 40.000 76.000 - 72.73% -
464.h264ref (avg) 90.00% 40.000 76.000 - 72.73% -
471.omnetpp 80.77% 52.000 94.000 - 32.39% -
471.omnetpp (avg) 80.77% 52.000 94.000 - 32.39% -
471.omnetpp (max) 56.10% 82.000 128.000 - 7.56% -
470.lbm (max) (percentage) 45.29% 1.700 2.470 - 4016.67% -
447.dealII (avg) (percentage) 24.74% 0.970 1.210 - 32.97% -
483.xalancbmk (avg) (percentage) 11.63% 0.430 0.480 - 11.63% -
433.milc (avg) (percentage) 2.91% 1.030 1.060 - 27.71% -

Performance Regressions - Score Δ Previous Current σ Δ (B) σ (B)
nb_total_tests -38.71% 31.000 19.000 - 0.00% -
nb_compile_successful -29.63% 27.000 19.000 - 0.00% -
nb_execution_successful -29.63% 27.000 19.000 - 0.00% -

Performance Improvements - sample variability Δ Previous Current σ Δ (B) σ (B)
400.perlbench (max) (percentage) -99.71% 3.500 0.010 - -98.57% -
400.perlbench (avg) (percentage) -99.25% 2.650 0.020 - -96.08% -
462.libquantum (unexplained diff vs previous) (percentage) -99.01% 1.010 0.010 - 0.00% -
400.perlbench (max) -98.51% 67.000 1.000 - -98.70% -
473.astar (max) (percentage) -97.34% 3.760 0.100 - 25.00% -
462.libquantum (max) (percentage) -96.94% 5.550 0.170 - 325.00% -
400.perlbench -96.00% 50.000 2.000 - -96.43% -
400.perlbench (avg) -96.00% 50.000 2.000 - -96.43% -
429.mcf (max) (percentage) -95.96% 16.850 0.680 - 88.89% -
473.astar (avg) (percentage) -94.05% 1.680 0.100 - 42.86% -
462.libquantum (avg) (percentage) -93.75% 2.560 0.160 - 45.45% -
429.mcf (avg) (percentage) -92.79% 6.380 0.460 - 24.32% -
473.astar (max) -92.43% 185.000 14.000 - 40.00% -
470.lbm (unexplained diff vs previous) (percentage) -91.86% 0.860 0.070 - 0.00% -
403.gcc (max) (percentage) -90.91% 3.740 0.340 - -15.00% -
403.gcc (avg) (percentage) -85.55% 2.560 0.370 - 19.35% -
473.astar -82.93% 82.000 14.000 - 55.56% -
473.astar (avg) -82.93% 82.000 14.000 - 55.56% -
401.bzip2 (avg) (percentage) -81.33% 0.750 0.140 - 40.00% -
450.soplex (max) (percentage) -80.71% 7.050 1.360 - 23.64% -
433.milc (unexplained diff vs previous) (percentage) -80.61% 0.980 0.190 - 850.00% -
450.soplex (avg) (percentage) -79.17% 3.120 0.650 - 20.37% -
401.bzip2 (max) (percentage) -78.95% 0.760 0.160 - 220.00% -
444.namd (unexplained diff vs previous) (percentage) -78.33% 0.600 0.130 - 0.00% -
445.gobmk (unexplained diff vs previous) (percentage) -76.92% 0.260 0.060 - 0.00% -
445.gobmk (max) (percentage) -69.57% 1.610 0.490 - -18.33% -
471.omnetpp (max) (percentage) -66.90% 2.870 0.950 - -5.00% -
470.lbm (unexplained diff vs previous) -66.67% 27.000 9.000 - 0.00% -
471.omnetpp (avg) (percentage) -61.33% 1.810 0.700 - 16.67% -
429.mcf (max) -60.98% 305.000 119.000 - 75.00% -
445.gobmk (avg) (percentage) -59.26% 0.810 0.330 - 3.13% -
453.povray (max) (percentage) -59.19% 3.700 1.510 - 42.45% -
433.milc (max) (percentage) -52.29% 2.620 1.250 - -32.43% -
458.sjeng (max) (percentage) -50.57% 0.870 0.430 - 86.96% -
458.sjeng (avg) (percentage) -50.00% 0.540 0.270 - 17.39% -
482.sphinx3 (max) (percentage) -47.23% 5.230 2.760 - 300.00% -
445.gobmk (unexplained diff vs previous) -42.86% 14.000 8.000 - 0.00% -
464.h264ref (avg) (percentage) -39.13% 0.920 0.560 - 51.35% -
482.sphinx3 (avg) (percentage) -34.17% 1.990 1.310 - 43.96% -
429.mcf -30.43% 115.000 80.000 - 15.94% -
429.mcf (avg) -30.43% 115.000 80.000 - 15.94% -
483.xalancbmk (max) (percentage) -29.73% 0.370 0.260 - -49.02% -
445.gobmk (max) -28.74% 87.000 62.000 - -11.43% -
453.povray (avg) (percentage) -27.56% 1.270 0.920 - 26.03% -
464.h264ref (max) (percentage) -22.70% 1.410 1.090 - 303.70% -
401.bzip2 -15.00% 20.000 17.000 - 30.77% -
401.bzip2 (avg) -15.00% 20.000 17.000 - 30.77% -
445.gobmk -4.65% 43.000 41.000 - 10.81% -
445.gobmk (avg) -4.65% 43.000 41.000 - 10.81% -
458.sjeng (max) -4.62% 65.000 62.000 - 77.14% -
458.sjeng -2.50% 40.000 39.000 - 11.43% -
458.sjeng (avg) -2.50% 40.000 39.000 - 11.43% -

Added Tests - Code Size
483.xalancbmk

Added Tests - Execution
483.xalancbmk

Added Tests - sample variability
403.gcc (unexplained diff vs previous)
403.gcc (unexplained diff vs previous) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)
471.omnetpp (unexplained diff vs previous)
471.omnetpp (unexplained diff vs previous) (percentage)
483.xalancbmk
483.xalancbmk (avg)
483.xalancbmk (max)
483.xalancbmk (unexplained diff vs previous)
483.xalancbmk (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size Δ (B) Baseline Current σ (B) Δ σ
473.astar 1.17% 23789.000 24068.000 - 1.86% -

Performance Regressions - Execution Δ (B) Baseline Current σ (B) Δ σ
483.xalancbmk 24.38% 9633.000 11982.000 - 0.00% -
462.libquantum 21.18% 9659.000 11705.000 - 11723.23% -
470.lbm 20.27% 11046.000 13285.000 - 322.01% -
456.hmmer 19.44% 9122.000 10895.000 - 276.99% -
482.sphinx3 16.61% 23779.000 27729.000 - 3401.14% -
400.perlbench 13.30% 11085.000 12559.000 - 553.43% -
471.omnetpp 13.22% 11917.000 13493.000 - 366.89% -
464.h264ref 12.49% 12142.000 13659.000 - 209.59% -
473.astar 12.04% 13301.000 14902.000 - 202.09% -
445.gobmk 9.04% 11668.000 12723.000 - 134.31% -
447.dealII 5.74% 9102.000 9624.000 - 402.03% -

Performance Regressions - sample variability Δ (B) Baseline Current σ (B) Δ σ
470.lbm (max) 5366.67% 6.000 328.000 - 518.87% -
470.lbm (max) (percentage) 4016.67% 0.060 2.470 - 45.29% -
433.milc (unexplained diff vs previous) (percentage) 850.00% 0.020 0.190 - -80.61% -
433.milc (unexplained diff vs previous) 833.33% 3.000 28.000 - 133.33% -
462.libquantum (max) 533.33% 3.000 19.000 - 280.00% -
482.sphinx3 (max) 366.46% 164.000 765.000 - 1765.85% -
464.h264ref (max) 362.50% 32.000 148.000 - 138.71% -
462.libquantum (max) (percentage) 325.00% 0.040 0.170 - -96.94% -
464.h264ref (max) (percentage) 303.70% 0.270 1.090 - -22.70% -
482.sphinx3 (max) (percentage) 300.00% 0.690 2.760 - -47.23% -
401.bzip2 (max) 233.33% 6.000 20.000 - 0.00% -
401.bzip2 (max) (percentage) 220.00% 0.050 0.160 - -78.95% -
470.lbm 118.75% 64.000 140.000 - 324.24% -
470.lbm (avg) 118.75% 64.000 140.000 - 324.24% -
429.mcf (max) (percentage) 88.89% 0.360 0.680 - -95.96% -
458.sjeng (max) (percentage) 86.96% 0.230 0.430 - -50.57% -
470.lbm (avg) (percentage) 82.76% 0.580 1.060 - 0.00% -
462.libquantum 80.00% 10.000 18.000 - 800.00% -
462.libquantum (avg) 80.00% 10.000 18.000 - 800.00% -
456.hmmer (max) 79.41% 68.000 122.000 - 713.33% -
458.sjeng (max) 77.14% 35.000 62.000 - -4.62% -
429.mcf (max) 75.00% 68.000 119.000 - -60.98% -
464.h264ref 72.73% 44.000 76.000 - 90.00% -
464.h264ref (avg) 72.73% 44.000 76.000 - 90.00% -
482.sphinx3 68.06% 216.000 363.000 - 2320.00% -
482.sphinx3 (avg) 68.06% 216.000 363.000 - 2320.00% -
473.astar 55.56% 9.000 14.000 - -82.93% -
473.astar (avg) 55.56% 9.000 14.000 - -82.93% -
464.h264ref (avg) (percentage) 51.35% 0.370 0.560 - -39.13% -
456.hmmer (max) (percentage) 49.33% 0.750 1.120 - 103.64% -
482.sphinx3 (unexplained diff vs previous) 46.30% 162.000 237.000 - 7800.00% -
462.libquantum (avg) (percentage) 45.45% 0.110 0.160 - -93.75% -
482.sphinx3 (avg) (percentage) 43.96% 0.910 1.310 - -34.17% -
473.astar (avg) (percentage) 42.86% 0.070 0.100 - -94.05% -
453.povray (max) (percentage) 42.45% 1.060 1.510 - -59.19% -
447.dealII 41.46% 82.000 116.000 - 544.44% -
447.dealII (avg) 41.46% 82.000 116.000 - 544.44% -
453.povray (max) 40.26% 77.000 108.000 - 468.42% -
401.bzip2 (avg) (percentage) 40.00% 0.100 0.140 - -81.33% -
473.astar (max) 40.00% 10.000 14.000 - -92.43% -
483.xalancbmk 39.02% 41.000 57.000 - 0.00% -
483.xalancbmk (avg) 39.02% 41.000 57.000 - 0.00% -
447.dealII (avg) (percentage) 32.97% 0.910 1.210 - 24.74% -
471.omnetpp 32.39% 71.000 94.000 - 80.77% -
471.omnetpp (avg) 32.39% 71.000 94.000 - 80.77% -
401.bzip2 30.77% 13.000 17.000 - -15.00% -
401.bzip2 (avg) 30.77% 13.000 17.000 - -15.00% -
433.milc (avg) (percentage) 27.71% 0.830 1.060 - 2.91% -
433.milc 26.45% 121.000 153.000 - 1175.00% -
433.milc (avg) 26.45% 121.000 153.000 - 1175.00% -
453.povray (avg) (percentage) 26.03% 0.730 0.920 - -27.56% -
473.astar (max) (percentage) 25.00% 0.080 0.100 - -97.34% -
482.sphinx3 (unexplained diff vs previous) (percentage) 25.00% 0.680 0.850 - 123.68% -
453.povray 24.53% 53.000 66.000 - 1000.00% -
453.povray (avg) 24.53% 53.000 66.000 - 1000.00% -
429.mcf (avg) (percentage) 24.32% 0.370 0.460 - -92.79% -
450.soplex (max) (percentage) 23.64% 1.100 1.360 - -80.71% -
450.soplex (max) 23.53% 153.000 189.000 - 425.00% -
450.soplex (avg) (percentage) 20.37% 0.540 0.650 - -79.17% -
450.soplex 20.00% 75.000 90.000 - 500.00% -
450.soplex (avg) 20.00% 75.000 90.000 - 500.00% -
403.gcc 19.35% 31.000 37.000 - 3600.00% -
403.gcc (avg) 19.35% 31.000 37.000 - 3600.00% -
403.gcc (avg) (percentage) 19.35% 0.310 0.370 - -85.55% -
458.sjeng (avg) (percentage) 17.39% 0.230 0.270 - -50.00% -
471.omnetpp (avg) (percentage) 16.67% 0.600 0.700 - -61.33% -
429.mcf 15.94% 69.000 80.000 - -30.43% -
429.mcf (avg) 15.94% 69.000 80.000 - -30.43% -
483.xalancbmk (avg) (percentage) 11.63% 0.430 0.480 - 11.63% -
458.sjeng 11.43% 35.000 39.000 - -2.50% -
458.sjeng (avg) 11.43% 35.000 39.000 - -2.50% -
444.namd (avg) (percentage) 10.96% 2.280 2.530 - 216.25% -
445.gobmk 10.81% 37.000 41.000 - -4.65% -
445.gobmk (avg) 10.81% 37.000 41.000 - -4.65% -
444.namd 10.33% 213.000 235.000 - 4600.00% -
444.namd (avg) 10.33% 213.000 235.000 - 4600.00% -
456.hmmer 8.75% 80.000 87.000 - 690.91% -
456.hmmer (avg) 8.75% 80.000 87.000 - 690.91% -
471.omnetpp (unexplained diff vs previous) 8.33% 120.000 130.000 - 0.00% -
471.omnetpp (max) 7.56% 119.000 128.000 - 56.10% -
445.gobmk (avg) (percentage) 3.13% 0.320 0.330 - -59.26% -

Performance Improvements - Execution Δ (B) Baseline Current σ (B) Δ σ
401.bzip2 -10.00% 13962.000 12566.000 - 362.33% -
429.mcf -7.30% 18905.000 17524.000 - 865.51% -
458.sjeng -5.08% 15282.000 14506.000 - 93.98% -
453.povray -1.64% 7301.000 7181.000 - 1297.08% -

Performance Improvements - sample variability Δ (B) Baseline Current σ (B) Δ σ
400.perlbench (max) -98.70% 77.000 1.000 - -98.51% -
400.perlbench (max) (percentage) -98.57% 0.700 0.010 - -99.71% -
400.perlbench -96.43% 56.000 2.000 - -96.00% -
400.perlbench (avg) -96.43% 56.000 2.000 - -96.00% -
400.perlbench (avg) (percentage) -96.08% 0.510 0.020 - -99.25% -
483.xalancbmk (max) (percentage) -49.02% 0.510 0.260 - -29.73% -
483.xalancbmk (max) -36.73% 49.000 31.000 - 0.00% -
433.milc (max) -32.71% 269.000 181.000 - 465.62% -
433.milc (max) (percentage) -32.43% 1.850 1.250 - -52.29% -
445.gobmk (max) (percentage) -18.33% 0.600 0.490 - -69.57% -
444.namd (max) -16.70% 485.000 404.000 - 5671.43% -
444.namd (max) (percentage) -16.02% 5.180 4.350 - 299.08% -
403.gcc (max) -15.00% 40.000 34.000 - 1600.00% -
403.gcc (max) (percentage) -15.00% 0.400 0.340 - -90.91% -
445.gobmk (max) -11.43% 70.000 62.000 - -28.74% -
456.hmmer (avg) (percentage) -9.09% 0.880 0.800 - 100.00% -
447.dealII (max) (percentage) -7.91% 1.390 1.280 - 93.94% -
471.omnetpp (max) (percentage) -5.00% 1.000 0.950 - -66.90% -
471.omnetpp (unexplained diff vs previous) (percentage) -4.95% 1.010 0.960 - 0.00% -
447.dealII (max) -2.38% 126.000 123.000 - 925.00% -

Added Tests - sample variability
401.bzip2 (unexplained diff vs previous)
401.bzip2 (unexplained diff vs previous) (percentage)
403.gcc (unexplained diff vs previous)
403.gcc (unexplained diff vs previous) (percentage)
429.mcf (unexplained diff vs previous)
429.mcf (unexplained diff vs previous) (percentage)
444.namd (unexplained diff vs previous)
444.namd (unexplained diff vs previous) (percentage)
445.gobmk (unexplained diff vs previous)
445.gobmk (unexplained diff vs previous) (percentage)
447.dealII (unexplained diff vs previous)
447.dealII (unexplained diff vs previous) (percentage)
450.soplex (unexplained diff vs previous)
450.soplex (unexplained diff vs previous) (percentage)
453.povray (unexplained diff vs previous)
453.povray (unexplained diff vs previous) (percentage)
456.hmmer (unexplained diff vs previous)
456.hmmer (unexplained diff vs previous) (percentage)
458.sjeng (unexplained diff vs previous)
458.sjeng (unexplained diff vs previous) (percentage)
462.libquantum (unexplained diff vs previous)
462.libquantum (unexplained diff vs previous) (percentage)
464.h264ref (unexplained diff vs previous)
464.h264ref (unexplained diff vs previous) (percentage)
470.lbm (unexplained diff vs previous)
470.lbm (unexplained diff vs previous) (percentage)
473.astar (unexplained diff vs previous)
473.astar (unexplained diff vs previous) (percentage)
483.xalancbmk (unexplained diff vs previous)
483.xalancbmk (unexplained diff vs previous) (percentage)

Report Time: 0.39s
Execution (seconds)
Name Current %
400.perlbench 12559.000 553.43%
401.bzip2 12566.000 362.33%
403.gcc 10150.000 14610.14%
429.mcf 17524.000 865.51%
433.milc 14526.000 1087.74%
444.namd 9298.000 1291.92%
445.gobmk 12723.000 134.31%
447.dealII 9624.000 402.03%
450.soplex 13908.000 2621.72%
453.povray 7181.000 1297.08%
456.hmmer 10895.000 276.99%
458.sjeng 14506.000 93.98%
462.libquantum 11705.000 11723.23%
464.h264ref 13659.000 209.59%
470.lbm 13285.000 322.01%
471.omnetpp 13493.000 366.89%
473.astar 14902.000 202.09%
482.sphinx3 27729.000 3401.14%
483.xalancbmk 11982.000 -
Geometric Mean 12768.914 805.07%
Code Size (bytes)
Name Current %
400.perlbench 901163.000 2.05%
401.bzip2 44397.000 1.66%
403.gcc 2489749.000 3.15%
429.mcf 9249.000 -
433.milc 63155.000 1.69%
444.namd 143829.000 6.77%
445.gobmk 1850859.000 1.63%
447.dealII 325579.000 3.30%
450.soplex 224319.000 1.20%
453.povray 692210.000 3.02%
456.hmmer 113689.000 1.88%
458.sjeng 107040.000 1.83%
462.libquantum 17001.000 2.10%
464.h264ref 400126.000 3.95%
470.lbm 10273.000 -
471.omnetpp 490199.000 1.71%
473.astar 24068.000 1.86%
482.sphinx3 120454.000 2.43%
483.xalancbmk 2540035.000 -
Geometric Mean 170048.877 18.87%
Score
Name Current %
nb_compile_successful 19.000 -29.63%
nb_execution_successful 19.000 -29.63%
nb_total_tests 19.000 -38.71%
Geometric Mean 19.000 -32.80%
sample variability
Name Current %
400.perlbench 2.000 -96.00%
400.perlbench (avg) 2.000 -96.00%
400.perlbench (avg) (percentage) 0.020 -99.25%
400.perlbench (max) 1.000 -98.51%
400.perlbench (max) (percentage) 0.010 -99.71%
401.bzip2 17.000 -15.00%
401.bzip2 (avg) 17.000 -15.00%
401.bzip2 (avg) (percentage) 0.140 -81.33%
401.bzip2 (max) 20.000 -
401.bzip2 (max) (percentage) 0.160 -78.95%
401.bzip2 (unexplained diff vs previous) 12.000 1100.00%
401.bzip2 (unexplained diff vs previous) (percentage) 0.100 150.00%
403.gcc 37.000 3600.00%
403.gcc (avg) 37.000 3600.00%
403.gcc (avg) (percentage) 0.370 -85.55%
403.gcc (max) 34.000 1600.00%
403.gcc (max) (percentage) 0.340 -90.91%
403.gcc (unexplained diff vs previous) 31.000 -
403.gcc (unexplained diff vs previous) (percentage) 0.310 -
429.mcf 80.000 -30.43%
429.mcf (avg) 80.000 -30.43%
429.mcf (avg) (percentage) 0.460 -92.79%
429.mcf (max) 119.000 -60.98%
429.mcf (max) (percentage) 0.680 -95.96%
429.mcf (unexplained diff vs previous) 115.000 2775.00%
429.mcf (unexplained diff vs previous) (percentage) 0.660 200.00%
433.milc 153.000 1175.00%
433.milc (avg) 153.000 1175.00%
433.milc (avg) (percentage) 1.060 2.91%
433.milc (max) 181.000 465.62%
433.milc (max) (percentage) 1.250 -52.29%
433.milc (unexplained diff vs previous) 28.000 133.33%
433.milc (unexplained diff vs previous) (percentage) 0.190 -80.61%
444.namd 235.000 4600.00%
444.namd (avg) 235.000 4600.00%
444.namd (avg) (percentage) 2.530 216.25%
444.namd (max) 404.000 5671.43%
444.namd (max) (percentage) 4.350 299.08%
444.namd (unexplained diff vs previous) 12.000 200.00%
444.namd (unexplained diff vs previous) (percentage) 0.130 -78.33%
445.gobmk 41.000 -4.65%
445.gobmk (avg) 41.000 -4.65%
445.gobmk (avg) (percentage) 0.330 -59.26%
445.gobmk (max) 62.000 -28.74%
445.gobmk (max) (percentage) 0.490 -69.57%
445.gobmk (unexplained diff vs previous) 8.000 -42.86%
445.gobmk (unexplained diff vs previous) (percentage) 0.060 -76.92%
447.dealII 116.000 544.44%
447.dealII (avg) 116.000 544.44%
447.dealII (avg) (percentage) 1.210 24.74%
447.dealII (max) 123.000 925.00%
447.dealII (max) (percentage) 1.280 93.94%
447.dealII (unexplained diff vs previous) 75.000 -
447.dealII (unexplained diff vs previous) (percentage) 0.780 -
450.soplex 90.000 500.00%
450.soplex (avg) 90.000 500.00%
450.soplex (avg) (percentage) 0.650 -79.17%
450.soplex (max) 189.000 425.00%
450.soplex (max) (percentage) 1.360 -80.71%
450.soplex (unexplained diff vs previous) 22.000 -
450.soplex (unexplained diff vs previous) (percentage) 0.160 -
453.povray 66.000 1000.00%
453.povray (avg) 66.000 1000.00%
453.povray (avg) (percentage) 0.920 -27.56%
453.povray (max) 108.000 468.42%
453.povray (max) (percentage) 1.510 -59.19%
453.povray (unexplained diff vs previous) 21.000 -
453.povray (unexplained diff vs previous) (percentage) 0.290 -
456.hmmer 87.000 690.91%
456.hmmer (avg) 87.000 690.91%
456.hmmer (avg) (percentage) 0.800 100.00%
456.hmmer (max) 122.000 713.33%
456.hmmer (max) (percentage) 1.120 103.64%
456.hmmer (unexplained diff vs previous) 20.000 900.00%
456.hmmer (unexplained diff vs previous) (percentage) 0.180 157.14%
458.sjeng 39.000 -2.50%
458.sjeng (avg) 39.000 -2.50%
458.sjeng (avg) (percentage) 0.270 -50.00%
458.sjeng (max) 62.000 -4.62%
458.sjeng (max) (percentage) 0.430 -50.57%
458.sjeng (unexplained diff vs previous) 62.000 313.33%
458.sjeng (unexplained diff vs previous) (percentage) 0.430 115.00%
462.libquantum 18.000 800.00%
462.libquantum (avg) 18.000 800.00%
462.libquantum (avg) (percentage) 0.160 -93.75%
462.libquantum (max) 19.000 280.00%
462.libquantum (max) (percentage) 0.170 -96.94%
462.libquantum (unexplained diff vs previous) 1.000 -
462.libquantum (unexplained diff vs previous) (percentage) 0.010 -99.01%
464.h264ref 76.000 90.00%
464.h264ref (avg) 76.000 90.00%
464.h264ref (avg) (percentage) 0.560 -39.13%
464.h264ref (max) 148.000 138.71%
464.h264ref (max) (percentage) 1.090 -22.70%
464.h264ref (unexplained diff vs previous) 49.000 -
464.h264ref (unexplained diff vs previous) (percentage) 0.360 -
470.lbm 140.000 324.24%
470.lbm (avg) 140.000 324.24%
470.lbm (avg) (percentage) 1.060 -
470.lbm (max) 328.000 518.87%
470.lbm (max) (percentage) 2.470 45.29%
470.lbm (unexplained diff vs previous) 9.000 -66.67%
470.lbm (unexplained diff vs previous) (percentage) 0.070 -91.86%
471.omnetpp 94.000 80.77%
471.omnetpp (avg) 94.000 80.77%
471.omnetpp (avg) (percentage) 0.700 -61.33%
471.omnetpp (max) 128.000 56.10%
471.omnetpp (max) (percentage) 0.950 -66.90%
471.omnetpp (unexplained diff vs previous) 130.000 -
471.omnetpp (unexplained diff vs previous) (percentage) 0.960 -
473.astar 14.000 -82.93%
473.astar (avg) 14.000 -82.93%
473.astar (avg) (percentage) 0.100 -94.05%
473.astar (max) 14.000 -92.43%
473.astar (max) (percentage) 0.100 -97.34%
473.astar (unexplained diff vs previous) 15.000 650.00%
473.astar (unexplained diff vs previous) (percentage) 0.100 150.00%
482.sphinx3 363.000 2320.00%
482.sphinx3 (avg) 363.000 2320.00%
482.sphinx3 (avg) (percentage) 1.310 -34.17%
482.sphinx3 (max) 765.000 1765.85%
482.sphinx3 (max) (percentage) 2.760 -47.23%
482.sphinx3 (unexplained diff vs previous) 237.000 7800.00%
482.sphinx3 (unexplained diff vs previous) (percentage) 0.850 123.68%
483.xalancbmk 57.000 -
483.xalancbmk (avg) 57.000 -
483.xalancbmk (avg) (percentage) 0.480 11.63%
483.xalancbmk (max) 31.000 -
483.xalancbmk (max) (percentage) 0.260 -29.73%
483.xalancbmk (unexplained diff vs previous) 21.000 -
483.xalancbmk (unexplained diff vs previous) (percentage) 0.180 -
Geometric Mean 6.202 20.66%