Machine Info

Fields

hardware	None
os	None

Parameters

Run Info

Fields

Name	Value

Parameters

Name	Value
git_llvm	https://github.com/llvm/llvm-project/commit/f94ed6f7977305db8fa6b48a85c9db2b8cc4d3b3 (+ 2049 commits)
test_url	https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--llvm-arm-master-O3-build/174/

Report

Filter

llvm-arm-master-O3 test results

Run	Order	Start Time	Duration
Current	0174 (llvmorg-19-init-09063-gf94ed6f79773)	2024-04-11T22:27:27	0:00:00
Previous	0172 (no-change)	2024-04-01T02:49:23	0:00:00
Baseline	0015 (llvmorg-17-init-07405-g0f9842c21c76)	2023-04-07T17:38:29	0:00:00

Tests Summary

Status Group	#	# (B)
Performance Regressions	6	4
Performance Improvements	4	2
Added Tests	44	83
Unchanged Tests	384	349
Total Tests	438

Run-Over-Run Changes Detail

Performance Regressions - Execution	Δ	Previous	Current	σ	Δ (B)	σ (B)
500.perlbench_r	2.19%	15463.000	15802.000	-	0.00%	-
531.deepsjeng_r	1.47%	11437.000	11605.000	-	7.70%	-

Performance Regressions - sample variability	Δ	Previous	Current	σ	Δ (B)	σ (B)
557.xz_r (max)	50.00%	10.000	15.000	-	0.00%	-
557.xz_r (max) (percentage)	50.00%	0.100	0.150	-	0.00%	-
531.deepsjeng_r	1.12%	89.000	90.000	-	0.00%	-
531.deepsjeng_r (avg)	1.12%	89.000	90.000	-	0.00%	-

Performance Improvements - Execution	Δ	Previous	Current	σ	Δ (B)	σ (B)
511.povray_r	-1.49%	17732.000	17468.000	-	0.00%	-

Performance Improvements - sample variability	Δ	Previous	Current	σ	Δ (B)	σ (B)
505.mcf_r (max)	-6.90%	29.000	27.000	-	0.00%	-
531.deepsjeng_r (max) (percentage)	-4.85%	2.270	2.160	-	0.00%	-
531.deepsjeng_r (max)	-3.47%	259.000	250.000	-	0.00%	-

Added Tests - sample variability
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
508.namd_r (unexplained diff vs previous)
508.namd_r (unexplained diff vs previous) (percentage)
510.parest_r
510.parest_r (avg)
510.parest_r (avg) (percentage)
510.parest_r (max)
510.parest_r (max) (percentage)
510.parest_r (unexplained diff vs previous)
510.parest_r (unexplained diff vs previous) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
520.omnetpp_r (unexplained diff vs previous)
520.omnetpp_r (unexplained diff vs previous) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
525.x264_r (unexplained diff vs previous)
525.x264_r (unexplained diff vs previous) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
541.leela_r (unexplained diff vs previous)
541.leela_r (unexplained diff vs previous) (percentage)
557.xz_r (unexplained diff vs previous)
557.xz_r (unexplained diff vs previous) (percentage)

Run-Over-Baseline Changes Detail

Performance Regressions - Code Size	Δ (B)	Baseline	Current	σ (B)	Δ	σ
505.mcf_r	4.81%	30264.000	31720.000	-	0.20%	-
531.deepsjeng_r	3.36%	100020.000	103376.000	-	0.22%	-
557.xz_r	1.65%	184775.000	187815.000	-	0.00%	-

Performance Regressions - Execution	Δ (B)	Baseline	Current	σ (B)	Δ	σ
531.deepsjeng_r	7.70%	10775.000	11605.000	-	1.47%	-

Performance Improvements - Execution	Δ (B)	Baseline	Current	σ (B)	Δ	σ
557.xz_r	-3.03%	10353.000	10039.000	-	0.15%	-

Performance Improvements - Score	Δ (B)	Baseline	Current	σ (B)	Δ	σ
nb_execution_successful	433.33%	3.000	16.000	-	0.00%	-

Added Tests - Code Size
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - Execution
500.perlbench_r
502.gcc_r
508.namd_r
510.parest_r
511.povray_r
519.lbm_r
520.omnetpp_r
523.xalancbmk_r
525.x264_r
526.blender_r
538.imagick_r
541.leela_r
544.nab_r

Added Tests - sample variability
505.mcf_r
505.mcf_r (avg)
505.mcf_r (avg) (percentage)
505.mcf_r (max)
505.mcf_r (max) (percentage)
508.namd_r
508.namd_r (avg)
508.namd_r (avg) (percentage)
508.namd_r (max)
508.namd_r (max) (percentage)
508.namd_r (unexplained diff vs previous)
508.namd_r (unexplained diff vs previous) (percentage)
510.parest_r
510.parest_r (avg)
510.parest_r (avg) (percentage)
510.parest_r (max)
510.parest_r (max) (percentage)
510.parest_r (unexplained diff vs previous)
510.parest_r (unexplained diff vs previous) (percentage)
519.lbm_r
519.lbm_r (avg)
519.lbm_r (avg) (percentage)
519.lbm_r (max)
519.lbm_r (max) (percentage)
519.lbm_r (unexplained diff vs previous)
519.lbm_r (unexplained diff vs previous) (percentage)
520.omnetpp_r
520.omnetpp_r (avg)
520.omnetpp_r (avg) (percentage)
520.omnetpp_r (max)
520.omnetpp_r (max) (percentage)
520.omnetpp_r (unexplained diff vs previous)
520.omnetpp_r (unexplained diff vs previous) (percentage)
525.x264_r
525.x264_r (avg)
525.x264_r (avg) (percentage)
525.x264_r (max)
525.x264_r (max) (percentage)
525.x264_r (unexplained diff vs previous)
525.x264_r (unexplained diff vs previous) (percentage)
531.deepsjeng_r
531.deepsjeng_r (avg)
531.deepsjeng_r (avg) (percentage)
531.deepsjeng_r (max)
531.deepsjeng_r (max) (percentage)
541.leela_r
541.leela_r (avg)
541.leela_r (avg) (percentage)
541.leela_r (max)
541.leela_r (max) (percentage)
541.leela_r (unexplained diff vs previous)
541.leela_r (unexplained diff vs previous) (percentage)
557.xz_r
557.xz_r (avg)
557.xz_r (avg) (percentage)
557.xz_r (max)
557.xz_r (max) (percentage)

Report Time: 0.21s

Execution (seconds)

Name	Current	%
500.perlbench_r	15802.000	2.19%
502.gcc_r	9358.000	-
505.mcf_r	13916.000	-
508.namd_r	9111.000	-
510.parest_r	20811.000	-
511.povray_r	17468.000	-1.49%
519.lbm_r	15278.000	-
520.omnetpp_r	11772.000	-
523.xalancbmk_r	9424.000	-
525.x264_r	9587.000	-
526.blender_r	10615.000	-
531.deepsjeng_r	11605.000	1.47%
538.imagick_r	16715.000	-
541.leela_r	11033.000	-
544.nab_r	11614.000	-
557.xz_r	10039.000	-
Geometric Mean	12351.267	-

Code Size (bytes)

Name	Current	%
500.perlbench_r	2130519.000	-
502.gcc_r	8921755.000	-
505.mcf_r	31720.000	-
508.namd_r	640643.000	-
510.parest_r	8066547.000	-
511.povray_r	941886.000	-
519.lbm_r	13336.000	-
520.omnetpp_r	1719714.000	-
523.xalancbmk_r	4037263.000	-
525.x264_r	575701.000	-
526.blender_r	10703654.000	-
531.deepsjeng_r	103376.000	-
538.imagick_r	1817536.000	-
541.leela_r	148281.000	-
544.nab_r	192019.000	-
557.xz_r	187815.000	-
Geometric Mean	670842.743	-

Score

Name	Current	%
nb_compile_successful	16.000	-
nb_execution_successful	16.000	-
nb_total_tests	16.000	-
Geometric Mean	16.000	-

sample variability

Name	Current	%
505.mcf_r	22.000	-
505.mcf_r (avg)	22.000	-
505.mcf_r (avg) (percentage)	0.160	-
505.mcf_r (max)	27.000	-6.90%
505.mcf_r (max) (percentage)	0.200	-
508.namd_r	3.000	-
508.namd_r (avg)	3.000	-
508.namd_r (avg) (percentage)	0.040	-
508.namd_r (max)	3.000	-
508.namd_r (max) (percentage)	0.040	-
508.namd_r (unexplained diff vs previous)	4.000	-
508.namd_r (unexplained diff vs previous) (percentage)	0.040	-
510.parest_r	4.000	-
510.parest_r (avg)	4.000	-
510.parest_r (avg) (percentage)	0.020	-
510.parest_r (max)	4.000	-
510.parest_r (max) (percentage)	0.020	-
510.parest_r (unexplained diff vs previous)	5.000	-
510.parest_r (unexplained diff vs previous) (percentage)	0.020	-
519.lbm_r	19.000	-
519.lbm_r (avg)	19.000	-
519.lbm_r (avg) (percentage)	0.130	-
519.lbm_r (max)	19.000	-
519.lbm_r (max) (percentage)	0.130	-
519.lbm_r (unexplained diff vs previous)	20.000	-
519.lbm_r (unexplained diff vs previous) (percentage)	0.130	-
520.omnetpp_r	56.000	-
520.omnetpp_r (avg)	56.000	-
520.omnetpp_r (avg) (percentage)	0.480	-
520.omnetpp_r (max)	56.000	-
520.omnetpp_r (max) (percentage)	0.480	-
520.omnetpp_r (unexplained diff vs previous)	57.000	-
520.omnetpp_r (unexplained diff vs previous) (percentage)	0.480	-
525.x264_r	1.000	-
525.x264_r (avg)	1.000	-
525.x264_r (avg) (percentage)	0.020	-
525.x264_r (max)	1.000	-
525.x264_r (max) (percentage)	0.020	-
525.x264_r (unexplained diff vs previous)	2.000	-
525.x264_r (unexplained diff vs previous) (percentage)	0.020	-
531.deepsjeng_r	90.000	1.12%
531.deepsjeng_r (avg)	90.000	1.12%
531.deepsjeng_r (avg) (percentage)	0.780	-
531.deepsjeng_r (max)	250.000	-3.47%
531.deepsjeng_r (max) (percentage)	2.160	-4.85%
541.leela_r	12.000	-
541.leela_r (avg)	12.000	-
541.leela_r (avg) (percentage)	0.110	-
541.leela_r (max)	12.000	-
541.leela_r (max) (percentage)	0.110	-
541.leela_r (unexplained diff vs previous)	12.000	-
541.leela_r (unexplained diff vs previous) (percentage)	0.110	-
557.xz_r	12.000	-
557.xz_r (avg)	12.000	-
557.xz_r (avg) (percentage)	0.120	-
557.xz_r (max)	15.000	50.00%
557.xz_r (max) (percentage)	0.150	50.00%
557.xz_r (unexplained diff vs previous)	15.000	-
557.xz_r (unexplained diff vs previous) (percentage)	0.150	-
Geometric Mean	1.568	-68.58%

Show Delta
Show Previous Value
Show Standard Deviation
Show Median Absolute Deviation
Show All Values
Show All Samples
Show Sample Counts
Show Small Differences
Number of Comparison Runs
Show Report Graphs
Show Data Table
Hide Report By Default
Test Filter (regexp)
Test Min. Value Filter
Aggregation Function
Mann-Whitney test confidence level

Runs:

Compare To:

Fields

Parameters

Fields

Parameters

llvm-arm-master-O3 test results

Tests Summary

Run-Over-Run Changes Detail

Run-Over-Baseline Changes Detail