Machine Info

Fields

hardware	None
os	None

Parameters

Run Info

Fields

Name	Value

Parameters

Name	Value
git_binutils	https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=328e01595436e937a0cc00cc8451f17beda26799 (+ 28 commits)
git_gcc	https://github.com/gcc-mirror/gcc/commit/2d44ab221f64f01fc676be0da1a6774740d713c6 (+ 46 commits)
git_gdb	https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=a0a97b8cac56ea498bfcfa0afbbe943bf79b7894 (unchanged)
git_newlib	https://sourceware.org/git/?p=newlib-cygwin.git;a=commit;h=7863c07a9282b7aed358c5783976296adfa63505 (unchanged)
git_qemu	https://gitlab.com/qemu-project/qemu/commit/a3c3aaa846ad61b801e7196482dcf4afb8ba34e4 (unchanged)
test_url	https://ci.linaro.org/job/tcwg_gnu_embed_check_gcc--master-arm_eabi-build/453/

Report

Filter

master-arm_eabi test results

Run	Order	Start Time	Duration
Current	0453 (2-changed-components)	2023-11-09T10:28:23	0:00:00
Previous	0033 (2-changed-components)	2023-06-07T03:56:44	0:00:00
Baseline	0011 (5-changed-components)	2023-06-05T17:27:31	0:00:00

Tests Summary

Status Group	#	# (B)
Performance Regressions	9	12
Performance Improvements	21	17
Added Tests	19	19
Unchanged Tests	896	897
Total Tests	945

Run-Over-Run Changes Detail

Performance Regressions - UNSUPPORTED	Δ	Previous	Current	σ	Δ (B)	σ (B)
gcc.UNSUPPORTED	5.09%	8256.000	8676.000	-	5.09%	-
TOTAL.UNSUPPORTED	2.36%	20562.000	21047.000	-	0.36%	-
g++.UNSUPPORTED	1.11%	11193.000	11317.000	-	1.12%	-

Performance Regressions - XFAIL	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.XFAIL	2083.33%	6.000	131.000	-	670.59%	-
g++.XFAIL	44.17%	1055.000	1521.000	-	44.17%	-
TOTAL.XFAIL	31.62%	1980.000	2606.000	-	29.46%	-
gcc.XFAIL	3.81%	919.000	954.000	-	3.81%	-

Performance Regressions - XPASS	Δ	Previous	Current	σ	Δ (B)	σ (B)
g++.XPASS	130.00%	10.000	23.000	-	130.00%	-
TOTAL.XPASS	93.75%	16.000	31.000	-	72.22%	-

Performance Improvements - FAIL	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.FAIL	-99.73%	4145.000	11.000	-	-97.61%	-
TOTAL.FAIL	-96.49%	4411.000	155.000	-	-99.17%	-
g++.FAIL	-48.15%	189.000	98.000	-	-48.15%	-
gcc.FAIL	-40.26%	77.000	46.000	-	-40.26%	-

Performance Improvements - PASS	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.PASS	230.69%	4650.000	15377.000	-	12.78%	-
TOTAL.PASS	6.81%	352236.000	376221.000	-	-1.60%	-
g++.PASS	4.87%	198525.000	208197.000	-	4.89%	-
gcc.PASS	2.41%	149061.000	152647.000	-	2.43%	-

Performance Improvements - UNRESOLVED	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.UNRESOLVED	-99.93%	4366.000	3.000	-	-99.98%	-
g++.UNRESOLVED	-97.99%	149.000	3.000	-	-97.99%	-

Performance Improvements - UNSUPPORTED	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.UNSUPPORTED	-5.30%	1113.000	1054.000	-	61.41%	-

Performance Improvements - all	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.all	4.30%	383572.000	400063.000	-	-9.56%	-

Performance Improvements - bad	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.bad	-99.84%	8244.000	13.000	-	-97.95%	-
TOTAL.bad	-97.85%	8794.000	189.000	-	-99.49%	-
gcc.bad	-74.26%	202.000	52.000	-	-74.26%	-
g++.bad	-64.37%	348.000	124.000	-	-64.37%	-

Performance Improvements - fails	Δ	Previous	Current	σ	Δ (B)	σ (B)
TOTAL.fails	-97.83%	8709.000	189.000	-	-99.46%	-

Performance Improvements - good	Δ	Previous	Current	σ	Δ (B)	σ (B)
libstdc++.good	187.09%	5769.000	16562.000	-	15.79%	-
TOTAL.good	6.70%	374778.000	399874.000	-	-1.34%	-
g++.good	4.87%	210773.000	221035.000	-	4.88%	-
gcc.good	2.55%	158236.000	162277.000	-	2.58%	-

Added Tests - ERROR
TOTAL.ERROR
g++.ERROR
gcc.ERROR
libstdc++.ERROR

Added Tests - KFAIL
TOTAL.KFAIL
g++.KFAIL
gcc.KFAIL
libstdc++.KFAIL

Added Tests - OTHER
TOTAL.OTHER
g++.OTHER
gcc.OTHER
libstdc++.OTHER

Added Tests - UNRESOLVED
gcc.UNRESOLVED
libstdc++.UNRESOLVED

Added Tests - UNTESTED
TOTAL.UNTESTED
g++.UNTESTED
gcc.UNTESTED
libstdc++.UNTESTED

Added Tests - flaky
TOTAL.flaky

Run-Over-Baseline Changes Detail

Performance Regressions - PASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.PASS	-1.60%	382336.000	376221.000	-	6.81%	-

Performance Regressions - UNSUPPORTED	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.UNSUPPORTED	61.41%	653.000	1054.000	-	-5.30%	-
gcc.UNSUPPORTED	5.09%	8256.000	8676.000	-	5.09%	-
g++.UNSUPPORTED	1.12%	11192.000	11317.000	-	1.11%	-

Performance Regressions - XFAIL	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.XFAIL	670.59%	17.000	131.000	-	2083.33%	-
g++.XFAIL	44.17%	1055.000	1521.000	-	44.17%	-
TOTAL.XFAIL	29.46%	2013.000	2606.000	-	31.62%	-
gcc.XFAIL	3.81%	919.000	954.000	-	3.81%	-

Performance Regressions - XPASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
g++.XPASS	130.00%	10.000	23.000	-	130.00%	-
TOTAL.XPASS	72.22%	18.000	31.000	-	93.75%	-

Performance Regressions - all	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.all	-9.56%	442374.000	400063.000	-	4.30%	-

Performance Regressions - good	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.good	-1.34%	405320.000	399874.000	-	6.70%	-

Performance Improvements - FAIL	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.FAIL	-99.17%	18705.000	155.000	-	-96.49%	-
libstdc++.FAIL	-97.61%	460.000	11.000	-	-99.73%	-
g++.FAIL	-48.15%	189.000	98.000	-	-48.15%	-
gcc.FAIL	-40.26%	77.000	46.000	-	-40.26%	-

Performance Improvements - PASS	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.PASS	12.78%	13634.000	15377.000	-	230.69%	-
g++.PASS	4.89%	198496.000	208197.000	-	4.87%	-
gcc.PASS	2.43%	149024.000	152647.000	-	2.41%	-

Performance Improvements - UNRESOLVED	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.UNRESOLVED	-99.98%	16578.000	3.000	-	-99.93%	-
g++.UNRESOLVED	-97.99%	149.000	3.000	-	-97.99%	-

Performance Improvements - bad	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.bad	-99.49%	37054.000	189.000	-	-97.85%	-
libstdc++.bad	-97.95%	634.000	13.000	-	-99.84%	-
gcc.bad	-74.26%	202.000	52.000	-	-74.26%	-
g++.bad	-64.37%	348.000	124.000	-	-64.37%	-

Performance Improvements - fails	Δ (B)	Baseline	Current	σ (B)	Δ	σ
TOTAL.fails	-99.46%	35217.000	189.000	-	-97.83%	-

Performance Improvements - good	Δ (B)	Baseline	Current	σ (B)	Δ	σ
libstdc++.good	15.79%	14304.000	16562.000	-	187.09%	-
g++.good	4.88%	210743.000	221035.000	-	4.87%	-
gcc.good	2.58%	158199.000	162277.000	-	2.55%	-

Added Tests - ERROR
TOTAL.ERROR
g++.ERROR
gcc.ERROR
libstdc++.ERROR

Added Tests - KFAIL
TOTAL.KFAIL
g++.KFAIL
gcc.KFAIL
libstdc++.KFAIL

Added Tests - OTHER
TOTAL.OTHER
g++.OTHER
gcc.OTHER
libstdc++.OTHER

Added Tests - UNRESOLVED
gcc.UNRESOLVED
libstdc++.UNRESOLVED

Added Tests - UNTESTED
TOTAL.UNTESTED
g++.UNTESTED
gcc.UNTESTED
libstdc++.UNTESTED

Added Tests - flaky
TOTAL.flaky

Report Time: 0.19s

PASS

	Name	Current	%
	TOTAL.PASS	376221.000	6.81%
	g++.PASS	208197.000	4.87%
	gcc.PASS	152647.000	2.41%
	libstdc++.PASS	15377.000	230.69%
	Geometric Mean	116444.598	39.56%

XPASS

	Name	Current	%
	TOTAL.XPASS	31.000	93.75%
	g++.XPASS	23.000	130.00%
	gcc.XPASS	6.000	-
	libstdc++.XPASS	2.000	-
	Geometric Mean	9.618	1628.14%

FAIL

	Name	Current	%
	TOTAL.FAIL	155.000	-96.49%
	g++.FAIL	98.000	-48.15%
	gcc.FAIL	46.000	-40.26%
	libstdc++.FAIL	11.000	-99.73%
	Geometric Mean	52.654	-92.67%

XFAIL

	Name	Current	%
	TOTAL.XFAIL	2606.000	31.62%
	g++.XFAIL	1521.000	44.17%
	gcc.XFAIL	954.000	3.81%
	libstdc++.XFAIL	131.000	2083.33%
	Geometric Mean	838.940	156.08%

KFAIL

	Name	Current	%
	TOTAL.KFAIL	0.000	-
	g++.KFAIL	0.000	-
	gcc.KFAIL	0.000	-
	libstdc++.KFAIL	0.000	-
	Geometric Mean	0.000	-

UNSUPPORTED

	Name	Current	%
	TOTAL.UNSUPPORTED	21047.000	2.36%
	g++.UNSUPPORTED	11317.000	1.11%
	gcc.UNSUPPORTED	8676.000	5.09%
	libstdc++.UNSUPPORTED	1054.000	-5.30%
	Geometric Mean	6831.568	-

UNRESOLVED

	Name	Current	%
	TOTAL.UNRESOLVED	3.000	-99.93%
	g++.UNRESOLVED	3.000	-97.99%
	gcc.UNRESOLVED	0.000	-
	libstdc++.UNRESOLVED	0.000	-
	Geometric Mean	0.017	-100.00%

UNTESTED

	Name	Current	%
	TOTAL.UNTESTED	0.000	-
	g++.UNTESTED	0.000	-
	gcc.UNTESTED	0.000	-
	libstdc++.UNTESTED	0.000	-
	Geometric Mean	0.000	-

ERROR

	Name	Current	%
	TOTAL.ERROR	0.000	-
	g++.ERROR	0.000	-
	gcc.ERROR	0.000	-
	libstdc++.ERROR	0.000	-
	Geometric Mean	0.000	-

OTHER

	Name	Current	%
	TOTAL.OTHER	0.000	-
	g++.OTHER	0.000	-
	gcc.OTHER	0.000	-
	libstdc++.OTHER	0.000	-
	Geometric Mean	0.000	-

good

	Name	Current	%
	TOTAL.good	399874.000	6.70%
	g++.good	221035.000	4.87%
	gcc.good	162277.000	2.55%
	libstdc++.good	16562.000	187.09%
	Geometric Mean	124147.647	34.72%

bad

	Name	Current	%
	TOTAL.bad	189.000	-97.85%
	g++.bad	124.000	-64.37%
	gcc.bad	52.000	-74.26%
	libstdc++.bad	13.000	-99.84%
	Geometric Mean	63.090	-95.80%

fails

	Name	Current	%
	TOTAL.fails	189.000	-97.83%
	Geometric Mean	189.000	-97.83%

flaky

	Name	Current	%
	TOTAL.flaky	0.000	-
	Geometric Mean	0.000	-

all

	Name	Current	%
	TOTAL.all	400063.000	4.30%
	Geometric Mean	400063.000	4.30%