2025-10-24 00:20:37,252 INFO: Started file logging. [in app.py:269]
2025-10-24 01:21:54,102 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 01:24:34,437 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 02:19:39,619 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 04:11:05,545 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 04:11:10,196 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-24 04:11:11,230 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-24 06:20:05,683 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 07:21:02,185 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 08:21:52,372 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 12:21:09,219 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 12:22:16,662 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 13:20:00,040 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 14:26:26,694 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 16:09:10,566 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 16:09:57,039 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 18:15:55,026 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 19:16:14,558 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 21:13:27,564 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 21:17:42,153 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 22:09:32,385 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-24 22:45:23,270 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-25 00:04:21,741 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 00:04:38,918 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 00:04:44,268 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 02:03:24,583 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 03:03:11,967 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 03:27:02,087 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 03:27:07,543 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 03:27:07,848 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 03:27:08,174 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 03:27:08,518 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 04:04:57,445 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 06:04:34,757 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 08:04:26,766 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 08:05:52,546 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:03:43,386 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:03:54,556 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:04:41,352 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:04:53,650 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:25:10,477 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 09:25:17,698 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 09:25:18,049 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 09:25:18,600 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 09:25:20,567 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-25 10:04:51,698 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 11:03:43,693 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 11:04:42,826 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-25 14:07:16,777 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 03:25:17,328 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 03:25:19,925 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-26 03:25:20,981 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-26 03:25:21,339 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-26 03:25:21,698 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-26 20:42:47,670 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,691 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,722 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,751 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,779 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,790 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,818 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 20:42:47,841 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,499 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,519 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,548 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,577 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,603 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,615 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,641 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:38,662 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,683 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,702 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,731 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,758 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,787 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,798 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,826 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:17:54,848 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,545 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,564 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,636 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,665 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,691 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,702 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,727 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-26 21:19:11,748 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 01:16:15,832 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 01:16:18,808 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 01:16:20,689 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 03:53:45,068 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,087 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,116 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,143 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,170 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,181 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,208 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 03:53:45,230 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 04:29:47,880 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 07:32:27,530 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 11:39:46,729 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-27 12:37:58,797 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 12:38:52,630 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 13:30:55,822 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 13:34:49,627 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 18:24:00,483 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 19:52:19,822 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 20:05:54,730 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 20:05:55,858 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 20:05:56,546 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 20:05:56,913 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 22:08:36,646 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-27 22:08:39,998 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 22:08:40,746 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 22:08:41,437 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-27 22:08:42,144 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-28 03:57:39,675 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-28 04:29:28,656 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-28 04:29:35,538 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-28 06:26:41,763 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-28 10:26:41,802 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-28 12:27:05,109 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-28 17:22:54,029 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-29 00:32:48,853 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 294, in v4_machine_latest
    return v4_redirect(v4_url_for('.v4_run', id=run.id, **request.args))
                                                ^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'
 [in app.py:134]
2025-10-29 00:39:51,112 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 08:19:33,207 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 08:19:36,332 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 08:19:36,684 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 08:19:37,371 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 08:19:37,722 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 09:00:30,419 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-29 10:23:29,449 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 11:43:18,633 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 11:50:37,933 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 12:34:47,861 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 12:34:48,553 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 12:34:49,274 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 12:34:50,057 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 12:53:49,154 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-29 13:08:37,419 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 13:08:38,100 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 13:08:39,071 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 13:08:40,783 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 13:21:42,018 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 13:25:06,837 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 13:27:49,343 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 14:06:29,982 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 14:28:29,413 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 16:42:15,520 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 16:53:00,325 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-29 18:51:28,084 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 18:51:32,426 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 18:51:33,517 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 22:01:38,359 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-29 22:46:43,097 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-29 22:46:46,581 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 22:46:47,450 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 22:46:48,675 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 22:46:49,836 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-29 23:42:01,646 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 02:34:06,536 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 584, in v4_run
    return render_template("v4_run.html", **data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 133, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 96, in <module>
    <td>{{cr.samples|length}}</td>
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/layout.html", line 255, in <module>
    {% block sidebar %}{% endblock %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/templates/v4_run.html", line 124, in <module>
    {% if comparison_neighboring_runs %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'lnt.server.ui.views.V4RequestInfo object' has no attribute 'comparison_neighboring_runs'
 [in app.py:134]
2025-10-30 03:43:01,251 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 08:40:59,781 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 10:16:38,989 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 10:16:41,449 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-30 10:16:43,807 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-30 10:16:44,360 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-30 10:16:45,560 ERROR: Internal Error [in __init__.py:309]
Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 471, in wrapper
    resp = resource(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/views.py", line 84, in view
    return self.dispatch_request(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 581, in dispatch_request
    resp = meth(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 76, in wrap
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/api.py", line 441, in get
    .join(ts.Run) \
     ^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
           ^^^^^^^^^^^
  File "<string>", line 2, in _join
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/sqlalchemy/orm/query.py", line 2747, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
2025-10-30 15:23:31,140 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 294, in v4_machine_latest
    return v4_redirect(v4_url_for('.v4_run', id=run.id, **request.args))
                                                ^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'
 [in app.py:134]
2025-10-30 17:06:39,244 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 17:57:00,767 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 313, in v4_machine_compare
    return v4_redirect(v4_url_for('.v4_run', id=machine_1_run.id,
                                                ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'
 [in app.py:134]
2025-10-30 17:59:06,899 ERROR: Exception: Traceback (most recent call last):
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask_restful/__init__.py", line 270, in error_router
    return original_handler(e)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/sandbox/lib/python3.12/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/decorators.py", line 54, in wrap
    return f(**args)
           ^^^^^^^^^
  File "/home/tcwg-buildslave/tcwg-lnt-01/lntserver/llvm-lnt/lnt/server/ui/views.py", line 313, in v4_machine_compare
    return v4_redirect(v4_url_for('.v4_run', id=machine_1_run.id,
                                                ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'id'
 [in app.py:134]
2025-10-30 18:08:31,439 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 19:00:34,474 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]
2025-10-30 19:01:16,857 INFO: timer: 'get_last_order_for_machine' 0.00 sec [in commands.py:22]