From patchwork Fri Jan 9 18:31:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1391 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767983555; bh=zarmVXiNtrEtZXh/fK78NsL9AIMYoPW2VlMVdfo53ao=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=w2kB24hk8iMuaK2LqGgcEaiBbd1NTndMdOuV6NFdItApfh5wKyKQVaT+nSD7exjnk 1SAfLbn+3P/FUraa7xvg6tIe7ry7R6JUFMmTfOBrXCvMeKG1hFS9YAaQnvETIxDw57 d8j+SPShuR1bozlp/wDc8EO8C9/saB/sOvRa57xYKiquk22QjHyxxSAr1pgk2j2aY/ ZUmaNUZsBgvukDjeAKrDSOu4kWlBEdX/1p+5+d9xwB9PTgwQeSKE8544/k2Ar6P81J QVL3M7DvmVCUKmtvuudHxFnZ/xvse7p31J1c21nzqMqITXPNVIUEPA8TWfy0X71cjK NpOrY/hDUciKA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7A0FB69221 for ; Fri, 9 Jan 2026 11:32:35 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Ly6ac3aQdGHp for ; Fri, 9 Jan 2026 11:32:35 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767983555; bh=zarmVXiNtrEtZXh/fK78NsL9AIMYoPW2VlMVdfo53ao=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=w2kB24hk8iMuaK2LqGgcEaiBbd1NTndMdOuV6NFdItApfh5wKyKQVaT+nSD7exjnk 1SAfLbn+3P/FUraa7xvg6tIe7ry7R6JUFMmTfOBrXCvMeKG1hFS9YAaQnvETIxDw57 d8j+SPShuR1bozlp/wDc8EO8C9/saB/sOvRa57xYKiquk22QjHyxxSAr1pgk2j2aY/ ZUmaNUZsBgvukDjeAKrDSOu4kWlBEdX/1p+5+d9xwB9PTgwQeSKE8544/k2Ar6P81J QVL3M7DvmVCUKmtvuudHxFnZ/xvse7p31J1c21nzqMqITXPNVIUEPA8TWfy0X71cjK NpOrY/hDUciKA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6937A69216 for ; Fri, 9 Jan 2026 11:32:35 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767983553; bh=z6FeyXiaPsRQDBzjpd70n4IF6wDqiuFBZ6DU1FhtDG8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fV5SII1MzKGN3rvpXkcVLoPwXiEK8UCbweQ4OzknZAmtapOf88vsFzfsseH7UiWu8 ZeSOsvfBk8Z7+ic2EaY0AO/2gg7ECIDkDQT4aBI+vAK44lAm2IbLBNHFLSmhoEh60n we4aNcjhkYGKQFVKK76iks/v62c5DTPYGXwd30ma4vJIwzgsFGfJgrpdlZkR05tQpP DOjWr291hH8vzMXfgSllPNmZlXj1zz/oy5l3UO58ltBiwq2Ms78pdNHR/Pgw+AU5lz wwDS73I7iYXD7jqwyESmAlJSoHQwey4A3i3ytfw6LPnWjBfcu7BB6ooTIWO7pxeNnK FQjUK9NQtXWSw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id E80566920A; Fri, 9 Jan 2026 11:32:33 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id rcA2m_tukLkx; Fri, 9 Jan 2026 11:32:33 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767983549; bh=NU1uUlAWOXBg5PAV0K1mMtWVcsDFMfo+x3Oqd9AmHsM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GLhhE5lYjw/ZitlEZrVWVQ040YRNLMVCEvBGLyk3jscBQq0yxBXRoSMSTpbHvuqW0 q9B3qe2YoTqvhHmpkclXGB+VOw5ErwKm00uihQ9idPbINCgzYR5kWidbIOtcBpcvnz B3G6eqSy7gpcqrj18ELojuP3N59m5RcOwhGUoKfeKjL+5kaw0jEsvie7T1AUMGcEmD WbTwm/qfvweADzqaPKk+4oHFM464ay8z0h6W2UcqhgDctI69LlnQb3cXqmq9tS+MJa RGyeBkXPk+d13fOmPDYc5oDzoAvxQvS3qGFuzuuRNAYro2/T44Qy4h5qTryn+Af2pT /KAc/caTnFu1w== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 670D469209; Fri, 9 Jan 2026 11:32:29 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Fri, 9 Jan 2026 11:31:05 -0700 Message-ID: <20260109183116.3262115-14-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260109183116.3262115-1-sjg@u-boot.org> References: <20260109183116.3262115-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: SJHAEYEQPELVK7SLIDJGD3FVWFMIV4DC X-Message-ID-Hash: SJHAEYEQPELVK7SLIDJGD3FVWFMIV4DC X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 13/18] buildman: Test _show_not_built() and toolchain errors List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add testToolchainErrors() functional test to verify that boards with missing toolchains are reported in the summary display, grouped by architecture. Add TestShowNotBuilt unit test class with tests for: - All boards built (no output) - Some boards not built (reports missing boards) - No boards built (reports all boards) Update the docstring for _show_not_built() to clarify that it reports boards missing from board_dict. Note that in practice this is unlikely to trigger since get_result_summary() creates an outcome for every board. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- tools/buildman/builder.py | 5 ++++ tools/buildman/func_test.py | 25 ++++++++++++++++ tools/buildman/main.py | 1 + tools/buildman/test_builder.py | 53 ++++++++++++++++++++++++++++++++++ 4 files changed, 84 insertions(+) diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index 3cb14af279f..99aac80d95e 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -1938,6 +1938,11 @@ class Builder: def _show_not_built(board_selected, board_dict): """Show boards that were not built + This reports boards that are in board_selected but have no outcome in + board_dict. In practice this is unlikely to happen since + get_result_summary() creates an outcome for every board, even if just + OUTCOME_UNKNOWN. + Args: board_selected (dict): Dict of selected boards, keyed by target board_dict (dict): Dict of boards that were built, keyed by target diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index efd66796a65..21c700aa073 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -590,6 +590,31 @@ Some images are invalid''' f"No tool chain found for arch '{brd.arch}'"]) fd.close() + def testToolchainErrors(self): + """Test that toolchain errors are reported in the summary + + When toolchains are missing, boards fail to build. The summary + should report which boards had errors, grouped by architecture. + """ + self.setupToolchains() + # Build with missing toolchains - only sandbox will succeed + self._RunControl('-b', TEST_BRANCH, '-o', self._output_dir) + + # Now show summary - should report boards with errors + terminal.get_print_test_lines() # Clear + self._RunControl('-b', TEST_BRANCH, '-o', self._output_dir, '-s', + clean_dir=False) + lines = terminal.get_print_test_lines() + text = '\n'.join(line.text for line in lines) + + # Check that boards with missing toolchains are shown with errors + # The '+' prefix indicates new errors for these boards + self.assertIn('arm:', text) + self.assertIn('board0', text) + self.assertIn('board1', text) + self.assertIn('powerpc:', text) + self.assertIn('board2', text) + def testBranch(self): """Test building a branch with all toolchains present""" self._RunControl('-b', TEST_BRANCH, '-o', self._output_dir) diff --git a/tools/buildman/main.py b/tools/buildman/main.py index 449263b48e8..18809d843c6 100755 --- a/tools/buildman/main.py +++ b/tools/buildman/main.py @@ -56,6 +56,7 @@ def run_tests(skip_net_tests, debug, verbose, args): test_builder.TestPrintFuncSizeDetail, test_builder.TestPrepareThread, test_builder.TestPrepareWorkingSpace, + test_builder.TestShowNotBuilt, test_builder.TestPrepareOutputSpace, 'buildman.toolchain']) diff --git a/tools/buildman/test_builder.py b/tools/buildman/test_builder.py index b17d7942e1d..d31c0080863 100644 --- a/tools/buildman/test_builder.py +++ b/tools/buildman/test_builder.py @@ -339,6 +339,59 @@ class TestPrepareWorkingSpace(unittest.TestCase): mock_prepare_thread.assert_any_call(1, True) +class TestShowNotBuilt(unittest.TestCase): + """Tests for Builder._show_not_built()""" + + def setUp(self): + """Set up test fixtures""" + terminal.set_print_test_mode() + + def tearDown(self): + """Clean up after tests""" + terminal.set_print_test_mode(False) + + def test_all_boards_built(self): + """Test when all selected boards were built""" + board_selected = {'board1': None, 'board2': None} + board_dict = {'board1': None, 'board2': None} + + terminal.get_print_test_lines() # Clear + builder.Builder._show_not_built(board_selected, board_dict) + lines = terminal.get_print_test_lines() + + # No output when all boards were built + self.assertEqual(len(lines), 0) + + def test_some_boards_not_built(self): + """Test when some boards were not built""" + board_selected = {'board1': None, 'board2': None, 'board3': None} + board_dict = {'board1': None} # Only board1 was built + + terminal.get_print_test_lines() # Clear + builder.Builder._show_not_built(board_selected, board_dict) + lines = terminal.get_print_test_lines() + + self.assertEqual(len(lines), 1) + self.assertIn('Boards not built', lines[0].text) + self.assertIn('2', lines[0].text) # Count of not-built boards + self.assertIn('board2', lines[0].text) + self.assertIn('board3', lines[0].text) + + def test_no_boards_built(self): + """Test when no boards were built""" + board_selected = {'board1': None, 'board2': None} + board_dict = {} # No boards built + + terminal.get_print_test_lines() # Clear + builder.Builder._show_not_built(board_selected, board_dict) + lines = terminal.get_print_test_lines() + + self.assertEqual(len(lines), 1) + self.assertIn('Boards not built', lines[0].text) + self.assertIn('board1', lines[0].text) + self.assertIn('board2', lines[0].text) + + class TestPrepareOutputSpace(unittest.TestCase): """Tests for Builder._prepare_output_space() and _get_output_space_removals()"""