From patchwork Mon Dec 29 16:06:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1107 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024420; bh=9+zkuFj45eXVfdKhaIIPsuh+a98JduRUqSvEVyYWLnA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=wgAutjZQWJlGeJsNq+idFeRwtfANJ2v5AsJvD3rxwTu1guSvYT+rdeTziOprlYmnb 6gyQ03KwZliLygcdJGu0abSk1oU1xSysjCYX2XzgRMdme1RwVZegy7+kJx/VHIfdWX mw2LhjvJLV9n+w8vpNiB1ZT7LGBB8DRXo9wAMt6O5BFMYFYyngFK/Po+2YjtxJfBum HgVBc9eTdMPoxK7eNs1uFBMYVQxF3pVGtH859JnwIRJYFl3r9ukrJhxTcyDGHknJJz DcYJUmV4dAv8uukMeNt0DvDnp5pc0RhN0wSqTDILLgaDhlSevS1HtoAyo25/9BCrnJ JQhajXNb+L8cQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CCA2068CAC for ; Mon, 29 Dec 2025 09:07:00 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 0MlOqNszcSIj for ; Mon, 29 Dec 2025 09:07:00 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024419; bh=9+zkuFj45eXVfdKhaIIPsuh+a98JduRUqSvEVyYWLnA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=H/9OwjSYlKUAfBF8C40e2fkDg0RmDmvX/gbcTbEK+KgmsWe61AhlvzomMM6XHj9M6 lC4XULFxwegQTt9GoncKXSNj989eAeAj51meWymuJ9OcHibpEk8y8Nk9asFn+AqdRw o4hVaxEDa7B/SJhPNnuudjxpgLAAR3/Zy9J9Li46B9k0acfd3SeSzEG5SQ9wNEFL/0 Lbkz1GtEGSqxKFXoszoRmOfsyiIhjpYyrdRGBCjQa4LIiXBQv/j8hMJYWRCBTjdAMI kEfbpE1Sv+c1Bulk2ZjxR+nlOQ4C9ikqxjwzJFIZhyr3mCHmzch+GcyRzfgkwzxHmd qarbrzetqLsNA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 8AF8D68F31 for ; Mon, 29 Dec 2025 09:06:59 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024416; bh=JBAO//IO0tWN/k9hqBj/ITw6PiWMNCv6HGfjBpQ9TCI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YEtmEg+EN63qASAStPvLzmUX+MGes0OellHy5wMOnXk/KN6v2poLJ0BKj11qmmlmU +3HvMFUJ8G8tLVkIuCRbLtsMHfGb1lmjYO+4l/UlWZsa/H3ByDKP303VyK603/PgCT SZEOfqlPfLnsibMnigowh7qzqeHGFqcjQogb151fIQR5QppQ26o0iKaAySzgtvshYj MFX2MTjRNNudzK1zeWbPIkkXPFYz12H37HJ9OHAsb9we5En4q1TeeDQnsCP1+AYYBT H3MyzV42ueTOmrG1oq9ErMrfkY1mgyiKhFQTdRjE/tJpCB7gRc651tIpsCktLTfFLq R2sM1l9hW0QwA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5DCD168CAC; Mon, 29 Dec 2025 09:06:56 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id jrkAjk0iFwaK; Mon, 29 Dec 2025 09:06:56 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024416; bh=s/2by+IzxTtQ6C2Q+qrlYKnL0cmarKYJP4sULWCD7UA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rQKWSFqJK3lTankH3cxjakYCIY9g48G+9ynAE3IKm7JAWwk79Db1IO84JO3p4QNPO fem+FC/zPYHs/AHYwLtGXF54yjztu+oso0E9YaljaiqomHOrEfKazR3gMQJMWsJHsI +ZTEgIt7oQOR7OhNlT7799HBGz7eNv+p87D8L5BsY4CkNsoLyJxufSORcAm+fMKlwj cmCN+WPGt7s1Q3NnHOTHZzr97gS7DwiTR/UhQpzPLUQN9h7SjEtrEXxAccXpZFYh7j J4LYbWOmcDC0R00oZOu0Vkq5nCkq+IYZ9GkeNaDanWivECOY2iTggR+h1bX+eyd0zw 3dfuK+MUKirNQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id A2D9264E24; Mon, 29 Dec 2025 09:06:55 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:07 -0700 Message-ID: <20251229160611.3899708-10-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 4NMF26EB25Q5L2A6ILPMJ2KBCYKY4V7Y X-Message-ID-Hash: 4NMF26EB25Q5L2A6ILPMJ2KBCYKY4V7Y X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 9/9] test: Add parallel test execution support List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Running the full test suite takes a long time. It would be useful to distribute tests across multiple sandbox instances to speed up testing. Add support for running tests in parallel across multiple sandbox instances. Each worker runs a subset of tests based on its worker ID. Add -P: option to the ut command where n is the total number of workers and w is this worker's ID (0 to n-1). Tests are distributed by index modulo number of workers. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/develop/py_testing.rst | 28 ++++++++++++++++++++++++++++ doc/usage/cmd/ut.rst | 8 +++++++- include/test/test.h | 4 ++++ test/cmd_ut.c | 16 +++++++++++++++- test/test-main.c | 4 ++++ 5 files changed, 58 insertions(+), 2 deletions(-) diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst index 303c07040b3..cfb8f959e12 100644 --- a/doc/develop/py_testing.rst +++ b/doc/develop/py_testing.rst @@ -171,6 +171,34 @@ require a large amount of refactoring, e.g. with more use of pytest fixtures. The code-coverage tests are omitted since they cannot run in parallel due to a Python limitation. +Parallel C unit tests +~~~~~~~~~~~~~~~~~~~~~ + +The ``ut`` command supports distributing tests across multiple sandbox +instances using the ``-P`` flag. This is useful when running tests directly +from the command line without pytest. + +To run tests in parallel across 4 workers:: + + # Terminal 1 + /tmp/b/sandbox/u-boot -T -c "ut -P4:0 dm" + + # Terminal 2 + /tmp/b/sandbox/u-boot -T -c "ut -P4:1 dm" + + # Terminal 3 + /tmp/b/sandbox/u-boot -T -c "ut -P4:2 dm" + + # Terminal 4 + /tmp/b/sandbox/u-boot -T -c "ut -P4:3 dm" + +The format is ``-P:`` where ``n`` is the total number of workers and +``w`` is this worker's ID (0 to n-1). Tests are distributed by index modulo +the number of workers, so each worker runs a disjoint subset. + +This can be combined with other flags, e.g. ``-EP4:0`` to emit result lines +while running as worker 0 of 4. + Testing under a debugger ~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index 731579e5b1d..99419c0d384 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-Efmr] [-R] [-I:] [ | all []] [...] + ut [-Efmr] [-R] [-I:] [-P:] [ | all []] [...] ut [-s] info Description @@ -44,6 +44,12 @@ test causes another test to fail. If the one test fails, testing stops immediately. +-P : + Run as worker `` of `` parallel workers. Tests are distributed by + index modulo number of workers, so each worker runs a disjoint subset of + tests. This allows running tests in parallel across multiple sandbox + instances. + -R Preserve console recording on test failure. Normally when a test fails, console recording is disabled so error messages go directly to output. diff --git a/include/test/test.h b/include/test/test.h index 74225a70e54..56e25f6fa9d 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -89,6 +89,8 @@ struct ut_arg { * @of_other: Live tree for the other FDT * @runs_per_test: Number of times to run each test (typically 1) * @force_run: true to run tests marked with the UTF_MANUAL flag + * @workers: Number of parallel workers, 0 if not sharding tests + * @worker_id: ID of this worker (0 to workers-1) * @old_bloblist: stores the old gd->bloblist pointer * @soft_fail: continue execution of the test even after it fails * @expect_str: Temporary string used to hold expected string value @@ -121,6 +123,8 @@ struct unit_test_state { struct device_node *of_other; int runs_per_test; bool force_run; + int workers; + int worker_id; void *old_bloblist; bool soft_fail; char expect_str[1024]; diff --git a/test/cmd_ut.c b/test/cmd_ut.c index d6e591916ce..a35ac69434d 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -257,6 +257,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) bool keep_record = false; bool emit_result = false; int runs_per_text = 1; + int workers = 0, worker_id = 0; struct suite *ste; char *name; int ret; @@ -285,6 +286,15 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) if (!strchr(test_insert, ':')) return CMD_RET_USAGE; goto next_arg; + case 'P': { + const char *colon = strchr(str + 1, ':'); + + if (!colon) + return CMD_RET_USAGE; + workers = dectoul(str + 1, NULL); + worker_id = dectoul(colon + 1, NULL); + goto next_arg; + } case 'R': keep_record = true; break; @@ -304,6 +314,8 @@ next_arg: ut_init_state(&uts); uts.keep_record = keep_record; uts.emit_result = emit_result; + uts.workers = workers; + uts.worker_id = worker_id; name = argv[0]; select_name = cmd_arg1(argc, argv); @@ -349,11 +361,13 @@ next_arg: } U_BOOT_LONGHELP(ut, - "[-Efmrs] [-R] [-I:] [ [...]] - run unit tests\n" + "[-Efmrs] [-R] [-I:] [-P:] [ [...]]\n" + " - run unit tests\n" " -E Emit result line after each test\n" " -r Number of times to run each test\n" " -f/-m Force 'manual' tests to run as well\n" " -I Test to run after other tests have run\n" + " -P: Run as worker of parallel workers\n" " -R Preserve console recording on test failure\n" " -s Show all suites with ut info\n" " Test suite to run (or comma-separated list)\n" diff --git a/test/test-main.c b/test/test-main.c index 2524a154186..a2c4e32423b 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -790,6 +790,10 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix, !test_matches(prefix, test_name, select_name)) continue; + /* Skip tests not assigned to this worker */ + if (uts->workers && upto % uts->workers != uts->worker_id) + continue; + if (test->flags & UTF_MANUAL) { int len;