From patchwork Sun Dec 14 17:54:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 914 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765734986; bh=Kjidm9z/1+ZAFuCI0kF/zbQe5YBmJJ0jJY8nJKoeZjA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=AzSs2pEWjYIDV0lSc3UnDskCY4V4l3StdqNQ5OFIabg2TteZUNNmuFTeHem0Lyo+M ji8dh5RUOBJYbaRxk5vag7Uyg3XbUgTZPpfL+JS0yLN/VUUZniKZ5ivpXoZ2prQKYH TMAZADoYq9Y6b8AVB0utlUhqsF/asiStJex9bQSsmj9SYV/ZyV/pCIUPt2HNmsDIPp V03vnVGpqWq8kN4quDJFm/+QBZoBrnlg7ixT50u4ZEAkZJZoIzLm1zavD0opMp2c1e ZwZdHsdg243EPHAJQzd9dIS/Vdxdu9fD0fzYBXFNKPvy7Eee70GNeJdzye5sVKfkY6 FKBKsYdFL6xmQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 79D2068A91 for ; Sun, 14 Dec 2025 10:56:26 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id NlXgNIwCrjqp for ; Sun, 14 Dec 2025 10:56:26 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765734984; bh=Kjidm9z/1+ZAFuCI0kF/zbQe5YBmJJ0jJY8nJKoeZjA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=DJ0QDYmApF6gy4KuOhYWjOZq1rmG3Bm3FSn3vmPfdOgHV0FbWvgqQvDiUIZT6m148 pFWX748gucfZnK2ICfAxVM6WCDq+DWXGOhfSyAUykvmV60+Sb+7Ws9Eln6PZ/ctXMh Kd8O40C6+EL7YwxvkLVqiP0jvNdqoxoZOJElJ7aJk7n8ZmeKbRc8J3H+fS0BTlRSfn cxZ9b1SU/vvkG4iHRSca3ZpJQoHC5dr0Sc6Nc+gcWRc2nB9LAh/57DegfWXzG6y9mw cL99WNrEY74tCFBMUkh6jC1dOHznXojhjyAzS9ijLrNgHPIYnTlz7S2eOz582ogfbj 6LHyNBM/7oSWw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id DC0FB68ADF for ; Sun, 14 Dec 2025 10:56:24 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765734982; bh=PfiVZHeRs3EsMW/O32ROycf/19qudrdYWmUyVc5l+6o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iNQNX1ZLATLLsdUeKcpup8TXNTLnNFYb/Xxy29g4jL73vI4NiiOqsTLGewuGkou2E NWea7MyPnK+xBMdhd9zVzA+yuRENoj5N3/+KiDpZq2HiMK7w9xCeaKnVTJ5H4XrO7y qYPdVFE10n5qN73ooiZ/F8MbonFbRIUrFk736PiY2TgARUJSjvCO+oUj6Gh8HNvWIa ubYLabG0UrqwSiODWWHTKwuj+jy6BQFjRAEu5WZ/ikDnBaEl0YAqcTLp1QhQ5WBDpK 68FiMgeyQnLKYWCPWCLQYa57klttfYVtyvLUfxVy4oXHRu1+ea0V3BYV+rgjzxdhI4 nCevSZOQ+Xoww== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0554B68A91; Sun, 14 Dec 2025 10:56:22 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id 8bJeIqjI3YPP; Sun, 14 Dec 2025 10:56:21 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1765734981; bh=z1LoqfKvIzpUFgZq2RPYajvfyI3qxK6wlqcgzW1JhG0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UAZ3ixtZIkAJ6PCAEJgljcrxnbF/mA3aqO3KBuKq/K3+hI0EfyLSzAGI/gf032iJC 7RlCVkBjT8cEs89on8gAfnUqZup5QKZYq9w1kuFIuvVshquqC8MgD2YLGFPv1N2Cuc I4+OKL80S0a1q6kzYTU1qGgdf+ESKE2PJ1y0Is7peWD8SwimHUbl3SP8ithvluEayy eXBZDAPNlyhngFzzxpE/OuW1ykciMb47gN532FB55h071BnwaYhMqflRe9pRCrw2cm /sVh1LtmXRnoBZygJ2hce2zpKTAiwdspSG8Tggoq0RrSMja7Sqt9iS2JO3Pi4PWURr ffTA8k49Qc+HQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 517E268A84; Sun, 14 Dec 2025 10:56:21 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Sun, 14 Dec 2025 10:54:43 -0700 Message-ID: <20251214175449.3799539-22-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251214175449.3799539-1-sjg@u-boot.org> References: <20251214175449.3799539-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: SUJFOS37HNFGQBD5JKBIC2HV75QSUG3R X-Message-ID-Hash: SUJFOS37HNFGQBD5JKBIC2HV75QSUG3R X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , Claude X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 21/21] test: Add documentation for test parameters List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add a description of how test parameters work. This helps to make it easier to write C tests which need setup to be done in Python. Co-developed-by: Claude Signed-off-by: Simon Glass --- doc/develop/tests_writing.rst | 41 +++++++++++++++++++++++++++++++++++ doc/usage/cmd/ut.rst | 11 ++++++++-- 2 files changed, 50 insertions(+), 2 deletions(-) diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst index 6842533e2c6..d322cf60e26 100644 --- a/doc/develop/tests_writing.rst +++ b/doc/develop/tests_writing.rst @@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the Python test. +Passing arguments to C tests +---------------------------- + +Sometimes a C test needs parameters from Python, such as filenames or expected +values that are generated at runtime. The test-argument feature allows this. + +Use the `UNIT_TEST_ARGS` macro to declare a test with arguments:: + + static int my_test_norun(struct unit_test_state *uts) + { + const char *filename = ut_str(0); + int count = ut_int(1); + + /* test code using filename and count */ + + return 0; + } + UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite, + { "filename", UT_ARG_STR }, + { "count", UT_ARG_INT }); + +Each argument definition specifies a name and type: + +- `UT_ARG_STR` - string argument, accessed via `ut_str(n)` +- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)` +- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)` + (use `1` for true, any other value for false) + +Arguments are passed on the command line in `name=value` format:: + + ut -f my_suite my_test_norun filename=/path/to/file count=42 + +From Python, you can call the test like this:: + + cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}' + ubman.run_command(cmd) + +This approach combines Python's flexibility for setup (creating files, +generating values) with C's speed and debuggability for the actual test logic. + + How slow are Python tests? -------------------------- diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index d8c3cbf496c..a26ee6ad7de 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-r] [-f] [-I:] [ | all []] [...] + ut [-r] [-f] [-R] [-I:] [ | all []] [...] ut [-s] info Description @@ -37,10 +37,17 @@ test causes another test to fail. If the one test fails, testing stops immediately. +-R + Preserve console recording on test failure. Normally when a test fails, + console recording is disabled so error messages go directly to output. + This flag keeps recording enabled, which is useful when testing the test + framework itself. + args Optional arguments to pass to the test, in `name=value` format. These are used by tests declared with `UNIT_TEST_ARGS()` which define expected - argument names and types. + argument names and types. See :ref:`develop/tests_writing:passing arguments + to c tests` for details. Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it includes a near-complete set of emulators, no code-size limits, many CONFIG