From patchwork Mon Dec 29 16:05:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1099 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024391; bh=GWobDRP2atI2tNU1VyJjIa8mvc5Kzc3enR5wWD3cO/A=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=V89u9byqpGCZM/vlqRH6lv0oQf7EeqOWrDMlJM+mZ7mrGBWkE7qP749m5pUcDbvCb OBuMa/K83LfIPIAS2LZMxmdhHC4C6u495phlokSwUeKEtrQdHrJhjKxLt01gdepVwJ U4rVtJYSFL8gqWXKDUA/f2KPEFXKReEptZUiB0DB6KOISwM2F/rTo91IwTNNJNIhPx U7LlB+2M0C3sAMDnc08/t4aIJGAaWtPY7kLcjT8aY9tqQUzBVvY45qWktRek/CZRJN tO8pSqcutweuozVewIZ5xGJlonUDYvKTWF0zmw748Rz3Bfk49LZDXu0jkvQdJXJKTy PMd9borQnTUfQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 9FD2768F3B for ; Mon, 29 Dec 2025 09:06:31 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id lwAyGqtpldja for ; Mon, 29 Dec 2025 09:06:31 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024391; bh=GWobDRP2atI2tNU1VyJjIa8mvc5Kzc3enR5wWD3cO/A=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=V89u9byqpGCZM/vlqRH6lv0oQf7EeqOWrDMlJM+mZ7mrGBWkE7qP749m5pUcDbvCb OBuMa/K83LfIPIAS2LZMxmdhHC4C6u495phlokSwUeKEtrQdHrJhjKxLt01gdepVwJ U4rVtJYSFL8gqWXKDUA/f2KPEFXKReEptZUiB0DB6KOISwM2F/rTo91IwTNNJNIhPx U7LlB+2M0C3sAMDnc08/t4aIJGAaWtPY7kLcjT8aY9tqQUzBVvY45qWktRek/CZRJN tO8pSqcutweuozVewIZ5xGJlonUDYvKTWF0zmw748Rz3Bfk49LZDXu0jkvQdJXJKTy PMd9borQnTUfQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 8E36168F2D for ; Mon, 29 Dec 2025 09:06:31 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024390; bh=Nj7M7RnPksUX2q0Hyz4i0Dsg4bBRlu6De0HYvjhpNwo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IjnFNd3ZVIJz7EUjnHlvWuXkbmEPDSEKClmoekTTnou039WE3J+hRUKsrXOnBPbjW nAv4vzi6AJSYlWaoywn5OmTptaqTf8VHqEO2b/Ge8tEWY0j2M1jAZkKdNHXV7kA/vK mWcf0lg+8S90Xcvx3SHhLlkrZ1dWpdiaHrEit6Nu9/dLjXulR9QHN7Wjmctf//6RTC 5dJjVZwb6K73F8HAEKGeqG3rXHu0AlgX4qdOYh80VJwZuuDbzEHxxyPimyR694hA1S nY2aNDK3dIpBuEQKg6/rFa6uUikPemnsT1vCDk4dpmvh6ZvbIqrWDT2oyCuVm7L5GK hlwsSuSzn7gtw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 19DB368B78; Mon, 29 Dec 2025 09:06:30 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id nD_N6DaUVqu5; Mon, 29 Dec 2025 09:06:30 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024385; bh=QQknSc698mkdtQpgng18qaxi4LDMmzG5gcssvvMuBxY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vn/P/okRe32xZyXUkdP7+2k/RLzcWMnnwpmOjmNFNMBcHLyNWfKEJacgXj+BLhm8R /+Q3Nh/DJyyiGtR6fdHXbiccO20kWM7JxeprKBc7Lp3EBY15qe1vezjgwPzHT415zk eLyC8Lnqa0mCetLg8FkFZiUM9ZrmKTUPVqfbbK6ZhjT0+8msDwINTrrFuDN2X8ESLO qP5bikORXAXvTealhhHQYgbA5Wr9DPkccFHv0WXiTLeRKYlEH+8lbgUsvCIYTxkRKa gSwMrQLHEA+v4Ty3UaE1t56XxOcb11/VkEHUraWSLl+haOyVmssMjHUjOuDBaV4qfi OiDdz93vggutQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 79D6564E2A; Mon, 29 Dec 2025 09:06:25 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:05:59 -0700 Message-ID: <20251229160611.3899708-2-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: T5MD3OHQ5LXQ36UXL3HJFZESDPFY4GL5 X-Message-ID-Hash: T5MD3OHQ5LXQ36UXL3HJFZESDPFY4GL5 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 1/9] test: Move disk images to persistent-data directory List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass It is annoying to have disk images in the source directory since it clutters up the working space. Remove cur_dir=True from DiskHelper calls so disk images are written to the persistent-data directory instead. Move scsi.img too (used by the bootstd tests) and mmc6.img (used by the MBR tests. Add a few comments as to where the images are used. This keeps the source tree clean and puts disk images in the same place as other test data. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- drivers/mmc/sandbox_mmc.c | 13 +++++++++++-- drivers/scsi/sandbox_scsi.c | 13 +++++++++++-- drivers/usb/emul/sandbox_flash.c | 12 ++++++++++-- test/py/img/armbian.py | 2 +- test/py/img/common.py | 2 +- test/py/img/efi.py | 2 +- test/py/tests/test_ut.py | 11 +++++++---- 7 files changed, 42 insertions(+), 13 deletions(-) diff --git a/drivers/mmc/sandbox_mmc.c b/drivers/mmc/sandbox_mmc.c index f49cb4b146a..74427db7e06 100644 --- a/drivers/mmc/sandbox_mmc.c +++ b/drivers/mmc/sandbox_mmc.c @@ -170,11 +170,20 @@ static int sandbox_mmc_probe(struct udevice *dev) int ret; if (plat->fname) { - ret = os_map_file(plat->fname, OS_O_RDWR | OS_O_CREAT, + const char *fname = plat->fname; + char buf[256]; + + /* + * Try persistent data directory first, then fall back to the + * filename as given (for absolute paths or current directory) + */ + if (!os_persistent_file(buf, sizeof(buf), plat->fname)) + fname = buf; + ret = os_map_file(fname, OS_O_RDWR | OS_O_CREAT, (void **)&priv->buf, &priv->size); if (ret) { log_err("%s: Unable to map file '%s'\n", dev->name, - plat->fname); + fname); return ret; } priv->csize = priv->size / SIZE_MULTIPLE - 1; diff --git a/drivers/scsi/sandbox_scsi.c b/drivers/scsi/sandbox_scsi.c index 3c451313109..97afeddc2e9 100644 --- a/drivers/scsi/sandbox_scsi.c +++ b/drivers/scsi/sandbox_scsi.c @@ -104,9 +104,18 @@ static int sandbox_scsi_probe(struct udevice *dev) info->block_size = SANDBOX_SCSI_BLOCK_LEN; if (priv->pathname) { - priv->fd = os_open(priv->pathname, OS_O_RDONLY); + const char *pathname = priv->pathname; + char buf[256]; + + /* + * Try persistent data directory first, then fall back to the + * pathname as given (for absolute paths or current directory) + */ + if (!os_persistent_file(buf, sizeof(buf), priv->pathname)) + pathname = buf; + priv->fd = os_open(pathname, OS_O_RDONLY); if (priv->fd >= 0) { - ret = os_get_filesize(priv->pathname, &info->file_size); + ret = os_get_filesize(pathname, &info->file_size); if (ret) return log_msg_ret("sz", ret); } diff --git a/drivers/usb/emul/sandbox_flash.c b/drivers/usb/emul/sandbox_flash.c index 25d968e91c7..82aa7062865 100644 --- a/drivers/usb/emul/sandbox_flash.c +++ b/drivers/usb/emul/sandbox_flash.c @@ -339,11 +339,19 @@ static int sandbox_flash_probe(struct udevice *dev) struct sandbox_flash_plat *plat = dev_get_plat(dev); struct sandbox_flash_priv *priv = dev_get_priv(dev); struct scsi_emul_info *info = &priv->eminfo; + const char *pathname = plat->pathname; + char buf[256]; int ret; - priv->fd = os_open(plat->pathname, OS_O_RDWR); + /* + * Try persistent data directory first, then fall back to the + * pathname as given (for absolute paths or current directory) + */ + if (!os_persistent_file(buf, sizeof(buf), plat->pathname)) + pathname = buf; + priv->fd = os_open(pathname, OS_O_RDWR); if (priv->fd >= 0) { - ret = os_get_filesize(plat->pathname, &info->file_size); + ret = os_get_filesize(pathname, &info->file_size); if (ret) return log_msg_ret("sz", ret); } diff --git a/test/py/img/armbian.py b/test/py/img/armbian.py index e1ae9b0aae9..db7fb43ddf6 100644 --- a/test/py/img/armbian.py +++ b/test/py/img/armbian.py @@ -126,7 +126,7 @@ booti ${kernel_addr_r} ${ramdisk_addr_r} ${fdt_addr_r} utils.run_and_log_no_ubman(log, f'echo here {kernel} {symlink}') os.symlink(kernel, symlink) fsh.mk_fs() - img = DiskHelper(config, mmc_dev, 'mmc', True) + img = DiskHelper(config, mmc_dev, 'mmc') img.add_fs(fsh, DiskHelper.EXT4) img.create() fsh.cleanup() diff --git a/test/py/img/common.py b/test/py/img/common.py index 547066b24a5..25edc84944e 100644 --- a/test/py/img/common.py +++ b/test/py/img/common.py @@ -84,7 +84,7 @@ def setup_extlinux_image(config, log, devnum, basename, vmlinux, initrd, dtbdir, fsh.mk_fs() - img = DiskHelper(config, devnum, basename, True) + img = DiskHelper(config, devnum, basename) img.add_fs(fsh, DiskHelper.VFAT, bootable=True) ext4 = FsHelper(config, 'ext4', max(1, part2_size - 30), prefix=basename, diff --git a/test/py/img/efi.py b/test/py/img/efi.py index 8239b4c247f..4cb55c957ec 100644 --- a/test/py/img/efi.py +++ b/test/py/img/efi.py @@ -31,7 +31,7 @@ def setup_efi_image(config): fsh.mk_fs() - img = DiskHelper(config, devnum, 'flash', True) + img = DiskHelper(config, devnum, 'flash') img.add_fs(fsh, DiskHelper.VFAT) img.create() fsh.cleanup() diff --git a/test/py/tests/test_ut.py b/test/py/tests/test_ut.py index b9ba240c848..70a748a31f9 100644 --- a/test/py/tests/test_ut.py +++ b/test/py/tests/test_ut.py @@ -34,7 +34,8 @@ from img.localboot import setup_localboot_image def test_ut_dm_init(ubman): """Initialize data for ut dm tests.""" - fn = ubman.config.source_dir + '/testflash.bin' + # This is used by flash-stick@0 in test.py + fn = ubman.config.persistent_data_dir + '/testflash.bin' if not os.path.exists(fn): data = b'this is a test' data += b'\x00' * ((4 * 1024 * 1024) - len(data)) @@ -47,8 +48,8 @@ def test_ut_dm_init(ubman): with open(fn, 'wb') as fh: fh.write(data) - # Create a file with a single partition - fn = ubman.config.source_dir + '/scsi.img' + # Create a file with a single partition (used by /scsi in test.dts) */ + fn = ubman.config.persistent_data_dir + '/scsi.img' if not os.path.exists(fn): data = b'\x00' * (2 * 1024 * 1024) with open(fn, 'wb') as fh: @@ -56,11 +57,13 @@ def test_ut_dm_init(ubman): utils.run_and_log( ubman, f'sfdisk {fn}', stdin=b'type=83') + # These two are used by test/dm/host.c FsHelper(ubman.config, 'ext2', 2, '2MB').mk_fs() FsHelper(ubman.config, 'fat32', 1, '1MB').mk_fs() + # This is used by test/cmd/mbr.c mmc_dev = 6 - fn = os.path.join(ubman.config.source_dir, f'mmc{mmc_dev}.img') + fn = os.path.join(ubman.config.persistent_data_dir, f'mmc{mmc_dev}.img') data = b'\x00' * (12 * 1024 * 1024) with open(fn, 'wb') as fh: fh.write(data) From patchwork Mon Dec 29 16:06:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1100 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024394; bh=f2Xn115/ps/4igV0Eq/o6EprwboDGo4MOmJeKROkkjg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=FmJk3RBAM1X2KCmA0Bb29fbyAgUFCI4d8AOvYZ00xXkqZqcHrQJUIaC7QTsjRN2IL 4CRVsgql427DZp+ZNstr8QMpwk1Fp8TdDtuCvsmL4KqptEIa+msQelHuGCpp35/X8R +nQWhjHgu8SjMr0jFZQKlvB9OleGimYVMAPJAVY5qfcD2R5qV1RsTq4Fk0paykPsu7 4lvMa18o5qcpCRToU0H6Ry5IJyRK2FWBmTFPfpcfj5Hfvfd4LHXM8gyq5/dcUCPaXL 7Hw9Vz1qkjNF+idApZFQOt18omAbEhjn/UPVi8qCTJ/cJMcFzdtGiNvyRoPV4UkYqW hwcv0qjod/nzA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0EF1568CAD for ; Mon, 29 Dec 2025 09:06:34 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id xjqROSPtEeHo for ; Mon, 29 Dec 2025 09:06:34 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024393; bh=f2Xn115/ps/4igV0Eq/o6EprwboDGo4MOmJeKROkkjg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=h9aeJtbcF9/Yfra85JKkSbCp4pigifDg2C1S057KFY0yQh8AyFJJZgO2nRMC3jF7C 7GPXCZs9jKR+Z7uwBNm5JV9e8Y8/MY5pGeEKEomUCpNsY0tHL6tktFD8w4RaNwRPr1 TQpKcAcF/3MnslPkyB+Bvcsr30zt2fSfJI9m4MuRPoAoXgTcA/p9JoAs+46w6AflkI eWblWt/Ui0oTVB6BhBKE/1fv1Ibt++W7GukDqNHioYtk7F4yte3Dr7S9qnitTKxnae ksCaWqIO6WNeyPDJRLFbkkrmLBB3QkDFSdDXHkxLFr/x/g4TJ2lsY04VDP6k73CJ8K ReK5Ba9OyKJTg== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 7C70E68CAC for ; Mon, 29 Dec 2025 09:06:33 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024391; bh=Cy4k8dftu3PaxpmSjvDjta5fKx9dvqEh1A6h5zgWQYM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YxmbZBoYSR2NhwajOPBWj6EesBwiJ/W3fXUen8/eiJvQXTe+b4Bkg5AB1gGR3jvri Zq75M6JcXns6o3AZ/R5t8u2POOza0l1WCPTddk4rJSiqIFzhhwiLSfmaRkJwaCSbdE OuV6kEau4T0JtF7fA36Mc6vVe4H+evUmU0GL2yCw9AyJtHggpxhLLDdODEzkKRxt7n /lgtAWXS58u4f41opds/qK9s9PTHgSp7jOxx7ghgnde2+1YPxo+sVbRssXtymQrv9Y pyJSdS900zJXMspTKTVik6vQazA2PW5LmtV61Iz5j5YNRruj0KvoVeJQrhMmZq6wKY I8MtIF+UQR5Wg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 0049C68CAC; Mon, 29 Dec 2025 09:06:31 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id lNNRkt2xcLW6; Mon, 29 Dec 2025 09:06:30 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024390; bh=eKV3Kd8GAlnPdNS7+oH7g+iKUJ4TwiFH5hGy0LaqEzc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uWBztzRD69FyZAYmmhoR9yDjjcSE/zopHCDNyUTk56JPlul2iDA3ydVqDd+qvVbOM G+Ut3Nrw60ZqDmrwDwqr/mhuGvIp/ebnzPOd72Z/DGxBdbJ9LK2YRYOQAPcof10Y21 xtLegt1lcT4sPvOq89Xqq1MhMTKB0/plYZB27Civ25O3fOFbPeF8f3HykW1i9mrWqX 8M591VJNRplU3RujDpObazU/jf2e4rhvlh8QnK1p+s0Ux6syNTXQtxraslB5+pBzlv eJalMaimkjF1Tw9lX3AGeFHt4QYsH/awiT6pGVUxISiKKpRv9zdvcJSAl0UruHCzEI vlyReV/H0WVTw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 5C18B64E2A; Mon, 29 Dec 2025 09:06:30 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:00 -0700 Message-ID: <20251229160611.3899708-3-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: A467KRBBZS6JQZETIJO7CLCCFPLDPH4E X-Message-ID-Hash: A467KRBBZS6JQZETIJO7CLCCFPLDPH4E X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 2/9] test: Write the SPI image to the persistent-data directory List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass It is annoying to have disk images in the source directory since it clutters up the working space. Move spi.bin (used by the SPI tests) into the persistent-data directory, update the driver and add a comment. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- drivers/mtd/spi/sandbox.c | 14 +++++++++++--- test/dm/sf.c | 4 +++- test/py/tests/test_ut.py | 2 +- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/drivers/mtd/spi/sandbox.c b/drivers/mtd/spi/sandbox.c index d9afe77e52d..0b56312bb85 100644 --- a/drivers/mtd/spi/sandbox.c +++ b/drivers/mtd/spi/sandbox.c @@ -126,7 +126,9 @@ static int sandbox_sf_probe(struct udevice *dev) struct dm_spi_slave_plat *slave_plat; struct udevice *bus = dev->parent; const char *spec = NULL; + const char *filename; struct udevice *emul; + char buf[256]; int ret = 0; int cs = -1; @@ -170,10 +172,16 @@ static int sandbox_sf_probe(struct udevice *dev) if (sandbox_sf_0xff[0] == 0x00) memset(sandbox_sf_0xff, 0xff, sizeof(sandbox_sf_0xff)); - sbsf->fd = os_open(pdata->filename, 02); + /* + * Try persistent data directory first, then fall back to the + * filename as given (for absolute paths or current directory) + */ + filename = pdata->filename; + if (!os_persistent_file(buf, sizeof(buf), pdata->filename)) + filename = buf; + sbsf->fd = os_open(filename, 02); if (sbsf->fd < 0) { - printf("%s: unable to open file '%s'\n", __func__, - pdata->filename); + log_err("Unable to open file '%s'\n", filename); ret = -EIO; goto error; } diff --git a/test/dm/sf.c b/test/dm/sf.c index 3684d021709..15a95a2d41e 100644 --- a/test/dm/sf.c +++ b/test/dm/sf.c @@ -27,10 +27,12 @@ static int dm_test_spi_flash(struct unit_test_state *uts) uint map_size; ulong map_base; uint offset; + char pathname[256]; int i; src = map_sysmem(0x20000, full_size); - ut_assertok(os_write_file("spi.bin", src, full_size)); + ut_assertok(os_persistent_file(pathname, sizeof(pathname), "spi.bin")); + ut_assertok(os_write_file(pathname, src, full_size)); ut_assertok(uclass_first_device_err(UCLASS_SPI_FLASH, &dev)); dst = map_sysmem(0x20000 + full_size, full_size); diff --git a/test/py/tests/test_ut.py b/test/py/tests/test_ut.py index 70a748a31f9..da36898d803 100644 --- a/test/py/tests/test_ut.py +++ b/test/py/tests/test_ut.py @@ -42,7 +42,7 @@ def test_ut_dm_init(ubman): with open(fn, 'wb') as fh: fh.write(data) - fn = ubman.config.source_dir + '/spi.bin' + fn = ubman.config.persistent_data_dir + '/spi.bin' if not os.path.exists(fn): data = b'\x00' * (2 * 1024 * 1024) with open(fn, 'wb') as fh: From patchwork Mon Dec 29 16:06:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1101 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024398; bh=eolqTrP5X3VrX3m4ch6f8C1OzLfwL3MSQF2F7wrXb/Y=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Vfu/ensnfUqNOecO/xXrc/wbP9IrgZM70TB0ufbGIlkswsaxDd/s2kyuTjJTNoNAP sBZJOc7NruNIUtGZYI9D3u77y/buM/QXSgDUYZZVB7FwHjyxP5qhrggF9/9HBsKxZP 8wdy4nFkO70n5CbB9Auapf54JGke8Ieqz+6YYcv0Tllw7WDLUgijeqoLvrOBi0Nfky TRetxn06EYRWjPQHL6poRmrwqKYAnrrQGLuCrabBTrVbAODJKakhqV7nWdeLkx1gbH Q6daNkTsEW9TNmCsu5W/83ZIkieXDj/NAoOcBDYv3z8pnPvzc+uJxXqj1CbiZQFeyb VPH5zLWUIrzFw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2253568F2D for ; Mon, 29 Dec 2025 09:06:38 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id Oe36--V4AuI5 for ; Mon, 29 Dec 2025 09:06:38 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024398; bh=eolqTrP5X3VrX3m4ch6f8C1OzLfwL3MSQF2F7wrXb/Y=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Vfu/ensnfUqNOecO/xXrc/wbP9IrgZM70TB0ufbGIlkswsaxDd/s2kyuTjJTNoNAP sBZJOc7NruNIUtGZYI9D3u77y/buM/QXSgDUYZZVB7FwHjyxP5qhrggF9/9HBsKxZP 8wdy4nFkO70n5CbB9Auapf54JGke8Ieqz+6YYcv0Tllw7WDLUgijeqoLvrOBi0Nfky TRetxn06EYRWjPQHL6poRmrwqKYAnrrQGLuCrabBTrVbAODJKakhqV7nWdeLkx1gbH Q6daNkTsEW9TNmCsu5W/83ZIkieXDj/NAoOcBDYv3z8pnPvzc+uJxXqj1CbiZQFeyb VPH5zLWUIrzFw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 113AA68CAD for ; Mon, 29 Dec 2025 09:06:38 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024396; bh=c/mQbyd4nEBrddMss4NUF0Yeew5DTSXBBNVACX7XZ98=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LvvFMmTW687x3soSg8GvPbmM8mGa7TQMRbaC21jBZw0LG1EaiLviaLcQruGkRpMAr 0rhsxfIKDptWP7vN5K3lNsz+MCupSCqz7eBm6mcstIFEkvqgfs8ybb/W9aAhNyiKmB s7Vx+hJr/UgzykhFBJByqhKCfEURTv245ZAegMQQZKDCp4v1ngBhAUGegaSHL/psuT tsw9cGTUajVndm2DxBtlj0W4JkJqvgFU+9Xfqk/uQQkDZBImZBgzQSG7CxthLSmPz1 0V4LNYKUESP4pZDQL29juE7uzK3Wqjm87WhOG1mkkACFdheoYCwIsfg+F5sHuM9sLW WGXg4wU8Jducg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 4B3DB68CAD; Mon, 29 Dec 2025 09:06:36 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id kCn8egA3Pfgm; Mon, 29 Dec 2025 09:06:36 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024391; bh=bky0fu9+Z51/F7GmU7S8T9niYCJvZ/0rHX1AeBpe6gs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C6t+IfZZ7oMRM/GKVmRUBEj89hPXJ4bDuje4Hw18wzchipAL4+gFS8bDWl45WFBuy 0F4o+ZpWbEb7twkAYlGH/RZRCu53yhypT232bSqun+X9eAb0/VPaKo7yG2LeIOzulc wkvIt0M4srbD0NDuPeZvPhP9A8zTgLNldnCMb7ynq1yXkyyitexrCUg4ECdnJsAyA7 LZg5H5rkce5XyQHirru3fKYMstJTgCSSDoazcilncRBrSr/CQKullIuAfPn/i/0b/+ mWFxNSB0qAerszxNL6ulsog7i1QkADKOfHeglBRUQ28Av0pXzFr7+9M/EfKPlKHiFs QrA1WHxCzSQog== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 3F8BF64E2A; Mon, 29 Dec 2025 09:06:31 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:01 -0700 Message-ID: <20251229160611.3899708-4-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 7TRR46XJDHBJNDNYS4LCQ26HWSRLNMNI X-Message-ID-Hash: 7TRR46XJDHBJNDNYS4LCQ26HWSRLNMNI X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 3/9] test: py: Add --persist flag to keep test artefacts List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass When iterating on C test code, the Python fixtures that create disk images run each time, even though the images have not changed. This slows down the development cycle unnecessarily. Add a -P/--persist option to prevent cleanup of test-generated files like disk images. This allows re-running C tests directly, without re-running the Python fixture each time. Update the ext4l test to respect the persist flag. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/develop/py_testing.rst | 7 +++++++ test/py/conftest.py | 3 +++ test/py/tests/test_fs/test_ext4l.py | 4 ++-- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst index c65de27369b..303c07040b3 100644 --- a/doc/develop/py_testing.rst +++ b/doc/develop/py_testing.rst @@ -246,6 +246,13 @@ Command-line options sets the directory used to store persistent test data. This is test data that may be re-used across test runs, such as file-system images. +-P, --persist + prevents cleanup of test-generated files like disk images after the test run + completes. This is useful when iterating on C test code, allowing you to + re-run the C tests without re-running the Python fixture that creates the + test images. Note that this must be individually supported by each test, e.g. + with a check against u_boot_config.persist before removing images. + --timing shows a histogram of test duration, at the end of the run. The columns are: diff --git a/test/py/conftest.py b/test/py/conftest.py index 247d4e19094..b79080af961 100644 --- a/test/py/conftest.py +++ b/test/py/conftest.py @@ -100,6 +100,8 @@ def pytest_addoption(parser): help="Assume that U-Boot is ready and don't wait for a prompt") parser.addoption('--timing', default=False, action='store_true', help='Show info on test timing') + parser.addoption('-P', '--persist', default=False, action='store_true', + help='Persist test artifacts (do not clean up after tests)') def run_build(config, source_dir, build_dir, board_type, log): @@ -346,6 +348,7 @@ def pytest_configure(config): ubconfig.dtb = build_dir + '/arch/sandbox/dts/test.dtb' ubconfig.connection_ok = True ubconfig.timing = config.getoption('timing') + ubconfig.persist = config.getoption('persist') ubconfig.role = config.getoption('role') ubconfig.allow_exceptions = config.getoption('allow_exceptions') diff --git a/test/py/tests/test_fs/test_ext4l.py b/test/py/tests/test_fs/test_ext4l.py index 754c2cc69c4..3287f59fbe5 100644 --- a/test/py/tests/test_fs/test_ext4l.py +++ b/test/py/tests/test_fs/test_ext4l.py @@ -65,8 +65,8 @@ class TestExt4l: yield image_path - # Cleanup - if os.path.exists(image_path): + # Cleanup (skip if --persist flag is set) + if not u_boot_config.persist and os.path.exists(image_path): os.remove(image_path) def test_probe(self, ubman, ext4_image): From patchwork Mon Dec 29 16:06:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1102 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024402; bh=k2/zvLx0+UQ2Jl2RWFMfaL6L8Uh0zoWAmbCOJRxMnEo=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=YSmEVaFIkiuNuB/O2jstRlCTR9dGIJyEjucegYqL6aBKyfZEyp1PIBg754rwh/jft CNCqQSuLgolvGNSS4LYGeC+wgOwjh8BJsAIbvtfoMmOiRKTZopCX1F/LvTPycwXZKZ E1AlVuG9bc50N92itQQfBJHncDn9nGM4fnhy4FBcV4IHl9h5Nh+sOs3xXm+brWO9fO aaVNCDCcG6jj9bQEnJAwU0nUvCqOmV7E+CQ359vX30mokuUBLYBluE1s7adjd0520x wDvWAiLU+OtfieeTqVszma3YyzWoSon4Z9sCLb/CGziuEwts/362/Kuce5zYf2ls8f EgiZ0A7wP5NCw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id AED7568F31 for ; Mon, 29 Dec 2025 09:06:42 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id zqvOsBakQamn for ; Mon, 29 Dec 2025 09:06:42 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024402; bh=k2/zvLx0+UQ2Jl2RWFMfaL6L8Uh0zoWAmbCOJRxMnEo=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=YSmEVaFIkiuNuB/O2jstRlCTR9dGIJyEjucegYqL6aBKyfZEyp1PIBg754rwh/jft CNCqQSuLgolvGNSS4LYGeC+wgOwjh8BJsAIbvtfoMmOiRKTZopCX1F/LvTPycwXZKZ E1AlVuG9bc50N92itQQfBJHncDn9nGM4fnhy4FBcV4IHl9h5Nh+sOs3xXm+brWO9fO aaVNCDCcG6jj9bQEnJAwU0nUvCqOmV7E+CQ359vX30mokuUBLYBluE1s7adjd0520x wDvWAiLU+OtfieeTqVszma3YyzWoSon4Z9sCLb/CGziuEwts/362/Kuce5zYf2ls8f EgiZ0A7wP5NCw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 9DA4568CAD for ; Mon, 29 Dec 2025 09:06:42 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024400; bh=qDrNadD60DNcUWnWoOWxd0i04xM8EkE6HIClPyHFHyk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Xmv6JCxLcJwLq89FJNUtEKBkmozSq6qMAvD7WZdutQX8fEkR56IBoHptm4Sf8yHYz 0ydPnYXw2MpWNFjM3hRO8wakZ6o7v1HfNuzdwLiBTOSIA4KdjR7/rnI3XHF73qAoOv br+H0xE1pXGIprNhc7Q4BY713KgrcrWrltQOXcLRHIYvUMUgu1WrFppYDjJPeZyOfK EImSzW89LSQqXx1wQJRTgJDOyfUC80ayKmZME2GaiWZBWU+SUEjspWyPmUZj08GMn4 iW8hHa/TUX7cWvGvn8C1XlmD15RsPU5qWD/oXf7I18u5h59HOWX8S2ykTGAzlxEnw0 I3Qnt1PaIO9sQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 8F87664E2A; Mon, 29 Dec 2025 09:06:40 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id QivSBIHlYFy9; Mon, 29 Dec 2025 09:06:40 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024396; bh=ZxkAZyPH5p2G81ZeGJW15iwO2D13A4TcHWNSOf/cpkI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QqM+EPJTCHD5ZdSy1/rmn8EmPL8941W9eqj6kPaaKzSEmwTqkM4qBLsxgehSC2Bsh keEf1lKd977LC5RyLYwymBbCAhLfQgLOzbDeEBnK389YdM5xzMNNg1ORRoURl6Fnj/ Cfhg5KGuRRov+U4dTQsvJ/djJ22UpBj4aFjFUgVVV4E4Hq/4JWhTWZixIZhtQvBp0k C7Gmo2jNs7PrCZhmn6TwAcW6K6j9hxqccWBlL30U+PL8487XQyvuCny1gLr6o5yPuS O8M7nz+5R+vOUgqWUwiYZnrcQPdAYiLJ8Yg67gV3KTx7/vobXT19rXuaiUAenxLCo7 7ashwIwnzYAdQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 2033168CAC; Mon, 29 Dec 2025 09:06:36 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:02 -0700 Message-ID: <20251229160611.3899708-5-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: VOCI5TW7RTVZZHHW5ECZC7NDQXU2BUQL X-Message-ID-Hash: VOCI5TW7RTVZZHHW5ECZC7NDQXU2BUQL X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 4/9] test: py: Add run_ut() helper for manual unit tests List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Running manual unit tests (those with _norun suffix) involves a common pattern: building the ut command with the -f flag, running it, and checking for failures. This is verbose and error-prone. Add a run_ut() method to ConsoleBase that simplifies this. It handles the command construction, test arguments and failure checking automatically. Before: output = ubman.run_command( f'ut -f fs fs_test_ext4l_probe_norun fs_image={ext4_image}') assert 'failures: 0' in output After: ubman.run_ut('fs', 'fs_test_ext4l_probe', fs_image=ext4_image) Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- test/py/console_base.py | 27 +++++++++ test/py/tests/test_fit_print.py | 8 +-- test/py/tests/test_fs/test_basic.py | 87 +++++++++-------------------- test/py/tests/test_upl.py | 3 +- test/py/tests/test_vbe.py | 8 +-- 5 files changed, 58 insertions(+), 75 deletions(-) diff --git a/test/py/console_base.py b/test/py/console_base.py index 2bfd42746e4..86b6da1f5b0 100644 --- a/test/py/console_base.py +++ b/test/py/console_base.py @@ -485,6 +485,33 @@ class ConsoleBase(): output.append(self.run_command(cmd)) return output + def run_ut(self, suite, test, **kwargs): + """Run a manual unit test + + Run a unit test that has the _norun suffix, meaning it requires + external setup (like creating a disk image) before it can run. + + Args: + suite (str): Test suite name (e.g., 'fs') + test (str): Test name without _norun suffix + (e.g., 'fs_test_ext4l_probe') + **kwargs: Test arguments passed as key=value + (e.g., fs_image='/path/to/img') + + Returns: + str: Command output + + Raises: + AssertionError: If test reports failures + """ + args = ' '.join(f'{k}={v}' for k, v in kwargs.items()) + cmd = f'ut -f {suite} {test}_norun' + if args: + cmd += f' {args}' + output = self.run_command(cmd) + assert 'failures: 0' in output, f'Test {test} failed' + return output + def send(self, msg): """Send characters without waiting for echo, etc.""" self.run_command(msg, wait_for_prompt=False, wait_for_echo=False, diff --git a/test/py/tests/test_fit_print.py b/test/py/tests/test_fit_print.py index d8b034e9ce9..099a3a70591 100644 --- a/test/py/tests/test_fit_print.py +++ b/test/py/tests/test_fit_print.py @@ -261,9 +261,7 @@ def test_fit_print(ubman): build_test_fit(ubman, fit) # Run the C test which will load and verify this FIT - ubman.run_command('ut -f bootstd test_fit_print_norun') - result = ubman.run_command('echo $?') - assert '0' == result + ubman.run_ut('bootstd', 'test_fit_print') @pytest.mark.boardspec('sandbox') @@ -279,9 +277,7 @@ def test_fit_print_no_desc(ubman): utils.run_and_log(ubman, ['fdtput', '-d', fit, '/', 'description']) # Run the C test to check the missing description - ubman.run_command('ut -f bootstd test_fit_print_no_desc_norun') - result = ubman.run_command('echo $?') - assert '0' == result + ubman.run_ut('bootstd', 'test_fit_print_no_desc') @pytest.mark.boardspec('sandbox') @pytest.mark.buildconfigspec('fit_print') diff --git a/test/py/tests/test_fs/test_basic.py b/test/py/tests/test_fs/test_basic.py index 7f805d04dd5..174e2e074f4 100644 --- a/test/py/tests/test_fs/test_basic.py +++ b/test/py/tests/test_fs/test_basic.py @@ -16,39 +16,6 @@ from fstest_defs import SMALL_FILE, BIG_FILE from fstest_helpers import assert_fs_integrity -def run_c_test(ubman, fs_type, fs_img, test_name, small=None, big=None, - md5val=None): - """Run a C unit test with proper setup. - - Args: - ubman (ConsoleBase): U-Boot console manager - fs_type (str): Filesystem type (ext4, fat, fs_generic, exfat) - fs_img (str): Path to filesystem image - test_name (str): Name of C test function (without _norun suffix) - small (str): Filename of small test file (optional) - big (str): Filename of big test file (optional) - md5val (str): Expected MD5 value for verification (optional) - - Returns: - bool: True if test passed, False otherwise - """ - # Build the command with arguments - cmd = f'ut -f fs {test_name}_norun fs_type={fs_type} fs_image={fs_img}' - if small: - cmd += f' small={small}' - if big: - cmd += f' big={big}' - if md5val: - cmd += f' md5val={md5val}' - - # Run the C test - ubman.run_command(cmd) - - # Check result - result = ubman.run_command('echo $?') - return result.strip() == '0' - - @pytest.mark.boardspec('sandbox') @pytest.mark.slow class TestFsBasic: @@ -58,94 +25,92 @@ class TestFsBasic: """Test Case 1 - ls command, listing root and invalid directories""" fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 1 - ls'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_ls', - small=SMALL_FILE, big=BIG_FILE) + ubman.run_ut('fs', 'fs_test_ls', fs_type=fs_type, fs_image=fs_img, + small=SMALL_FILE, big=BIG_FILE) def test_fs2(self, ubman, fs_obj_basic): """Test Case 2 - size command for a small file""" fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 2 - size (small)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_small', - small=SMALL_FILE) + ubman.run_ut('fs', 'fs_test_size_small', fs_type=fs_type, + fs_image=fs_img, small=SMALL_FILE) def test_fs3(self, ubman, fs_obj_basic): """Test Case 3 - size command for a large file""" fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 3 - size (large)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big', - big=BIG_FILE) + ubman.run_ut('fs', 'fs_test_size_big', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE) def test_fs4(self, ubman, fs_obj_basic): """Test Case 4 - load a small file, 1MB""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 4 - load (small)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_small', - small=SMALL_FILE, md5val=md5val[0]) + ubman.run_ut('fs', 'fs_test_load_small', fs_type=fs_type, + fs_image=fs_img, small=SMALL_FILE, md5val=md5val[0]) def test_fs5(self, ubman, fs_obj_basic): """Test Case 5 - load, reading first 1MB of 3GB file""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 5 - load (first 1MB)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_first', - big=BIG_FILE, md5val=md5val[1]) + ubman.run_ut('fs', 'fs_test_load_big_first', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE, md5val=md5val[1]) def test_fs6(self, ubman, fs_obj_basic): """Test Case 6 - load, reading last 1MB of 3GB file""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 6 - load (last 1MB)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_big_last', - big=BIG_FILE, md5val=md5val[2]) + ubman.run_ut('fs', 'fs_test_load_big_last', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE, md5val=md5val[2]) def test_fs7(self, ubman, fs_obj_basic): """Test Case 7 - load, 1MB from the last 1MB in 2GB""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 7 - load (last 1MB in 2GB)'): - assert run_c_test(ubman, fs_type, fs_img, - 'fs_test_load_big_2g_last', - big=BIG_FILE, md5val=md5val[3]) + ubman.run_ut('fs', 'fs_test_load_big_2g_last', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE, md5val=md5val[3]) def test_fs8(self, ubman, fs_obj_basic): """Test Case 8 - load, reading first 1MB in 2GB""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 8 - load (first 1MB in 2GB)'): - assert run_c_test(ubman, fs_type, fs_img, - 'fs_test_load_big_2g_first', - big=BIG_FILE, md5val=md5val[4]) + ubman.run_ut('fs', 'fs_test_load_big_2g_first', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE, md5val=md5val[4]) def test_fs9(self, ubman, fs_obj_basic): """Test Case 9 - load, 1MB crossing 2GB boundary""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 9 - load (crossing 2GB boundary)'): - assert run_c_test(ubman, fs_type, fs_img, - 'fs_test_load_big_2g_cross', - big=BIG_FILE, md5val=md5val[5]) + ubman.run_ut('fs', 'fs_test_load_big_2g_cross', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE, md5val=md5val[5]) def test_fs10(self, ubman, fs_obj_basic): """Test Case 10 - load, reading beyond file end""" fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 10 - load (beyond file end)'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_load_beyond', - big=BIG_FILE) + ubman.run_ut('fs', 'fs_test_load_beyond', fs_type=fs_type, + fs_image=fs_img, big=BIG_FILE) def test_fs11(self, ubman, fs_obj_basic): """Test Case 11 - write""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 11 - write'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write', - small=SMALL_FILE, md5val=md5val[0]) + ubman.run_ut('fs', 'fs_test_write', fs_type=fs_type, + fs_image=fs_img, small=SMALL_FILE, md5val=md5val[0]) assert_fs_integrity(fs_type, fs_img) def test_fs12(self, ubman, fs_obj_basic): """Test Case 12 - write to "." directory""" fs_type, fs_img, _ = fs_obj_basic with ubman.log.section('Test Case 12 - write (".")'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dot') + ubman.run_ut('fs', 'fs_test_write_dot', fs_type=fs_type, + fs_image=fs_img) assert_fs_integrity(fs_type, fs_img) def test_fs13(self, ubman, fs_obj_basic): """Test Case 13 - write to a file with '/./'""" fs_type, fs_img, md5val = fs_obj_basic with ubman.log.section('Test Case 13 - write ("./")'): - assert run_c_test(ubman, fs_type, fs_img, 'fs_test_write_dotpath', - small=SMALL_FILE, md5val=md5val[0]) + ubman.run_ut('fs', 'fs_test_write_dotpath', fs_type=fs_type, + fs_image=fs_img, small=SMALL_FILE, md5val=md5val[0]) assert_fs_integrity(fs_type, fs_img) diff --git a/test/py/tests/test_upl.py b/test/py/tests/test_upl.py index c79c32adf0b..f2b69078cf1 100644 --- a/test/py/tests/test_upl.py +++ b/test/py/tests/test_upl.py @@ -33,5 +33,4 @@ def test_upl_handoff(ubman): assert 'UPL state: active' == output # Check the FIT offsets look correct - output = ubman.run_command('ut upl -f upl_test_info_norun') - assert 'failures: 0' in output + ubman.run_ut('upl', 'upl_test_info') diff --git a/test/py/tests/test_vbe.py b/test/py/tests/test_vbe.py index 4ccf4fb937b..411ed429605 100644 --- a/test/py/tests/test_vbe.py +++ b/test/py/tests/test_vbe.py @@ -127,9 +127,7 @@ def test_vbe_extlinux_fit_no_oem(ubman): fname = os.path.join(ubman.config.persistent_data_dir, 'vbe0.img') ubman.run_command(f'host bind 0 {fname}') - ubman.run_command('ut -f bootstd vbe_test_abrec_no_oem_norun') - result = ubman.run_command('echo $?') - assert '0' == result + ubman.run_ut('bootstd', 'vbe_test_abrec_no_oem') @pytest.mark.boardspec('sandbox') def test_vbe_extlinux_fit_oem(ubman): @@ -137,6 +135,4 @@ def test_vbe_extlinux_fit_oem(ubman): fname = os.path.join(ubman.config.persistent_data_dir, 'vbe1.img') ubman.run_command(f'host bind 0 {fname}') - ubman.run_command('ut -f bootstd vbe_test_abrec_oem_norun') - result = ubman.run_command('echo $?') - assert '0' == result + ubman.run_ut('bootstd', 'vbe_test_abrec_oem') From patchwork Mon Dec 29 16:06:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1103 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024404; bh=BkUXsdJffuUbGG4/y05DgqRGNyHoX4nblkdhyb20uL0=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=MPBTSBSSTvfZUGtIg/CYbfnCs5fsq/wLdNdKtjqLNa8K8dPSzzQIJANAlbcgXWvuc SjrMRNEQsq3NMnchZwPwMXvuWuxIuzl+nuFty1kHDMXJnNrEw46qVkpbVHiibWeG0S KHGBuMKf/iX1tI0of3Wjd7QvSClG8s2DmSTbNDo7NAjMcae8Ip+Tr/iFc5ry/qTwzy OfikSEqn7OnmksjIAOzZlvmkrYbWxOo1gQIPwtzDYYG99Vi9eWh7NQcOqqjyJBx21o SjPJN/tGyy/FC0GZTVhH+SsFSRw09pny6XD8UGpsFZcSAFO+kuUxy/dMiugQAP+pjR D8nQdexPjwKoQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 329E964E2A for ; Mon, 29 Dec 2025 09:06:44 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id jYRFXFQj56eo for ; Mon, 29 Dec 2025 09:06:44 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024403; bh=BkUXsdJffuUbGG4/y05DgqRGNyHoX4nblkdhyb20uL0=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=M7K93EWVRp4jHlbcWV4YgRFooGY7731m2AU9tohoXWAXzn2K048QecQQmzai6lJHH JRdRitFYV6/TqQU4xOnsbP7wOEV5NY4w785I9LDmTC/t1/wp+UzbaDx+jjIy10UfFm FDGovlE03JHSPAOMMQ153klCdpMPo6eTA4jp8h/rikSZWIm5vSZ3sYrgMIZaK8lktS OKhMNYkw25jZbltpi5FArdMuCYPH8lcmXMehwE6BobsQUtsjEDHpX52rm5YAcKe8a7 0UZxqAW8hSIQlC2fHRhWiM7VhKc1ZdrgLjsPDW6nMpsebb78KckUZKonAJ6WHFcuq0 6I3OuNxRL/4Gw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 36B9868F2E for ; Mon, 29 Dec 2025 09:06:43 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024401; bh=j3EIO9nd7Xxb9GgVtIT/OTk41QLBX2y0G5To6WvB6V0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PuBbrcfuNB918RlN8ynAnlWD3xYf7UxdmgAOjWVgwz7XrtzFQbKGGvRy7bouhhSAn yjfFumBfXMxRmeyuoByse9fD0K8EVU46OnihGQfYT1hH6n33njeAQr/3+aFyUfMPpU URVWBJSp4dyCGlMRL2RpA8owpF+U80nWhRK9BDYYtM0uGpF8F5zGxJQaUWA2CE/Pem tMIaYa+F6U9MRpyUJfPvulvlX0d232/yu79XqG8EybG2kUjJu725qRL54hR8g2LWVZ fMDszelqVCBJQy5DoJmUOvLbnVYptjzwZMQVKwOCLUXwNqmBKNSE1cOxZUoUoDIw7p FgsSwMHoWyW8A== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 78F7B68CAC; Mon, 29 Dec 2025 09:06:41 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id OTIOff-CtWRA; Mon, 29 Dec 2025 09:06:41 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024401; bh=+nW4H7r85ATAW6wrBTgBjNTURanMpqUSqu3P5Aob1IE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wgjd/waQ5PKXztZXuJju3k4IsezEjTk4Raiq2p0yh6jYljnRi6Cyi7PN3s88UHCgz 3fvAHMvultIxmwegAsxWOc8G5XqgXJUFZRKSk9viFsZQTviiC85nhTgQSrGVF+yV85 0LQ0iyhMZbGsuAaZgBiYx14SPic+I3+0Iv7i4VbL8or0+mfTr5YCmkrt2pa3dF03PB JBq7mugh8DeE4IZVGBAA1lRy8yC9z96SEiHEFQm3F/itx36WlFhIAJOTHbidLqOqoV +Zir+g60ewdxtpDF7eSyImgcwyU3RDO+/xMsYC/TbgRdUPydelWZl1/CtUuVcnmB1V rTLEaC/wDLRBw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id C7DB968B78; Mon, 29 Dec 2025 09:06:40 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:03 -0700 Message-ID: <20251229160611.3899708-6-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 6T675KGL2SD4X4NS4M7ZNVHFXWHJWA3Y X-Message-ID-Hash: 6T675KGL2SD4X4NS4M7ZNVHFXWHJWA3Y X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 5/9] test: ut: Add -m as alias for the -f flag List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add -m as an alias for the -f flag which forces manual tests to run. This provides consistency with external test runners that will use -m for "manual" tests. Also update the documentation to explain what manual tests are, and fix a typo ("types" -> "times") in the -r description. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/usage/cmd/ut.rst | 10 ++++++---- test/cmd_ut.c | 5 +++-- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index a26ee6ad7de..07cd970da6c 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-r] [-f] [-R] [-I:] [ | all []] [...] + ut [-fmr] [-R] [-I:] [ | all []] [...] ut [-s] info Description @@ -26,11 +26,13 @@ suite test Speciifes a particular test to run, within a suite, or all suites --f - Forces running of a manual test. +-f, -m + Force running of manual tests. Manual tests have the `_norun` suffix and + are normally skipped because they require external setup (e.g., creating + disk images from Python/pytest). -r - Specifies the number of types to run each test + Specifies the number of times to run each test -I : Test to run after other tests have run. This is used to find which test diff --git a/test/cmd_ut.c b/test/cmd_ut.c index 8d8a2d763a2..050f7ee6caf 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -272,6 +272,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) runs_per_text = dectoul(str + 2, NULL); break; case 'f': + case 'm': force_run = true; break; case 'I': @@ -340,9 +341,9 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) } U_BOOT_LONGHELP(ut, - "[-rs] [-f] [-R] [-I:] [ [...]] - run unit tests\n" + "[-fmrs] [-R] [-I:] [ [...]] - run unit tests\n" " -r Number of times to run each test\n" - " -f Force 'manual' tests to run as well\n" + " -f/-m Force 'manual' tests to run as well\n" " -I Test to run after other tests have run\n" " -R Preserve console recording on test failure\n" " -s Show all suites with ut info\n" From patchwork Mon Dec 29 16:06:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1104 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024409; bh=iN4pgl81ktR49OVjQCNAzF6Q66V2HtR2wWriaZ06TIg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Z4Yh8OzaKjir/NkyvZ9abWIlLE4wghaSrTgexZbmJWLHrA+ijNgW9f9nR4XACmE5r py9l6t5Vq/KPlOo9iS72KoaEAayo+enHz5LeAPgi6IgWMCjSs8t6Nz2EHav2N9j6W7 uH4oZMV3ElSJ0UXVITpVvEqiZNdQCq9FLbaQSYf0t+CPSv60lQBCVpxEIyJbEcf/25 AFx0T2fBa0qR+eny7VrGypKBFzYjmNwuKnIUNQosEczYVBqdoz+/bBcPuzH87u8Afv kLg02Tst2zNj9iyW4O4XO07hWrbYtl1ShX32xMXcbFO2O/w/q3b+aTtAruyk25VMbN dBaIIi43et9dw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 3EF5968F2C for ; Mon, 29 Dec 2025 09:06:49 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id ndOX21oXKT2S for ; Mon, 29 Dec 2025 09:06:49 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024409; bh=iN4pgl81ktR49OVjQCNAzF6Q66V2HtR2wWriaZ06TIg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=Z4Yh8OzaKjir/NkyvZ9abWIlLE4wghaSrTgexZbmJWLHrA+ijNgW9f9nR4XACmE5r py9l6t5Vq/KPlOo9iS72KoaEAayo+enHz5LeAPgi6IgWMCjSs8t6Nz2EHav2N9j6W7 uH4oZMV3ElSJ0UXVITpVvEqiZNdQCq9FLbaQSYf0t+CPSv60lQBCVpxEIyJbEcf/25 AFx0T2fBa0qR+eny7VrGypKBFzYjmNwuKnIUNQosEczYVBqdoz+/bBcPuzH87u8Afv kLg02Tst2zNj9iyW4O4XO07hWrbYtl1ShX32xMXcbFO2O/w/q3b+aTtAruyk25VMbN dBaIIi43et9dw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 2E52768CAC for ; Mon, 29 Dec 2025 09:06:49 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024406; bh=ArJIKOEWbdITvqWal4ScFayvOoD1q72b17nE/NApgr0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CV1kbX2yYgEfmp2N4VHt/7/vkG97epK+PJYUgHsexL11sPiwxDMO1bJVu+5uLDduz 5o2rRRva+KNKzEOshCxTjo4v1SgiqfsN1s/exlItnhuDcJFnM5JPW3ExrXP6h1JgWh tdy0Iv6Xq9mmpIzooedrdNvB64feSstEkAzlK5NtWafDaJgS9Ccfe15ptvbSDFMGOB knjbgN15OGsKCgcFGvkSn8LuZ6+6/a6yZpmTnIlh72sVbf7nLLbfRge3SoZUbD7acg 8SfQAAF+o7KXckGCv3zVFsLRB6joKyK27iKJtB5DsWCrMc4XF5bDcd0wAAmDc/jUJM uobpeyZjHwGNg== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 6F8FF68CAC; Mon, 29 Dec 2025 09:06:46 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id m-SNVYFoO__x; Mon, 29 Dec 2025 09:06:46 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024402; bh=ffbmLaCsROtGJXJ+WxseFeziI54SBMPT6NMG+YicSic=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MzLKq5iIomCuBt1p7uHf0nkXeHCo2CLm8vB2oEUG8arDDxGpI1PMxfL9KLgrgby4B 8q3UUUKivTpKNcisjO7U84iZoqgEj7SqYR8BNEbA3d6CQ3A/QyYQr2NbxXBssycp+W za0o7Nc8qOzarbMAeKFzs/1B53h3a60fVXIEXGsGLuxo0BbIG6bUmnQnWRBFU+DAe8 UMgH+E9NjLnnoA0Gun2ZaNo5Hcf9/AR4r+RrGCbihqzvYtrUtBNdjEW6ir7aICo2bd dOKF6ou2fI1yER/2eIUzfXUFYMPfwLWcg/w2IEjW9uHR5VvCodEScD7bQPQPYqdNIL zx9zxJK6ZiT3w== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id B765E68B78; Mon, 29 Dec 2025 09:06:41 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:04 -0700 Message-ID: <20251229160611.3899708-7-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: BVO7P3FQPBNWF6AHWDWI3CTAB3K6QPP3 X-Message-ID-Hash: BVO7P3FQPBNWF6AHWDWI3CTAB3K6QPP3 X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 6/9] test: ut: Refactor argument processing to use a loop List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The current argument-parsing logic uses switch (str[1]) which only processes the second character of each argument. This prevents combining multiple single-character flags in one argument (e.g., -fm). Refactor the code to use a for loop that iterates through all characters in the argument. For flags that take a value (like -r and -I), use goto to skip the rest of the argument after processing. This allows combined flags like -fmR instead of requiring -f -m -R. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- test/cmd_ut.c | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/test/cmd_ut.c b/test/cmd_ut.c index 050f7ee6caf..37144242099 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -267,26 +267,29 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) while (argc > 0 && *argv[0] == '-') { const char *str = argv[0]; - switch (str[1]) { - case 'r': - runs_per_text = dectoul(str + 2, NULL); - break; - case 'f': - case 'm': - force_run = true; - break; - case 'I': - test_insert = str + 2; - if (!strchr(test_insert, ':')) - return CMD_RET_USAGE; - break; - case 'R': - keep_record = true; - break; - case 's': - show_suites = true; - break; + for (str++; *str; str++) { + switch (*str) { + case 'r': + runs_per_text = dectoul(str + 1, NULL); + goto next_arg; + case 'f': + case 'm': + force_run = true; + break; + case 'I': + test_insert = str + 1; + if (!strchr(test_insert, ':')) + return CMD_RET_USAGE; + goto next_arg; + case 'R': + keep_record = true; + break; + case 's': + show_suites = true; + break; + } } +next_arg: argv++; argc--; } From patchwork Mon Dec 29 16:06:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1105 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024413; bh=JiAv5qqx5vn84wpf+Zx0eUNS3CvWihDBm2CwBj/NNOg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=jUkkngwtk3MOz84lnZx3STnO4+7ONCI3nR6xQGApBJ8i32v+n73/WC7BL8MRpNKAT PvBE8mtaMrg19vIkJt3yTpjfSFDC9k+d0hIzsk3SB+W5RJLUv6uffHaWit7zafnB+X E+moNSY84Vt2/rzB/7BadxJsaFrskoPsU439IhpN65ex8Y7Tybfnr8Perqf1Nk6rRh BdoavGrO5ohmwknnEG7VMJXokvkBR1sHH1ZXLUnDBlcjb248sv74iecL2JWPAOR294 8rY2QkOjQX7+/g4oCzBP2vmfLHdhynBiWv1CvASj2UjjD7Nz9RXj23EGx6TpxLZI/C vk/tofDdcC1dQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CA60568CAD for ; Mon, 29 Dec 2025 09:06:53 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id v_E0z5BjTqJe for ; Mon, 29 Dec 2025 09:06:53 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024413; bh=JiAv5qqx5vn84wpf+Zx0eUNS3CvWihDBm2CwBj/NNOg=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=jUkkngwtk3MOz84lnZx3STnO4+7ONCI3nR6xQGApBJ8i32v+n73/WC7BL8MRpNKAT PvBE8mtaMrg19vIkJt3yTpjfSFDC9k+d0hIzsk3SB+W5RJLUv6uffHaWit7zafnB+X E+moNSY84Vt2/rzB/7BadxJsaFrskoPsU439IhpN65ex8Y7Tybfnr8Perqf1Nk6rRh BdoavGrO5ohmwknnEG7VMJXokvkBR1sHH1ZXLUnDBlcjb248sv74iecL2JWPAOR294 8rY2QkOjQX7+/g4oCzBP2vmfLHdhynBiWv1CvASj2UjjD7Nz9RXj23EGx6TpxLZI/C vk/tofDdcC1dQ== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id B8B5D68F2D for ; Mon, 29 Dec 2025 09:06:53 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024411; bh=HR8n6FR/q5pkNytm4grZ1iLj3SQoDyyJzjxBRa36lX0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RviJJEnh8lOV8gIPvNIF8WNaDcj3nB22gKtqoMCIzKPwjhz5T5Nl4uElsqsiP/LpL oM7c5og0+YHVsU0y5geTtVchJ36qXASNvNcGS1drrsZzodE4teMnMN019crhdwelz0 sEnoHqMF1T2e0/tqyzdSWcYi6DZMS+9Pn0X0cTNqWQo31vynnoLCg0P7AhbkcgWZxH HIbqQ7XuWf3xfaPn/LUVgdZTszXThKFAVXbvbyvTAlACUSXXBpi4AXVTW2Ud7swOd5 i5KVs3zVFM5uIHzWMSmgoTCzdp8/ZU9HNc8ktpq4VMakVG6C9il4916vl8N2JaXfs3 BpdlIAjbJRd1Q== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id A18B668CAD; Mon, 29 Dec 2025 09:06:51 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id fIlXqkcTOJA9; Mon, 29 Dec 2025 09:06:51 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024406; bh=CpQ3XVHnq4WAxXRWS4KkiN8oCLiO8GoTP8yP4Ee18bw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Pmfg10dTJktUsmkk7q3gw1DKwtsRbdbiWfLdMoG+XTSkund7ek1f4BaOdcEEIrJNO rGe1qJGh774W1LmRMtBRwtIdhaiPQuBvmWE115QLrqMyy4j4UjwDF/kWXk7o/+nI+Z taRux1+qJdpcXefMHM0daZ6pWdnZMXWCldbRgGQG/6sYAzIikgKIAznbxu7PfLVmtK ky5VOV9ADHzWf43cgSy5Q59oumhsTzyytTSZArqeIFtyAz8CqnPeB3/rgmgOV10eIO it44k+sl3uY0zZ7WA/fPdCj576Nt2ffwGgJQAwdTgd4EewrosPtDDy1ceAXw6vbjPr g+glf9Vf6hldw== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 64C2F64E2A; Mon, 29 Dec 2025 09:06:46 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:05 -0700 Message-ID: <20251229160611.3899708-8-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: GDLA5J7GGNR6VKT5RE4KRXBNV65NMRMM X-Message-ID-Hash: GDLA5J7GGNR6VKT5RE4KRXBNV65NMRMM X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 7/9] test: Add a flag to emit per-test result lines List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass The ut command shows test output but does not provide a machine-readable indication of whether each individual test passed or failed. External tools must rely on heuristics like scanning for failure patterns in the output. Add a -E flag that emits an explicit result line after each test: Result: PASS: test_name: file.c Result: FAIL: test_name: file.c This allows tools to reliably determine per-test pass/fail status without fragile pattern matching. The flag is optional to maintain backward compatibility with existing scripts. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/usage/cmd/ut.rst | 7 ++++++- include/test/test.h | 2 ++ test/cmd_ut.c | 8 +++++++- test/test-main.c | 9 +++++++++ 4 files changed, 24 insertions(+), 2 deletions(-) diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index 07cd970da6c..c40928dece9 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-fmr] [-R] [-I:] [ | all []] [...] + ut [-Efmr] [-R] [-I:] [ | all []] [...] ut [-s] info Description @@ -26,6 +26,11 @@ suite test Speciifes a particular test to run, within a suite, or all suites +-E + Emit a result line after each test, in the format + `Result: PASS|FAIL|SKIP: : `. This is useful for + automated parsing of test results. + -f, -m Force running of manual tests. Manual tests have the `_norun` suffix and are normally skipped because they require external setup (e.g., creating diff --git a/include/test/test.h b/include/test/test.h index 5ae90e39e00..74225a70e54 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -97,6 +97,7 @@ struct ut_arg { * @arg_count: Number of parsed arguments * @arg_error: Set if ut_str/int/bool() detects a type mismatch * @keep_record: Preserve console recording when ut_fail() is called + * @emit_result: Emit result line after each test completes * @priv: Private data for tests to use as needed */ struct unit_test_state { @@ -128,6 +129,7 @@ struct unit_test_state { int arg_count; bool arg_error; bool keep_record; + bool emit_result; char priv[UT_PRIV_SIZE]; }; diff --git a/test/cmd_ut.c b/test/cmd_ut.c index 37144242099..d6e591916ce 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -255,6 +255,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) bool show_suites = false; bool force_run = false; bool keep_record = false; + bool emit_result = false; int runs_per_text = 1; struct suite *ste; char *name; @@ -269,6 +270,9 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) for (str++; *str; str++) { switch (*str) { + case 'E': + emit_result = true; + break; case 'r': runs_per_text = dectoul(str + 1, NULL); goto next_arg; @@ -299,6 +303,7 @@ next_arg: ut_init_state(&uts); uts.keep_record = keep_record; + uts.emit_result = emit_result; name = argv[0]; select_name = cmd_arg1(argc, argv); @@ -344,7 +349,8 @@ next_arg: } U_BOOT_LONGHELP(ut, - "[-fmrs] [-R] [-I:] [ [...]] - run unit tests\n" + "[-Efmrs] [-R] [-I:] [ [...]] - run unit tests\n" + " -E Emit result line after each test\n" " -r Number of times to run each test\n" " -f/-m Force 'manual' tests to run as well\n" " -I Test to run after other tests have run\n" diff --git a/test/test-main.c b/test/test-main.c index c9e164da678..2524a154186 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -624,6 +624,7 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test, { const char *fname = strrchr(test->file, '/') + 1; const char *note = ""; + int old_fail_count; int ret; if ((test->flags & UTF_DM) && !uts->of_live) @@ -639,6 +640,7 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test, if (ret) return ret; + old_fail_count = uts->cur.fail_count; uts->arg_error = false; ret = test->func(uts); if (ret == -EAGAIN) @@ -650,6 +652,13 @@ static int ut_run_test(struct unit_test_state *uts, struct unit_test *test, ut_set_state(NULL); + if (uts->emit_result) { + bool passed = uts->cur.fail_count == old_fail_count; + + printf("Result: %s: %s: %s%s\n", passed ? "PASS" : "FAIL", + test_name, fname, note); + } + return 0; } From patchwork Mon Dec 29 16:06:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1106 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024418; bh=73wbTWQA2y46vWPUbDC6zt+IBxY57oSq3g/Ay1zcJ5g=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gHd0OEo6yRIcNkb5PSJbdikXzPjOV09V5wzleQc0cb/0WgGaYiQKnrm6uK5DF87HV wQH0aUIE8602ro16oOJEk+5Vhuo9WYB93HnZGfwcWd8Qbk13UKugCiKwYFSP+t3p7f ttKqgy+5+9CCGjhW7QGOWQTs+rjVjrInIXt4nc4KTkYxEwfYFGWflJGHW+xHbJodwI LyLTBl4JW2VFpbAGMgeoPpCRmAWygCWpNEDlxQ0GwJJ9QlXIiC/x5CgWp1nJFWaQJZ wY6xIRkCjwvfi4R6i2udK/40bt+ng9MZTyzSGDrQm7Apc4VN2Z84ZjLJWvxEZjMUZw r+p2wDJHZQ2jw== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 62D5A68F31 for ; Mon, 29 Dec 2025 09:06:58 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 1Xhi6LENZh6k for ; Mon, 29 Dec 2025 09:06:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024418; bh=73wbTWQA2y46vWPUbDC6zt+IBxY57oSq3g/Ay1zcJ5g=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=gHd0OEo6yRIcNkb5PSJbdikXzPjOV09V5wzleQc0cb/0WgGaYiQKnrm6uK5DF87HV wQH0aUIE8602ro16oOJEk+5Vhuo9WYB93HnZGfwcWd8Qbk13UKugCiKwYFSP+t3p7f ttKqgy+5+9CCGjhW7QGOWQTs+rjVjrInIXt4nc4KTkYxEwfYFGWflJGHW+xHbJodwI LyLTBl4JW2VFpbAGMgeoPpCRmAWygCWpNEDlxQ0GwJJ9QlXIiC/x5CgWp1nJFWaQJZ wY6xIRkCjwvfi4R6i2udK/40bt+ng9MZTyzSGDrQm7Apc4VN2Z84ZjLJWvxEZjMUZw r+p2wDJHZQ2jw== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 516F768F2C for ; Mon, 29 Dec 2025 09:06:58 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024416; bh=+isfZIuKHzwwz+H8Hm9gFaozFYyzGYS6NLw+gN0ZlY8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X9eBnC6/8gXrbmWcMCfppsWlRbuiydh+VnU4oPmSuCqa5ygw4z2B5Ccf4VaT3DYOH fEbWztyvSddFDm3Bwayk+mjV1iBme1vOnzLpVjJg8u/BoJtqi04jpmKXAsORJYkMFM jhPmnfNotuv9TOpMpoAel28OhzDnDc7MVgA+XfN8HCqZkM7dt6VjOC6tKwvv9pApMd 4Y3SHREX88BR4gHLDnenPMQljDftaCKd/E/fSXL7zO5f9Wj4ROeOzRkiq6YRZAcoi/ YkMMxKWFG6rTxf7vyZlL2cmjG1EfFlHMUF/ELUH3TGA/NrqZr+ickkepc4TjUiYD1X bI9QxhyDX4fRQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 39B9868CAD; Mon, 29 Dec 2025 09:06:56 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id j4C5WvXqh_f4; Mon, 29 Dec 2025 09:06:56 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024411; bh=eIYcXrdV3Bmwj2fgSvJlF4I0AfAUz4Lk2kaWE0ehp/M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sxNd9k2zrSXhgN4QUFckjNzSajpnj/6eOMKpBIA3CMZNzSDqyOHLPIWQBG9bOpcFt HKeRjhdGQMmCe380fwjAYN68xxA8Wfduzj7MR+iW176k+DGCihyMWrWtzC1jbAuJI4 Tq6xLPZxkskaII6TB6Z7aIdTvereohharNtRurAknPSQgeqIwpcHTCMjjeg/scs2hC yXPT4RfZRbfX16a0UlRtqKpmm9AttokRIYxi6hcTTGC28neQeRXSG32lSwDGoBfhDR hGUPaxBP9HUQHPsFBY5S/g96vCKpjRO/SSOZSBlsilZZwdsfjXb97I+eiugS2oITL4 YeBrR4/Xbv42w== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id 0A96C68CAC; Mon, 29 Dec 2025 09:06:50 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:06 -0700 Message-ID: <20251229160611.3899708-9-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: GU3KCOVFJL6WT5SPP4LYPFOA7BXJRBQC X-Message-ID-Hash: GU3KCOVFJL6WT5SPP4LYPFOA7BXJRBQC X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 8/9] doc: ut: Document return value List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Add documentation for the return value of the ut command: - Returns 0 on success if all tests pass - Returns 1 on failure if any test fails - Skipped tests do not cause a failure Also explain when tests may be skipped and how to detect skipped tests programmatically using the -E flag. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/usage/cmd/ut.rst | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index c40928dece9..731579e5b1d 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -84,6 +84,23 @@ To specify a list of suites to run, can also be a comma-separated list. See :ref:`develop/tests_writing:writing c tests` for more information on how to write unit tests. +Return Value +------------ + +The `ut` command returns 0 (success) if all tests pass, or 1 (failure) if any +test fails. + +Skipped tests do not cause a failure return. Tests may be skipped for several +reasons: + +- Manual tests (with `_norun` suffix) are skipped unless `-f` or `-m` is used +- Tests requiring features not available on the current platform (e.g., + `UTF_OTHER_FDT` on non-sandbox, console recording disabled) +- Tests that explicitly request to be skipped by returning `-EAGAIN` + +To detect skipped tests programmatically, use the `-E` flag and check for +`Result: SKIP:` lines in the output. + ut all ~~~~~~ From patchwork Mon Dec 29 16:06:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Glass X-Patchwork-Id: 1107 Return-Path: X-Original-To: u-boot-concept@u-boot.org Delivered-To: u-boot-concept@u-boot.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024420; bh=9+zkuFj45eXVfdKhaIIPsuh+a98JduRUqSvEVyYWLnA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=wgAutjZQWJlGeJsNq+idFeRwtfANJ2v5AsJvD3rxwTu1guSvYT+rdeTziOprlYmnb 6gyQ03KwZliLygcdJGu0abSk1oU1xSysjCYX2XzgRMdme1RwVZegy7+kJx/VHIfdWX mw2LhjvJLV9n+w8vpNiB1ZT7LGBB8DRXo9wAMt6O5BFMYFYyngFK/Po+2YjtxJfBum HgVBc9eTdMPoxK7eNs1uFBMYVQxF3pVGtH859JnwIRJYFl3r9ukrJhxTcyDGHknJJz DcYJUmV4dAv8uukMeNt0DvDnp5pc0RhN0wSqTDILLgaDhlSevS1HtoAyo25/9BCrnJ JQhajXNb+L8cQ== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id CCA2068CAC for ; Mon, 29 Dec 2025 09:07:00 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 0MlOqNszcSIj for ; Mon, 29 Dec 2025 09:07:00 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024419; bh=9+zkuFj45eXVfdKhaIIPsuh+a98JduRUqSvEVyYWLnA=; h=From:To:Date:In-Reply-To:References:CC:Subject:List-Id: List-Archive:List-Help:List-Owner:List-Post:List-Subscribe: List-Unsubscribe:From; b=H/9OwjSYlKUAfBF8C40e2fkDg0RmDmvX/gbcTbEK+KgmsWe61AhlvzomMM6XHj9M6 lC4XULFxwegQTt9GoncKXSNj989eAeAj51meWymuJ9OcHibpEk8y8Nk9asFn+AqdRw o4hVaxEDa7B/SJhPNnuudjxpgLAAR3/Zy9J9Li46B9k0acfd3SeSzEG5SQ9wNEFL/0 Lbkz1GtEGSqxKFXoszoRmOfsyiIhjpYyrdRGBCjQa4LIiXBQv/j8hMJYWRCBTjdAMI kEfbpE1Sv+c1Bulk2ZjxR+nlOQ4C9ikqxjwzJFIZhyr3mCHmzch+GcyRzfgkwzxHmd qarbrzetqLsNA== Received: from mail.u-boot.org (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 8AF8D68F31 for ; Mon, 29 Dec 2025 09:06:59 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024416; bh=JBAO//IO0tWN/k9hqBj/ITw6PiWMNCv6HGfjBpQ9TCI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YEtmEg+EN63qASAStPvLzmUX+MGes0OellHy5wMOnXk/KN6v2poLJ0BKj11qmmlmU +3HvMFUJ8G8tLVkIuCRbLtsMHfGb1lmjYO+4l/UlWZsa/H3ByDKP303VyK603/PgCT SZEOfqlPfLnsibMnigowh7qzqeHGFqcjQogb151fIQR5QppQ26o0iKaAySzgtvshYj MFX2MTjRNNudzK1zeWbPIkkXPFYz12H37HJ9OHAsb9we5En4q1TeeDQnsCP1+AYYBT H3MyzV42ueTOmrG1oq9ErMrfkY1mgyiKhFQTdRjE/tJpCB7gRc651tIpsCktLTfFLq R2sM1l9hW0QwA== Received: from localhost (localhost [127.0.0.1]) by mail.u-boot.org (Postfix) with ESMTP id 5DCD168CAC; Mon, 29 Dec 2025 09:06:56 -0700 (MST) X-Virus-Scanned: Debian amavis at Received: from mail.u-boot.org ([127.0.0.1]) by localhost (mail.u-boot.org [127.0.0.1]) (amavis, port 10026) with ESMTP id jrkAjk0iFwaK; Mon, 29 Dec 2025 09:06:56 -0700 (MST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=u-boot.org; s=default; t=1767024416; bh=s/2by+IzxTtQ6C2Q+qrlYKnL0cmarKYJP4sULWCD7UA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rQKWSFqJK3lTankH3cxjakYCIY9g48G+9ynAE3IKm7JAWwk79Db1IO84JO3p4QNPO fem+FC/zPYHs/AHYwLtGXF54yjztu+oso0E9YaljaiqomHOrEfKazR3gMQJMWsJHsI +ZTEgIt7oQOR7OhNlT7799HBGz7eNv+p87D8L5BsY4CkNsoLyJxufSORcAm+fMKlwj cmCN+WPGt7s1Q3NnHOTHZzr97gS7DwiTR/UhQpzPLUQN9h7SjEtrEXxAccXpZFYh7j J4LYbWOmcDC0R00oZOu0Vkq5nCkq+IYZ9GkeNaDanWivECOY2iTggR+h1bX+eyd0zw 3dfuK+MUKirNQ== Received: from u-boot.org (unknown [73.34.74.121]) by mail.u-boot.org (Postfix) with ESMTPSA id A2D9264E24; Mon, 29 Dec 2025 09:06:55 -0700 (MST) From: Simon Glass To: U-Boot Concept Date: Mon, 29 Dec 2025 09:06:07 -0700 Message-ID: <20251229160611.3899708-10-sjg@u-boot.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251229160611.3899708-1-sjg@u-boot.org> References: <20251229160611.3899708-1-sjg@u-boot.org> MIME-Version: 1.0 Message-ID-Hash: 4NMF26EB25Q5L2A6ILPMJ2KBCYKY4V7Y X-Message-ID-Hash: 4NMF26EB25Q5L2A6ILPMJ2KBCYKY4V7Y X-MailFrom: sjg@u-boot.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Heinrich Schuchardt , Simon Glass , "Claude Opus 4 . 5" X-Mailman-Version: 3.3.10 Precedence: list Subject: [Concept] [PATCH 9/9] test: Add parallel test execution support List-Id: Discussion and patches related to U-Boot Concept Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: Simon Glass Running the full test suite takes a long time. It would be useful to distribute tests across multiple sandbox instances to speed up testing. Add support for running tests in parallel across multiple sandbox instances. Each worker runs a subset of tests based on its worker ID. Add -P: option to the ut command where n is the total number of workers and w is this worker's ID (0 to n-1). Tests are distributed by index modulo number of workers. Co-developed-by: Claude Opus 4.5 Signed-off-by: Simon Glass --- doc/develop/py_testing.rst | 28 ++++++++++++++++++++++++++++ doc/usage/cmd/ut.rst | 8 +++++++- include/test/test.h | 4 ++++ test/cmd_ut.c | 16 +++++++++++++++- test/test-main.c | 4 ++++ 5 files changed, 58 insertions(+), 2 deletions(-) diff --git a/doc/develop/py_testing.rst b/doc/develop/py_testing.rst index 303c07040b3..cfb8f959e12 100644 --- a/doc/develop/py_testing.rst +++ b/doc/develop/py_testing.rst @@ -171,6 +171,34 @@ require a large amount of refactoring, e.g. with more use of pytest fixtures. The code-coverage tests are omitted since they cannot run in parallel due to a Python limitation. +Parallel C unit tests +~~~~~~~~~~~~~~~~~~~~~ + +The ``ut`` command supports distributing tests across multiple sandbox +instances using the ``-P`` flag. This is useful when running tests directly +from the command line without pytest. + +To run tests in parallel across 4 workers:: + + # Terminal 1 + /tmp/b/sandbox/u-boot -T -c "ut -P4:0 dm" + + # Terminal 2 + /tmp/b/sandbox/u-boot -T -c "ut -P4:1 dm" + + # Terminal 3 + /tmp/b/sandbox/u-boot -T -c "ut -P4:2 dm" + + # Terminal 4 + /tmp/b/sandbox/u-boot -T -c "ut -P4:3 dm" + +The format is ``-P:`` where ``n`` is the total number of workers and +``w`` is this worker's ID (0 to n-1). Tests are distributed by index modulo +the number of workers, so each worker runs a disjoint subset. + +This can be combined with other flags, e.g. ``-EP4:0`` to emit result lines +while running as worker 0 of 4. + Testing under a debugger ~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index 731579e5b1d..99419c0d384 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -11,7 +11,7 @@ Synopsis :: - ut [-Efmr] [-R] [-I:] [ | all []] [...] + ut [-Efmr] [-R] [-I:] [-P:] [ | all []] [...] ut [-s] info Description @@ -44,6 +44,12 @@ test causes another test to fail. If the one test fails, testing stops immediately. +-P : + Run as worker `` of `` parallel workers. Tests are distributed by + index modulo number of workers, so each worker runs a disjoint subset of + tests. This allows running tests in parallel across multiple sandbox + instances. + -R Preserve console recording on test failure. Normally when a test fails, console recording is disabled so error messages go directly to output. diff --git a/include/test/test.h b/include/test/test.h index 74225a70e54..56e25f6fa9d 100644 --- a/include/test/test.h +++ b/include/test/test.h @@ -89,6 +89,8 @@ struct ut_arg { * @of_other: Live tree for the other FDT * @runs_per_test: Number of times to run each test (typically 1) * @force_run: true to run tests marked with the UTF_MANUAL flag + * @workers: Number of parallel workers, 0 if not sharding tests + * @worker_id: ID of this worker (0 to workers-1) * @old_bloblist: stores the old gd->bloblist pointer * @soft_fail: continue execution of the test even after it fails * @expect_str: Temporary string used to hold expected string value @@ -121,6 +123,8 @@ struct unit_test_state { struct device_node *of_other; int runs_per_test; bool force_run; + int workers; + int worker_id; void *old_bloblist; bool soft_fail; char expect_str[1024]; diff --git a/test/cmd_ut.c b/test/cmd_ut.c index d6e591916ce..a35ac69434d 100644 --- a/test/cmd_ut.c +++ b/test/cmd_ut.c @@ -257,6 +257,7 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) bool keep_record = false; bool emit_result = false; int runs_per_text = 1; + int workers = 0, worker_id = 0; struct suite *ste; char *name; int ret; @@ -285,6 +286,15 @@ static int do_ut(struct cmd_tbl *cmdtp, int flag, int argc, char *const argv[]) if (!strchr(test_insert, ':')) return CMD_RET_USAGE; goto next_arg; + case 'P': { + const char *colon = strchr(str + 1, ':'); + + if (!colon) + return CMD_RET_USAGE; + workers = dectoul(str + 1, NULL); + worker_id = dectoul(colon + 1, NULL); + goto next_arg; + } case 'R': keep_record = true; break; @@ -304,6 +314,8 @@ next_arg: ut_init_state(&uts); uts.keep_record = keep_record; uts.emit_result = emit_result; + uts.workers = workers; + uts.worker_id = worker_id; name = argv[0]; select_name = cmd_arg1(argc, argv); @@ -349,11 +361,13 @@ next_arg: } U_BOOT_LONGHELP(ut, - "[-Efmrs] [-R] [-I:] [ [...]] - run unit tests\n" + "[-Efmrs] [-R] [-I:] [-P:] [ [...]]\n" + " - run unit tests\n" " -E Emit result line after each test\n" " -r Number of times to run each test\n" " -f/-m Force 'manual' tests to run as well\n" " -I Test to run after other tests have run\n" + " -P: Run as worker of parallel workers\n" " -R Preserve console recording on test failure\n" " -s Show all suites with ut info\n" " Test suite to run (or comma-separated list)\n" diff --git a/test/test-main.c b/test/test-main.c index 2524a154186..a2c4e32423b 100644 --- a/test/test-main.c +++ b/test/test-main.c @@ -790,6 +790,10 @@ static int ut_run_tests(struct unit_test_state *uts, const char *prefix, !test_matches(prefix, test_name, select_name)) continue; + /* Skip tests not assigned to this worker */ + if (uts->workers && upto % uts->workers != uts->worker_id) + continue; + if (test->flags & UTF_MANUAL) { int len;