[Concept,32/35] malloc: Skip backtrace when stack is corrupted

Message ID 20251210000737.180797-33-sjg@u-boot.org
State New
Headers
Series malloc: Add heap debugging commands and mcheck caller tracking |

Commit Message

Simon Glass Dec. 10, 2025, 12:07 a.m. UTC
  From: Simon Glass <simon.glass@canonical.com>

When the stack is corrupted (e.g., by the stack protector test),
collecting a backtrace during malloc causes a crash because the
backtrace code walks the invalid stack frames.

Update __stack_chk_fail() to set the flag before calling panic()

Also update stackprot_test() to set the flag before intentionally
corrupting the stack. This is needed because of the printf() in the
test: on sandbox printf() results in truetype allocations due to the
console output.

These fixes allow the stack protector test to pass with mcheck enabled.

Co-developed-by: Claude <noreply@anthropic.com>
Signed-off-by: Simon Glass <simon.glass@canonical.com>
---

 cmd/stackprot_test.c | 7 +++++++
 common/stackprot.c   | 6 ++++++
 2 files changed, 13 insertions(+)
  

Patch

diff --git a/cmd/stackprot_test.c b/cmd/stackprot_test.c
index e7ff4a06158..d7fbc3ecca0 100644
--- a/cmd/stackprot_test.c
+++ b/cmd/stackprot_test.c
@@ -4,6 +4,7 @@ 
  */
 
 #include <command.h>
+#include <malloc.h>
 
 static int do_test_stackprot_fail(struct cmd_tbl *cmdtp, int flag, int argc,
 				  char *const argv[])
@@ -14,6 +15,12 @@  static int do_test_stackprot_fail(struct cmd_tbl *cmdtp, int flag, int argc,
 	 */
 	char a[128];
 
+	/*
+	 * Disable backtrace collection before corrupting the stack.
+	 * Otherwise, any malloc (e.g., from printf/font rendering) will
+	 * attempt to collect a backtrace from the corrupted stack and crash.
+	 */
+	malloc_backtrace_skip(true);
 	memset(a, 0xa5, 512);
 
 	printf("We have smashed our stack as this should not exceed 128: sizeof(a) = %zd\n",
diff --git a/common/stackprot.c b/common/stackprot.c
index 4e3297b7d00..408cd6d1e05 100644
--- a/common/stackprot.c
+++ b/common/stackprot.c
@@ -4,6 +4,7 @@ 
  */
 
 #include <asm/global_data.h>
+#include <malloc.h>
 
 DECLARE_GLOBAL_DATA_PTR;
 
@@ -13,6 +14,11 @@  void __stack_chk_fail(void)
 {
 	void *ra;
 
+	/*
+	 * When the stack is corrupted, backtrace collection will crash.
+	 * Skip it before calling panic().
+	 */
+	malloc_backtrace_skip(true);
 	ra = __builtin_extract_return_addr(__builtin_return_address(0));
 	panic("Stack smashing detected in function:\n%p relocated from %p",
 	      ra, ra - gd->reloc_off);