diff mbox series

[Concept,22/24] test/py: Speed up VT100-filtering in expect()

Message ID 20260103011908.149445-23-sjg@u-boot.org
State New
Headers
Series Malloc debugging and test/py improvements |

Commit Message

Simon Glass Jan. 3, 2026, 1:18 a.m. UTC 
  From: Simon Glass <simon.glass@canonical.com>

The VT100 escape sequence filter is applied to the entire buffer on
every read iteration, causing O(n^2) performance. With 1.6MB of output
from verbose unit tests and 1KB reads, this results in ~1600 iterations,
each processing an ever-growing buffer.

Fix this by filtering only the newly received data before appending it
to the buffer.

To test this, dm_test_host() was modified to do 10 malloc_dump() calls,
thus producing a lot of output. The impact of this patch is:

  total time 6s  -> 4.5s
  total CPU  18s -> 13s

Note: Various other approaches were tried, including removing the
concatenation to self.buf and updating expect() to only consider recent
text. None of these yielded useful results.

Signed-off-by: Simon Glass <simon.glass@canonical.com>
---

 test/py/console_base.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
diff mbox series

Patch

diff --git a/test/py/console_base.py b/test/py/console_base.py
index 6d117413aa3..c4472420c31 100644
--- a/test/py/console_base.py
+++ b/test/py/console_base.py
@@ -813,11 +813,13 @@  class ConsoleBase():
                 c = self.p.receive(RECV_BUF_SIZE)
                 if self.logfile_read:
                     self.logfile_read.write(c)
-                self.buf += c
+                # Filter VT100 escapes from new data only, not entire buffer.
+                # This avoids O(n^2) behaviour when receiving large output.
                 # count=0 is supposed to be the default, which indicates
                 # unlimited substitutions, but in practice the version of
                 # Python in Ubuntu 14.04 appears to default to count=2!
-                self.buf = self.re_vt100.sub('', self.buf, count=1000000)
+                c = self.re_vt100.sub('', c, count=1000000)
+                self.buf += c
         finally:
             if self.logfile_read:
                 self.logfile_read.flush()