[mastercactapus]

This PR adds two new commands:

• MatchLine that takes a regular expression and waits until the current line matches
• MatchScreen that takes a regular expression and waits until the "screen" matches

The purpose is to allow waiting for things like network or serial devices that might have more variable delays while using VHS.

---

MatchLine example

Output "out.gif"

Type "sudo apt update" Enter
MatchLine "password"

Type "hunter2" Enter # fake password :D
MatchLine "^>"

Type "sudo apt dist-upgrade" Enter
MatchLine "^>"

Sleep 1

MatchScreen example

Output "out.gif"

Set Width 1280
Set Height 720

Type "telnet telehack.com" Enter
MatchScreen "command list"

Type "starwars" Enter
MatchScreen "A long time ago"

Ctrl+C
MatchLine "^\."

Type "exit" Enter
MatchLine "^>"

Sleep 1

Additional Info

• A few string(s[0]) + strings.ToLower(s[1:]) places were changed to a toCamel helper with unit test (fixes issue with MUTLI_WORD commands formatting)
• A .Buffer() method was added to *VHS to allow getting the current buffer as text (screen)
• A .CurrentLine() method was added to *VHS to allow getting the current line as text
• .SaveOutput() was updated to use the shared method (largely the same code as before)
• Possibly closes Wait for command to finish before continuing #70

[rbergmanaf]

I've been experimenting with this and it's a useful feature, thanks for creating this!

I've noticed though that it would benefit from a timeout option in the event the expected pattern isn't encountered?

[mastercactapus]

@rbergmanaf Yeah, I could see using that; should it be optional or required? If optional, should it have a default? It seems odd to have something hang forever by default. Something short, like 3 seconds, and then you can manually raise it for long commands.

If the default is shorter, then it's more likely a user will put their value in for longer-running commands -- the last thing we want is a script that takes a long time failing halfway through because the user didn't realize there was a 30-second timeout or something.

MatchLine <regex> [timeout]

I considered using @ like Type (e.g., MatchLine@1s), but it doesn't read well.

Alternatively, since it is a waiting command, we could just make it mandatory.

[rbergmanaf]

I don't feel strongly, but do like the idea of a short default. @ would probably be the least surprising given the precedent in Type, but as long as it's documented I imagine either way being acceptable personally. Maybe the charm team would have a preference?

[maaslalani]

I agree on the time out being a great option, I agree on the default of 3 seconds as well having the user manually bump it if they know it will be longer.

Additionally, I would probably have this renamed to Wait (possibly WaitFor / WaitUntil) so it reads a bit like english commands:

Type 'echo "Hello"'
Enter
Wait /hello/ 5s

[maaslalani]

This is great! I'm really excited about this feature, just a bit concerned about the naming. I don't know if MatchLine and MatchScreen are the best names.

I also think waiting for the prompt i.e. MatchLine "^>" will be a very popular incantation of the command. I wonder if it's worth having a Wait command that simply Sleeps until the prompt shows up.

[maaslalani]

Thanks for this extraction ❤️

[maaslalani]

Nice, this is great! 👌

[maaslalani]

This is great! I'm really excited about this feature, just a bit concerned about the naming. I don't know if MatchLine and MatchScreen are the best names.

I also think waiting for the prompt i.e. MatchLine "^>" will be a very popular incantation of the command. I wonder if it's worth having a Wait command that simply Sleeps until the prompt shows up.

[rbergmanaf]

A simplified command like Wait that just waits for the next prompt to appear without needing an explicit expression for that one case does seem really convenient. It's the main way we are using it at this point, though we have some other cases that benefit from also having the ability to customize the regex.

[mastercactapus]

I like how Wait reads as well as implies what it does -- waits for a thing. Follow the what rather than the how.

I think WaitForLine is pretty clear, not sure if WaitForScreen reads as well, though. What about WaitOnScreen?

As for the common case of waiting for prompts, we could potentially do a Wait variation that uses WaitForLine but with a Set-able PROMPT variable or something like that with a sane default. Line & Screen could require a pattern, and Wait would not accept one to prevent the overlap.

[maaslalani]

I would propose something like the following:

Wait[+Context][@timeout] [regex]

Wait for prompt

Wait # simply wait for the prompt to appear (default timeout of 5 seconds)

Wait with timeout.

Wait@3m # wait for a maximum of 3 minutes for the prompt to appear

Wait with regex, we can decided whether the default is a screen or line.

Wait /hello/ # wait for hello to appear

Wait with context

Wait+Screen /hello/ # wait for hello to appear on the screen.
Wait+Line /hello/ # wait for hello to appear on the line.

Wait with all options:

Wait+Screen@1m /hello/

What are your thoughts? I think this syntax is nicer since it limits the commands but I'm very open to having different API that is potentially nicer?

[mastercactapus]

Context is a nice idea, is that used elsewhere or would this be a new syntax?

As for defaulting, I think we should set the defaults for our "wait for prompt" use case.

Set WaitPattern />$/ # default wait pattern
Set WaitTimeout 5s   # default wait timeout

Wait # can be used in most cases to wait for the next prompt
Wait+Line # identical to the above
Wait@1m # for longer-running commands
Wait /custom-pattern/

Wait+Screen /dialog title/ # match anywhere on the screen

[maaslalani]

As for defaulting, I think we should set the defaults for our "wait for prompt" use case.

Yup, sounds good!

Context is a nice idea, is that used elsewhere or would this be a new syntax?

This + syntax is used with the Ctrl and Alt commands, i.e.:

Ctrl+L
Alt+Enter

In terms of implementation, the Command would be Wait and the context would be the Options.

[mastercactapus]

Okay, that all sounds reasonable, and thanks for the insight on context.

The last bit to clarify is regex parsing -- we can probably piggyback on readString and pass / as the endChar with a new token type REGEX to keep things clear.

Does all sound good?

[maaslalani]

The new REGEX token sounds great!

[mastercactapus]

Excellent, I'll switch this PR to a draft until I'm able to get those changes pushed up for review

[maaslalani]

Yeah the 5s wait timeout also caught me off guard. I think it should probably be either 15 or 30 seconds by default.

[fenio]

Definitely. If someone tries to find something that will allow to work with commands that finish in unpredictable time then it's probably way more than 5 seconds ;)
I'd say 30s as default should fit most of the use cases.
And maybe a bit more clear statement that it timed out. I didn't write down how exactly it fails in case it exceeds timeout but it definitely wasn't clearly communicated and probably should have been.
Anyway even with default set to 0.1s it's still something I want to be included in main branch ;)
This is just killer feature.

[mastercactapus]

I updated the default to 15s for now, though I'm happy to adjust it if it should be higher. I expect use cases to vary a lot, so I'm not sure of the best value.

[fenio]

So how about merge? What are the blockers for it?

[3v1n0]

Maybe documentation should be updated too?

[gmile]

FWIW, I also cloned astercactapus:match branch and ran the following tape (found earlier in this thread) - it worked perfectly:

Details

# Where should we write the GIF?
Output demo.gif
Set Shell fish

# Set up a 1200x600 terminal with 46px font.
Set FontSize 25
Set Width 1500
Set Height 1000
Set BorderRadius 10

Type "secator x httpx testphp.vulnweb.com"
Sleep 500ms
Enter
Wait

[kyletaylored]

I just tried running this locally and ran into this error (slightly modified the test script to run traceroute instead)

➜  vhs git:(adaeead) ./vhs ../example.tape                                                  
File: ../example.tape
Output .gif demo.gif
Set FontSize 25
Set Width 1500
Set Height 1000
Set BorderRadius 10
Type traceroute testphp.vulnweb.com
Sleep 500ms
Enter 1
Wait Line
failed to execute command: timeout waiting for "Line" to match >$; last value was:  6  * ae1.3515.edge1.dallas2.net.lumen.tech (4.69.209.114)  5.908 ms
recording failed

I ran a different tape, got a similar error:

failed to execute command: timeout waiting for "Line" to match >$; last value was: 

[davixcky]

Hey I was really curious about this feature, any update?

[meowgorithm]

Alright, so let's get this one done. I think we're there in terms of API and it should just be a matter of vetting the execution. Thanks for your patience with this one, everyone.

[caarlos0]

code-wise this lgtm! thanks everyone for the effort in this! ❤️

that said, one thing is that waiting on a line will not always work, e.g.

Type "sleep 5; echo hello"
Enter
Wait+Line@10s /hello/

This might fail because at the time the line was check it was already just the prompt (>).

I guess there's no way around it though...

But I thought this would work:

Type "sleep 2; echo hello; sleep 1"
Enter
Wait+Line@10s /hello/

and i still got

failed to execute command: timeout waiting for "Line hello" to match hello; last value was: >

maybe we should get the last line that isn't the prompt?

[mastercactapus]

@caarlos0 Wait (Line variant) looks for a "ready" indicator prefix on the current line, as determined by the cursor:

• echo prints "hello" followed by a newline, so the current line never contains "hello".
• Wait is intended to detect command completion, not specific output, and just checks behind the cursor.

Solutions for your examples:

• Use printf instead of echo to avoid the newline.
• Use the multiline variant to search the entire buffer: Wait+Screen@10s /hello\n/

Both of these should also work over serial or SSH sessions.

---

While potentially confusing, the current behavior serves the intended purpose of detecting when execution has finished (as best as possible). Alternatives like line-by-line processing of the buffer could introduce instability with large outputs since we can only scan what fits in the buffer (e.g., logs, large files), so we'll need to make a tradeoff somewhere on complexity/behavior. Determining if we're checking the same line twice would also be challenging, especially because the buffer can scroll as new lines are fed in.

Conversely, tools like expect (which inspired this idea) can be deterministic and behave as you're expecting (no pun intended); however, that's only possible because they operate directly on a stream (i.e., a child process). Our options here are limited by operating through a "looking glass" to ttyd.

[caarlos0]

right, yeah, that makes sense!

cool, so, IMO, we can merge this.

@meowgorithm any additional remarks?

[meowgorithm]

LGTM. Let’s get it done.

[caarlos0]

thanks everyone! <3