Search scrollback

[mitchellh]

A major missing feature is the ability to search scrollback, i.e. cmd+f on Mac, ctrl+F on Linux. This issue can be implemented in multiple steps, not just one giant PR:

• Core search functionality in src/terminal (Naive search internals (core only) #2885)
• Make core search work simultaneously to PageList read/write to allow multi-threading
• Implement multi-threaded search, including data modeling for notifications of matches
• Search UI, glue in apprt

We can also support search only in certain modes as long as the apprt glue is all there for someone to come along and finish it up. For example, only macOS, Linux+GTK, etc.

[amadeus]

Would be cool to fake what terminal emulator in vim does -- where you can pause the existing session ctrl+w N, and move around the buffer with vim commands for searching and highlighting, then press i to return back into the session. Here's an example video (not saying the commands should be the same to pull this off, just using that as an example):

CleanShot.2023-07-07.at.00.09.42.mp4

[andrewrk]

Looking forward to this feature - I missed it yesterday when I had a lot of output from a command and wanted to search for the word "fail".

[vancluever]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

This also lets you take advantage of the pager's features and can be tailored to a certain user's preference without minimal load on our side.

[gpanders]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

FYI Ghostty has this too, bound to Cmd-Shift-J on macOS and Ctrl-Shift-J on Linux.

It works slightly differently than Kitty in that it puts the temp file path to the captured scrollback on the command line rather than opening it in a pager directly.

[benwaffle]

Once we have search, searching via regex would be really nice to have. I've been finding myself search for exit [^0] a lot recently.

[mattpolzin]

Kitty has a a pretty novel approach here that I've actually really enjoyed - ctrl+shift+h actually sends the scrollback to a pager (e.g. less). I've found this much less confusing in comparison to others (like iTerm's).

FYI Ghostty has this too, bound to Cmd-Shift-J on macOS and Ctrl-Shift-J on Linux.

It works slightly differently than Kitty in that it puts the temp file path to the captured scrollback on the command line rather than opening it in a pager directly.

One nice thing about Kitty's implementation is you can even use it as a program is actively running and potentially still outputting text; new text doesn't automatically get fed into your pager but you don't need to wait until a command finishes to use the pager functionality.

I'm not immediately sure if that would be equivalent in behavior to

1. Pause current execution
2. Send to background
3. Open ghostty's scrollback file in pager
4. On quit, bring task to foreground if still running

[pjz]

I'm used to screen/tmux behavior here: ctrl-A,[ puts you into 'scrollback mode' where you can then use the keyboard (vi-keys) to scroll up and down, and then even hilight some text and put it into a clipboard where you can then paste it with ctrl-A,].

I don't think it updates the buffer while scrolling, though... but I don't see why it couldn't just extend it downward.

[yankcrime]

Personally I'd rather this stuck to OS-native conventions for search functionality as well as look and feel. I don't think implementation-specific idiosyncrasies are a good idea since a lot of folks will have differing preferences and expectations.

The implementation along the lines of what's provided by Terminal.app or iTerm would be fine since that easily lets you search the scrollback buffer in a way that doesn't affect any additional output.

[andrewrk]

Some prior art:

It... suffices.

[peteschaffner]

@qwerasd205 Are you working on this now? I'd be happy to help out on the macOS UI front.

(I mentioned this in Discord but just surfacing it here too since I rarely check over there.)

[qwerasd205]

I am working on laying some groundwork for this with a basic implementation of the search functionality - focused mainly on proper threading for the search and creating a good interface for the UI code to talk to with minimal coupling, so that in the future we can improve the performance of the search (if necessary) by changing the algorithm and/or data structures used for the searching. I'll open a draft PR once I've got the first "technically functioning" version of the code.

Right now it's in too early of a state to really create UI for.

[ferdinandyb]

Personally, "native" in a terminal means "vi-like" to me :) I also think kitty's approach is pretty flexible and can probably be used along side a more classic ctrl-f as well. I also quite like contour's vim-like normal mode tbh.

But why I actually wanted to comment is related to generic search, but a bit more specific: sometimes you actually see what you need on screen, but want to quickly copy/paste/execute it without a mouse. Back when I was using kitty I used hints a lot, and in tmux I use tmux-butler. Behind the scenes these use cases could be regex-searches constrained to the currently visible text, although obviously the UI would be quite different. Imho especially quickly opening links in the browser and copying paths to the prompt without a mouse is really useful.

[sluongng]

I wanted to file a feature request for something like Kitty Hints like @ferdinandyb mentioned.
https://sw.kovidgoyal.net/kitty/kittens/hints/

However, after a bit of searching, I figured this issue is a prerequisite for such a hinting system.

[andrewrk]

More prior art:

Konsole has a pretty decent implementation if you want something to draw inspiration from.

[rmacklin]

Re:

I wanted to file a feature request for something like Kitty Hints like @ferdinandyb mentioned. https://sw.kovidgoyal.net/kitty/kittens/hints/

However, after a bit of searching, I figured this issue is a prerequisite for such a hinting system.

Mitchell confirmed on Discord that this issue isn't a prerequisite for a "hints" feature - so I've gone ahead and created a separate issue to specifically discuss that feature request:

• Feature Request: "Hints" mode to select and act on arbitrary text patterns visible in the terminal #2012

[qwerasd205]

I am working on laying some groundwork for this with a basic implementation of the search functionality - focused mainly on proper threading for the search and creating a good interface for the UI code to talk to with minimal coupling, so that in the future we can improve the performance of the search (if necessary) by changing the algorithm and/or data structures used for the searching. I'll open a draft PR once I've got the first "technically functioning" version of the code.

Right now it's in too early of a state to really create UI for.

I got sidetracked from this shortly after starting work on it, but to summarize the architecture I landed on for the search thread:

Search Thread

Data

• Search results: Doubly linked list of spans represented by pairs of pins
• Active area search results: Separate results list for content in the active area, since it needs to be repopulated whenever the active area's content changes (dirty tracking logic could be added in the future if necessary)
• Search pattern: Struct containing info used to perform the search (in my prototype it's just a string and a flag for whether to be case insensitive)

Messages

• Search pattern updated; if new pattern is different from old pattern, reset search results, and if currently mid-search, reset search back to start (either top or bottom of scrollback, depending on which direction the search is performed in)
• Begin search; start searching, intermittently yielding to the event loop. In my prototype I use a line iterator. If no iterator exists, create a new one, take a line from it, apply the pattern, add any results to the results list, notify the surface - if the iterator is out, we're done searching, notify the surface of this fact - then check the mailbox again.
• Stop search; self explanatory.
• Active area updated; if not currently searching, do nothing - otherwise throw out current active area results list, apply the search pattern to all lines in the active area, update the results list, notify the surface.

Whenever the surface is notified of new search results, it updates its display accordingly, reading from the results list should be thread safe.

[Stono]

I've setup my new ghostty terminal and am really enjoying it. Unfortunately, wont be daily driving it until this functionality is ready. Key part of my workflow

Yeah I second this. I've really being enjoying Ghostty but found the inability to do cmd+f pretty debilitating in my day to day workflow. It's like the only missing piece of the puzzle for me! I'm kinda sad to have to go back to iterm for now!

[jdx]

this issue is the most thumbed several times over. There are likely hundreds of people subscribed to it. It's not helpful to post messages here simply saying it "doesn't fit your workflow". @mitchellh knows this is necessary. He'll work on it when he can I'm certain. Let's try to keep the discussion to helpful information.

[Stono]

I was actually, primarily, complimenting the awesome terminal this is because it quite literally is almost perfect and I'm quite excited by that. As the issue has been open for 18 months I perhaps wrongly made an assumption that it probably wasn't moving forward (at least any time soon) so was just adding my perspective to it.

I'm sorry for sharing my experience with Ghostty and I'm sorry you took such offence to that mistake. It's such a shame GitHub (and tech in general) is so hostile these days, people just seem to assume the worst in each other.

Merry Christmas!

[jdx]

I apologize if that came across as hostile, it was not meant to, nor what is it targeted at you since you're not the only one in recent days to have done this. My goal is simply to keep the discussion here helpful since it has a lot of eyeballs on it and we don't want to add to the noise.

[Stono]

That is completely fair, and I understand - thanks. 🤝

[eaverdeja]

keybind = super+f=write_screen_file:open

Did anyone manage to get this to open in vim/nvim? I have nvim as my default editor on MacOS, but for some reason ghostty still opens the buffer in the default MacOS text edit.

[martensson]

keybind = super+f=write_screen_file:open

Did anyone manage to get this to open in vim/nvim? I have nvim as my default editor on MacOS, but for some reason ghostty still opens the buffer in the default MacOS text edit.

It uses the default editor of MacOS, not your $EDITOR. There is an OS setting to modify it globally from the default (need to Google it), but probably better to change it to MacVim rather than a CLI-tool in this case.

[eaverdeja]

Thanks @martensson ! I got it to work with Neovide by setting the default editor via Launch Services:

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add '{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.neovide.neovide";
}'

[elCarlosSantiago]

Thanks @martensson ! I got it to work with Neovide by setting the default editor via Launch Services:

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add '{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.neovide.neovide";
}'

Great solution, officially daily driving it now. Here's the command for vscode, can be done for any editor.

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add
'{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.microsoft.VSCode";
}'

[justyn]

Regarding the implementation of search in Ghostty.

Different terminals have different approaches to search. For example, Wezterm has both Copy Mode and Quick Select Mode.

I personally would really value something that emulated my (Lazy)Vim configuration - not just basic Vim motions but custom textobjects, seeking with flash.nvim etc. Obviously I can get part of the way there with write_selection_file but it is a bit clunky and doesn't update with new output.

Given that it would be bloated for Ghostty implement all the different approaches, could it instead hand over to an external tool for the search? That would allow experimentation and innovation in this area without affecting Ghostty's code.

Perhaps there is something more streamlined than dumping the scrollback to a file then opening it in an editor... I haven't figured out the mechanics (a pointer? an emulated file?). Or is write_scrollback_file the only reasonable way it can be done?

[zxt-tzx]

Thanks @martensson ! I got it to work with Neovide by setting the default editor via Launch Services:

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add '{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.neovide.neovide";
}'

Great solution, officially daily driving it now. Here's the command for vscode, can be done for any editor.

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add
'{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.microsoft.VSCode";
}'

not sure if this applies more generally, but I needed to restart my Mac for this to take effect. thanks for the suggestion!

[github-cloud-cd]

On MacOS the defaults command requires you to log out and back in for it to take effect. A reboot will also work.

[rmacklin]

Regarding the implementation of search in Ghostty.

Different terminals have different approaches to search. For example, Wezterm has both Copy Mode and Quick Select Mode.

For quick select mode, there is a feature request here (the title is based on the analogous feature in kitty):

• "Hints" mode to quickly select and act on arbitrary text patterns visible in the terminal #2394

[Stono]

For some reason I couldn't get the commands above to work, but duti from brew helped:

brew install duti
duti -s com.microsoft.VSCode public.plain-text all

I didn't need to log back in for this either.

If you need to find the plist identifier for your desired app, do:

/usr/libexec/PlistBuddy -c 'Print CFBundleIdentifier' /Applications/Visual\ Studio\ Code.app/Contents/Info.plist

I failed to get it to open anything not installed as an Application though (for example i have nvim installed via brew).

[mmerickel]

As a new user coming from iterm+tmux and trying to use ghostty as a replacement the main hangup I've run into is around scrollback management. With tmux I can use ctrl-a-[ to enter a scrollback mode where I can use ? and / to search the scrollback, as well as hjkl to navigate, ctrl-u to page up and ctrl-d to page down, and space to start a selection, and enter to finish a selection, copying it to the clipboard. I think something like this would require a new scope for key-bindings like scrollback alongside the current all and global scopes.

I would really love this modal capability in ghostty with the added benefit that whatever y'all do would also work well with the mouse where tmux tends to struggle.

[escluse]

Great solution, officially daily driving it now. Here's the command for vscode, can be done for any editor.

 defaults write com.apple.LaunchServices/com.apple.launchservices.secure LSHandlers -array-add
'{
    LSHandlerContentType = "public.plain-text";
    LSHandlerRoleAll = "com.microsoft.VSCode";
}'

not sure if this applies more generally, but I needed to restart my Mac for this to take effect. thanks for the suggestion!

You can do this using Finder:

1/ select "Get Info" on any .txt, etc. file
2/ change the "Open With" default to any macOS Application

However, this doesn't work for $EDITOR and it's not worth creating a .app wrapper bundle given that it launches an entirely new instance.

[m4saurabh]

I like the way kitty does it, it exposes the scrollback buffer and you pipe that to any program, like less and more importantly neovim so you can move around with vim bindings without creating a subpar version of them.

[tmccombs]

FWIW, wezterm has a feature similar to kitty's which allows you to send the buffer to an external program. But also has it's own search and navigation modes.

[leo-paz]

I just want this to work like warp :)

[m4saurabh]

@leo-paz you should share a video of it

[JonasGruenwald]

I like the way kitty does it, it exposes the scrollback buffer and you pipe that to any program, like less and more importantly neovim so you can move around with vim bindings without creating a subpar version of them.

I think it's really nice that ghostty does tabs via UI like most other software (file explorer, browser etc.) using the common OS hotkey for it, rather than pushing users towards something like tmux or zellij.

Similarly I'd prefer if scrollback search was done via UI as well, so cmd/ctrl+f works the same as in all other software, rather than pushing you to pipe to another utility for searching, even if it's how you would otherwise commonly search a text buffer from within the terminal.

It's just nice to have common functionality work the same across all software in my opinion, makes things intuitive to use.

But I suppose nothings speaks against also exposing the scrollback buffer for piping in addition to that.

[bofm]

iTerm highlights the searched pattern in live mode. This is handy when tailing some log and want to highlight something important but also not throw away the rest of the log as grep would do it.

[justyn]

I like the way kitty does it, it exposes the scrollback buffer and you pipe that to any program, like less and more importantly neovim so you can move around with vim bindings without creating a subpar version of them.

Thanks @m4saurabh for the hint on this Kitty functionality.

From there I also discovered this incredible Neovim plugin which integrates with the Kitty scrollback buffer and provides some extra handy functionality.

Here is part of a demo by the author (starting at 9:05) showing switching seamlessly between live Kitty and the scrollback buffer within Neovim. Note the little flash of the text redrawing but other than that you wouldn't know that you had left the terminal!

Most relevant to this discussion is that the author also details here (same video from 4:50) various approaches to getting the scrollback buffer into Neovim, starting with writing to a file then eventually settling on Kitty's built-in remote calls like get-text.

It will be great if Ghostty can also allow for this kind of deep integration which offers the best of both worlds of native terminal and your raw custom Neovim power, plugins and all.

But I suppose nothings speaks against also exposing the scrollback buffer for piping in addition to that.

I completely agree this kind of capability should be alongside rather than instead of built-in search.

In the meantime thanks to the great discussion in this thread I think I'm switching to Kitty for a while :-)

[joe-chandler]

iTerm highlights the searched pattern in live mode. This is handy when tailing some log and want to highlight something important but also not throw away the rest of the log as grep would do it.

Indeed, with how much time i spend SSHd into servers, this feature pairs well with all of the other bazillions of utilities i have to use on live tailing of screaming logs. On OSX, i had to go back to iTerm2. Still using Ghostty on my personal laptop.

[quinncomendant]

Another use for scrollback search is to highlight matching strings to make screenshots that I send to colleagues. This is extra useful with regex search, to allow highlighting different types of strings. I use this often – here's an example screenshot.

[withnale]

The ability to search scrollback and the ability to pipe scrollback to arbitrary commands are two different pieces of functionality. It would be nice to have both but this card is specifically about the former.

[CanRau]

iTerm highlights the searched pattern in live mode. This is handy when tailing some log and want to highlight something important but also not throw away the rest of the log as grep would do it.

use(d) this all the time, currently trying to main ghostty tho 🤓 and hit the missing search feature pretty quickly 😅

[tuananh]

Another use for scrollback search is to highlight matching strings to make screenshots that I send to colleagues. This is extra useful with regex search, to allow highlighting different types of strings. I use this often – here's an example screenshot.

agreed. this is a very common use case.