Skip to content

Instantly share code, notes, and snippets.

@flatcap

flatcap/browser.md

Created December 2, 2017 18:58
Show Gist options
  • Save flatcap/7ef35a0be2ed45bbfc07397428b37189 to your computer and use it in GitHub Desktop.
Save flatcap/7ef35a0be2ed45bbfc07397428b37189 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
browser.md

Fixing NeoMutt's Browser

Discussions

I suggest we try, as much as possible to focus our work on browser.c/h and not altering the code of any other file. This way, we can work in a specific branch and have a specific architecture in the file that will merge more efficiently. Then, when we merge, we can split the content of the file into many smaller files if that is relevant, of course.

Yes, or a separate file completely. New file, new history.

Ah, that's a nice suggestion!

Then as it matures, we can change one user of the old browser to the improved browser.

Goals, ideas

In a new file/directory, rebuild a clean browser application for mutt.

  1. Keep all functionalities of the current mutt browser
  2. Refactor all the code to get something coherent.
  3. Get rid of as much ifdef as possible (being meta is the key)
  4. The browser is a browser, it doesn't give any signification to its display. It displays and select, that's all.
  5. Get coherent key bindings
  6. It has features (sorting, listing, unselectable entries, tagging, …)
  7. mutt_select_file or else are browsing functions, but high-level ones, these functions are specific and should be defined to rely on more basic functions of the browser. That would reduce the redundancy in the code.

Current Browsers

Below are all the ways in which NeoMutt browses things.

Shared Fields

  • %>X right justify
  • %|X pad to the end of the line
  • *%X soft-fill

Address Browser

Allow the user to select an address (from an external query). Note: There is no sorting option

Options

  • NONE

Functions

  • "A" query-append
  • "Q" query
  • "a" create-alias
  • "m" mail

Fields for $query_format

  • %a destination address
  • %c current entry number
  • %e extra information
  • %n destination name
  • %t tagged?

Alias Browser

Allow the user to select an alias. Note: There is no sorting option

Options

  • NONE

Functions

  • select-entry
  • select-entry
  • tag-entry
  • "d" delete-entry
  • "u" undelete-entry

Fields for $alias_format

  • %a alias name
  • %f flags
  • %n index number
  • %r address which alias expands to
  • %t tagged?

Attachment Browser

Allow the user to select an attachment. Note: Many of the functions apply to the email, not the attachment.

Options

  • NONE

Functions

  • "^D" toggle-disposition
  • "^E" edit-encoding
  • "^F" forget-passphrase
  • "" view-attach
  • "" view-attach
  • "^O" rename-attachment
  • "^T" edit-type
  • "^Xe" edit-file
  • "f" edit-from
  • "k" attach-key
  • "A" attach-message
  • "C" copy-file
  • "D" detach-file
  • "E" edit-headers
  • "F" filter-entry
  • "G" get-attachment
  • "P" postpone-message
  • "R" rename-file
  • "S" smime-menu
  • "T" tag-entry
  • "U" update-encoding
  • "a" attach-file
  • "b" edit-bcc
  • "c" edit-cc
  • "d" edit-description
  • "e" edit-message
  • "f" edit-fcc
  • "h" display-toggle-weed
  • "i" ispell
  • "l" print-entry
  • "m" edit-mime
  • "n" new-mime
  • "p" pgp-menu
  • "r" edit-reply-to
  • "s" edit-subject
  • "t" edit-to
  • "u" toggle-unlink
  • "w" write-fcc
  • "y" send-message
  • "|" pipe-entry

Fields for $attach_format

  • %C charset
  • %c requires charset conversion
  • %D deleted flag
  • %d description
  • %e MIME content-transfer-encoding
  • %F filename in content-disposition header
  • %f filename
  • %I disposition
  • %m major MIME type
  • %M MIME subtype
  • %n attachment number
  • %Q qualifies for attachment counting
  • %s size
  • %t tagged flag
  • %T graphic tree characters
  • %u unlink (=to delete) flag
  • %X number of MIME parts

Filesystem Browser

Allow the user to select a file or directory.

Options

  • Select file only
  • Select dir only
  • Select file or dir
  • Show hidden files/dirs
  • Single selection
  • Multiple selections
  • File mask (filtering)
  • Sorting

Functions

  • toggle-mailboxes
  • view-file
  • "." buffy-list
  • "=" goto-folder
  • "@" display-filename
  • "C" create-mailbox
  • "N" select-new
  • "O" sort-reverse
  • "T" toggle-subscribed (IMAP)
  • "c" change-dir
  • "d" delete-mailbox (IMAP)
  • "m" enter-mask
  • "o" sort
  • "r" rename-mailbox (IMAP)
  • "s" subscribe (IMAP/NNTP)
  • "u" unsubscribe (IMAP/NNTP)
  • "y" exit

Fields for $folder_format

  • %C current file number
  • %d date/time folder was last modified
  • %D date/time folder was last modified using $date_format.
  • %F file permissions
  • %f filename (with suffix [/@*])
  • %g group name (or numeric gid, if missing)
  • %l number of hard links
  • %m number of messages in the mailbox
  • %N N if mailbox has new mail, blank otherwise
  • %n number of unread messages in the mailbox
  • %s size in bytes
  • %t "*" if the file is tagged, blank otherwise
  • %u owner name (or numeric uid, if missing)

Mailbox Browser

Allow the user to select a network mailbox. Note: Currently, this uses $folder_format.

Options

  • Sorting
  • Toggle between mailbox and filesystem views

Functions

  • toggle-mailboxes
  • view-file
  • "." buffy-list
  • "=" goto-folder
  • "@" display-filename
  • "C" create-mailbox
  • "N" select-new
  • "O" sort-reverse
  • "T" toggle-subscribed (IMAP)
  • "c" change-dir
  • "d" delete-mailbox (IMAP)
  • "m" enter-mask
  • "o" sort
  • "r" rename-mailbox (IMAP)
  • "s" subscribe (IMAP/NNTP)
  • "u" unsubscribe (IMAP/NNTP)
  • "y" exit

Fields for $folder_format

  • %C current file number
  • %d date/time folder was last modified
  • %D date/time folder was last modified using $date_format.
  • %F file permissions
  • %f filename (with suffix [/@*])
  • %g group name (or numeric gid, if missing)
  • %l number of hard links
  • %m number of messages in the mailbox
  • %N N if mailbox has new mail, blank otherwise
  • %n number of unread messages in the mailbox
  • %s size in bytes
  • %t "*" if the file is tagged, blank otherwise
  • %u owner name (or numeric uid, if missing)

Message Browser

Allow the user to select a message, e.g. for attaching. Note: This spawns a the main index, so the list of possible functions is huge.

Options

  • Sorting

Functions

  • "^D" delete-thread
  • "^E" edit-type
  • "^F" forget-passphrase
  • "" next-new-then-unread
  • "" display-message
  • "^K" extract-keys
  • "" display-message
  • "^N" sidebar-next
  • "^O" sidebar-open
  • "^P" sidebar-prev
  • "^R" read-thread
  • "^T" untag-pattern
  • "^U" undelete-thread
  • "" previous-new-then-unread
  • "P" check-traditional-pgp
  • "V" collapse-all
  • "c" change-folder-readonly
  • "d" delete-subthread
  • "e" resend-message
  • "k" mail-key
  • "l" show-limit
  • "n" next-subthread
  • "p" previous-subthread
  • "r" read-subthread
  • "s" decode-save
  • "t" tag-thread
  • "u" undelete-subthread
  • "v" collapse-thread
  • "" display-message
  • "#" break-thread
  • "$" sync-mailbox
  • "%" toggle-write
  • "&" link-threads
  • "." buffy-list
  • "@" display-address
  • "C" copy-message
  • "D" delete-pattern
  • "F" flag-message
  • "G" imap-fetch-mail
  • "J" next-entry
  • "C" copy-message
  • "D" delete-pattern
  • "F" flag-message
  • "G" imap-fetch-mail
  • "J" next-entry
  • "K" previous-entry
  • "L" list-reply
  • "N" toggle-new
  • "O" sort-reverse
  • "P" parent-message
  • "Q" query
  • "R" recall-message
  • "T" tag-pattern
  • "U" undelete-pattern
  • "V" show-version
  • "W" clear-flag
  • "Y" edit-label
  • "a" create-alias
  • "b" bounce-message
  • "c" change-folder
  • "d" delete-message
  • "e" edit
  • "f" forward-message
  • "g" group-reply
  • "h" display-toggle-weed
  • "j" next-undeleted
  • "k" previous-undeleted
  • "l" limit
  • "m" mail
  • "o" sort-mailbox
  • "p" print-message
  • "q" quit
  • "r" reply
  • "s" save-message
  • "u" undelete-message
  • "v" view-attachments
  • "w" set-flag
  • "x" exit
  • "|" pipe-message
  • "~" mark-message
  • "" next-undeleted
  • "" previous-undeleted

Fields for $index_format

  • %a address of the author
  • %A reply-to address
  • %b filename of the message folder
  • %B the list to which the letter was sent
  • %c number of characters (bytes) in the message
  • %C current message number
  • %d date and time of the message in sender's time zone
  • %D date and time of the message in the local time zone
  • %e current message number in thread
  • %E number of messages in current thread
  • %f sender (address + real name)
  • %F author name, or recipient name
  • %g message labels
  • %H spam attribute(s)
  • %I initials of author
  • %i message-id of the current message
  • %K the list to which the letter was sent
  • %l number of lines in the message
  • %L "To "
  • %m total number of message in the mailbox
  • %M number of hidden messages
  • %N message score
  • %n author's real name
  • %O original save folder
  • %P progress indicator for the built-in pager
  • %q newsgroup name
  • %r list of "To:" recipients
  • %R list of "Cc:" recipients
  • %s subject of the message
  • %S single character status of the message
  • %t "To:" field
  • %T character from the $to_chars string
  • %u user (login) name of the author
  • %v first name of the author
  • %W name of organization of author
  • %x "X-Comment-To:" field
  • %X number of attachments
  • %y "X-Label:" field, if present
  • %Y "X-Label:" field, if present
  • %Z a three character set of message status flags.
  • %zs message status flags
  • %zc message crypto flags
  • %zt message tag flags
  • %{fmt} date and time of the message is converted to sender's time zone
  • %[fmt] date and time of the message is converted to the local time zone
  • %(fmt) local date and time when the message was received.
  • % current local time

PGP Key Browser

Allow the user to select a PGP key. Note: There is no sorting option

Options

  • NONE

Functions

  • "%" view-name
  • "c" verify-key

Fields for $pgp_entry_format

  • %a algorithm
  • %c capabilities
  • %f flags
  • %k key id
  • %l key length
  • %n number
  • %p protocol
  • %t trust/validity
  • %u user id
  • %[] date of the key

Design Decisions

What do we want to display?

  • List of single-line items
    • Simple Mutt-like
    • Use format string
  • List of multi-line items
    • e.g. PGP keys (may need variable number of lines):
      • Name
      • Expiry, validity, etc
    • Formatting string
      • One with embedded '\n'
      • Two, second is optional
    • Colouring
      • One. Coloured by field
      • Two, second is optional
    • Does selection highlight first line, or all lines?
      • Depends on the caller, then can mark lines selectable
    • Do we display partial multi-line rows?
  • Header Row
    • Use format string to arrange header in the order of the fields
    • Highlighting of Header Row field could indicate sort field
  • Separators
    • Blank - for visual clarity
    • Titled - for distinguishing groups of data
    • Inserted by the code, not the user

How do we want to display lines?

  • Flat list
    • Simple, understandable
  • Nested
    • Shallow, 1 or 2 levels of nesting
    • Deep: show entire tree
      • Threaded emails
      • Nested mime attachments
  • Paged
    • Display entire list
      • Caller must *have *the entire list first
      • For large mailboxes, this could be many thousands of items
    • Display subset of list
      • Caller can prepare the visible list as needed
      • Need clear visual feedback when the list is incomplete
      • How is sorting handled?

What does the Browser know about the lines?

  • Nothing
    • Browser has ordered list of items
    • Caller has formatted them
    • Caller has themed them
    • Some are selectable
  • Knows about separate fields and their types
    • Browser could offer sorting

How do we style the Browser?

Attributes: foreground colour, background colour, style, e.g. underline

  • Browser
    • Panel. Default colours for the entire panel
    • Header row (if handled by the browser)
      • Normal
      • Sort field
    • Selection
      • Current selection
      • Cursor position
      • Tagged (if handled by the browser)
  • Line
    • Caller sets attributes for the entire (multi-)line
    • Caller sets attributes per line (or sub-line)
  • Field
    • Caller sets attributes for an entire field
    • Caller sets attributes for part(s) of a field
  • Search
    • Highlight a matching search field (even if the string isn't visible on screen)

Folding Nested Items

  • Fold levels:
    • Default fold level (like vim)
    • Open/close folds manually
    • All open by default?
    • Caller dependent
  • What can we fold?
    • nested items, e.g. directories
    • multiline rows, e.g. pgp keys
  • Need clear visual feedback

Search/Limit API

If the browser doesn't know about fields, then the searching needs to be done by the caller.

  • Search
    • On full fields (the on-screen fields might be truncated)
    • Search in either direction
    • Optionally wrap around
  • Highlight
    • Optional
    • Highlight entire matching field (especially if the field is truncated)
    • Highlight match within a field
    • Matches:
      • No highlight, just move the cursor
      • Highlight current match
      • Highlight all matches (like vim's hlsearch)
  • Limit
    • Like search
    • Optional highlighting of fields
    • Caller hides non-matching rows

**Example: **Search for 'the'

  • Highlight field
    • Match visible: "weather"
    • Match not visible: "today's wea"
  • Sub-field
    • Match visible: "weather"

Glossary

  • attributes - underline, bold, reverse, etc
  • browser - select from a list of items, displayed as rows
  • caller - code that creates the entries and handles callbacks from the browser
  • colour - foreground, background
  • column - same field of multiple items in a list
  • cursor - currently selected item
  • entry/ies - object representing one item
  • field - piece of data belonging to an item
  • header - non-selectable row in the browser
  • hidden - item in list that shouldn't be displayed
  • item - object (in a list) to be selected
  • line - one screen line, see row
  • list - collection of items
  • panel - window containing the browser
  • row - one item on screen, may use several lines
  • selection - the chosen item
  • separator - a non-selectable line in the list
  • style - colour or attributes
  • visible - item in list that shouldn't be displayed
╭─────────────┬───────────────────────────────────────────────────────────────────╮
│ Header      │ Column1  Column2         Column3            Column4               │
│             ├───────────────────────────────────────────────────────────────────┤
             
│             ├───────────────────────────────────────────────────────────────────┤
│ Simple Row  │Field1    Field2          Field3             Field4                │
│             ├───────────────────────────────────────────────────────────────────┤
             
│             ├───────────────────────────────────────────────────────────────────┤
│ Complex Row │Field1    Field2          Field3             Field4                │
│  2nd line   │    Some other formatted data                                      │
│  3rd line   │    More data still                                                │
│             ├───────────────────────────────────────────────────────────────────┤
Raw
notes.md

Unsorted Notes

We want a standard way to interact with a list of objects.

There's a big overlap between the:

  • "browser" allows the user to select a mailbox or a file
  • "menu" allows the user to select a key, an alias

browser.c} has over 50 #ifdefs There are significant amounts of imap, nntp and notmuch code in it (and 1/3 of imap's ifdefs).

The browser shouldn't know what it's displaying.

Browser needs a format_string

  • default behaviour/expandos for padding, etc
  • overrides for caller-specific fields

Browser needs navigation keys

  • defaults - up, down, select, etc
  • override - for caller-specific actions
  • needs to tie these into the help line

Browser needs colours

  • default - app-wide indicator, highlight, etc
  • override - caller-specific colours

Browser's list attributes

  • selection mode: 1, many (tagging)
  • callback(s) for events, keys, etc
  • display title/header line?

Browser obj has

  • (optional) list of column names
  • (list of) callback(s)

Browser item attributes

  • item is (in)visible
  • item is (un)selectable
  • item is selectED (perhaps to prepopulate a list)
  • sequence number: for "unsorting" a list

Browser callback for actions

  • SELECTION_CHANGED
  • KEYPRESS (allowing override)
  • WINCH? (window resized?)

Keypress callback returns

  • HANDLED
  • NOT_HANDLED
  • ACTION_X?
    • isn't that the same as NOT_HANDLED?
    • X = browser default behaviours

API for caller to manipulate browser

  • no callbacks during API usage?
  • opaque pointer to browser data

Item types (things in a browser list)

  • data
  • title/header (need colour)
  • separator (need colour)

sorting callback

  • build in some defaults? (+reverse)
  • ascii
  • numeric
  • date
  • how do we select a field generically?

Requirements from mutt

  • MuttWindow
  • WINCH from getch()
  • colours
  • some global variables

Consider lua integration

  • if exists DOMAIN_browser()? call it
  • else call code callback?
  • pass lua array (or same opaque ptr)

lua_init: register callback for DOMAIN_browser?

Who handles toggle mailboxes/files

able to enter directories e.g.

  • imap INBOX
  • maildir cur

mutt_select_file() mutt_enter_fname()

action on timeout/hooks? notification of buffy changes status bar notifications

  • "you're in the browser"
  • or a helpline title?

how does the help page know the keybindings?

  • generic list, it's not smart

var $debug_use_new_browser new browser can be enabled piecemeal (one caller at a time)

external formatting external colouring should the browser know about columns, or just rows column headings could indicate sort order headings => browser is aware of columns/fields unless external header drawing external colouring needs to be able to colour entire line -> allows "color ~T" and other custom colours

is limiting done by the browser or the caller? probably the caller. the browser doesn't understand the data caller can mark entries "hidden" to filter

multi-line: do we store the "sub-lines" as children? (just like nesting) probably simpler for sorting purposes

keep raw fields in the browser entries for searching purposes allow the browser to search? it would need to understand field types (unless we coerce all the types to string)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment