How to get `git imap-send` to accept any email

git-imap-send-all-the-emails.md

Email workflows with Git

Since Git was designed for the mailing-list heavy workflow of the Linux kernel development community, it has commands that make it easy to transform a commit (also known as a patch) into an email and vice-versa. For example, git format-patch, git am, git send-email and git imap-send.

While git send-email can be used to send a list of patches prepared using git format-patch to a development mailing list via an SMTP server, git imap-send is designed to upload git format-patch-prepared patches to an IMAP folder, so that they can then be sent using a regular mail client (maybe after tweaking the cover letter or the in-patch commentaries).

(Ab)using git imap-send to upload any email

Even if you want to keep up-to-date with the development mailing list of your favorite project, you might not want to subscribe to the mailing list, to avoid cluttering your inbox. After all, most development mailing list have a web interface which can be used to browse the list, such as https://lore.kernel.org/git/, the archive for the Git mailing list. Some mailing list software (like public-inbox, which powers lore.kernel.org) also provide bridges to other technologies like feed syndication (Atom) or Usenet (NNTP).

But if you're not subscribed to the mailing list, and see an interesting thread that you'd like to answer to, how can you do it easily ? If the web archive provides a way to download messages, you can import them into your favorite mail client, but that can be hard to automate when you are using a desktop email client (and not old-school mutt or alpine).

So why not use git imap-send ? That's the plan, but since it's designed to work hand-in-hand with git format-patch, it expects a certain ordering of email headers ("From: ", "Date :" and "Subject :" must appear in that order). So a simple

curl https://lore.kernel.org/git/<message-id>/raw | git imap-send

might work if the email corresponding to <message-id> is a patch created by git format-patch, but it probably won't work if it's a regular email (say, a bug report or user question that you wish to answer to).

To make git imap-send happy, we simply have to shuffle the email headers in the expected order before piping in the mbox. I use this little Python script that I name git-in to conveniently place it in the git namespace :

#!/usr/bin/env python3

import sys
import os
import mailbox
import urllib.request
import tempfile
import argparse
import gzip

# Parse input
parser = argparse.ArgumentParser(
           prog='git in', 
           description="download raw or gzip'ed mbox from given URL and print messages with headers reordered for `git imap-send`",
           epilog='example usage: git in <url> | git imap-send')
parser.add_argument("url", help="URL to download mbox from")
args = parser.parse_args()

# Check if the mbox is compressed with gzip
must_decompress = False
gz_ext = args.url[-3:]
if gz_ext == '.gz':
  must_decompress = True

# Create and open a temporary file
(mbox_file, mbox_path) = tempfile.mkstemp('.mbox', 'git-in-')

# Download the mbox file from `url` and save it to the temporary file
# https://stackoverflow.com/a/7244263/
with urllib.request.urlopen(args.url) as response, open(mbox_path, 'wb') as out_file:
    data = response.read() # a `bytes` object
    if must_decompress:
      out_file.write(gzip.decompress(data))
    else:
      out_file.write(data)

# Print messages with ordered headers
for message in mailbox.mbox(mbox_path):
    # Get 'From:', 'Date:' and 'Subject:' header values 
    from_header = message['from']
    date = message['date']
    subject = message['subject']
    # Delete the 3 headers
    del message['from']
    del message['date']
    del message['subject']
    # Write the 3 headers in the order `git imap-send` expects
    message['From'] = from_header
    message['Date'] = date
    message['Subject'] = subject
    # Print unixfrom and message
    print('From ' + message.get_from())
    print(message)

os.remove(mbox_path)

To upload an email or a collection of emails in a gzip'ed mbox to the IMAP inbox you configured for git imap-send, simply do :

git in https://lore.kernel.org/git/<message-id>/raw | git imap-send  # single email
git in https://lore.kernel.org/git/<message-id>/t.mbox.gz | git imap-send  # whole thread

Working with public-inbox instances with b4

If the mailing list you are interested in is hosted on a public-inbox instance (like lore.kernel.org), there is a tool called b4, developed by contributors from the Linux kernel project, that can be used to do what the git in script above demonstrates, and much more. To download a whole thread, you can use the Message ID of any message in the thread:

# Import the whole thread to the git-imap-send—configured IMAP folder
b4 mbox -o - <message-id> | git imap-send

b4 also has a lot more useful features, like detecting and downloading only patches in a specific thread, which is very useful for local code review:

# Apply the most recent version of a patch series to the current branch
b4 am -o - <message-id> | git am

git-in

#!/usr/bin/env python3

import sys
import os
import mailbox
import urllib.request
import tempfile
import argparse
import gzip

# Parse input
parser = argparse.ArgumentParser(
           prog='git in', 
           description="download raw or gzip'ed mbox from given URL and print messages with headers reordered for `git imap-send`",
           epilog='example usage: git in <url> | git imap-send')
parser.add_argument("url", help="URL to download mbox from")
args = parser.parse_args()

# Check if the mbox is compressed with gzip
must_decompress = False
gz_ext = args.url[-3:]
if gz_ext == '.gz':
  must_decompress = True

# Create and open a temporary file
(mbox_file, mbox_path) = tempfile.mkstemp('.mbox', 'git-in-')

# Download the mbox file from `url` and save it to the temporary file
# https://stackoverflow.com/a/7244263/
with urllib.request.urlopen(args.url) as response, open(mbox_path, 'wb') as out_file:
    data = response.read() # a `bytes` object
    if must_decompress:
      out_file.write(gzip.decompress(data))
    else:
      out_file.write(data)

# Print messages with ordered headers
for message in mailbox.mbox(mbox_path):
    # Get 'From:', 'Date:' and 'Subject:' header values 
    from_header = message['from']
    date = message['date']
    subject = message['subject']
    # Delete the 3 headers
    del message['from']
    del message['date']
    del message['subject']
    # Write the 3 headers in the order `git imap-send` expects
    message['From'] = from_header
    message['Date'] = date
    message['Subject'] = subject
    # Print unixfrom and message
    print('From ' + message.get_from())
    print(message)

os.remove(mbox_path)

You should probably be using tempfile library, instead of creating randomized name and then using this name for a temporary file to save contents of email or thread you want to respond to.

Alternative solution would be to use Usenet newsreader software like Gnus package in Emacs, or Pan, or email client that supports NNTP news like Thunderbird, together with Usenet news <-> mailing list bridge service, like public-inbox.org, or lore.kernel.org (or old gmane).

@jnareb thanks for the suggestion to use the tempfile library. I've just tweaked the script to use it. It should be more cross-platform this way.

And thanks for the pointer to other alternatives:)

I added a mention of Atom/NNTP bridges, and a little section on b4: https://gist.github.com/phil-blain/d350e91959efa6e7afce60e74bf7e4a8#working-with-public-inbox-instances-with-b4