Recently, given the economic hardships presented by the current pandemic,
coupled with the fact that I have started making my own money, I have started to
keep track of my finances. I have always known that I would use Ledger, the
command line account tool, to manage my money. Ledger was created by an Emacs
maintainer, which hints to a very good Emacs integration, and indeed
=ledger-mode= offers a great environment for editing, reporting and modifying
Ledger files. But even though it is a pretty complete major mode, I missed being
able to organize it my way. Sure you can use outline-minor-mode or even
=outshine=, but I wanted to have it in Org-mode.
I should also note that, from now on, I’ll be referring to editing a source
block in a buffer that has all of the content of the final tangled file as
editing a source block with context. I actually named it
org-contextual-src-mode.
Keep in mind that the title says “Literate Ledger Programming(?)”, but this is (probably) reproducible with any language that you desire. It’s just that I use it mainly for Ledger and, at the point of writing this, have only seen real use for Ledger.
Well, because there is some value in using ledger-mode (and, thinking about it
a little, in other modes as well) with the context of the full file when editing
it. In ledger-mode one can, for example, generate reports and get the balance
for a single account, as well as having completion based on your previous
postings and whatnot. But mostly I guess its because I’ve always wanted to be
able to edit a part of a bigger file in Org-mode, and this was the turning point
for me to take a jab at it.
I like to have Org-mode help me organize things in my life, and I am having more and more of a tendency to do all things in Emacs and from the things I do in Emacs, the more I can integrate with Org-mode the better. I have also wrote about how I have Org-mode manage my templates previously, and this is going in sort of a similar direction.
It might be worth noting that most of the benefits from using what I describe
here can be achieved with setting the variable ledger-master-file on something
like .dir-locals.el. This variable being set tells ledger to use that file for
reports and balances, instead of the current buffer.
I’ll try to be brief. I overrode org-edit-src-code with an :around advice
that tests if the current element is a source block[fn:1], which is necessary
since org-edit-src-code is also used for example blocks, and if the language
of the source block is in org-contextual-src-langs which is a variable that
holds a list of languages that should consider the context of the whole tangled
file when editing. In particular, "ledger" is a member of this list, so is
"latex" in my particular case.
Then, after that’s established, I tangle all source blocks related to the file
of the current block, if any (if none, just call the usual org-edit-src-code),
and then create a new buffer with the contents of the tangled file, but narrowed
to the result of searching the current block’s content in the file. The tangling
is necessary since you might be in the middle of editing the block and haven’t
tangled yet, in which case the resulting file is incomplete and the search will
fail.
It is pretty simple:
- Add the file
org-contextual-src.elto yourload-pathand call(require 'org-contextual-src). - Set the variable
org-contextual-src-langsto the languages you would like to have the context when editing (by default it is set to'("ledger" "latex")). - Call
org-contextual-src-modein an Org buffer. The minor mode is buffer-local, so you would call it only on the buffers you would like to have this behavior. Although, if you always want it, it can be achieved with(add-hook 'org-mode-hook #'org-contextual-src-mode).
I’ll try to explain my thought process so that you could replicate what I have achieved in your own way, if you wish to do so. I’ll also be providing a gist with this document in Org-mode and the Emacs Lisp file with the whole code.
As a first step, I took a look at what function is being called when issuing the
keybinding C-c ' in a source block. Turns out it is org-edit-special, which
is a catch-all function to edit all sorts of blocks and structures within
Org-mode. From that, I looked through the definition to figure out the function
responsible for editing source blocks, and it so happens that it is
org-edit-src-code. This function takes care of editing source blocks and
example blocks, but I am only interested in the source block part.
In my org-contextual-edit-src (the name I gave for the wrapper of
org-edit-src-code), I mimmick most of the behavior of the original function.
One minor difference is that the first let-bind gets the information for the
:tangle header argument and tries to resolve the file
;; (let (...
(block-begin (and babel-info (nth 5 babel-info)))
(tangle (and babel-info (cdr (assq :tangle (nth 2 babel-info))))) ; (ref:tangle)
(file (cond ; (ref:file)
((string= tangle "yes")
(expand-file-name
(concat (file-name-sans-extension (buffer-name)) "."
(or (cdr (assoc lang org-babel-tangle-lang-exts))
lang))))
((string= tangle "no") nil)
((> (length tangle) 0)
(expand-file-name tangle))))
;; ...The big difference comes when a few conditions are met, in order (at the time of writing this), but not limited to:
- The argument
CODEis nil.This argument is only used when calling
org-babel-expand-src-block. In this function, the contents of the source block are expanded (noweb references, etc.) and they are passed toorg-edit-src-codeto be edited. Since we won’t be doing any editing in this case, just call the original function. - The current block is a source block.
As mentioned previously,
org-edit-src-codealso handles example blocks, but we are only interested in source blocks. - The language of the current block is a member of
org-contextual-src-langs.This variable consists of a list of strings, which in turn are Org Babel languages, for which I would like to have contextual editing. I do so because if in a same file I have some Emacs Lisp source blocks and some Ledger source blocks, I wouldn’t want to have contextual editing for the Emacs Lisp blocks, but would do for the Ledger blocks, so enabling
org-contextual-src-modewould give me the desired behavior. - The header argument
:nowebisno.I also don’t want to edit blocks that have noweb references with their context. I rely on searching for the block’s contents in the tangled file in order to narrow the editing buffer to the correct place, so editing a block– with its context–that has a noweb reference, would require me to either expand the reference, in which case the final result that should be written to the source block is going to be the full expanded reference, which goes in the opposite direction of the intentions of noweb references.
- If a file to be tangled has been identified.
This is done either through the header argument =:tangle= set to either
yesor a filename, or by giving an ID to the source block (=#+name= property or =:noweb-ref= header argument.) I’ll get into a bit more detail later.
If all of these are met, we then tangle all of the blocks for the tangle file of the block at point.
Now comes the most relevant part.
This is the point where I need to tangle the file. We only ACTUALLY need to
tangle if the file was modified, because since we are going to use the contents
of the source block to search on the tangled file, that means that if the
current Org buffer is modified (checked with (buffer-modified-p)), the
resulting tangled file is probably out of date, so update it.
;; tangle only the file associated with the current block if the buffer
;; was not saved
(when (buffer-modified-p)
(save-excursion
(goto-char block-begin)
(org-babel-tangle '(16))))Ok so what is really the difference between the usual approach to editing source
blocks? It’s actually pretty simple. We save the value of the buffer name for
the editing buffer, which is just like the original org-edit-src-code, the
contents of the tangled file and the contents of the source block (note here
that the original function has an and to check if CODE was given, but we don’t
need to do that because of the first point above) in variables on a let*-bind.
I store them here because they are used as arguments to org-src--edit-element,
but I need some of them after the execution of this function, so I found it
easier to just store them on a let and use the values after the execution.
(let* ((buffer-name (or edit-buffer-name
(org-src--construct-edit-buffer-name
(buffer-name) lang)))
;; the contents of the tangled file (if we got to here, we have it)
(file-contents (with-temp-buffer
(insert-file-contents file)
(buffer-substring-no-properties
(point-min) (point-max))))
;; the actual contents of the source block.
(contents (nth 2 (org-src--contents-area elt))))
;; (org-src--edit-element ...)Inside this let, the first thing we do is call org-src--edit-element. The
call is very similar to how org-edit-src-code does it, the only differences
being that buffer-name is used instead of the or, since we saved that in a
variable, and the contents we give this function is the contents of the tangled
file (file-contents).
After the execution of org-src--edit-element, our cursor is (supposedly) on
the newly generated edit buffer, with the appropriate name and buffer-local
variables, and the contents of the whole file; now all that is left is for us to
narrow to the contents of the block:
;; narrow to the relevant part
(with-current-buffer buffer-name
(let* ((contents-end (search-forward contents))
(contents-beg (search-backward contents)))
(narrow-to-region contents-beg contents-end)
;; enable it in the editing buffer ; (ref:enable-mode)
(org-contextual-src-mode t)))As a small side note, I activate the mode in the editing buffer because I use the fact that this is a contextual editing buffer in other parts of it.
One caveat I found when first implementing this is that if I had, for example, a single source block with a bunch of noweb references to other blocks in other places, if I tried to edit one of these blocks with its context, it would fail, and that forced me to place the blocks in the order in which I wished they would appear in the file. But, since this is Emacs and we’re dealing with Emacs Lisp, of course there is a workaround.
Before[fn:2] checking these points, I check if the file is null, e.g., I
couldn’t figure out the file to be tangled from current source block. When that
is the case, I have a function that gets the name or noweb reference for the
current block, and searchs for <<name-or-ref>>. For each match, check if the
we are currently on a source block (it could be an internal link) and if so,
extract the file to be tangled. Yeah, this function retrieves the file from the
first match of a source block, but I couldn’t figure out how to do it in
general, and this seems like it would cover 99.5% of the cases.
(let* (;; ...
(name-or-ref (or (and (not (string= "" name)) name) ; (ref:name-or-ref)
;; or it has a noweb-ref (name has precedence)
(cdr (assq :noweb-ref (nth 2 babel-info)))))
;; ...
)
;; ...
(when (not (string= "no" match-tangle))
(setq file (expand-file-name ; (ref:file-noweb)
(cond
((string= match-tangle "yes")
(concat
(file-name-sans-extension (buffer-name)) "."
(or (cdr (assoc lang org-babel-tangle-lang-exts))
lang)))
((> (length match-tangle) 0) match-tangle))
block-begin (nth 5 babel-info))))
(list file block-begin))Some other functions had to be advised for it to work. One of those functions
was org-edit-src-exit. This function is what is usually bound to C-c ' in
the dedicated editing buffer, and is responsible for finishing the editing
session, while writing the edited content back on the source block in the Org
file. It doesn’t differ much from the original function, but we have to add an
advice arround it, so we need to check when to use our modified version, and
when to use the original functoh, like so
<<advice-check>>
(defun org-contextual-src-edit-exit (orig-fun &rest args)
(if (or org-contextual-src--no-context-p
(not org-contextual-src-mode))
(apply orig-fun args)
;; ...I only had to change two little things on my custom function: 1. the variable
coordinates is populated with org-contextual-src--coordinates, which gets
the coordinates in which to put the cursor at the Org buffer, in relation to the
a beginning and end position, which are, in the original org-edit-src-exit, 1
as the beginning position, which corresponds to the beginning of the buffer,
regardless of narrowing, and (point-max), which is the position of the end of
the buffer; however, in our version we would like to consider only the text
inside the narrowing, so we simply substitute the 1 for (point-min), and the
org-contextual-src--coordinates behaves just like org-src--coordinates but
without wrapping it with a org-with-wide-buffer; and 2. set the variable
org-contextual-src--no-context-p to nil. More about this variable below.
Another one of those functions was org-src--contents-for-write-back, which is
a function that returns the contents that should be written back to the block.
This function had to be advised because it was getting the contents of the whole
buffer, unwidened, which would mean that the whole final file would be written
back to the block. That is obviously not what we want, we want to get the
contents of the visible part of the buffer, since we have it already narrowed.
I also had to check if we are using the context or not just like here. The rest
of the function is exactly like the original org-src--contents-for-write-back,
with the exception that the variable contents is initialized with
(buffer-string) instead of (org-with-wide-buffer (buffer-string)), similarly
to org-contextual-src--coordinates. It is used when either saving or exiting,
that’s why we couldn’t change the behavior of it only in the case that we are
calling org-contextual-src-edit-exit in a buffer that is considering the
context, like we did with org-contextual-src--coordinates.
<<before-advice>>
There is only one other function which I have advised, but this one is much
simpler. When you issue a command inside a source block, Org actually executes
it in a dedicated edit buffer, just like if you have called org-edit-src-code,
but invisibly. Because of that, we have to override this behavior in case we are
on a block that should be executed with the context. That is because, if
everytime we issue a command in a source block we have to actually tangle it,
given that we are going to call our org-contextual-edit-src, which tangles all
blocks related to the final file, it would be really annoying to edit any source
block in the Org file itself, but sometimes for quick edits its just easier to
change it right there. The function that does this,
org-babel-do-key-sequence-in-edit-buffer, has a :before advice that sets
org-contextual-src--no-context-p to t, that’s why all of the functions that
are advices check if the mode is on, or if this variable is true.
In order to make it easier to activate this behavior on specific buffers, I gathered the advicing of the functions in a buffer-local minor mode. The body of the minor mode is:
(if org-contextual-src-mode
(progn
(advice-add 'org-edit-src-code :around #'org-contextual-src-edit-code)
(advice-add 'org-edit-src-exit :around #'org-contextual-src-edit-exit)
(advice-add 'org-src--contents-for-write-back
:around #'org-contextual-src--contents-for-write-back)
(advice-add 'org-babel-do-key-sequence-in-edit-buffer
:before
#'org-contextual-src-do-key-sequence-in-edit-buffer-advice)
(dolist (func org-contextual-src--narrow-functions) ; (ref:disable-narrow)
(advice-add `,func :around #'org-contextual-src-narrow-advice)))
(advice-remove 'org-edit-src-code #'org-contextual-edit-src)
(advice-remove 'org-edit-src-exit #'org-contextual-edit-src-exit)
(advice-remove 'org-src--contents-for-write-back
#'org-src--contextual-contents-for-write-back)
(advice-remove 'org-babel-do-key-sequence-in-edit-buffer
#'org-babel-do-key-sequence-in-edit-buffer-ad)
(dolist (func org-contextual-src--narrow-functions)
(advice-add `,func :around #'org-contextual-src-narrow-advice)))<<disable-narrowing>>
I guess I should mention that I also disable any narrowing function inside the
contextual editing buffer, because that would mess up how we get the contents to
be written in the block with our org-contextual-src--contents-for-write-back.
And done!
And this is basically it. I know it might have been a lot to digest, and I
recognize that I may be wordy most of the time, but I wanted to share this
little hack with the world because I feel that many would like to keep their
ledger journal in Org-mode but not being able to access the context of the
whole file have turned them away.
There are some caveats with it, though. The biggest one is that if these functions ever get changed I would have to reflect the change and adapt the modified code to work as I have intended, though I don’t think this would be much of a problem since this part of Org-mode seems stable enough. Another one of those is that one cannot call any of the narrowing functions, as mentioned above. Another one is that it takes a couple of milliseconds before the editing buffer shows up because of the tangling. But despite them, I am enjoying very much having sort of “the best of both worlds.”
One minor issue that I haven’t yet solved, and don’t know what is causing it, is
that calling save-buffer saves the entire file to the source block, instead of
just the visible contents of the buffer. In the editing buffer, C-x C-s gets
bound to org-edit-src-save, which works as desired without having to advise or
anything like that (because we’ve already advised
org-src--contents-for-write-back), but calling save-buffer behaves like I
mentioned, which might not be ideal.
Ps.: I know that one shouldn’t contribute anything relevant in web forums like Reddit, but I haven’t had the time to setup my blog yet so I’m just posting here for now.
[fn:2] I check it before checking the points becuase one of the checks I didn’t mention is if the file is null, in which case no tangling is happening and we should just use the original function.
[fn:1] I don’t think it would be beneficial to have it working with inline source blocks, but that can be arranged.