Roxygen3 design

roxy3_vit.md

The process

1. Roxygen parses the file(s) and creates an object roxyPackage which is a roxygen representation of the package and is essentially a list of RoxyBlock objects. Each RoxyBlock object contains two slots, one @doc of class RoxyDoc and another one @object of class ANY, such that any object can be fit inside. In other words RoxyBlock, is an abstract representation of:

##' My cool function
##'
##' @title 
##' @param a 
##' @examples 
foo <- function(a = 34) a^5

RoxyBlock can also contain @name slot (to hold "foo" above), but may be it's not necessary.

2. Each RoxyDoc is an abstract representation of the documentation block:

##' My cool function
##'
##' @title cool_function
##' @param a 
##' @examples bla bla

And is essentially a list of RoxyTag objects. RoxyTag is a root class containing at least one slot @text. All tags are represented by their own classes:

setClass("RoxyTag", list(text = "character"))
setClass("tagTitle", contains = "RoxyTag")
setClass("tagParam", contains = "RoxyTag")
setClass("tagSuperParam", list(super = "numeric"), contains = "tagParam")

When Roxygen fist parses the file it reads the documentation block as above and tokenises it into tag objects according to the name @title, @param etc. That is each object's text property is populated by the corresponding text, nothing more.

At this stage, tags are raw. They have not been processed as yet.

3. Next stage is the processing of the tags. There is a processTag method which is defined for each tag class. It should return a transformed object of the same class as the input one. Most of the tags don't need processing so the generic method is just identity.

setGeneric("processTag",
           def = function(tag, object, roxydoc, roxypackage) standardGeneric("processTag"),
           signature = c("tag", "object"), # for time being donn't dispatch on roxydox and roxypackage
           useAsDefault =  function(tag, object, roxydoc, roxypackage) tag)

Usually this method would populate other slots which are specific to every tag, but it might have a global effect as well. How is this implemented is a another good question :). It depends on the situation I guess.

One option is to have a roxyPackage objects stored globally. And then every processTag method can access that object and modify in place. This way one processTag call can prepare other roxyBlocks for subsequent processTag invocations. Then there is no need to pass roxypackage argument to processTag at all.

Another option is to implement a local exchange mechanism only for those tags which need that. That is add an additional slot of class "environment" and populate it with the same environment on the initialization of the RoxyTag objects.

4. Once all the RoxyDoc objects are processed , they are stored again in a RoxyPackage object which is again a collection of RoxyBlock objects which contain an target object (@object) and a RoxyDoc (@doc) object which is now a collection of all (already processed) RoxyTag objects.

5. Generating documentation. You need one method for each type of the documentation outRd, outHTML, outMarkDown. Which is dispatched on two objects first is a documentation (be it RoxyDoc or RoxyTag) and another one is the documented object itself.

setGeneric("outRd", def = function(doc, object) standardGeneric("outRd"))

6. Other functionality like namespace generation and description, can be implemented just as functions taking one argument, a RoxyPackage object.

7. Other generics like, roxyTemaplate, roxyUpdate could be defined on RoxyDoc or RoxyTag objects correspondingly.

To extend a tag system, a user will have to define a new derived class for his tag setClass("MyCoolUsageTag", list(coolSlot = "numeric", contains = "tagUsage")) and define the parseTag and outRd methods for the tag and the objects which should have this tag in the documentation.

Observation: In the above proposal RoxyPackage, RoxyBlock, RoxyDoc and RoxyTag objects are used in for dual purpose. First to store the row output from a preparser, and then, in a second pos-processing stage they are used as a storage for processed tags. This is probably not a big deal, as parsed objects are a super entities of raw objects, so the same RoxyTag class can be used as a representation of an raw tag, as well as the processed tag..