Skip to content

Instantly share code, notes, and snippets.

@marcomalva

marcomalva/markdown-first-steps.md

Last active July 22, 2021 17:20
Show Gist options
  • Save marcomalva/2ba154741cf2f3c5a9824eeab2badeeb to your computer and use it in GitHub Desktop.
Save marcomalva/2ba154741cf2f3c5a9824eeab2badeeb to your computer and use it in GitHub Desktop.
Download ZIP
[markdown-first-steps] Markdown First Steps #markdown #introduction
Raw
markdown-first-steps.md

%First Steps with Markdown %Marco Malva %2015/12/22 - 2021/05/18

Introduction

The format traditionally Markdown is now rebranded as commonmark. One can read more on commonmark here http://commonmark.org/

The commonly used suffix for markdown files is .md

Markdown works great for smaller documents, the available extensions are great, and the documents can be written with a simple text editor (e.g. vim) or a WYSIWYG editor such as typora. However,

  • not all extensions work for all target formats, which is a bummer
  • not all features are supported by the editors
  • lack of standard across the tools and target formats is an issue
  • not all editors are supported on all platforms, e.g. lack on Raspberry Pie.
  • the CSS Style makes a huge difference in the resulting HTML
  • PDF and HTML output do not support the same set of extensions

If you are looking into writing longer technical documents markdown might not be the best solution:

In this case, Sphinx and AsciiDoctor will most likely serve you better:

As for scientific writing with Markdown:

Other options for this purpose are:

Objective

This document started out to cover the basics of Markdown but was gradually extended to included advanced topics such as:

  • pretty format tables
  • convert CSV files to tables
  • math expressions
  • embed UML and other charts
  • embed plots
  • filter to embed CSV files
  • various UI editors

The motivation is to enable the reader to unlock some of the power of Markdown while still keeping the scope contained.

Some side aspects are tested and illustrated in small sample markdown files in the current folder.

History

I added a lot of stuff, mostly related to filter, on 2019/01/13.

I added some details on editors on 2021/05/08.

I added PDF generation on 2021/05/08.

I added various CSS templates on 2021/05/09.

Polished writing on 2021/05/18.

Conversion to HTML

The tool to convert Markdown documents to HTML, PDF, etc. is called pandoc. Its documentation is a worthwhile read, see http://pandoc.org/

To convert as markdown files into HTML with pandoc I use a command line similar to the one shown below:

# works for standard stuff
pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc

# works w/ math equations, CSV inline, CSV import, etc
pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter -F R-pandoc -F pandoc-placetable -F pandoc-pyplot

Some of the pandoc extensions (e.g. embed CSV files) do not work w/ some/most/all (?) editors.

A CSS file resulting in nicer HTML is provided here: pandoc.css. It lacks the highlight of the current line in a table but is otherwise superior.

Other CSS files:

More CSS related links are curated here: css-links.md

Conversion to PDF

The command to convert this file to a nice looking PDF:

pandoc markdown-first-steps.md -o markdown-first-steps.pdf --from markdown --template eisvogel --listings -F mermaid-filter -F R-pandoc -F pandoc-placetable -F pandoc-pyplot -F pandoc-csv2table

This requires the Eisvogel Latex Template for pandoc, see GitHub - Wandmalfarbe/pandoc-latex-template: A pandoc LaTeX template to convert markdown files to PDF or LaTeX.

In any case, the creation of PDF from Markdown is a very vast subject that is only covered rudimentary in this document.

Google Sites

One can copy & paste the HTML output of pandoc into Google sites, though only without the --toc option. I add the TOC via "Insert | Table of contents" in Google sites and choose a width of 1250 points instead of the default 250.

The pandoc command is:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css

If you have images in your document, those images do not copy&paste properly, and one must add them manually to the Google site. Maybe we can find a better way to import the HTML page.

Bitbucket

Bitbucket renders markdown documents correctly, though it does not seem to understand the optional title and author. In a Markdown document the title, the author, and the date may be added in the first three lines if those start with a percent (%) character.

Tables

There are various tools to convert a CSV or Excel table into a Markdown table. I prefer to embed CSV files directly, but the required extensions do not work in every setup, see section [](# Embed CSV Files - pandoc-csv2table)

One such online conversion tool is: http://www.tablesgenerator.com/markdown_tables

There is also a tool one can install and run locally, but it is limited to CSV files. I like it for confidential data and for automation. The tool is called csvtomd and is python3 based. One can download it here: https://github.com/mplewis/csvtomd.git.

mkdir -p
git clone https://github.com/mplewis/csvtomd.git

You can test it with python3 ${HOME}/dev/public/csv/csvtomd/csvtomd/csvtomd.py test.csv

I created a shell script (${HOME}/bin/csvtomd) to make it work:

#!/bin/bash
# python3 /usr/bin/csvtomd $@
python3 ${HOME}/dev/public/csv/csvtomd/csvtomd/csvtomd.py $@

This project is no longer under development as the developer moved on to a Java Script based solution:

mkdir -p ~/dev/public/csv
cd ~/dev/public/csv

git clone https://github.com/mplewis/csvtomd-web.git
cd csvtomd-web

npm install
npm run unit
npm run dev

The command above runs a local web-server that converts the data on http://localhost:8080, but it seems to have no CLI version right now. Anyway, the python based version is still working fine.

Tables: CSS for Nicer Formatting

For nicer formatting of tables I used a CSS that I downloaded from here: https://gist.github.com/1882525.git. I have that public git repository downloaded to my local folder ${HOME}/tools/pandoc/. It renders tables more nicely, such as alternating colors with rows.

A CSS file resulting in overall nicer HTML is provided here: pandoc.css. It lacks the highlight of the current line in a table but is otherwise superior to the other CSS file.

CSS Stylesheet Installation

mkdir -p ${HOME}/tools/pandoc/
cd ${HOME}/tools/pandoc/
git clone https://gist.github.com/1882525.git

Sample Tables

Tables Are Cool
col 3 is right-aligned $1600
col 2 is centered $12
zebra stripes are neat $1

The sample from the test.csv:

Name Position Wanted
Andromedus, Darrow au Leader Yes
Augustus, Victoria au Accomplice Yes

Math

Math Sample 1

Here are some sample equations copied from the pandoc math sample (math.text): https://pandoc.org/demo/math.text.

$a^2 + b^2 = c^2$

$v(t) = v_0 + \frac{1}{2}at^2$

$\gamma = \frac{1}{\sqrt{1 - v^2/c^2}}$

$\exists x \forall y (Rxy \equiv Ryx)$

$e^x = \sum_{n=0}^\infty \frac{x^n}{n!} = \lim_{n\rightarrow\infty} (1+x/n)^n$

They can be rendered by adding any of the following switches, --mathml, --webtex, --mathjax, --katex, i.e. with any of the following commands:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc --mathml

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --katex

The rendering result differs, as does the output file size, which varies between 18 KiB, 21 KiB, and 2000 KiB.

It seems that those work also w/o --self-contained , in which case the file sizes are 16 KiB, 15 KiB, and 15 KiB.

The --katex did only work after I upgraded pandoc to version 2.5 from https://github.com/jgm/pandoc/releases/tag/2.5 (before the upgrade the version installed from the default repository was version 2.2.1) and after I installed Katex https://katex.org/docs/node.html:

sudo npm -g install katex

The --mathjax did not render the math correctly, nor did it help to omit the flag.

Please note that the above does not render work correctly in ghostwriter.

Math Sample 2

This sample is borrowed from https://stackedit.io/app# and has inline math expressions, explicit expressions, and note blocks, all of which also work with ghostwriter but the inline math expression.

The Gamma function satisfying $\Gamma(n) = (n-1)!\quad\forall n\in\mathbb N$ is via the Euler integral

$$ \Gamma(z) = \int_0^\infty t^{z-1}e^{-t}dt,. $$

You can find more information about LaTeX mathematical expressions here.

Math Sample 3+4

Here is two equations to be rendered in Pandoc in ghostwriter which does requires to include a java script from mathjax (idea seen here: https://github.com/wereturtle/ghostwriter/issues/81):

<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>
<script src='https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'></script>

$$ \Gamma(z) = \int_0^\infty t^{z-1}e^{-t}dt,. $$

And the second one, which only works properly in $$ block and not [[ ... ]] block:

$$ [\ {x}_{1,2}=\frac{-b\pm \sqrt{{b}^{2}-4ac}}{2a} \] $$

That is all I have for math right now.

Editors

The Markdown editors are an ever-expanding list of applications, each with its strong suit and draw back.

Haroopad

Today (2021/05/08), I came across haroopad, and it may yet displace typora:

  • it does support vim mode
  • it renders the Math and Mermaid fine

On the negative side:

  • it does not support ~~~{.bash} and similar notations for code blocks, so one must use ~~~bash instead, but that seems to the case for most editors
  • some of the advanced features, like embedding CSV files, do not work

Typora

Since I discovered typora, it has replaced ghostwriter as my favorite WYSIWYG Markdown editor.

I like:

  • single view
  • build-in spell check is decent, not as good as Gramarly in code but decent
  • it renders Math and Mermaid fine
  • some of the advanced features, like embedding CSV files, seems to work

Ghostwriter

Initially, I mostly used ghostwriter for its real-time preview of the HTML. It is fair but lacks syntax highlighting in source code blocks and requires some special considerations w.r.t. math expressions.

The Github page of ghostwriter is: https://github.com/wereturtle/ghostwriter.

Vim

Most Markdown documents start their life in vim.

Code

I used code because of the unofficial Grammarly plugin for advanced spell-check.

Stackedit

There is also an online editor under https://stackedit.io/editor, which seems to be more powerful than ghostwriter but works less well with the local file system as one has to go through import/export to read/write files from/to the local file system.

It can also be installed as a Google Chrome extension <> and run from a local node:

mkdir -p ~/tools/markdown/
cd ~/tools/markdown/

git clone https://github.com/benweet/stackedit.git
cd stackedit

npm install
npm start

The above makes the stackedit available under localhost:8080.

MarkVis

Another online editor which supports graph is markvis: https://github.com/geekplux/markvis.

It can run locally via node after installing it as follows:

git clone https://github.com/geekplux/markvis
cd markvis
npm install

There is derivate work such as markvis-bar that allows to create SVG/PNG charts such as https://github.com/geekplux/markvis-bar:

# installation
git clone https://github.com/geekplux/markvis-bar.git
cd markvis-bar
npm install

# exection
npm run example

The last command creates SVG/PNG/HTML bar-chart files under the examples folder.

Markdown Plus

The markdown-plus https://mdp.tylingsoft.com/ seems to be only available for Windows and Mac.

It has additional features such as charts (line charts, bar charts, etc.) via dagre-d3 (https://github.com/dagrejs/dagre-d3).

UML & Other Charts

The ability to add UML and other charts in a Markdown document is something I just now (2019/01/12) discovered while looking into stackedit. In the past, I used apps like dot, plantuml to create UML and other diagrams, but I might switch to embed them directly in Markdown documents.

One can use mermaid https://mermaidjs.github.io/ for Markdown documents. It also has a CLI interface https://github.com/mermaidjs/mermaid.cli.

Requirements

You need to install a recent version of mermaid https://mermaidjs.github.io/, https://github.com/raghur/mermaid-filter and https://www.npmjs.com/package/mermaid-filter. I ran into some permission problems, but --unsafe-perm=true solved it for me:

sudo npm install --global mermaid-filter --unsafe-perm=true

Now add -F mermaid-filter to the pandoc command line:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter;

So do not install the three year old version (https://www.npmjs.com/package/pandoc-mermaid-filter):

sudo npm install --global git+https://github.com/nishimaki10/pandoc-mermaid-filter.git

And the following command line:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F pandoc-mermaid;

Graph

This example is borrowed from https://www.npmjs.com/package/pandoc-mermaid-filter. Originally, it did not render properly with pandoc but it works with:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter -F R-pandoc

It also shows nicely in typora and haroopad:

graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;
Loading

UML

This example borrowed from the slackedit welcome page and, of course, works fine with this editor, i.e. it even worked before I installed mermaid. However, they did not work right off the bat with ghostwriter or pandoc.

You can render UML diagrams using Mermaid. For example, this will produce a sequence diagram:

sequenceDiagram
Alice ->> Bob: Hello Bob, how are you?
Bob-->>John: How about you John?
Bob--x Alice: I am good thanks!
Bob-x John: I am good thanks!
Note right of John: Bob thinks a long<br/>long time, so long<br/>that the text does<br/>not fit on a row.

Bob-->Alice: Checking with John...
Alice->John: Yes... John, how are you?
Loading

And this will produce a flow chart:

graph LR
A[Square Rect] -- Link text --> B((Circle))
A --> C(Round Rect)
B --> D{Rhombus}
C --> D
Loading

Plots & Charts

Python Plot (pandoc-pyplot)

Using python plot: https://github.com/LaurentRDC/pandoc-pyplot

Installation

cabal install pandoc-pyplot

Usage

Just add filter -F pandoc-pyplot to the pandoc command line:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter -F R-pandoc -F pandoc-placetable -F pandoc-pyplot

Example

A simple figure with:

import matplotlib.pyplot as plt

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('This is an example figure')

The above does not display correctly with typora or haroopad. In haroopad the remainder is screwed up b/c of it, which is why I had to modify the line from {plot_target=my_figure.jpg plot_alt="This is a simple figure"}

R-pandoc Filter

Sample R Plot

The R-plot requires the R-pandoc filter, i.e. the command line switch -F R-pandoc:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter -F R-pandoc

Here is a the R-plot output:

require(stats)
D = 150
T = 10
t = seq(0, 80, 0.01)
x = -D*exp(-(t/T))+D
v = (D/T)*exp(-(t/T))
plot(t, x, type="l", main="position through time", xlab="time (s)", ylab="position (m)", xlim=c(0,80), ylim=c(0, D+10),  xaxs = "i", yaxs = "i")

It was created with the following R script:

require(stats)
D = 150
T = 10
t = seq(0, 80, 0.01)
x = -D*exp(-(t/T))+D
v = (D/T)*exp(-(t/T))
plot(t, x, type="l", main="position through time", xlab="time (s)", ylab="position (m)", xlim=c(0,80), ylim=c(0, D+10),  xaxs = "i", yaxs = "i")

The above does not render correctly in typora or haroopad.

R Plot Installation (Filter R-pandoc)

References:

Getting R-pandoc to work was a little bit convoluted as it required to

  • install cabal
  • upgrade cabal
  • change environment variable PATH to use upgraded cabal
  • install R-pandoc
  • optional install of Haskell tool stack

The R-pandoc did not work out-of-the-box, I had to modify the dependency file to get it to work.

As far as the installation and update are concerned I followed the instructions as laid out here: https://www.haskell.org/cabal/index.html#troubleshooting. An alternative way to install the latest version of cabal directly can be found here: https://github.com/haskell/ghcup.

The installation of cabal:

sudo apt-get install cabal
sudo apt-get install cabal-install

The upgrade of cabal:

cabal update
cabal install Cabal
cabal install cabal-install

The change to PATH which are persisted in ~/.bashrc and ~/.zshrc:

export PATH=${HOME}/.cabal/bin:${PATH}
export PATH=${HOME}/.local/bin:${PATH}

The installation of the R-pandoc filter from the source:

mkdir -p cd ~/tools/pandoc/
cd ~/tools/pandoc/

git clone https://github.com/cdupont/R-pandoc.git
cd R-pandoc

~/.cabal/bin/cabal install

I also installed Haskell tool stack as per https://docs.haskellstack.org/en/stable/README/:

curl -sSL https://get.haskellstack.org/ | sh

Other Filters

The pandoc CLI can be extended with filters, see:

There are many more filters available for pandoc.

Embed CSV Files (pandoc-csv2table)

The section about pandoc-csv2table was removed as it did not work and created failures when creating PDF documents.

The pandoc-placetable is a possible alternative.

Embed CSV Files - pandoc-placetable

The pandoc-placetable filter allows embedding CSV files as tables: https://github.com/mb21/pandoc-placetable

Installation

The installation:

cabal install pandoc-placetable

Usage

This filter can be used by adding the filter -F pandoc-placetable:

pandoc markdown-first-steps.md -o markdown-first-steps.html -s --self-contained --highlight-style tango -c ~/tools/pandoc/1882525/buttondown.css --toc  --webtex -F mermaid-filter -F R-pandoc -F pandoc-placetable

Examples

A CSV file reference example

An inline example:

some,values
even,"with spaces"

Other Filters

Links

Original Links

Scientific Writing

PDF

PDF Template

Text Box

Info Blocks

Alternatives

Markdown-it

Jupyter

Sphinx / MyST

AsciiDoctor

Misc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment