Skip to content

Instantly share code, notes, and snippets.

@jeroen
Last active March 4, 2022 07:18
Show Gist options
  • Save jeroen/89073a6387e7169a40ec to your computer and use it in GitHub Desktop.
Save jeroen/89073a6387e7169a40ec to your computer and use it in GitHub Desktop.
Getting shit to work

Getting shit to work

Disclaimers

  • No authority
  • Official reference is writing r extentions (joke extentions)
  • If you thought writing regular r packages was tricky, hold on
  • External libraries add a layer of complexity
  • Portable packages need to work with different compilers and systems.
  • Warning: People are opiniated and not always polite.
  • Other people might tell you different things. I just tell you how I like to do it.
  • Fine to mess around with pakcages on Github.
  • Always thoroughly test before submitting to CRAN.

Resources

Example packages

Jeroen's Packages with anticonf scripts.

Package library Lang link type
commonmark cmark C bundled
jsonlite yajl C bundled
curl libcurl + winhttp.dll C system
openssl openssl C system
mongolite mongo-c-driver C bundled
V8 libv8 C++ system
xml2 libxml2 C system
RMySQL libmysqlclient C system

Other packages

Package library Lang link type
RSQLite sqlite C bundled
git2r libgit2 C bundled
stringi ICU C++ bundled
RProtoBuf libprotobuf C++ system

Bit of context

  • R's package management system is modeled after Debian.
  • DCF, configure, make
  • Changed lot over time.
  • CRAN now has lots of checks and requirements.

System linking versus bundling

  • Important choice. Both have pro's and con's.

Bundling entire library:

  • Include the external C/C++ library with the package
  • Need to configure for each system.
  • Need to keep updating upstream library.
  • Very difficult for complex packages
  • Requires more disk space (not an argument)

Link to system library

  • Dynamic linking: library has to be present on the build server and the user system
  • Static linking: library has to be present only on the build server
  • If a suitable version is availble, this is usually preferable
  • The distro deals with config, dependency libraries, etc.

Make and configure

Flow:

  • configure or configure.win (if any)
  • make (via Makevars or Makevars.win if any)
  • sources the R code
  • copy everything and try to load

Make

  • Preferred over configure
  • Executes some rules to make files
  • Access to everything in Makeconf
  • Default: compile and link *.c and *.cpp

Configure scripts:

  • Any scripting
  • Always start with #!/bin/bash
  • Hard to maintain. Makevars is more elegant.

Unix/Linux

  • Start here because this is what R was made for. This is where things make sense, sort of.
  • Install system libraries via pkg manager
  • Dynamic linking: cran does not supply binary packages for Linux
  • Debian/Ubuntu: -dev packages, rpm: -devel
  • EC2 testing (video for Fedora / Debian / RHEL).
  • VirtualBox for Solaris: opencsw

Mac OSX

  • Also Unix (BSD based)
  • Clang instead of gcc.
  • No native packager. But brew is really nice.
  • No brew: just like linux (e.g. curl)
  • To brew (everything else)
  • Autobrew:
    • build server (CRAN)
    • user without brew
  • Be careful: avoid dynamic linking!
  • Static linking: self contained package.

Windows: the big hackery

  • Tool chain: mingw-w64 gcc with some msys tools
  • configure.win and Makevars.win

Linking to system DLL files

  • Can't use lib files. Need an import lib file (libsomething.dll.a)
  • For many system files these are included with mingw. Yay.
  • For others you need a .def file and DLL tool. See makevars
  • If you need to include the .dll file with your package, copy it to ./src during installation.

Steps before submitting to CRAN

  • R CMD check --as-cran
  • Submit to win builder
  • Check on Fedora and Debian/Ubuntu (e.g. via EC2)
  • Check Solaris on VirtualBox
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment