Skip to content

Instantly share code, notes, and snippets.

@TerryE

TerryE/review-53.md

Last active September 14, 2019 16:13
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save TerryE/22c28d6622aaf54033eceb14752e77f2 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save TerryE/22c28d6622aaf54033eceb14752e77f2 to your computer and use it in GitHub Desktop.
Download ZIP
Review Notes on Changes to app/lua53
Raw
review-53.md

Review Notes on Changes to app/lua53

These are my notes following my own review of the lua53 commit diff file:

Reasons for functional changes

  • The type LUA_TTABLE now has subtypes LUA_TTBLRAM and LUA_TTBLROF, with handling of these subtypes following the model adopted for strings being split into short and long subtypes. In general the variant coding for table subtypes is managed as low as possible in the ltable.c routine. The new ROTable is a cut down version of Table which only includes the fields used in ROTables.

  • The getXXX(o) access macros replace (o)->XXX field accesses for lu_byte fields in records that could be in constant (flash-based) memory.

  • Functional support for dynamically loaded libraries removed from lauxlib.c, ldo.c, and loadlib.c.

  • On firmware builds all ROM modules and base functions are in the ROTable ROM. The metatable of _G is _G and both __index and ROM point to this ROTable. On luac.cross builds ROM only contains the ROM modules, with its meta __index pointing to a separate baselib ROTable. This gives the fastest runtime performance in the case of firmware builds, but luac.cross builds can now be linked using standard linker defaults without any GCC, MSVC, etc. botches. In both cases all ROM tables and base functions are resolved through the global environment, _G.

  • The new 5.3 string cache is replaced by a unified keycache which is used for both string and ROTable entry caching. The hash algo is now prime multiplier based to allow the use of a 2^n KEYCACHE_N number of rows. This avoid the need for an expensive modulus call during cache lookup.

  • LCD style compressed line info is implemented as standard through lcode.c and ldebug.c.

  • The math library is now pretty complete.

  • ldblib.c now contains a pretty complete debug implementation (less debug.debug for firmware builds as this would require take-over of stdin).

  • Error reporting is not directed to stderr, but instead the error string is posted as an upval to the C closure error reporter helper. This then calls the Lua error reporter as a separate task.

  • lapi.c and lauxlib.c implement the API and interface changes listed below.

  • lobject.c includes a fast implementation of luaO_ceillog2() using an asm("nsau %0, %1;") instruction which is a lot faster and avoids the need for a byte lookup table.

  • The lstate.c seed aglo is fixed rather than using randomisation.

  • lua.c is a complete reimplementation more closely following the current NodeMCU 5.1 lua.c implementation. This is as a result of architectural drivers arising from its context and being initiated within the startup sequence of the IoT embedded runtime.

    1. Processing is based on a single threaded event loop model. The Lua interactive mode processes input lines from a stdin pipe. This must be handled on a line by line basis, and other Lua tasks can interleave any multiline processing, so the standard doREPL approach doesn't work.
    2. Most OS services and environment processing are supported so much of the standard functionality is irrelevant and is stripped out for simplicity.
    3. stderr and stdout redirection aren't offered as an SDK service, so this is handled in the baselib print function and errors are sent to print.

  • luaconf.h has been reworked to reflect the IoT implementation.

  • locales support has been removed from lvm.c. Ditto caching of last closure in Proto records.

  • ltest.c has its Memcontrol structure extended to include a double linked list. This allows me to set H/W watchpoints on dangling blocks in host gdb to work out what blocks aren't being GCed.

  • The test suite has disabled tests which aren't appropriate (e.g. dynamic loading). Even so I need to do a second pass to make sure that I haven't disabled valid, but failing tests.

API and Interface Changes

  • lua_gc has an extra parameter option LUA_GCSETMEMLIMIT to set the ECG memory limit.

  • lua_pushrotable, void lua_pushrotable(lua_State *L, const ROTable *p) can be used to push a ROtable onto the Lua Stack.

  • lua_createrotable, void lua_createrotable(lua_State *L, ROTable *t, const ROTable_entry *e, ROTable *mt) is used the create a ROTable header for the specified ROTable_entry vector. This is only required for linker-marshalled entry vectors as the ROTable is normally generated by the LROT macro declarations.

  • lua_getstate, lua_State *lua_getstate(void). Returns the L0 state. Used in C modules CBs to call the Lua VM.

  • lua_getcache, KeyCache *lua_getcache(int cacheline) used internally to access the string and table key cache.

  • lua_getstrings, int lua_getstrings(lua_State *L, int opt) returns a table of strings in the specified string table (0 = RAM, 1 = LFS ROM)

  • lua_freeheap, int lua_freeheap(void) returns the amount of free heap.

  • luaL_rometatable, int luaL_rometatable(lua_State *L, const char* tname, const ROTable *p) shorthand for lua_pushrotable() and luaL_newmetatable(); used to associate a ROTable metatable with the entry tname in the Lua registry.

  • luaL_pcallx, int luaL_pcallx(lua_State *L, int narg, int nres). This is designed as a plug-in replacement for lua_call(L, n, m) used to call Lua CBs in the module event routines. Unlike lua_call(), this does a protected call and returns a call status. In the case of the called routine throwing an error, instead of the VM panicing and restarting, the error handler collects a full traceback and posts a separate task with this error string as an upval. The error reporter then calls the users reporter function, which can then print or log the error, and continue or restart as required.

  • luaL_posttask, int luaL_posttask(lua_State* L, int prio) Post the task popped from the stack at the specified priority.

The build will compile in the Lua Test suite if LUA_ENABLE_TEST and LUA_USE_HOST are defined -- that is for test luac.cross builds only.

To be implemented

  • lgc.c to honour the lua_gc() option LUA_GCSETMEMLIMIT

  • Replace all lua_call() uses in the app/modules files with the luaL_pcallx() variant.

  • Support loading both standard (format=0) and NodeMCU (format=10) dump formats. Also use the head to determin the int and float sizes.

  • LFS implementation

  • Write up a howto on using the LROT macros.

Known Issues

  • debug.getstrings() raises a E:M 32784 not enough memory error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment