A Lisp interpreter in C, part 1: symbols

README.md

Blog 2020/1/9

<- previous | index | next ->

A Lisp interpreter in C, part 1: symbols

Let's learn how to write a Lisp interpreter in C!

In part 1, we implement a basic echoing REPL (read-eval-print-loop) which thinks everything is a symbol.

This first post is a bit heavy, because there's a lot of basic guts we have to set up just to get a working REPL.

The REPL

repl() reads one Lisp "form", evaluates it, prints the result, then loops.

A Lisp form is the smallest unit of syntax which results in a value. A Lisp form is an atom (a number, symbol, string, etc.) or a list. Of course, a list can contain other lists, so a form can be quite large!

To prompt or not to prompt?

We want our repl to be a bit smart about when it should display a prompt (> ).

$ ./lisp
>

It should prompt the user when run interactively,

$ ./lisp
> (+ 1 1)
2
> (+ 2 3)
5
>

but it shouldn't print any prompts if input is being piped in.

$ cat input.txt
(+ 1 1)
(+ 2 3)
$ cat input | ./lisp
2
5
$

Additionally, if multiple Lisp forms are entered on a single line, it shouldn't print another prompt until it runs out of forms to process.

$ ./lisp
> 1 2 3
1
2
3
>

Here is repl():

repl.c:

/* Starts a read-eval-print loop.
Loops until EOF or I/O error.
Returns 0 or error. */
int repl() {
    FBuff* fbp;
    int err = new_fbuff(&fbp, stdin);
    if (err) {
        return err;
    }

    while (true) {
        fbuff_skip_buffered_ws(fbp);

        if (should_display_prompt(fbp)) {
            /* display the prompt. */
            int err = fputs("> ", stdout);
            if (err == EOF) {
                free_fbuff(fbp);
                err = errno;
                errno = 0;
                return err;
            }
        }

        /* read the next Lisp form. */
        Form* formp;
        err = read_form(fbp, &formp);
        if (err) {
            if (err == EOF) {
                break;
            } else {
                int err2 = fprintf(stderr, "Error %d.\n", err);
                if (err2 < 0) {
                    free_fbuff(fbp);
                    return err2;
                }
                continue;
            }
        }

        /* evaluate the form. */
        Form* resultp;
        err = eval_form(formp, &resultp);
        if (err) {
            int err2 = fprintf(stderr, "Error %d.\n", err);
            if (err2 < 0) {
                free_fbuff(fbp);
                return err2;
            }
            continue;
        }

        /* print the result. */
        err = print_form(resultp, stdout);
        if (err) {
            free_fbuff(fbp);
            return err;
        }
        err = fputs("\n", stdout);
        if (err == EOF) {
            free_fbuff(fbp);
            err = errno;
            errno = 0;
            return err;
        }

        /* loop. */
        continue;
    }

    free_fbuff(fbp);
    return 0;
}

Yuck, so verbose!

Because C doesn't support exceptions, the pedantic error handling ends up making the code very verbose.

Here's the same code but with error handling elided:

repl.c (error handling elided):

int repl() {
    FBuff* fbp;
    new_fbuff(&fbp, stdin);

    while (true) {
        fbuff_skip_buffered_ws(fbp);

        if (should_display_prompt(fbp)) {
            /* display the prompt. */
            fputs("> ", stdout);
        }

        /* read the next Lisp form. */
        Form* formp;
        read_form(fbp, &formp);

        /* evaluate the form. */
        Form* resultp;
        eval_form(formp, &resultp);

        /* print the result. */
        print_form(resultp, stdout);
        fputs("\n", stdout);

        /* loop. */
        continue;
    }

    free_fbuff(fbp);
    return 0;
}

Note: I'm using a bit of Hungarian notation: pointers have names which end in 'p'.

repl() calls the core REPL functions:

• read_form
• eval_form
• print_form

and some FBuff functions:

• new_fbuff
• free_fbuff

and some minutiae related to getting the prompt behavior we described above:

• should_display_prompt
• fbuff_skip_buffered_ws

The reader

A Lisp "reader" reads enough input to parse the next Lisp "form".

This reader is very basic: it turns every token into a Lisp symbol.

reader.c (error handling elided):

/* Read one Lisp form from fp intp formpp.
Returns 0 or EOF or errno or error code. */
int read_form(FBuff* fbp, Form** formpp) {
    int buffsize = 100;
    char buff[buffsize];
    char* buffp = buff;
    char ch1;

    /* read a token. */
    fbuff_get_token(fbp, &buffp, buffsize);
    ch1 = *buffp;

    /* we've reached the end of input. */
    if (ch1 == '\0') {
        return EOF;
    } else {
        /* assume every token is a symbol. */
        Symbol* symp;
        new_symbol(&symp, buffp);
        *formpp = (Form*)symp;
        return 0;
    }
}

The reader is built upon the tokenizer:

• fbuff_get_token

and Symbol creation:

• new_symbol

The tokenizer

_reader.c (error handling elided):

/* Advances fbp far enough to read one token of input.
Writes the token contents to *buffpp.
Returns 0, EOF, errno, or an error code. */
static int fbuff_get_token(FBuff* fbp, char** buffpp, size_t buffsize) {
    char ch;
    size_t bufflen = buffsize - 1;
    char* cursor = *buffpp;

    /* discard any leading whitespace. */
    fbuff_discard_ws(fbp);

    /* a token must be at least one char in length. */
    fbuff_getch(fbp, &ch);
    *cursor = ch;
    cursor++;

    /* the rest of the chars. */
    while (true) {
        fbuff_getch(fbp, &ch);

        /* we've reached the end of this token. */
        if (is_ch_delim(ch)) {
            fbuff_ungetch(fbp, ch);
            *cursor = '\0';
            break;

        /* this char is part of the token. */
        } else {
            *cursor = ch;
            cursor++;
            continue;
        }
    }
    return 0;
}

The tokenizer is built upon character-oriented functions:

• fbuff_getch
• fbuff_ungetch
• is_ch_delim
• fbuff_discard_ws

FBuff

FBuff is a struct and a set of functions which read input in a line-oriented fashion, but presents a character-oriented interface to the tokenizer.

reader.h:

/* A line-oriented FILE* buffer. */
struct FBuff_ {
    FILE* fp;
    char* buffp;
    size_t size;
    size_t len;
    char* nextp;
};
typedef struct FBuff_ FBuff;

int new_fbuff(FBuff** fbpp, FILE* fp);
void free_fbuff(FBuff* fbp);

The line-oriented interface consists of fbuff_getline:

reader.c (error handling elided):

/* Reads the next line into fbp.
Returns 0, EOF, or errno. */
static int fbuff_getline(FBuff* fbp) {
    ssize_t result = getline(&(fbp->buffp), &(fbp->size), fbp->fp);
    if (feof(fbp->fp)) {
        return EOF;
    } else {
        fbp->len = result;
        fbp->nextp = fbp->buffp;
        return 0;
    }
}

The character-oriented interface consists of fbuff_getch and fbuff_ungetch:

reader.c (error handling elided):

/* Reads and consumes the next character into chp from fbp.
Returns 0, EOF, or errno. */
static int fbuff_getch(FBuff* fbp, char* chp) {
    if (is_fbuff_eol(fbp)) {
        fbuff_getline(fbp);
    }
    char ch = *(fbp->nextp);
    (fbp->nextp)++;
    *chp = ch;
    return 0;
}

/* Pushes ch back into fbp.
Asserts if used incorrectly. */
static void fbuff_ungetch(FBuff* fbp, char ch) {
    assert(fbp->nextp > fbp->buffp);
    fbp->nextp--;
    *(fbp->nextp) = ch;
}

/* Is fbp at the end of the current line? */
bool is_fbuff_eol(FBuff* fbp) {
    return fbp->len == 0 || fbp->nextp == fbp->buffp + fbp->len;
}

The evaluator

A Lisp evaluator determines the value of a Lisp form.

This evaluator is a simple no-op:

eval.c:

/* Evaluates formp into resultpp.
Returns 0. */
int eval_form(Form* formp, Form** resultpp) {
    /* for now, all forms evaluate to themselves. */
    if (is_symbol(formp)) {
        *resultpp = formp;
        return 0;

    /* unsupported form. */
    } else {
        assert(false);
    }
}

The printer

A Lisp printer prints a canonical representation of a Lisp form.

This printer only knows how to print Lisp symbols.

For now, the printer will also indicate the type of the form (e.g. Symbol: foo), which will allow us to peek at the internal state of our interpreter and verify its behavior.

printer.c (error-handling elided):

/* Prints the Form in formp into fp.
Returns 0 or errno. */
int print_form(Form* formp, FILE* fp) {
    if (is_symbol(formp)) {
        Symbol* symp = (Symbol*)formp;
        return print_symbol(symp, fp);
    } else {
        assert(false);
    }
}

/* Prints the Symbol in symp into fp.
Returns 0 or errno. */
static int print_symbol(Symbol* symp, FILE* fp) {
    fprintf(fp, "Symbol: %s", symp->valuep);
    return 0;
}

Lisp forms

A Lisp symbol is represented as struct:

forms.h:

struct Symbol_ {
    FormType type;
    char* valuep;
};
typedef struct Symbol_ Symbol;

int new_symbol(Symbol** sympp, const char* sp);
bool is_symbol(Form* formp);

Additionally, we can cast Lisp forms to a type-erased representation:

forms.h:

/* A type-erased Lisp form. */
struct Form_ {
    FormType type;
};
typedef struct Form_ Form;

We use an enum to keep track of the Lisp for types:

forms.h:

/* The list of Lisp form types. */
enum FormType_ {
    _Type_UNINITIALIZED = 0,
    TypeSymbol = 10,
};
typedef enum FormType_ FormType;

Try it out

Give it a try! Click the "Download Zip" button at the top-right of this page.

Build the interpreter using make:

$ make lisp

Fire it up:

$ ./lisp 
> hello world
Symbol: hello
Symbol: world
> (+ 1 1)
Symbol: (+
Symbol: 1
Symbol: 1)
>

As you can see, literally everything is considered a symbol.

You can also pipe into it:

$ echo 1 2 3 | ./lisp 
Symbol: 1
Symbol: 2
Symbol: 3

There are a few other commands implemented in the Makefile:

$ make test
$ make clean

I've run the tests on macOS, linux, and win7/cygwin.

Next time

In part 2 we will add support for C long integers!

errors.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#ifndef _ERRCODE_H_
#define _ERRCODE_H_

enum ErrCode_ {
    /* not an error. */
    E_success = 0,

    /* A catch-all "unknown" error. */
    /* Note: we start at 10,000 to skip past the errno range. */
    E_unknown = 10000,

    /* We ran out of buffer while reading a token. */
    E_file_get_token__buff_overflow = 10010,
};
typedef enum ErrCode_ ErrCode;

#endif

eval.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#include "eval.h"
#include "printer.h"
#include <assert.h>


/* Evaluates formp into resultpp.
Returns 0. */
int eval_form(Form* formp, Form** resultpp) {
    /* for now, all forms evaluate to themselves. */
    if (is_symbol(formp)) {
        *resultpp = formp;
        return 0;

    /* unsupported form. */
    } else {
        assert(false);
    }
}

eval.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#ifndef _EVAL_H_
#define _EVAL_H_

#include "forms.h"

int eval_form(Form* formp, Form** resultpp);

#endif

forms.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

/* All of the Lisp forms. */

#include "forms.h"
#include <stdlib.h>
#include <sys/errno.h>
#include <assert.h>
#include <string.h>



/* Symbol */


/* Malloc's a Symbol into sympp and copies sp into it. */
int new_symbol(Symbol** sympp, const char* sp) {
    Symbol* symp = malloc(sizeof(Symbol));
    if (symp == NULL) {
        int err = errno;
        errno = 0;
        return err;
    }
    symp->type = TypeSymbol;

    size_t len = strlen(sp);
    symp->valuep = malloc(len + 1);
    if (symp->valuep == NULL) {
        free(symp);
        int err = errno;
        errno = 0;
        return err;
    }
    strcpy(symp->valuep, sp);

    *sympp = symp;
    return 0;
}


/* Is formp a Symbol? */
bool is_symbol(Form* formp) {
    return formp->type == TypeSymbol;
}

forms.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

/* All of the Lisp forms. */

#ifndef _FORM_H_
#define _FORM_H_

#include <stdbool.h>


/* The list of Lisp form types. */
enum FormType_ {
    _Type_UNINITIALIZED = 0,

    TypeSymbol = 10,
};
typedef enum FormType_ FormType;


/* A type-erased Lisp form. */
struct Form_ {
    FormType type;
};
typedef struct Form_ Form;


/* Symbol */

struct Symbol_ {
    FormType type;
    char* valuep;
};
typedef struct Symbol_ Symbol;

int new_symbol(Symbol** sympp, const char* sp);
bool is_symbol(Form* formp);


#endif

main.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#include "repl.h"
#include "forms.h"
#include <stdlib.h>
#include <stdio.h>


int main() {
    int err = repl();
    if (err) {
        fprintf(stderr, "Error %d.\n", err);
        return err;
    } else {
        return EXIT_SUCCESS;
    }
}

Makefile

CC=gcc -g -std=c99 -Wall -Werror -D_POSIX_C_SOURCE=200809L

lisp: main.o forms.o reader.o eval.o printer.o repl.o
	$(CC) -o lisp *.o

main.o: main.c
	$(CC) -c main.c

forms.o: forms.h forms.c
	$(CC) -c forms.c

reader.o: reader.h reader.c
	$(CC) -c reader.c

eval.o: eval.h eval.c
	$(CC) -c eval.c

printer.o: printer.h printer.c
	$(CC) -c printer.c

repl.o: repl.h repl.c
	$(CC) -c repl.c

clean:
	rm -f *.o lisp

test: lisp
	./run_tests.sh

.PHONY: clean

printer.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#include "printer.h"
#include <assert.h>


/* Prints the Symbol in symp into fp.
Returns 0 or errno. */
static int print_symbol(Symbol* symp, FILE* fp) {
    int err = fprintf(fp, "Symbol: %s", symp->valuep);
    if (err < 0) {
        return err;
    } else {
        return 0;
    }
}


/* Prints the Form in formp into fp.
Returns 0 or errno. */
int print_form(Form* formp, FILE* fp) {
    if (is_symbol(formp)) {
        Symbol* symp = (Symbol*)formp;
        return print_symbol(symp, fp);
    } else {
        assert(false);
    }
}

printer.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#ifndef _PRINT_H_
#define _PRINT_H_

#include "forms.h"
#include <stdio.h>

int print_form(Form* formp, FILE* fp);

#endif

reader.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#include "reader.h"
#include "errors.h"
#include <stdlib.h>
#include <sys/errno.h>
#include <assert.h>
#include <stdbool.h>
#include <ctype.h>
#include <string.h>


/* Creates a new FBuff.
Returns 0 or errno. */
int new_fbuff(FBuff** fbpp, FILE* fp) {
    FBuff* fbp = malloc(sizeof(FBuff));
    if (fbp == NULL) {
        int err = errno;
        errno = 0;
        return err;
    }

    fbp->fp = fp;
    fbp->buffp = NULL;
    fbp->nextp = NULL;
    fbp->size = 0;
    fbp->len = 0;

    *fbpp = fbp;
    return 0;
}


/* Frees fbp. */
void free_fbuff(FBuff* fbp) {
    free(fbp->buffp);
    free(fbp);
}


/* Reads the next line into fbp.
Returns 0, EOF, or errno. */
static int fbuff_getline(FBuff* fbp) {
    ssize_t result = getline(&(fbp->buffp), &(fbp->size), fbp->fp);
    if (result == -1) {
        if (feof(fbp->fp)) {
            return EOF;
        } else {
            result = errno;
            errno = 0;
            return result;
        }
    } else {
        fbp->len = result;
        fbp->nextp = fbp->buffp;
        return 0;
    }
}


/* Is fbp at the end of the current line? */
bool is_fbuff_eol(FBuff* fbp) {
    return fbp->len == 0 || fbp->nextp == fbp->buffp + fbp->len;
}


/* Reads and consumes the next character into chp from fbp.
Returns 0, EOF, or errno. */
static int fbuff_getch(FBuff* fbp, char* chp) {
    if (is_fbuff_eol(fbp)) {
        int err = fbuff_getline(fbp);
        if (err) {
            return err;
        }
    }
    char ch = *(fbp->nextp);
    (fbp->nextp)++;
    *chp = ch;
    return 0;
}


/* Pushes ch back into fbp.
Asserts if used incorrectly. */
static void fbuff_ungetch(FBuff* fbp, char ch) {
    assert(fbp->nextp > fbp->buffp);
    fbp->nextp--;
    *(fbp->nextp) = ch;
}


/* Is ch considered whitespace?
Note: commas are considered whitespace. */
static bool is_ch_ws(char ch) {
    return isspace(ch) || ch == ',';
}


/* Advances fbp past any leading whitespace.
Note: commas are considered whitespace.
Returns 0, EOF, or errno. */
static int fbuff_discard_ws(FBuff* fbp) {
    int err;
    char ch;

    while (true) {
        err = fbuff_getch(fbp, &ch);
        if (err) {
            return err;
        } else if (is_ch_ws(ch)) {
            continue;
        } else {
            fbuff_ungetch(fbp, ch);
            break;
        }
    }
    return 0;
}


/* Advances fbp past any whitespace in the current line. */
void fbuff_skip_buffered_ws(FBuff* fbp) {
    while (!is_fbuff_eol(fbp) && is_ch_ws(*(fbp->nextp))) {
        fbp->nextp++;
    }
}


/* Does ch indicate the end of a token? */
static bool is_ch_delim(char ch) {
    return is_ch_ws(ch);
}


/* Advances fbp far enough to read one token of input.
Writes the token contents to *buffpp.
Returns 0, EOF, errno, or an error code. */
static int fbuff_get_token(FBuff* fbp, char** buffpp, size_t buffsize) {
    int err;
    char ch;
    size_t bufflen = buffsize - 1;
    char* cursor = *buffpp;

    /* discard any leading whitespace. */
    err = fbuff_discard_ws(fbp);
    if (err) {
        return err;
    }

    /* a token must be at least one char in length. */
    err = fbuff_getch(fbp, &ch);
    if (err) {
        return err;
    }
    *cursor = ch;
    cursor++;

    /* the rest of the chars. */
    while (true) {
        size_t len = cursor - *buffpp;

        /* we have run out of room. */
        if (len == bufflen) {
            return E_file_get_token__buff_overflow;
        }

        err = fbuff_getch(fbp, &ch);
        /* we've reached EOF. return what we have so far. */
        if (err == EOF) {
            *cursor = '\0';
            break;

        /* there was an error reading from fp. */
        } else if (err != 0) {
            return err;

        /* we've reached the end of this token. */
        } else if (is_ch_delim(ch)) {
            fbuff_ungetch(fbp, ch);
            *cursor = '\0';
            break;

        /* this char is part of the token. */
        } else {
            *cursor = ch;
            cursor++;
            continue;
        }
    }
    return 0;
}


/* Read one Lisp form from fp intp formpp.
Returns 0 or EOF or errno or error code. */
int read_form(FBuff* fbp, Form** formpp) {
    int err;
    int buffsize = 100;
    char buff[buffsize];
    char* buffp = buff;
    char ch1;

    /* read a token. */
    err = fbuff_get_token(fbp, &buffp, buffsize);
    if (err) {
        return err;
    }
    ch1 = *buffp;

    /* we've reached the end of input. */
    if (ch1 == '\0') {
        return EOF;
    } else {
        /* assume every token is a symbol. */
        Symbol* symp;
        err = new_symbol(&symp, buffp);
        if (err) {
            return err;
        } else {
            *formpp = (Form*)symp;
            return 0;
        }
    }
}

reader.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#ifndef _READ_H_
#define _READ_H_

#include "forms.h"
#include <stdio.h>


/* A line-oriented FILE* buffer. */
struct FBuff_ {
    FILE* fp;
    char* buffp;
    size_t size;
    size_t len;
    char* nextp;
};
typedef struct FBuff_ FBuff;

int new_fbuff(FBuff** fbpp, FILE* fp);
void free_fbuff(FBuff* fbp);
bool is_fbuff_eol(FBuff* fbp);
void fbuff_skip_buffered_ws(FBuff* fbp);


int read_form(FBuff* fbp, Form** formpp);


#endif

repl.c

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#include "repl.h"
#include "reader.h"
#include "printer.h"
#include "eval.h"
#include <stdbool.h>
#include <sys/errno.h>
#include <stdio.h>
#include <unistd.h>


/* Is fp a tty? */
bool is_file_tty(FILE* fp) {
    int result = isatty(fileno(fp));
    if (result == 0) {
        errno = 0;
        return false;
    } else {
        return true;
    }
}


/* Should the REPL prompt be displayed? */
bool should_display_prompt(FBuff* fbp) {
    if (!is_file_tty(fbp->fp)) {
        return false;
    } else {
        return is_fbuff_eol(fbp);
    }
}


/* Starts a read-eval-print loop.
Loops until EOF or I/O error.
Returns 0 or error. */
int repl() {
    FBuff* fbp;
    int err = new_fbuff(&fbp, stdin);
    if (err) {
        return err;
    }

    while (true) {
        fbuff_skip_buffered_ws(fbp);

        if (should_display_prompt(fbp)) {
            /* display the prompt. */
            int err = fputs("> ", stdout);
            if (err == EOF) {
                free_fbuff(fbp);
                err = errno;
                errno = 0;
                return err;
            }
        }

        /* read the next Lisp form. */
        Form* formp;
        err = read_form(fbp, &formp);
        if (err) {
            if (err == EOF) {
                break;
            } else {
                int err2 = fprintf(stderr, "Error %d.\n", err);
                if (err2 < 0) {
                    free_fbuff(fbp);
                    return err2;
                }
                continue;
            }
        }

        /* evaluate the form. */
        Form* resultp;
        err = eval_form(formp, &resultp);
        if (err) {
            int err2 = fprintf(stderr, "Error %d.\n", err);
            if (err2 < 0) {
                free_fbuff(fbp);
                return err2;
            }
            continue;
        }

        /* print the result. */
        err = print_form(resultp, stdout);
        if (err) {
            free_fbuff(fbp);
            return err;
        }
        err = fputs("\n", stdout);
        if (err == EOF) {
            free_fbuff(fbp);
            err = errno;
            errno = 0;
            return err;
        }

        /* loop. */
        continue;
    }

    free_fbuff(fbp);
    return 0;
}

repl.h

/* This file is Copyright (C) 2019 Jason Pepas. */
/* This file is released under the terms of the MIT License. */
/* See https://opensource.org/licenses/MIT */

#ifndef _REPL_H_
#define _REPL_H_

int repl();

#endif

run_tests.sh

#!/bin/bash

set -e -o pipefail

for f in `ls test*.input`
do
    base=$(basename $f .input)
    echo $base
    cat ${base}.input | ./lisp > /tmp/${base}.out
    diff -urN ${base}.expected /tmp/${base}.out
done

echo "all tests passed"

test1.expected

Symbol: 1
Symbol: 1
Symbol: 2
Symbol: 3
Symbol: 1.2
Symbol: foo
Symbol: "foo"
Symbol: "line
Symbol: one
Symbol: line
Symbol: two"
Symbol: true
Symbol: nil

test1.input

1
1 2 3
1.2
foo
"foo"
"line one
line two"
true
nil

test2.expected

Symbol: ()
Symbol: (
Symbol: )
Symbol: (1)
Symbol: (1
Symbol: 2)
Symbol: (1
Symbol: 2)
Symbol: (1(2()))

test2.input

()
(
)
(1)
(1 2)
(1
2)
(1(2()))

test3.expected

Symbol: ;comment
Symbol: ;
Symbol: comment
Symbol: 1;comment
Symbol: 1
Symbol: ;
Symbol: comment
Symbol: (1(;comment
Symbol: 2;comment
Symbol: ))

test3.input

;comment
; comment
1;comment
1 ; comment
(1(;comment
2;comment
))

Man wow. You are totally right. Ignore my suggestion. Ha. Yup I am sitting here staring at your code and I had not seen ssize_t before. Just checked my man page and you are right I am wrong!

No worries! I actually didn't know about ssize_t myself before starting this journey :)