Skip to content

Instantly share code, notes, and snippets.

@nemith
Created December 14, 2014 04:50
Show Gist options
  • Select an option

  • Save nemith/11bbb4c7a16c1a1e25dc to your computer and use it in GitHub Desktop.

Select an option

Save nemith/11bbb4c7a16c1a1e25dc to your computer and use it in GitHub Desktop.
input
/* comment */
modifier: keyword value2 value3 value4;
statement2;
statement3 value;
section {
statement4;
section 2 {
statement5 value5;
}
}
 bbennett  ~  go run test.go
(tokenBlockComment) '/* comment */'
(tokenModifier) 'modifier'
(tokenKeyword) 'keyword'
(tokenValue) 'value2'
(tokenValue) 'value3'
(tokenValue) 'value4'
(tokenEndStatement) ';'
(tokenKeyword) 'statement2'
(tokenEndStatement) ';'
(tokenKeyword) 'statement3'
(tokenValue) 'value'
(tokenEndStatement) ';'
(tokenKeyword) 'section'
(tokenValue) ''
(tokenSectionStart) '{'
(tokenKeyword) 'statement4'
(tokenEndStatement) ';'
(tokenKeyword) 'section'
(tokenValue) '2'
(tokenValue) ''
(tokenSectionStart) '{'
(tokenKeyword) 'statement5'
(tokenValue) 'value5'
(tokenEndStatement) ';'
(tokenSectionEnd) '}'
(tokenSectionEnd) '}'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment