lazar/tia-lsp
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Compare commits

base: lazar/tia-lsp:v0.1.1
Branches Tags
lazar/tia-lsp:main lazar/tia-lsp:develop
lazar/tia-lsp:v0.1.7 lazar/tia-lsp:v0.1.6 lazar/tia-lsp:v0.1.5 lazar/tia-lsp:v0.1.2 lazar/tia-lsp:v0.1.1 lazar/tia-lsp:v0.1.0
..
compare: lazar/tia-lsp:v0.1.2
Branches Tags
lazar/tia-lsp:main lazar/tia-lsp:develop
lazar/tia-lsp:v0.1.7 lazar/tia-lsp:v0.1.6 lazar/tia-lsp:v0.1.5 lazar/tia-lsp:v0.1.2 lazar/tia-lsp:v0.1.1 lazar/tia-lsp:v0.1.0
1 Commits
v0.1.1 .. v0.1.2
Author SHA1 Message Date
lazar fb55915147 feat: major grammar upgrade for SCL spec conformance (Phase 1) 
Comprehensive grammar rewrite to conform to SCL specs. Improves parse
success rate from 0% to 90% (187 of 208 reference project files).

Case-insensitive keywords:
- Add ci() helper for case-insensitive token matching
- All keywords (TYPE, STRUCT, VAR, IF, etc.) now match any case
- All type names (BOOL, Int, Dint, etc.) now match any case

New grammar constructs:
- RETURN statement
- GOTO statement
- AUTHOR/FAMILY/NAME block headers
- VAR DB_SPECIFIC section
- ARRAY[*] (variable-length arrays)
- Multidimensional ARRAY[a..b, c..d]
- REF_TO <type>
- VARIANT type
- Anonymous STRUCT...END_STRUCT as a type
- Compound assignment operators (+=, -=, *=, /=, &=, |=, ^=)
- Chained assignments (lvalue := lvalue := expression)
- Bit access (lvalue.%X0)
- Method calls (lvalue.field(...))
- Struct/array initializers ((TRUE, FALSE, ...), (name := value))
- END_STRUCT; (semicolon after END_STRUCT in UDTs)
- Typed DB associated block name (DATA_BLOCK "Name" : "UDT")
- BOM (Byte Order Mark) handling
- TITLE as single-token statement (avoids keyword/identifier conflict)

New type_builtin aliases:
- LWORD, LINT, ULINT, UDINT, DATE, TIME_OF_DAY, DATE_AND_TIME
- S5TIME, LTIME, DTL, LTOD, LDT, LTIME_OF_DAY, DATE_AND_LTIME

New literal formats:
- Binary (2#...), Octal (8#...)
- BOOL#TRUE/FALSE
- Typed integers (INT#, SINT#, DINT#, etc.)
- Date/time literals (DATE#, TOD#, DT#, LTIME#, S5TIME#, DTL#)
- WSTRING# and STRING# typed string literals
- C# char literal
- Time units with case-insensitive matching (T#1S, T#2s)

Updated constructs:
- CASE labels: comma-separated lists, string labels
- Function call parameters: interleaved input (:=) and output (=>) params
- FOR loop variable: accepts prefixed_identifier (#tempVar)
- Array bounds: accept string/identifier constants (Array[0.."MAX"])
- Author/Family values: accept unquoted text with hyphens/slashes

Known remaining issues (21 files, 10%):
- Triple-nested parentheses in expressions
- Some multi-line function call parameter patterns
- END_REGION with trailing region name
- Complex chained field access in assignments
 2026-07-18 19:19:03 +02:00
5 changed files with 105863 additions and 75562 deletions
Expand all files Collapse all files
grammar.js
+381 -195
View File
@@ -1,3 +1,10 @@
// Case-insensitive token helper: matches any case combination of a word.
function ci(word) {
return token(new RegExp(word.split('').map(c =>
c.toUpperCase() === c.toLowerCase() ? c : '[' + c.toUpperCase() + c.toLowerCase() + ']'
).join('')));
}
module.exports = grammar({
name: 'scl',
@@ -8,91 +15,138 @@ module.exports = grammar({
$.c_style_comment
],
conflicts: $ => [
[$.case_item],
[$.region_statement],
[$.region_name],
[$.case_statement],
[$.fb_call, $.function_call],
[$.fb_parameter, $.parameter],
[$.var_declaration, $.var_temp_declaration],
[$.var_declaration, $.var_constant_declaration],
[$.var_declaration, $.var_retain_declaration],
[$.var_declaration, $.var_non_retain_declaration]
],
conflicts: $ => [
[$.case_item],
[$.region_statement],
[$.case_statement],
[$.fb_call, $.function_call],
[$.fb_parameter, $.parameter],
[$.var_declaration, $.var_temp_declaration],
[$.var_declaration, $.var_constant_declaration],
[$.var_declaration, $.var_retain_declaration],
[$.var_declaration, $.var_non_retain_declaration],
[$.var_declaration, $.var_db_specific_declaration],
[$.type, $.struct_type],
[$.expression, $.struct_initializer]
],
rules: {
source_file: $ => repeat(
choice(
$.organization_block,
$.function_block,
$.function,
$.type_definition,
$.data_block,
$.statement
source_file: $ => seq(
optional($.bom),
repeat(
choice(
$.organization_block,
$.function_block,
$.function,
$.type_definition,
$.data_block,
$.statement
)
)
),
// Title statement: matches the entire "TITLE = <text>" line as a single token
title_statement: $ => token(seq(
/[Tt][Ii][Tt][Ll][Ee]/,
/\s*=\s*/,
/[^\n\r]+/
)),
// Author/Family value: unquoted text with hyphens, underscores, slashes, etc.
// E.g. L-TECH, Siemens_Digital_Industries, Manz/Siemens (without quotes)
author_value: $ => token(repeat1(/[a-zA-Z0-9_\-\/]/)),
// --- Block definitions ---
type_definition: $ => seq(
alias('TYPE', $.keyword),
alias(ci('TYPE'), $.keyword),
field('name', choice($.string, $.identifier)),
optional(seq(alias('TITLE', $.keyword), '=', choice($.string, $.identifier))),
optional($.title_statement),
optional($.attribute_list),
optional(seq(alias('VERSION', $.keyword), ':', $.number)),
alias('STRUCT', $.keyword),
optional(seq(alias(ci('NAME'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('AUTHOR'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('FAMILY'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('VERSION'), $.keyword), ':', $.number)),
alias(ci('STRUCT'), $.keyword),
repeat(choice($.var_item, $.block_comment)),
alias('END_STRUCT', $.keyword),
alias('END_TYPE', $.keyword)
alias(ci('END_STRUCT'), $.keyword),
optional(';'),
alias(ci('END_TYPE'), $.keyword)
),
data_block: $ => seq(
alias('DATA_BLOCK', $.keyword),
alias(ci('DATA_BLOCK'), $.keyword),
field('name', choice($.string, $.identifier)),
optional(seq(alias('TITLE', $.keyword), '=', choice($.string, $.identifier))),
optional($.title_statement),
optional($.attribute_list),
optional(seq(alias('VERSION', $.keyword), ':', $.number)),
optional(seq(alias(ci('NAME'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('AUTHOR'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('FAMILY'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('VERSION'), $.keyword), ':', $.number)),
optional(choice(
alias('NON_RETAIN', $.keyword),
alias('RETAIN', $.keyword)
alias(ci('NON_RETAIN'), $.keyword),
alias(ci('RETAIN'), $.keyword)
)),
optional($.var_declaration),
alias('BEGIN', $.keyword),
// Associated FB/UDT name (for typed DBs: appears on its own line)
optional(field('associated', $.string)),
optional(choice(
$.var_declaration,
$.var_db_specific_declaration
)),
alias(ci('BEGIN'), $.keyword),
repeat($.statement),
alias('END_DATA_BLOCK', $.keyword)
alias(ci('END_DATA_BLOCK'), $.keyword)
),
organization_block: $ => seq(
alias('ORGANIZATION_BLOCK', $.keyword),
alias(ci('ORGANIZATION_BLOCK'), $.keyword),
field('name', $.string),
optional(seq(alias('TITLE', $.keyword), '=', choice($.string, $.identifier))),
optional($.title_statement),
optional($.attribute_list),
optional(seq(alias('VERSION', $.keyword), ':', $.number)),
optional(seq(alias(ci('NAME'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('AUTHOR'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('FAMILY'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('VERSION'), $.keyword), ':', $.number)),
repeat($.var_declaration),
alias('BEGIN', $.keyword),
alias(ci('BEGIN'), $.keyword),
repeat($.statement),
alias('END_ORGANIZATION_BLOCK', $.keyword)
alias(ci('END_ORGANIZATION_BLOCK'), $.keyword)
),
function_block: $ => seq(
alias('FUNCTION_BLOCK', $.keyword),
alias(ci('FUNCTION_BLOCK'), $.keyword),
field('name', $.string),
optional(seq(alias('TITLE', $.keyword), '=', choice($.string, $.identifier))),
optional($.title_statement),
optional($.attribute_list),
optional(seq(alias('VERSION', $.keyword), ':', $.number)),
repeat($.var_declaration),
alias('BEGIN', $.keyword),
optional(seq(alias(ci('NAME'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('AUTHOR'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('FAMILY'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('VERSION'), $.keyword), ':', $.number)),
repeat(choice(
$.var_declaration,
$.var_temp_declaration,
$.var_constant_declaration,
$.var_retain_declaration,
$.var_non_retain_declaration,
$.var_db_specific_declaration,
$.block_comment
)),
alias(ci('BEGIN'), $.keyword),
repeat($.statement),
alias('END_FUNCTION_BLOCK', $.keyword)
alias(ci('END_FUNCTION_BLOCK'), $.keyword)
),
function: $ => seq(
alias('FUNCTION', $.keyword),
alias(ci('FUNCTION'), $.keyword),
field('name', $.string),
optional(seq(':', $.type)),
optional(seq(alias('TITLE', $.keyword), '=', choice($.string, $.identifier))),
optional($.title_statement),
optional($.attribute_list),
optional(seq(alias('VERSION', $.keyword), ':', $.number)),
optional(seq(alias(ci('NAME'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('AUTHOR'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('FAMILY'), $.keyword), ':', choice($.string, $.author_value))),
optional(seq(alias(ci('VERSION'), $.keyword), ':', $.number)),
repeat(choice(
$.var_declaration,
$.var_temp_declaration,
@@ -101,11 +155,13 @@ module.exports = grammar({
$.var_non_retain_declaration,
$.block_comment
)),
alias('BEGIN', $.keyword),
alias(ci('BEGIN'), $.keyword),
repeat($.statement),
alias('END_FUNCTION', $.keyword)
alias(ci('END_FUNCTION'), $.keyword)
),
// --- Attributes ---
attribute_list: $ => seq(
'{',
$.attribute,
@@ -122,132 +178,163 @@ module.exports = grammar({
attribute_value: $ => choice(
$.string,
alias('TRUE', $.constant),
alias('FALSE', $.constant),
alias(ci('TRUE'), $.constant),
alias(ci('FALSE'), $.constant),
$.identifier
),
// --- VAR section declarations ---
var_declaration: $ => seq(
choice(
alias('VAR_INPUT', $.keyword),
alias('VAR_OUTPUT', $.keyword),
alias('VAR_IN_OUT', $.keyword),
alias('VAR', $.keyword),
alias('VAR_TEMP', $.keyword),
seq(alias('VAR', $.keyword), alias('CONSTANT', $.keyword)),
seq(alias('VAR', $.keyword), alias('RETAIN', $.keyword)),
seq(alias('VAR', $.keyword), alias('NON_RETAIN', $.keyword)),
alias('VAR_CONSTANT', $.keyword),
alias('VAR_RETAIN', $.keyword),
alias('VAR_NON_RETAIN', $.keyword)
alias(ci('VAR_INPUT'), $.keyword),
alias(ci('VAR_OUTPUT'), $.keyword),
alias(ci('VAR_IN_OUT'), $.keyword),
alias(ci('VAR'), $.keyword),
alias(ci('VAR_TEMP'), $.keyword),
seq(alias(ci('VAR'), $.keyword), alias(ci('CONSTANT'), $.keyword)),
seq(alias(ci('VAR'), $.keyword), alias(ci('RETAIN'), $.keyword)),
seq(alias(ci('VAR'), $.keyword), alias(ci('NON_RETAIN'), $.keyword)),
alias(ci('VAR_CONSTANT'), $.keyword),
alias(ci('VAR_RETAIN'), $.keyword),
alias(ci('VAR_NON_RETAIN'), $.keyword)
),
repeat(choice($.var_item, $.block_comment)),
alias('END_VAR', $.keyword)
alias(ci('END_VAR'), $.keyword)
),
var_temp_declaration: $ => seq(
alias('VAR_TEMP', $.keyword),
alias(ci('VAR_TEMP'), $.keyword),
repeat(choice($.var_item, $.block_comment)),
alias('END_VAR', $.keyword)
alias(ci('END_VAR'), $.keyword)
),
var_constant_declaration: $ => seq(
choice(
seq(alias('VAR', $.keyword), alias('CONSTANT', $.keyword)),
alias('VAR_CONSTANT', $.keyword)
seq(alias(ci('VAR'), $.keyword), alias(ci('CONSTANT'), $.keyword)),
alias(ci('VAR_CONSTANT'), $.keyword)
),
repeat(choice($.var_item, $.block_comment)),
alias('END_VAR', $.keyword)
alias(ci('END_VAR'), $.keyword)
),
var_retain_declaration: $ => seq(
choice(
seq(alias('VAR', $.keyword), alias('RETAIN', $.keyword)),
alias('VAR_RETAIN', $.keyword)
seq(alias(ci('VAR'), $.keyword), alias(ci('RETAIN'), $.keyword)),
alias(ci('VAR_RETAIN'), $.keyword)
),
repeat(choice($.var_item, $.block_comment)),
alias('END_VAR', $.keyword)
alias(ci('END_VAR'), $.keyword)
),
var_non_retain_declaration: $ => seq(
choice(
seq(alias('VAR', $.keyword), alias('NON_RETAIN', $.keyword)),
alias('VAR_NON_RETAIN', $.keyword)
seq(alias(ci('VAR'), $.keyword), alias(ci('NON_RETAIN'), $.keyword)),
alias(ci('VAR_NON_RETAIN'), $.keyword)
),
repeat(choice($.var_item, $.block_comment)),
alias('END_VAR', $.keyword)
alias(ci('END_VAR'), $.keyword)
),
var_db_specific_declaration: $ => seq(
seq(alias(ci('VAR'), $.keyword), alias(ci('DB_SPECIFIC'), $.keyword)),
repeat(choice($.var_item, $.block_comment)),
alias(ci('END_VAR'), $.keyword)
),
// --- Variable items ---
var_item: $ => seq(
field('name', $.identifier),
optional($.attribute_list),
':',
field('type', $.type),
optional(seq(':=', $.expression)),
optional(seq(':=', choice($.expression, $.struct_initializer))),
';'
),
// --- Types ---
type: $ => choice(
alias('BOOL', $.type_builtin),
alias('Bool', $.type_builtin),
alias('INT', $.type_builtin),
alias('Int', $.type_builtin),
alias('int', $.type_builtin),
alias('DINT', $.type_builtin),
alias('DInt', $.type_builtin),
alias('dint', $.type_builtin),
alias('SINT', $.type_builtin),
alias('SInt', $.type_builtin),
alias('sint', $.type_builtin),
alias('USINT', $.type_builtin),
alias('USInt', $.type_builtin),
alias('usint', $.type_builtin),
alias('UINT', $.type_builtin),
alias('UInt', $.type_builtin),
alias('uint', $.type_builtin),
alias('WORD', $.type_builtin),
alias('Word', $.type_builtin),
alias('word', $.type_builtin),
alias('DWORD', $.type_builtin),
alias('DWord', $.type_builtin),
alias('dword', $.type_builtin),
alias('BYTE', $.type_builtin),
alias('Byte', $.type_builtin),
alias('byte', $.type_builtin),
alias('REAL', $.type_builtin),
alias('Real', $.type_builtin),
alias('real', $.type_builtin),
alias('LREAL', $.type_builtin),
alias('LReal', $.type_builtin),
alias('lreal', $.type_builtin),
alias('TIME', $.type_builtin),
alias('Time', $.type_builtin),
alias('time', $.type_builtin),
alias('TOD', $.type_builtin),
alias('Tod', $.type_builtin),
alias('tod', $.type_builtin),
alias('DT', $.type_builtin),
alias('Dt', $.type_builtin),
alias('dt', $.type_builtin),
alias('String', $.type_builtin),
alias('STRING', $.type_builtin),
alias('string', $.type_builtin),
seq(choice(alias('String', $.type_builtin), alias('STRING', $.type_builtin), alias('string', $.type_builtin)), '[', $.number, ']'),
alias('WString', $.type_builtin),
alias('WSTRING', $.type_builtin),
alias('wstring', $.type_builtin),
seq(choice(alias('WString', $.type_builtin), alias('WSTRING', $.type_builtin), alias('wstring', $.type_builtin)), '[', $.number, ']'),
alias('Char', $.type_builtin),
alias('CHAR', $.type_builtin),
alias('char', $.type_builtin),
alias('WChar', $.type_builtin),
alias('WCHAR', $.type_builtin),
alias('wchar', $.type_builtin),
seq(alias('ARRAY', $.keyword), '[', $.number, '..', $.number, ']', alias('OF', $.keyword), $.type),
$.type_builtin,
$.array_type,
$.ref_type,
$.struct_type,
$.string_type,
$.identifier,
$.string
),
type_builtin: $ => choice(
alias(ci('BOOL'), $.type_builtin),
alias(ci('BYTE'), $.type_builtin),
alias(ci('WORD'), $.type_builtin),
alias(ci('DWORD'), $.type_builtin),
alias(ci('LWORD'), $.type_builtin),
alias(ci('SINT'), $.type_builtin),
alias(ci('USINT'), $.type_builtin),
alias(ci('INT'), $.type_builtin),
alias(ci('UINT'), $.type_builtin),
alias(ci('DINT'), $.type_builtin),
alias(ci('UDINT'), $.type_builtin),
alias(ci('LINT'), $.type_builtin),
alias(ci('ULINT'), $.type_builtin),
alias(ci('REAL'), $.type_builtin),
alias(ci('LREAL'), $.type_builtin),
alias(ci('CHAR'), $.type_builtin),
alias(ci('WCHAR'), $.type_builtin),
alias(ci('TIME'), $.type_builtin),
alias(ci('LTIME'), $.type_builtin),
alias(ci('S5TIME'), $.type_builtin),
alias(ci('DATE'), $.type_builtin),
alias(ci('TIME_OF_DAY'), $.type_builtin),
alias(ci('TOD'), $.type_builtin),
alias(ci('LTOD'), $.type_builtin),
alias(ci('LTIME_OF_DAY'), $.type_builtin),
alias(ci('DATE_AND_TIME'), $.type_builtin),
alias(ci('DT'), $.type_builtin),
alias(ci('LDT'), $.type_builtin),
alias(ci('DATE_AND_LTIME'), $.type_builtin),
alias(ci('DTL'), $.type_builtin),
alias(ci('VARIANT'), $.type_builtin)
),
string_type: $ => seq(
choice(
alias(ci('STRING'), $.type_builtin),
alias(ci('WSTRING'), $.type_builtin)
),
optional(seq('[', $.number, ']'))
),
array_type: $ => seq(
alias(ci('ARRAY'), $.keyword),
'[',
repeat1(seq(
choice(
seq(choice($.number, $.string, $.identifier), '..', choice($.number, $.string, $.identifier)),
'*'
),
optional(',')
)),
']',
alias(ci('OF'), $.keyword),
$.type
),
ref_type: $ => seq(
alias(ci('REF_TO'), $.keyword),
$.type
),
struct_type: $ => seq(
alias(ci('STRUCT'), $.keyword),
repeat(choice($.var_item, $.block_comment)),
alias(ci('END_STRUCT'), $.keyword)
),
// --- Statements ---
statement: $ => choice(
$.assignment,
$.fb_call,
@@ -258,6 +345,8 @@ module.exports = grammar({
$.repeat_statement,
$.region_statement,
$.function_call,
$.return_statement,
$.goto_statement,
$.exit_statement,
$.continue_statement,
$.empty_statement,
@@ -265,17 +354,18 @@ module.exports = grammar({
$.var_temp_declaration,
$.var_constant_declaration,
$.var_retain_declaration,
$.var_non_retain_declaration
$.var_non_retain_declaration,
$.var_db_specific_declaration
),
empty_statement: $ => ';',
// --- FB calls ---
fb_call: $ => seq(
field('instance', choice($.prefixed_identifier, $.identifier)),
'(',
optional(seq(
$.fb_parameter_list
)),
optional($.fb_parameter_list),
')'
),
@@ -290,19 +380,28 @@ module.exports = grammar({
$.expression
),
// --- Assignments (including compound and chained) ---
assignment: $ => seq(
field('target', $.lvalue),
':=',
field('value', $.expression),
choice(':=', '+=', '-=', '*=', '/=', '&=', '|=', '^='),
field('value', choice($.expression, $.chained_assignment)),
';'
),
chained_assignment: $ => prec.right(seq(
$.lvalue,
':=',
choice($.expression, $.chained_assignment)
)),
lvalue: $ => choice(
$.identifier,
$.prefixed_identifier,
$.field_access,
$.array_access,
$.function_name
$.function_name,
$.bit_access
),
function_name: $ => $.string,
@@ -313,37 +412,44 @@ module.exports = grammar({
array_access: $ => seq($.lvalue, '[', $.expression, ']'),
// Bit access: lvalue.%X0, lvalue.%X15, etc.
bit_access: $ => seq($.lvalue, '.', '%', /[A-Za-z]+/, $.number),
// --- Control flow ---
if_statement: $ => seq(
alias('IF', $.keyword),
alias(ci('IF'), $.keyword),
$.expression,
alias('THEN', $.keyword),
alias(ci('THEN'), $.keyword),
repeat($.statement),
repeat($.elsif_clause),
optional($.else_clause),
alias('END_IF', $.keyword)
alias(ci('END_IF'), $.keyword)
),
elsif_clause: $ => seq(
alias('ELSIF', $.keyword),
alias(ci('ELSIF'), $.keyword),
$.expression,
alias('THEN', $.keyword),
alias(ci('THEN'), $.keyword),
repeat($.statement)
),
else_clause: $ => seq(alias('ELSE', $.keyword), repeat($.statement)),
else_clause: $ => seq(alias(ci('ELSE'), $.keyword), repeat($.statement)),
case_statement: $ => seq(
alias('CASE', $.keyword),
alias(ci('CASE'), $.keyword),
$.expression,
alias('OF', $.keyword),
alias(ci('OF'), $.keyword),
repeat($.case_item),
optional($.else_clause),
alias('END_CASE', $.keyword),
optional(';')
alias(ci('END_CASE'), $.keyword)
),
case_item: $ => seq(
choice($.number, $.identifier, $.prefixed_identifier, $.range),
repeat1(seq(
choice($.number, $.string, $.identifier, $.prefixed_identifier, $.range),
optional(',')
)),
':',
repeat($.statement)
),
@@ -351,118 +457,195 @@ module.exports = grammar({
range: $ => seq($.number, '..', $.number),
for_statement: $ => seq(
alias('FOR', $.keyword),
$.identifier,
alias(ci('FOR'), $.keyword),
choice($.identifier, $.prefixed_identifier),
':=',
$.expression,
alias('TO', $.keyword),
alias(ci('TO'), $.keyword),
$.expression,
optional(seq(alias('BY', $.keyword), $.expression)),
alias('DO', $.keyword),
optional(seq(alias(ci('BY'), $.keyword), $.expression)),
alias(ci('DO'), $.keyword),
repeat($.statement),
alias('END_FOR', $.keyword)
alias(ci('END_FOR'), $.keyword)
),
while_statement: $ => seq(
alias('WHILE', $.keyword),
alias(ci('WHILE'), $.keyword),
$.expression,
alias('DO', $.keyword),
alias(ci('DO'), $.keyword),
repeat($.statement),
alias('END_WHILE', $.keyword)
alias(ci('END_WHILE'), $.keyword)
),
repeat_statement: $ => seq(
alias('REPEAT', $.keyword),
alias(ci('REPEAT'), $.keyword),
repeat($.statement),
alias('UNTIL', $.keyword),
alias(ci('UNTIL'), $.keyword),
$.expression,
alias('END_REPEAT', $.keyword)
alias(ci('END_REPEAT'), $.keyword)
),
region_statement: $ => seq(
alias('REGION', $.keyword),
alias(ci('REGION'), $.keyword),
$.region_name,
repeat($.statement),
alias('END_REGION', $.keyword),
alias(ci('END_REGION'), $.keyword),
optional($.region_name)
),
region_name: $ => repeat1(
choice(
$.identifier,
$.string,
'/',
'-'
)
region_name: $ => token(repeat1(choice(/[a-zA-Z0-9_\-\/]/, /[ \t]+/))),
return_statement: $ => alias(ci('RETURN'), $.keyword),
goto_statement: $ => seq(alias(ci('GOTO'), $.keyword), $.identifier),
// --- Function calls (with interleaved input/output params) ---
function_call: $ => choice(
seq(choice($.identifier, $.string), '(', optional($.parameter_list), ')'),
seq($.method_callee, '(', optional($.parameter_list), ')')
),
function_call: $ => seq(
choice($.identifier, $.string),
'(',
optional(seq(
$.parameter_list,
optional(seq('=>', $.identifier))
)),
')'
),
method_callee: $ => seq($.lvalue, '.', $.identifier),
parameter_list: $ => seq(
$.parameter,
repeat(seq(',', $.parameter))
),
parameter: $ => seq($.identifier, ':=', $.expression),
parameter: $ => choice(
prec(1, seq(field('name', $.identifier), ':=', $.expression)),
prec(1, seq(field('name', $.identifier), '=>', $.identifier))
),
// --- Expressions ---
expression: $ => choice(
$.number,
$.hex_number,
$.binary_number,
$.octal_number,
$.time_value,
$.date_literal,
$.bool_literal,
$.typed_int_literal,
$.wstring_literal,
$.char_literal,
$.identifier,
$.prefixed_identifier,
$.field_access,
$.array_access,
$.bit_access,
$.string,
$.function_call,
$.unary_expression,
$.binary_expression,
seq('(', $.expression, ')')
prec(1, seq('(', $.expression, ')'))
),
unary_expression: $ => prec.right(10, choice(
seq(alias('NOT', $.operator), $.expression),
seq(alias(ci('NOT'), $.operator), $.expression),
seq('+', $.expression),
seq('-', $.expression)
)),
binary_expression: $ => choice(
prec.left(2, seq($.expression, alias('OR', $.operator), $.expression)),
prec.left(3, seq($.expression, alias('XOR', $.operator), $.expression)),
prec.left(4, seq($.expression, alias('AND', $.operator), $.expression)),
prec.left(2, seq($.expression, alias(ci('OR'), $.operator), $.expression)),
prec.left(3, seq($.expression, alias(ci('XOR'), $.operator), $.expression)),
prec.left(4, seq($.expression, alias(ci('AND'), $.operator), $.expression)),
prec.left(5, seq($.expression, choice('=', '<>'), $.expression)),
prec.left(6, seq($.expression, choice('<', '>', '<=', '>='), $.expression)),
prec.left(7, seq($.expression, choice('+', '-'), $.expression)),
prec.left(8, seq($.expression, choice('*', '/', alias('MOD', $.operator)), $.expression))
prec.left(8, seq($.expression, choice('*', '/', alias(ci('MOD'), $.operator)), $.expression))
),
// Struct/array initializer: (TRUE, FALSE, 1, 2, ...) or ((1, 2), (3, 4)) or () or (name := value, ...)
struct_initializer: $ => prec(1, seq(
'(',
optional(seq(
choice(
$.struct_initializer,
seq(field('name', $.identifier), ':=', choice($.expression, $.struct_initializer)),
$.expression
),
repeat(seq(',', choice(
$.struct_initializer,
seq(field('name', $.identifier), ':=', choice($.expression, $.struct_initializer)),
$.expression
))),
optional(',')
)),
')'
)),
// --- Literals ---
number: $ => token(/\d+(\.\d+)?/),
hex_number: $ => token(seq('16#', /[0-9A-Fa-f_]+/)),
time_value: $ => token(seq('T#', /\d+(_?\d)*(ms|s|min|h)/)),
binary_number: $ => token(seq('2#', /[01_]+/)),
octal_number: $ => token(seq('8#', /[0-7_]+/)),
time_value: $ => token(seq(
/[Tt]#/,
repeat1(seq(/\d+/, choice(/[Nn][Ss]/, /[Uu][Ss]/, /[Mm][Ss]/, /[Ss]/, /[Mm][Ii][Nn]/, /[Hh]/, /[Dd]/))),
optional(/_[\d_]+[A-Za-z]+/)
)),
date_literal: $ => token(seq(
choice(
/[Dd][Aa][Tt][Ee]#/,
/[Tt][Oo][Dd]#/,
/[Dd][Tt]#/,
/[Ll][Tt][Ii][Mm][Ee]#/,
/[Ss]5[Tt][Ii][Mm][Ee]#/,
/[Ll][Dd][Tt]#/,
/[Ll][Tt][Oo][Dd]#/,
/[Dd][Tt][Ll]#/
),
/[\d:_\-]+/
)),
bool_literal: $ => token(seq(
/[Bb][Oo][Oo][Ll]#/,
choice('TRUE', 'FALSE', 'true', 'false', 'True', 'False')
)),
typed_int_literal: $ => token(seq(
choice(
/[Uu]?[Ss][Ii][Nn][Tt]#/,
/[Uu]?[Ii][Nn][Tt]#/,
/[Uu]?[Dd][Ii][Nn][Tt]#/,
/[Uu]?[Ll][Ii][Nn][Tt]#/,
/L#/,
/C#/,
/LW#/
),
/[\dA-Fa-f_#]+/
)),
string: $ => choice(
seq("'", repeat(choice(/[^'\n]/, "''")), "'"),
seq('"', repeat(choice(/[^"\n]/, '""')), '"')
),
string_type: $ => seq(
choice(alias('STRING', $.type_builtin), alias('String', $.type_builtin), alias('string', $.type_builtin), alias('WSTRING', $.type_builtin), alias('WString', $.type_builtin), alias('wstring', $.type_builtin)),
optional(seq('[', $.number, ']'))
),
wstring_literal: $ => token(seq(
choice(
/[Ww][Ss][Tt][Rr][Ii][Nn][Gg]#/,
/[Ss][Tt][Rr][Ii][Nn][Gg]#/
),
choice(
seq("'", repeat(choice(/[^'\n]/, "''")), "'"),
seq('"', repeat(choice(/[^"\n]/, '""')), '"')
)
)),
char_literal: $ => choice(
seq("'", /[^']/, "'"),
seq('WCHAR#', "'", /[^']/, "'")
seq(/[Ww][Cc][Hh][Aa][Rr]#/, "'", /[^']/, "'"),
seq('C#', "'", /[^']/, "'")
),
identifier: $ => /[a-zA-Z_][a-zA-Z0-9_]*/,
@@ -481,9 +664,12 @@ module.exports = grammar({
'*/)'
)),
exit_statement: $ => alias('EXIT', $.keyword),
exit_statement: $ => alias(ci('EXIT'), $.keyword),
continue_statement: $ => alias('CONTINUE', $.keyword)
continue_statement: $ => alias(ci('CONTINUE'), $.keyword),
// UTF-8 BOM (Byte Order Mark) - consumed at start of file
bom: $ => token('')
}
});
src/grammar.json
+2757 -1168
View File
File diff suppressed because it is too large Load Diff
src/node-types.json
+476 -18
View File
@@ -18,6 +18,37 @@
]
}
},
{
"type": "array_type",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "identifier",
"named": true
},
{
"type": "keyword",
"named": true
},
{
"type": "number",
"named": true
},
{
"type": "string",
"named": true
},
{
"type": "type",
"named": true
}
]
}
},
{
"type": "assignment",
"named": true,
@@ -36,6 +67,10 @@
"multiple": false,
"required": true,
"types": [
{
"type": "chained_assignment",
"named": true
},
{
"type": "expression",
"named": true
@@ -120,6 +155,25 @@
]
}
},
{
"type": "bit_access",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "lvalue",
"named": true
},
{
"type": "number",
"named": true
}
]
}
},
{
"type": "case_item",
"named": true,
@@ -147,6 +201,10 @@
{
"type": "statement",
"named": true
},
{
"type": "string",
"named": true
}
]
}
@@ -178,6 +236,34 @@
]
}
},
{
"type": "chained_assignment",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "chained_assignment",
"named": true
},
{
"type": "expression",
"named": true
},
{
"type": "lvalue",
"named": true
}
]
}
},
{
"type": "char_literal",
"named": true,
"fields": {}
},
{
"type": "continue_statement",
"named": true,
@@ -197,6 +283,16 @@
"type": "data_block",
"named": true,
"fields": {
"associated": {
"multiple": false,
"required": false,
"types": [
{
"type": "string",
"named": true
}
]
},
"name": {
"multiple": false,
"required": true,
@@ -221,7 +317,7 @@
"named": true
},
{
"type": "identifier",
"type": "author_value",
"named": true
},
{
@@ -240,6 +336,14 @@
"type": "string",
"named": true
},
{
"type": "title_statement",
"named": true
},
{
"type": "var_db_specific_declaration",
"named": true
},
{
"type": "var_declaration",
"named": true
@@ -325,6 +429,26 @@
"type": "binary_expression",
"named": true
},
{
"type": "binary_number",
"named": true
},
{
"type": "bit_access",
"named": true
},
{
"type": "bool_literal",
"named": true
},
{
"type": "char_literal",
"named": true
},
{
"type": "date_literal",
"named": true
},
{
"type": "expression",
"named": true
@@ -349,6 +473,10 @@
"type": "number",
"named": true
},
{
"type": "octal_number",
"named": true
},
{
"type": "prefixed_identifier",
"named": true
@@ -361,9 +489,17 @@
"type": "time_value",
"named": true
},
{
"type": "typed_int_literal",
"named": true
},
{
"type": "unary_expression",
"named": true
},
{
"type": "wstring_literal",
"named": true
}
]
}
@@ -482,6 +618,10 @@
"type": "keyword",
"named": true
},
{
"type": "prefixed_identifier",
"named": true
},
{
"type": "statement",
"named": true
@@ -513,11 +653,11 @@
"named": true
},
{
"type": "block_comment",
"type": "author_value",
"named": true
},
{
"type": "identifier",
"type": "block_comment",
"named": true
},
{
@@ -536,6 +676,10 @@
"type": "string",
"named": true
},
{
"type": "title_statement",
"named": true
},
{
"type": "type",
"named": true
@@ -587,7 +731,11 @@
"named": true
},
{
"type": "identifier",
"type": "author_value",
"named": true
},
{
"type": "block_comment",
"named": true
},
{
@@ -606,9 +754,33 @@
"type": "string",
"named": true
},
{
"type": "title_statement",
"named": true
},
{
"type": "var_constant_declaration",
"named": true
},
{
"type": "var_db_specific_declaration",
"named": true
},
{
"type": "var_declaration",
"named": true
},
{
"type": "var_non_retain_declaration",
"named": true
},
{
"type": "var_retain_declaration",
"named": true
},
{
"type": "var_temp_declaration",
"named": true
}
]
}
@@ -625,6 +797,10 @@
"type": "identifier",
"named": true
},
{
"type": "method_callee",
"named": true
},
{
"type": "parameter_list",
"named": true
@@ -651,6 +827,25 @@
]
}
},
{
"type": "goto_statement",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "identifier",
"named": true
},
{
"type": "keyword",
"named": true
}
]
}
},
{
"type": "if_statement",
"named": true,
@@ -694,6 +889,10 @@
"type": "array_access",
"named": true
},
{
"type": "bit_access",
"named": true
},
{
"type": "field_access",
"named": true
@@ -713,6 +912,25 @@
]
}
},
{
"type": "method_callee",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "identifier",
"named": true
},
{
"type": "lvalue",
"named": true
}
]
}
},
{
"type": "organization_block",
"named": true,
@@ -737,7 +955,7 @@
"named": true
},
{
"type": "identifier",
"type": "author_value",
"named": true
},
{
@@ -756,6 +974,10 @@
"type": "string",
"named": true
},
{
"type": "title_statement",
"named": true
},
{
"type": "var_declaration",
"named": true
@@ -766,9 +988,20 @@
{
"type": "parameter",
"named": true,
"fields": {},
"fields": {
"name": {
"multiple": false,
"required": true,
"types": [
{
"type": "identifier",
"named": true
}
]
}
},
"children": {
"multiple": true,
"multiple": false,
"required": true,
"types": [
{
@@ -832,19 +1065,19 @@
}
},
{
"type": "region_name",
"type": "ref_type",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": false,
"required": true,
"types": [
{
"type": "identifier",
"type": "keyword",
"named": true
},
{
"type": "string",
"type": "type",
"named": true
}
]
@@ -896,6 +1129,21 @@
]
}
},
{
"type": "return_statement",
"named": true,
"fields": {},
"children": {
"multiple": false,
"required": true,
"types": [
{
"type": "keyword",
"named": true
}
]
}
},
{
"type": "source_file",
"named": true,
@@ -905,6 +1153,10 @@
"multiple": true,
"required": false,
"types": [
{
"type": "bom",
"named": true
},
{
"type": "data_block",
"named": true
@@ -972,6 +1224,10 @@
"type": "function_call",
"named": true
},
{
"type": "goto_statement",
"named": true
},
{
"type": "if_statement",
"named": true
@@ -984,10 +1240,18 @@
"type": "repeat_statement",
"named": true
},
{
"type": "return_statement",
"named": true
},
{
"type": "var_constant_declaration",
"named": true
},
{
"type": "var_db_specific_declaration",
"named": true
},
{
"type": "var_declaration",
"named": true
@@ -1017,7 +1281,7 @@
"fields": {}
},
{
"type": "type",
"type": "string_type",
"named": true,
"fields": {},
"children": {
@@ -1025,7 +1289,56 @@
"required": true,
"types": [
{
"type": "identifier",
"type": "number",
"named": true
},
{
"type": "type_builtin",
"named": true
}
]
}
},
{
"type": "struct_initializer",
"named": true,
"fields": {
"name": {
"multiple": true,
"required": false,
"types": [
{
"type": "identifier",
"named": true
}
]
}
},
"children": {
"multiple": true,
"required": false,
"types": [
{
"type": "expression",
"named": true
},
{
"type": "struct_initializer",
"named": true
}
]
}
},
{
"type": "struct_type",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "block_comment",
"named": true
},
{
@@ -1033,7 +1346,30 @@
"named": true
},
{
"type": "number",
"type": "var_item",
"named": true
}
]
}
},
{
"type": "type",
"named": true,
"fields": {},
"children": {
"multiple": false,
"required": true,
"types": [
{
"type": "array_type",
"named": true
},
{
"type": "identifier",
"named": true
},
{
"type": "ref_type",
"named": true
},
{
@@ -1041,9 +1377,28 @@
"named": true
},
{
"type": "type",
"type": "string_type",
"named": true
},
{
"type": "struct_type",
"named": true
},
{
"type": "type_builtin",
"named": true
}
]
}
},
{
"type": "type_builtin",
"named": true,
"fields": {},
"children": {
"multiple": false,
"required": false,
"types": [
{
"type": "type_builtin",
"named": true
@@ -1079,11 +1434,11 @@
"named": true
},
{
"type": "block_comment",
"type": "author_value",
"named": true
},
{
"type": "identifier",
"type": "block_comment",
"named": true
},
{
@@ -1098,6 +1453,10 @@
"type": "string",
"named": true
},
{
"type": "title_statement",
"named": true
},
{
"type": "var_item",
"named": true
@@ -1147,6 +1506,29 @@
]
}
},
{
"type": "var_db_specific_declaration",
"named": true,
"fields": {},
"children": {
"multiple": true,
"required": true,
"types": [
{
"type": "block_comment",
"named": true
},
{
"type": "keyword",
"named": true
},
{
"type": "var_item",
"named": true
}
]
}
},
{
"type": "var_declaration",
"named": true,
@@ -1206,6 +1588,10 @@
{
"type": "expression",
"named": true
},
{
"type": "struct_initializer",
"named": true
}
]
}
@@ -1310,6 +1696,14 @@
"type": "\"\"",
"named": false
},
{
"type": "%",
"named": false
},
{
"type": "&=",
"named": false
},
{
"type": "'",
"named": false
@@ -1330,10 +1724,18 @@
"type": "*",
"named": false
},
{
"type": "*=",
"named": false
},
{
"type": "+",
"named": false
},
{
"type": "+=",
"named": false
},
{
"type": ",",
"named": false
@@ -1342,6 +1744,10 @@
"type": "-",
"named": false
},
{
"type": "-=",
"named": false
},
{
"type": ".",
"named": false
@@ -1354,6 +1760,10 @@
"type": "/",
"named": false
},
{
"type": "/=",
"named": false
},
{
"type": ":",
"named": false
@@ -1394,6 +1804,10 @@
"type": ">=",
"named": false
},
{
"type": "C#",
"named": false
},
{
"type": "[",
"named": false
@@ -1402,11 +1816,31 @@
"type": "]",
"named": false
},
{
"type": "^=",
"named": false
},
{
"type": "author_value",
"named": true
},
{
"type": "binary_number",
"named": true
},
{
"type": "block_comment",
"named": true,
"extra": true
},
{
"type": "bom",
"named": true
},
{
"type": "bool_literal",
"named": true
},
{
"type": "c_style_comment",
"named": true,
@@ -1416,6 +1850,10 @@
"type": "constant",
"named": true
},
{
"type": "date_literal",
"named": true
},
{
"type": "hex_number",
"named": true
@@ -1437,6 +1875,10 @@
"type": "number",
"named": true
},
{
"type": "octal_number",
"named": true
},
{
"type": "operator",
"named": true
@@ -1445,18 +1887,34 @@
"type": "prefix",
"named": true
},
{
"type": "region_name",
"named": true
},
{
"type": "time_value",
"named": true
},
{
"type": "type_builtin",
"type": "title_statement",
"named": true
},
{
"type": "typed_int_literal",
"named": true
},
{
"type": "wstring_literal",
"named": true
},
{
"type": "{",
"named": false
},
{
"type": "|=",
"named": false
},
{
"type": "}",
"named": false
src/parser.c
+102187 -74102
View File
File diff suppressed because it is too large Load Diff
src/tree_sitter/array.h
+20 -37
View File
@@ -60,7 +60,13 @@ extern "C" {
/// Free any memory allocated for this array. Note that this does not free any
/// memory allocated for the array's contents.
#define array_delete(self) _array__delete((self), (void *)(self)->contents, sizeof(*self))
#define array_delete(self) \
do { \
if ((self)->contents) ts_free((self)->contents); \
(self)->contents = NULL; \
(self)->size = 0; \
(self)->capacity = 0; \
} while (0)
/// Push a new `element` onto the end of the array.
#define array_push(self, element) \
@@ -130,12 +136,11 @@ extern "C" {
/// Swap one array with another
#define array_swap(self, other) \
do { \
struct Swap swapped_contents = _array__swap( \
(void *)(self)->contents, &(self)->size, &(self)->capacity, \
(void *)(other)->contents, &(other)->size, &(other)->capacity \
); \
(self)->contents = swapped_contents.self_contents; \
(other)->contents = swapped_contents.other_contents; \
void *_array_swap_tmp = (void *)(self)->contents; \
(self)->contents = (other)->contents; \
(other)->contents = _array_swap_tmp; \
_array__swap(&(self)->size, &(self)->capacity, \
&(other)->size, &(other)->capacity); \
} while (0)
/// Get the size of the array contents
@@ -188,12 +193,6 @@ extern "C" {
// The `Array` type itself was not altered as a solution in order to avoid breakage
// with existing consumers (in particular, parsers with external scanners).
/// This is not what you're looking for, see `array_delete`.
static inline void _array__delete(void *self, void *contents, size_t self_size) {
if (contents) ts_free(contents);
if (self) memset(self, 0, self_size);
}
/// This is not what you're looking for, see `array_erase`.
static inline void _array__erase(void* self_contents, uint32_t *size,
size_t element_size, uint32_t index) {
@@ -228,31 +227,15 @@ static inline void *_array__assign(void* self_contents, uint32_t *self_size, uin
return new_contents;
}
struct Swap {
void *self_contents;
void *other_contents;
};
/// This is not what you're looking for, see `array_swap`.
// static inline void _array__swap(Array *self, Array *other) {
static inline struct Swap _array__swap(void *self_contents, uint32_t *self_size, uint32_t *self_capacity,
void *other_contents, uint32_t *other_size, uint32_t *other_capacity) {
void *new_self_contents = other_contents;
uint32_t new_self_size = *other_size;
uint32_t new_self_capacity = *other_capacity;
void *new_other_contents = self_contents;
*other_size = *self_size;
*other_capacity = *self_capacity;
*self_size = new_self_size;
*self_capacity = new_self_capacity;
struct Swap out = {
.self_contents = new_self_contents,
.other_contents = new_other_contents,
};
return out;
static inline void _array__swap(uint32_t *self_size, uint32_t *self_capacity,
uint32_t *other_size, uint32_t *other_capacity) {
uint32_t tmp_size = *self_size;
uint32_t tmp_capacity = *self_capacity;
*self_size = *other_size;
*self_capacity = *other_capacity;
*other_size = tmp_size;
*other_capacity = tmp_capacity;
}
/// This is not what you're looking for, see `array_push` or `array_grow_by`.