文字コードは次の3種類に限定。
0 1 2 3 4 5 6 7 8
SourceCode: EndOfFile CodeElements EndOfFile CodeElements: WhiteSpace EndOfLine Comment Token
0 1 2 3 4 5 6 7
WhiteSpace: Space Space WhiteSpace Space: 0x0020 0x0009 0x000B 0x000C
0 1 2 3
EndOfLine: 0x000D 0x000A 0x000D 0x000A EndOfFile
0 1 2
EndOfFile: physical end of the file 0x0000 0x001A
1 2 3 4 5 6 7 8 9 10 11
#spanend #spanadd /* ネストできないコメントブロック */ #spanend #spanadd // 行コメント #spanend #spanadd /+ ネストできるコメントブロック /+ +/ +/ #spanend #spanadd
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Comment: "/*" Characters "*/" "//" Characters EndOfLine NestingBlockComment Characters: Character Character Characters NestingBlockComment: "/+" NestingBlockCommentCharacters "+/" NestingBlockCommentCharacters: NestingBlockCommentCharacter NestingBlockCommentCharacter NestingBlockCommentCharacters NestingBlockCommentCharacter: Character NestingBlockComment
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
Token: Identifier StringLiteral CharacterLiteral IntegerLiteral FloatLiteral Keyword "/" "/=" "." "&" "&=" "&&" "|" "|=" "||" "-" "-=" "--" "+" "+=" "++" "<" "<=" "<<" "<<=" ">" ">=" ">>=" ">>" "!" "!=" "(" ")" "[" "]" "{" "}" "?" "," ";" ":" "=" "==" "*" "*=" "%" "%=" "^" "^=" "~" "~="
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Identifier: IdentiferStart IdentiferStart IdentifierChars IdentifierChars: IdentiferChar IdentiferChar IdentifierChars IdentifierStart: EnglishAlpha UniversalAlpha IdentifierChar: IdentiferStart Digit "_" EnglishAlpha: a-zA-Z UniversalAlpha: あいうえお... Digit: 0-9
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
#spanend #spanadd StringLiteral: #spanend #spanadd WysiwygString #spanend #spanadd DoubleQuotedString #spanend #spanadd #spanend #spanadd WysiwygString: #spanend #spanadd r" WysiwygCharacters " Postfix(opt) #spanend #spanadd #spanend #spanadd WysiwygCharacters: #spanend #spanadd WysiwygCharacter #spanend #spanadd WysiwygCharacter WysiwygCharacters #spanend #spanadd #spanend #spanadd WysiwygCharacter: #spanend #spanadd Character #spanend #spanadd EndOfLine #spanend #spanadd #spanend #spanadd DoubleQuotedString: #spanend #spanadd " DoubleQuotedCharacters " Postfix(opt) #spanend #spanadd #spanend #spanadd DoubleQuotedCharacters: #spanend #spanadd DoubleQuotedCharacter #spanend #spanadd DoubleQuotedCharacter DoubleQuotedCharacters #spanend #spanadd #spanend #spanadd DoubleQuotedCharacter: #spanend #spanadd Character #spanend #spanadd EscapeSequence #spanend #spanadd EndOfLine #spanend #spanadd #spanend #spanadd EscapeSequence: #spanend #spanadd \' #spanend #spanadd \" #spanend #spanadd \? #spanend #spanadd \\ #spanend #spanadd \a (0x07) #spanend #spanadd \b (0x08) #spanend #spanadd \f (0x0C) #spanend #spanadd \n (0x0A) #spanend #spanadd \r (0x0D) #spanend #spanadd \t (0x09) #spanend #spanadd \v (0x0B) #spanend #spanadd #spanend #spanadd Postfix: #spanend #spanadd c #spanend #spanadd w #spanend #spanadd d #spanend #spanadd #spanend #spanadd