文字コードは次の3種類に限定。
0 1 2 3 4 5 6 7 8
SourceCode: EndOfFile CodeElements EndOfFile CodeElements: WhiteSpace EndOfLine Comment Token
0 1 2 3 4 5 6 7
WhiteSpace: Space Space WhiteSpace Space: 0x0020 0x0009 0x000B 0x000C
0 1 2 3
EndOfLine: 0x000D 0x000A 0x000D 0x000A EndOfFile
0 1 2
EndOfFile: physical end of the file 0x0000 0x001A
1 2 3
/* ネストできないコメントブロック */ // 行コメント /+ ネストできるコメントブロック /+ +/ +/
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Comment: "/*" Characters "*/" "//" Characters EndOfLine NestingBlockComment Characters: Character Character Characters NestingBlockComment: "/+" NestingBlockCommentCharacters "+/" NestingBlockCommentCharacters: NestingBlockCommentCharacter NestingBlockCommentCharacter NestingBlockCommentCharacters NestingBlockCommentCharacter: Character NestingBlockComment
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
Token: Identifier StringLiteral CharacterLiteral IntegerLiteral FloatLiteral Keyword "/" "/=" "." "&" "&=" "&&" "|" "|=" "||" "-" "-=" "--" "+" "+=" "++" "<" "<=" "<<" "<<=" ">" ">=" ">>=" ">>" "!" "!=" "(" ")" "[" "]" "{" "}" "?" "," ";" ":" "=" "==" "*" "*=" "%" "%=" "^" "^=" "~"
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Identifier: IdentiferStart IdentiferStart IdentifierChars IdentifierChars: IdentiferChar IdentiferChar IdentifierChars IdentifierStart: EnglishAlpha UniversalAlpha IdentifierChar: IdentiferStart Digit "_" EnglishAlpha: a-zA-Z UniversalAlpha: あいうえお... Digit: 0-9
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
StringLiteral: WysiwygString DoubleQuotedString WysiwygString: r" WysiwygCharacters " Postfix(opt) WysiwygCharacters: WysiwygCharacter WysiwygCharacter WysiwygCharacters WysiwygCharacter: Character EndOfLine DoubleQuotedString: " DoubleQuotedCharacters " Postfix(opt) DoubleQuotedCharacters: DoubleQuotedCharacter DoubleQuotedCharacter DoubleQuotedCharacters DoubleQuotedCharacter: Character EscapeSequence EndOfLine EscapeSequence: \' \" \? \\ \a (0x07) \b (0x08) \f (0x0C) \n (0x0A) \r (0x0D) \t (0x09) \v (0x0B) Postfix: c w d