Chapter 3

Lexical Structure

Overview

This chapter describes the lexical structure of Scaly: how source text is divided into tokens. The lexer transforms a sequence of characters into a sequence of tokens that the parser then processes.

Whitespace and Line Breaks

Whitespace characters (space, tab) separate tokens but are otherwise insignificant. Line breaks, however, are syntactically meaningful in Scaly — they can terminate statements and declarations.

A colon (:) can be used in place of a line break when multiple statements should appear on a single line.

Comments

Scaly supports two forms of comments:

Single-line comments start with a semicolon (;) and extend to the end of the line.
Multi-line comments start with ;* and end with *;.

; This is a single-line comment

;* This is a
   multi-line comment *;

Literals

Integer Literals

Integer literals represent whole numbers. They can be written in decimal or hexadecimal notation.

42        ; decimal
0x2A      ; hexadecimal (same value)

Floating-Point Literals

Floating-point literals represent decimal numbers with a fractional part or an exponent.

3.14159       ; pi
2.5e10        ; scientific notation
1.0e-5        ; small number

String Literals

String literals are enclosed in double quotes. The supported escape sequences are \", \\, \', \n, \r, \t, \0, and \uXXXX (four hex digits, encoded as UTF-8).

"Hello, World!"
"Line 1\nLine 2"      ; with newline escape
"Tab:\tValue"         ; with tab
"Quote: \"nested\""   ; escaped quotes
"\u0041"            ; Unicode escape - the letter A

Strings can span multiple lines without any special syntax — newlines are simply included as part of the string content:

"This is a string
that spans multiple
lines."

A backtick inside a string embeds an expression: the string is built from its parts, and each embedded value is written in the text form of its type — a String, a char, the bytes of a Slice[u8], an integer, a bool, or a floating-point number as the shortest text that reads back to it. A value of another type needs a conversion inside the backticks, and \` writes a backtick itself.

let n 42
let x 0.1
"The answer is `n`, not `n + 1`, and x is `x`"
                      ; The answer is 42, not 43, and x is 0.1

Character Literals

A character is written as a string literal of one character wherever a char is expected — a char parameter, a typed binding, or the cast as char. Escapes count as one character.

let a: char "a"        ; lowercase a
let nl "\n" as char    ; newline character

Boolean Literals

Boolean literals are true and false.

true
false

Operators and Special Symbols

The Equals Sign

Unlike many programming languages, Scaly reserves the equals sign (=) exclusively for equality comparison. It is not used for variable declarations or assignments.

; Variable declaration (NO equals sign)
var x 42                  ; declares x with value 42
let y: i32 100            ; with type annotation

; Assignment (its own sign)
x := 43                   ; assigns 43 to x

; Equality comparison
if x = 43                 ; tests if x equals 43
    print "x is 43"

This design eliminates the classic confusion between assignment and comparison that leads to bugs in C-style languages (if (x = 42) vs if (x == 42)).

Identifiers

Identifiers name variables, functions, types, and other entities. Scaly supports three forms of identifiers:

Alphanumeric identifiers start with a letter and continue with letters, digits, or underscores.
Operator identifiers consist of operator characters such as +, -, *, /.
String identifiers are enclosed in single quotes and can contain any characters.

myVariable        ; alphanumeric
calculate_total   ; with underscore
++                ; operator identifier
'my identifier'   ; string identifier (with spaces)

The form of an identifier does not by itself decide what it names. A variable, a function, or an operator may carry any of the three forms; an operator-character name such as ++ can name an ordinary function, and a string identifier such as 'divided by' can name an operator. What a name denotes is fixed by its declaration, not by how it is spelled (see the chapter on definitions).