# mojolint.tokenize

> A module of lint's API.

Logical lines: Mojo source with comments removed, string contents blanked, and bracketed continuations joined.

This is all the "parsing" `mojolint` does. A rule sees a `Line` — one
statement or header, its first physical line number, its indent, the code
text, and the identifier words in it — and matches on text. No AST, no
types, no dataflow. That ceiling is deliberate; see the README.

## Structs

### `Line`

```mojo
struct Line
```

One logical line of code.

Implements: `Copyable`, `Movable`

Fields:

- `lineno: Int`: 1-based number of the first physical line.
- `indent: Int`: Leading spaces of the first physical line.
- `code: String`: The text, comments stripped, every string literal reduced to `""`.
- `words: List[String]`: Identifier tokens of `code`, in order.
- `allow: List[String]`: Rule ids named in a `# lint: allow(...)` comment on or just above it.
- `starts: List[Int]`: Offset in `code` where each physical line begins; `starts[0]` is 0.
- `indents: List[Int]`: Leading spaces of each physical line, parallel to `starts`.

#### `locate`

```mojo
def locate(self, at: Int) -> Tuple[Int, Int]
```

The 1-based physical line and 1-based column of offset `at` in `code`, so a finding on a bracketed continuation points into the physical line it sits on.

#### `has_word`

```mojo
def has_word(self, word: String) -> Bool
```

#### `allows`

```mojo
def allows(self, rule: String) -> Bool
```

## Functions

### `handed_whole`

```mojo
def handed_whole(code: String, name: String) -> Bool
```

Does `code` pass the local `name` as a whole value somewhere?

True when an occurrence of `name` is followed by `)` or `,` (an argument)
or `.` (a method call on it). False when it is only read *through*: as
the base of `name[...]`, or not present at all. `Ctx.to(totals)` hands
`totals`; `Int(cells[unsafe_offset=2])` merely reads a cell of `cells`.

### `words_of`

```mojo
def words_of(code: String) -> List[String]
```

The identifier tokens of `code`, in order.

Numbers are skipped, so `1_000` never becomes a word; string contents are
already gone by the time this runs.

### `logical_lines`

```mojo
def logical_lines(source: String) -> List[Line]
```

Split `source` into logical lines.

Comments are removed first, then string literals are blanked, then
physical lines are joined while brackets are open. Blank lines are
dropped. A `# lint: allow(...)` comment on a line applies to that line;
on a line of its own it applies to the next code line.