Token Struct Definition

Your Task

Implement a complete Token structure and supporting types for a C compiler's lexer.

Requirements

1. Define a TokenType enum with at least these categories:

   • Keywords: TOKEN_INT, TOKEN_RETURN, TOKEN_IF, TOKEN_ELSE, TOKEN_WHILE
   • Operators: TOKEN_PLUS, TOKEN_MINUS, TOKEN_STAR, TOKEN_SLASH, TOKEN_ASSIGN
   • Comparisons: TOKEN_EQ, TOKEN_NE, TOKEN_LT, TOKEN_GT, TOKEN_LE, TOKEN_GE
   • Delimiters: TOKEN_LPAREN, TOKEN_RPAREN, TOKEN_LBRACE, TOKEN_RBRACE, TOKEN_SEMICOLON, TOKEN_COMMA
   • Literals: TOKEN_NUMBER, TOKEN_IDENTIFIER, TOKEN_STRING
   • Special: TOKEN_EOF, TOKEN_ERROR

2. Define a Token struct containing:

   • Token type
   • Lexeme (original source text)
   • Line and column numbers
   • Literal value (use a union for different types)

3. Implement helper functions:

   • token_create(): Allocate and initialize a token
   • token_destroy(): Free token memory
   • token_type_to_string(): Return human-readable type name

Success Criteria

• Code compiles with gcc -Wall -Wextra
• Token struct uses appropriate C types
• Memory management is correct (no leaks)