Tutorial 07: Data Structures (Vector & HashTable)

0
#
24
27.12.25 16:22

Tutorial 07: Data Structures (Vector & HashTable)

Overview

In this tutorial, we explore two fundamental data structures provided by FreshLib: Vectors (dynamic arrays) and HashTables (static Pearsons hash). These structures are essential for managing dynamic content and performing fast lookups in assembly applications.

Topics Covered

  1. Dynamic Arrays (Vectors) - Using the TArray structure.

  2. Array Lifecycle - Creation, allocation, and cleanup.

  3. Element Management - Adding, inserting, and deleting items.

  4. Static HashTables - Building and searching using PHashTable.

  5. Pearson's Hash - Understanding the case-insensitive hash used in FreshLib.

Dynamic Arrays (TArray)

FreshLib's TArray is a dynamic array (vector) that grows as needed. It stores a count of elements, current capacity, and item size.

struct TArray
  .count     dd ?           ; Count of elements in dynamic array
  .capacity  dd ?           ; Capacity of the array allocated memory.
  .itemsize  dd ?           ; Size of one element in dwords.
  .lparam    dd ?           ; User defined value.
  label .array dword        ; Start of actual data
ends

Essential TArray Functions

Function Purpose
CreateArray Creates a new array with specified item size.
AddArrayItems Adds one or more items to the end of the array.
InsertArrayItems Inserts items at a specific index.
DeleteArrayItems Removes items from the array.
GetArrayItem Retrieves a pointer to an item at an index.
FreeMem Used to free the array memory (as CreateArray uses heap).

NOTE

TArray elements are always dword-aligned. The itemsize argument is in bytes, but internal storage is dword-based.

HashTables (PHashTable)

FreshLib uses a static Pearson's hash table for fast keyword lookups. This pattern is extensively used in AsmBB for API command routing and template tag processing.

The PHashTable Macro

The PHashTable macro builds a static table of 256 items. It requires a Pearson's hash function table (tpl_func) to operate.

PHashTable myTable, tpl_func, \
    "command1", Proc1, \
    "command2", Proc2

Searching the Table

Searching is typically performed by a helper function like SearchInHashTable. It computes the hash of a string and lookups the corresponding entry in the table.

stdcall SearchInHashTable, hCommand, tableAPICommands
; Returns proc address in ECX if found, CF=1

Demo Programs

Demo 20: Vector Basics

File: demo20_vector_basics.asm

Demonstrates creating a dynamic array of string handles, adding items, and iterating through them. This pattern is used in AsmBB for tracking open SQLite statements.

Demo 21: HashTable Basics

File: demo21_hashtable_basics.asm

Demonstrates building a static lookup table for commands and searching it using a Pearson's hash. This is the exact pattern used for AsmBB's FastCGI command routing.

Critical Implementation Notes

CAUTION

Register Preservation: FreshLib procedures do NOT preserve EAX, ECX, EDX, or ESI. Any pointer returned by a procedure (e.g., StrPtr, GetArrayItem) must be saved before calling another procedure.

Pattern: Preserving Pointers Across Calls

; ❌ WRONG - EAX is clobbered after FileWriteString
stdcall StrPtr, ebx
stdcall FileWriteString, 1, msg_dash
stdcall FileWriteString, 1, eax     ; SEGFAULT! eax was destroyed

; ✅ CORRECT - Push/pop to preserve EAX
stdcall StrPtr, ebx
push    eax                         ; Save pointer
stdcall FileWriteString, 1, msg_dash
pop     eax                         ; Restore pointer
stdcall FileWriteString, 1, eax

Pattern: AddArrayItems Returns New Handle

When adding items, the array may be reallocated. Always update your handle from EDX:

stdcall StrDupMem, s_item1          ; EAX = string handle
push    eax
stdcall AddArrayItems, ebx, 1
mov     ebx, edx                    ; EDX = potentially new array handle
pop     dword [eax]                 ; EAX = pointer to new element

Building and Running

cd 07-data-structures
./build.sh

Reference

  • FreshLib Source: freshlib/data/arrays.asm

  • AsmBB Examples: asmbb/source/api.asm, asmbb/source/commands.asm

23
27.12.25 16:22
; _______________________________________________________________________________________
;|                                                                                       |
;| ..:: FreshLib Tutorials ::..                                                          |
;|_______________________________________________________________________________________|
;
;  Description: Tutorial 07, Demo 20: Vector (TArray) Basics.
;_________________________________________________________________________________________

include "%lib%/freshlib.inc"

LINUX_INTERPRETER equ './ld-musl-i386.so'

@BinaryType console, compact
LIB_MODE equ NOGUI

options.DebugMode = 0

include "%lib%/freshlib.asm"

; ========================================
; Initialized Data
; ========================================
iglobal
  msg_start  text "--- Demo 20: Vector Basics ---", 13, 10
  msg_added  text "Added items to the vector.", 13, 10
  msg_curr   text "Current items:", 13, 10
  msg_dash   text " - "
  msg_crlf   text 13, 10
  msg_done   text "Cleanup complete.", 13, 10

  s_item1    text "Item 1: FreshLib"
  s_item2    text "Item 2: Assembly"
  s_item3    text "Item 3: Performance"
endg

; ========================================
; Entry Point
; ========================================
start:
        InitializeAll
        stdcall FileWriteString, [STDOUT], msg_start

; 1. Create a dynamic array of dwords (string handles)
        stdcall CreateArray, 4
        jc      .finish
        mov     ebx, eax                ; EBX = Array handle

; 2. Add items
        stdcall StrDupMem, s_item1
        push    eax
        stdcall AddArrayItems, ebx, 1
        mov     ebx, edx
        pop     dword [eax]

        stdcall StrDupMem, s_item2
        push    eax
        stdcall AddArrayItems, ebx, 1
        mov     ebx, edx
        pop     dword [eax]

        stdcall StrDupMem, s_item3
        push    eax
        stdcall AddArrayItems, ebx, 1
        mov     ebx, edx
        pop     dword [eax]

        stdcall FileWriteString, [STDOUT], msg_added
        stdcall FileWriteString, [STDOUT], msg_curr

; 3. Iterate
        xor     esi, esi
.loop_print:
        cmp     esi, [ebx+TArray.count]
        jae     .loop_done

        stdcall GetArrayItem, ebx, esi
        mov     eax, [eax]              ; EAX = string handle
        stdcall StrPtr, eax             ; EAX = pointer to actual string

        push    eax                     ; PRESERVE THE POINTER
        stdcall FileWriteString, [STDOUT], msg_dash
        pop     eax                     ; RESTORE THE POINTER

        stdcall FileWriteString, [STDOUT], eax
        stdcall FileWriteString, [STDOUT], msg_crlf

        inc     esi
        jmp     .loop_print

.loop_done:

; 4. Cleanup
        xor     esi, esi
.loop_cleanup:
        cmp     esi, [ebx+TArray.count]
        jae     .cleanup_done
        stdcall GetArrayItem, ebx, esi
        stdcall StrDel, [eax]
        inc     esi
        jmp     .loop_cleanup

.cleanup_done:
        stdcall FreeMem, ebx
        stdcall FileWriteString, [STDOUT], msg_done
        stdcall FileWriteString, [STDOUT], msg_crlf

.finish:
        FinalizeAll
        stdcall TerminateAll, 0
22
27.12.25 16:22
; _______________________________________________________________________________________
;|                                                                                       |
;| ..:: FreshLib Tutorials ::..                                                          |
;|_______________________________________________________________________________________|
;
;  Description: Tutorial 07, Demo 21: HashTable Basics using PHashTable macro.
;_________________________________________________________________________________________

include "%lib%/freshlib.inc"

LINUX_INTERPRETER equ './ld-musl-i386.so'

@BinaryType console, compact
LIB_MODE equ NOGUI

options.DebugMode = 0

include "%lib%/freshlib.asm"

; ========================================
; Initialized Data
; ========================================
iglobal
  msg_start    text "--- Demo 21: HashTable Basics ---", 13, 10
  msg_explain  text "Using PHashTable for compile-time hash tables.", 13, 10, 13, 10
  msg_test1    text "Lookup 'GET':  "
  msg_test2    text "Lookup 'HELP': "
  msg_found    text "FOUND (Value: "
  msg_nf       text "NOT FOUND", 13, 10
  msg_crlf     text ")", 13, 10
  msg_complete text 13, 10, "Demo complete!", 13, 10
endg

; ========================================
; Entry Point
; ========================================
start:
        InitializeAll
        stdcall FileWriteString, [STDOUT], msg_start

; Demonstrate how the PHashTable macro creates a lookup table
        stdcall FileWriteString, [STDOUT], msg_explain

; Compute hash for "GET" manually (same as PHashTable does)
; The hash is computed by iterating through the string and XORing with the Pearson table
; For demonstration, we'll directly show how to look up a known key

        stdcall FileWriteString, [STDOUT], msg_test1

; Hash of "GET" (lowercase "get") is computed at compile time by PHashTable
; We can verify by looking up the value
        mov     eax, htGET                      ; htGET is a constant with the hash value
        mov     ecx, [tableCommands + 8*eax + 4] ; Get value from table
        test    ecx, ecx
        jz      .nf1

        push    ecx
        stdcall FileWriteString, [STDOUT], msg_found
        pop     ecx

        stdcall NumToStr, ecx, ntsDec or ntsUnsigned
        push    eax
        stdcall StrPtr, eax
        push    eax
        stdcall FileWriteString, [STDOUT], eax
        pop     eax
        pop     eax
        stdcall StrDel, eax
        stdcall FileWriteString, [STDOUT], msg_crlf
        jmp     .test2

.nf1:
        stdcall FileWriteString, [STDOUT], msg_nf

.test2:
        stdcall FileWriteString, [STDOUT], msg_test2
        mov     eax, htHELP
        mov     ecx, [tableCommands + 8*eax + 4]
        test    ecx, ecx
        jz      .nf2

        push    ecx
        stdcall FileWriteString, [STDOUT], msg_found
        pop     ecx

        stdcall NumToStr, ecx, ntsDec or ntsUnsigned
        push    eax
        stdcall StrPtr, eax
        push    eax
        stdcall FileWriteString, [STDOUT], eax
        pop     eax
        pop     eax
        stdcall StrDel, eax
        stdcall FileWriteString, [STDOUT], msg_crlf
        jmp     .done

.nf2:
        stdcall FileWriteString, [STDOUT], msg_nf

.done:
        stdcall FileWriteString, [STDOUT], msg_complete

.finish:
        FinalizeAll
        stdcall TerminateAll, 0

; ========================================
; Pearson's hash table (256 bytes)
; ========================================
tpl_func:
          db 148,  24,  71,  46, 179,   0, 106, 157,  70,   1, 102, 126, 120, 134, 151, 183
          db  47, 103,  92, 201,  62, 156,  13,  10, 254, 218, 248,  28,  85, 185, 245, 112
          db 236, 237, 150,  37, 172,  63, 203, 198, 116, 196,  25, 107, 239,  44,  15, 175
          db  38, 161, 140,  98,  33,  79,  99, 133,   2,  59, 174, 115,  69, 188,  51,  36
          db 229, 143, 231,  84,  22,  78,  89, 166, 104, 145,  34, 225, 180,  12, 230, 205
          db  97,  39,  49, 190, 182, 202,  17, 219, 176, 170,  80,  74,  73, 194,  41,  26
          db 243, 142,  60, 131, 244, 249, 119,  61, 138,  16,  77,  54, 210,  23,  29, 147
          db 204, 110, 130,  93, 213, 223, 146, 216, 123, 247,  90, 124,  75, 135,  57, 128
          db 226,  50, 224, 127, 139, 152, 193, 101, 207, 122,   9,  58, 184, 250,  96,  67
          db 241, 109,  87, 168, 187, 167, 240, 171, 233, 155, 173, 215,  43, 227, 160,   5
          db 246, 129,  20,  14, 100, 209, 137, 169,  68,  40,  42, 165, 121, 113, 200,  95
          db 189, 251,  82,  72, 154, 235, 195, 136, 186,  55, 212,  35,  32, 253,  66, 132
          db 214,  88, 192,   4,  31,  65, 114, 211,  19,  83,  21,   3,   6, 177,  76, 255
          db 252, 199, 141,  56, 105,  11, 117, 163, 220,  27, 222, 234, 178,  52,  64, 221
          db  18, 232, 164,  53, 206, 153, 118,  48, 158, 162, 159, 191, 242, 125, 144,  94
          db 108,  91, 197, 111,  30,   7,  81, 181, 149,  45, 217, 208,  86, 238,   8, 228

; ========================================
; Compile-time hash constants using the phash struc
; ========================================
htGET   phash tpl_func, "get"
htHELP  phash tpl_func, "help"

; The PHashTable macro creates a 256-slot table with key/value pairs
PHashTable tableCommands, tpl_func, \
          "help", 101, \
          "get",  102, \
          "exit", 103

Tutorial 07: Data Structures (Vector & HashTable)

0
#