LSTN

LSTN (Liquid Spiking Trigram Network)

LSTN is an experimental text generation engine that models language not as static probabilities, but as a dynamic and “liquid” neural network. Each trigram (3-character sequence) acts as an individual neuron within a continuous temporal dynamic system.

Core Concept

Instead of calculating a classic probability distribution, the model simulates a network where energy (or “voltage”) flows from neuron to neuron. When a neuron “fires”, it emits its final character and propagates its energy to subsequent neurons based on learned synaptic weights.

This architecture allows for the emergence of pseudo-organic text generation, introducing concepts of refractory period, energy decay, and dynamic resuscitation.

Mechanism and Equations

1. Training (Ouroboros Engraving)

The training text is split into sliding windows of 4 characters to connect contiguous trigrams. For a context window $C = [c_1, c_2, c_3, c_4]$, a synaptic link is created between the source neuron $N_{src} = (c_1, c_2, c_3)$ and the target neuron $N_{tgt} = (c_2, c_3, c_4)$.

The synaptic weight $W_{src \to tgt}$ is incremented by the learning rate $\eta$ (LEARNING_RATE). After compiling the corpus, each neuron’s weights are normalized locally: \(W_{src \to tgt} = W_{src \to tgt} \times \frac{2.0}{\max(W_{src})}\)

2. Generation Dynamics (Infinite Dream)

At each time step of generation, the network evolves:

• Organic Selection: Neurons whose voltage exceeds a threshold (0.1) and which are not in a refractory period are considered candidates. The energy of a candidate neuron is cubed to emphasize activation contrasts: \(E_i = V_i^3\) The winner is then randomly drawn proportionally to this energy $E_i$.

• Discharge and Propagation: If the central neuron exceeds the FIRE_THRESHOLD ($\ge 1.0$), it fires:
  1. Emission: It emits its final character.
  2. “Sniper silence”: The voltage of all other neurons in the system is set to 0.
  3. Transfer: It transfers its energy to its direct targets: $V_{tgt} = V_{tgt} + W_{src \to tgt} \times 2.5$.
  4. Rest: Its own potential drops to 0, and it enters a refractory period (REFRACTORY_TIME = 2), preventing immediate reactivation.

• Decay: At the end of the cycle, all active neurons have their potential naturally decrease: \(V_i(t+1) = V_i(t) \times \gamma\) Where $\gamma$ corresponds to the DECAY_RATE (0.92).

• Dynamic Defibrillator: If the network “dies” (no active candidate neuron), an emergency process is triggered. A universal energy pulse ($\Delta V = 0.5$) is injected into all neurons with outgoing connections to restart the flow.

Installation & Usage

This type of model is highly relevant for experiments in emergence (complexity arising from very simple local rules) or uninterrupted organic text flows.

1. Prerequisites (Rust)

Windows:

winget install --id Rustlang.Rustup -e
rustup default stable

Linux / macOS:

curl https://sh.rustup.rs -sSf | sh
source "$HOME/.cargo/env"

2. Execution

From the root of the project, you can launch or compile the model as follows:

Development Mode:

cargo run --bin lstn

Optimized Build (Release):

cargo build --release --bin lstn
# The executable will be available in target/release/

Adjustable Hyperparameters (src/main.rs)

The internal physics are driven by several fundamental constants:

• DECAY_RATE (0.92): Preservation of temporal inertia. A rate close to 1 preserves a broader activation spectrum.
• FIRE_THRESHOLD (1.0): Minimum action potential threshold triggering token emission.
• REFRACTORY_TIME (2): Duration of temporary invalidation of the recently fired neuron (fights local stuttering).
• LEARNING_RATE (1.0): Absolute increment during synaptic reinforcement.