LSTN

LSTN (Liquid Spiking Trigram Network)

An experimental text generation engine in Rust that models language as a spiking neural network. Each trigram (3-character sequence) is a neuron. Instead of computing a probability distribution, the engine simulates energy flowing between neurons: neurons fire, propagate energy to their neighbors based on learned synaptic weights, then enter a refractory period before they can fire again.

---

Core concept

Standard character-level models pick the next character from a static probability table. LSTN treats each trigram as a live node in a network whose state changes at every generation step.

Key dynamics:

• Firing: a neuron whose voltage exceeds FIRE_THRESHOLD emits its output character, propagates energy to connected neurons, and resets to zero.
• Refractory period: after firing, a neuron is silenced for REFRACTORY_TIME steps, preventing immediate re-activation and discouraging repetition.
• Decay: all active neurons lose voltage each step (V(t+1) = V(t) * DECAY_RATE), creating natural temporal forgetting.
• Defibrillator: if no neuron reaches the firing threshold (the network dies), a universal energy pulse is injected into all neurons with outgoing connections to restart generation.

---

Training

The corpus is split into sliding windows of 4 characters. For each window [c1, c2, c3, c4], a synaptic link from (c1,c2,c3) to (c2,c3,c4) is reinforced by LEARNING_RATE. After corpus ingestion, each neuron’s weights are locally normalized.

---

Hyperparameters (src/main.rs)

Constant	Default	Description
DECAY_RATE	0.92	Voltage decay per step (closer to 1 = broader activation)
FIRE_THRESHOLD	1.0	Minimum voltage to emit a character
REFRACTORY_TIME	2	Steps a fired neuron stays silent
LEARNING_RATE	1.0	Synaptic weight increment per training example

---

Building

# Run
cargo run --bin lstn

# Release build
cargo build --release --bin lstn

---

Star History

</a>

---

License

MIT.