GoAI - Neural Network Library

A lightweight, pure Go implementation of a feedforward neural network with backpropagation training. Perfect for learning neural network fundamentals or building small machine learning models directly in Go.

Features

• Modular Architecture: Clean separation between neurons, layers, and networks
• Backpropagation Training: Implements standard backpropagation algorithm for model training
• Sigmoid Activation: Uses sigmoid activation function for non-linearity
• Flexible Network Configuration: Create networks with arbitrary layer sizes
• Learning Rate Support: Adjustable learning rates for training control
• Delta-based Learning: Support for both direct corrections and delta propagation
• Binary Serialize/Deserialize: Persist network weights and biases to an io.Writer/io.Reader

API reference

Package network

• func NewNetwork(layerSizes []int) *Network

  • Create a new feedforward Network. layerSizes includes the input layer size and one entry per subsequent layer.

• func (n *Network) Randomize(rng float64)

  • Initialize all network weights and biases to values uniformly drawn from [-rng, rng].

• func (n *Network) Activate(inputs []float64) []float64

  • Run a forward pass through the network and return the output slice.

• func (n *Network) Correct(inputs []float64, expected []float64, learningRate float64) []float64

  • Perform a forward pass followed by backpropagation to update weights and biases. Returns the network output after the update.

• func (n *Network) Serialize(w io.Writer) error

  • Write the network weights and biases in big-endian binary form to w.

• func (n *Network) Deserialize(r io.Reader) error

  • Read the network weights and biases from r (big-endian binary) and restore the network state.

Package layer

• func NewLayer(neuronCount int, inputCount int) *NeuronLayer

  • Create a layer with neuronCount neurons, each expecting inputCount inputs.

• func (l *NeuronLayer) Randomize(rng float64)

  • Randomize all weights and biases in the layer.

• func (l *NeuronLayer) Activate(inputs []float64) []float64

  • Compute the outputs for every neuron in the layer given inputs.

• func (l *NeuronLayer) Correct(inputs []float64, outputs []float64, expected []float64, learningRate float64) []float64

  • Used for output layers: compute per-neuron deltas, apply weight updates, and return propagated errors for the previous layer.

• func (l *NeuronLayer) CorrectByErrors(inputs []float64, outputs []float64, errors []float64, learningRate float64) []float64

  • Used for hidden layers: accept a slice of propagated errors (one per neuron in this layer), update each neuron, and return the propagated errors to send to the preceding layer.

Package neuron

• func NewNeuron(weights []float64, bias float64) *Neuron

  • Construct a new Neuron with the provided weights and bias.

• func (n *Neuron) Randomize(rng float64)

  • Randomize the neuron's weights and bias to values in [-rng, rng].

• func (n *Neuron) Activate(inputs []float64) float64

  • Compute the neuron's output using a sigmoid activation on the weighted sum plus bias.

• func (n *Neuron) Correct(inputs []float64, output float64, expected float64, learningRate float64) float64

  • For output neurons: compute the local delta from (expected - output) * output * (1-output), apply weight/bias updates, and return the delta for backpropagation.

• func (n *Neuron) CorrectByError(inputs []float64, output, err float64, learningRate float64)

  • For hidden neurons: accept a propagated error value err (typically the sum of downstream deltas weighted by downstream connections), compute the local delta as err * output * (1-output), and apply weight/bias updates in-place.

Network

A complete feedforward neural network composed of multiple layers.

type Network struct {
    Layers []layer.NeuronLayer
}

Backpropagation details

Hidden layers are trained using propagated error terms from the next layer. During backpropagation each neuron receives a propagated error (the sum of next-layer deltas weighted by the downstream connection weights). The neuron's local delta is computed as propagatedError * output * (1 - output) (sigmoid derivative). In this repository the following methods implement these semantics:

• Neuron.CorrectByError: accepts a propagated error for a single neuron and updates its weights and bias in-place using the computed local delta and the provided learning rate.
• NeuronLayer.CorrectByErrors: accepts a slice of propagated errors for the layer and applies CorrectByError for each neuron, returning the propagated errors to be sent to the previous layer.

Example Usage

Basic Prediction

Train a simple 3-layer network to learn a XOR-like pattern:

package main

import (
	"fmt"
	"github.com/boolka/goai/pkg/network"
)

func main() {
	// Create a network with 3 input neurons, 3 hidden neurons, 3 output neurons
	net := network.NewNetwork([]int{3, 3, 3})
	
	// Randomize weights and biases
	net.Randomize(1.0)
	
	// Define training data
	inputs := []float64{10.0, 0.5, -20.0}
	expectedOutputs := []float64{1.0, 0.0, 1.0}
	learningRate := 0.01
	
	// Get prediction before training
	prediction := net.Activate(inputs)
	fmt.Printf("Before training: %v\n", prediction)
	
	// Train the network
	for i := 0; i < 10000; i++ {
		output := net.Correct(inputs, expectedOutputs, learningRate)
		
		// Print progress
		if i%1000 == 0 {
			fmt.Printf("Iteration %d: %v\n", i, output)
		}
	}
	
	// Get prediction after training
	finalPrediction := net.Activate(inputs)
	fmt.Printf("After training: %v\n", finalPrediction)
	fmt.Printf("Expected: %v\n", expectedOutputs)
}

Training Loop with Early Stopping

Train a network until it reaches desired accuracy:

package main

import (
	"fmt"
	"math"
	"github.com/boolka/goai/pkg/network"
)

func main() {
	// Create network
	net := network.NewNetwork([]int{2, 4, 1})
	net.Randomize(0.5)
	
	// Training parameters
	inputs := []float64{0.5, 0.3}
	expected := []float64{0.8}
	learningRate := 0.01
	tolerance := 1e-3
	
	// Training loop with early stopping
	converged := false
	iterations := 0
	maxIterations := 100000
	
	for iterations = 0; iterations < maxIterations; iterations++ {
		output := net.Correct(inputs, expected, learningRate)
		
		// Check if converged
		converged = true
		for i, out := range output {
			if math.Abs(out-expected[i]) > tolerance {
				converged = false
				break
			}
		}
		
		if converged {
			break
		}
		
		if iterations%10000 == 0 {
			fmt.Printf("Iteration %d | Output: %.6f | Error: %.6f\n", 
				iterations, output[0], math.Abs(output[0]-expected[0]))
		}
	}
	
	if converged {
		fmt.Printf("Training completed in %d iterations\n", iterations)
		fmt.Printf("Final output: %.6f\n", net.Activate(inputs)[0])
	} else {
		fmt.Printf("Training did not converge after %d iterations\n", maxIterations)
	}
}

Multiple Training Samples

Train on multiple samples in sequence:

package main

import (
	"fmt"

	"github.com/boolka/goai/pkg/network"
)

func main() {
	// Create network
	net := network.NewNetwork([]int{2, 128, 128, 128, 2})
	net.Randomize(1.0)

	trainingData := []struct {
		inputs   []float64
		expected []float64
	}{
		{[]float64{0, 1}, []float64{1, 0}},
		{[]float64{1, 0}, []float64{0, 1}},
		{[]float64{1, 1}, []float64{0, 0}},
		{[]float64{0, 0}, []float64{1, 1}},
	}

	learningRate := 0.04

	// Train for multiple epochs
	for epoch := 0; epoch < 5000; epoch++ {
		totalError := 0.0

		for _, sample := range trainingData {
			output := net.Correct(sample.inputs, sample.expected, learningRate)

			// Calculate error
			for i, out := range output {
				diff := out - sample.expected[i]
				totalError += diff * diff
			}
		}

		if epoch%500 == 0 {
			fmt.Printf("Epoch %d | Average Error: %.6f\n", epoch, totalError/float64(len(trainingData)))
		}
	}

	// Test predictions
	fmt.Println("\nFinal Predictions:")
	for _, sample := range trainingData {
		prediction := net.Activate(sample.inputs)
		fmt.Printf("Input: %v | Output: %v | Expected: %v\n",
			sample.inputs, prediction, sample.expected)
	}
}

API Reference

Network

• NewNetwork(layerSizes []int) *Network - Create a new network with specified layer sizes
• Randomize(rng float64) - Randomize all weights and biases
• Activate(inputs []float64) []float64 - Forward pass through network
• Correct(inputs []float64, expected []float64, learningRate float64) []float64 - Train on a single sample

Layer

• NewLayer(neuronCount int, inputCount int) *NeuronLayer - Create a layer
• Randomize(rng float64) - Randomize layer weights and biases
• Activate(inputs []float64) []float64 - Forward pass through layer
• Correct(inputs []float64, outputs []float64, expected []float64, learningRate float64) []float64 - Train layer and return previous-layer deltas
• CorrectByDelta(inputs []float64, outputs []float64, inDeltas []float64, learningRate float64) []float64 - Train with delta backprop and return previous-layer deltas

Neuron

• NewNeuron(weights []float64, bias float64) *Neuron - Create a neuron
• Randomize(rng float64) - Randomize weights and bias
• Activate(inputs []float64) float64 - Sigmoid activation
• Correct(inputs []float64, output float64, expected float64, learningRate float64) float64 - Train with output error and return delta
• CorrectByDelta(inputs []float64, output float64, delta float64, learningRate float64) - Train with propagated delta

I/O

• Serialize(f io.Writer) error - Serialize all weights and biases to binary form
• Deserialize(f io.Reader) error - Deserialize all weights and biases from binary form

How It Works

Forward Pass (Activation)

1. Each neuron computes a weighted sum of inputs plus bias
2. The sum is passed through a sigmoid activation function
3. The output is passed to the next layer

Backward Pass (Training)

1. Compute error at output layer
2. Calculate delta for each output neuron
3. Backpropagate deltas through hidden layers
4. Update weights and biases using deltas and learning rate

License

MIT License - see LICENSE file for details.