Training Neural Networks with Genetic Algorithms in Python

Introduction

This article focuses on the intersection of two topics — “genetic algorithms (GA) in general” and “gradient-based learning of neural networks (NN)” — namely Neuroevolution, the technique of using a GA to directly optimize an NN’s weights and biases.

The general theory behind GA’s selection, crossover, and mutation (schema theorem, etc.) is already derived in detail in Genetic Algorithms: Fundamentals and Python Implementation , so to avoid duplication this article instead digs into the mathematical properties specific to the NN weight space, namely:

1. What it looks like to formulate NN weight optimization as a GA search problem
2. How the permutation symmetry of hidden units destructively interacts with crossover (the Competing Conventions Problem)
3. How fitness scaling (\(1/\text{MSE}\) ) affects the selection pressure of roulette-wheel selection
4. An aliasing (reference-sharing) bug that is easy to introduce in an implementation, and its real-world impact

We examine these four points together with numbers obtained by actually running the code. For learning the same function via gradient-based methods (backpropagation), see Supervised Learning with Neural Networks in Python .

The target function to learn is:

Problem Setup: Formulating NN Weight Optimization as a GA Search Problem

The NN used in this article is a simple 3-layer feedforward network with 2 inputs, 2 hidden units, and 1 output. The network’s output can be written as

(with \(H=2\) and \(\sigma\) the sigmoid function). In GA terms, we treat all of this network’s parameters

as a single vector — one individual’s “gene.” The parameter count is \(2\times2 + 2 + 2\times1 + 1 = 9\) , so \(\theta\in\mathbb{R}^9\) . If we define the loss as the mean squared error over the training data

then the fitness used in the code is

Since \(1/x\) is monotonically decreasing for \(x>0\) , \(\arg\max_\theta \mathrm{fitness}(\theta) = \arg\min_\theta L(\theta)\) — so what the GA is maximizing is, in effect, the same thing as minimizing the NN’s training error.

Gradient methods (backpropagation) need to compute \(\partial L/\partial\theta\) via the chain rule (see the NN article for the derivation), whereas a GA only needs to evaluate \(L(\theta)\) from equation (3) through a forward pass, and requires no differentiability whatsoever. This makes it powerful in settings where the activation function is discontinuous or non-differentiable, or where an analytical gradient cannot even be defined (e.g., the search space is the network architecture itself). On the other hand, because it doesn’t use gradient information — i.e., “which direction improves things” — it is less sample-efficient (requires more evaluations to reach the same accuracy) than gradient-based methods.

Genetic Algorithm (GA)

GA is a search algorithm that mimics the mechanisms of biological evolution, particularly the principle of “survival of the fittest.” It represents candidate solutions as a population of “individuals (genes)” and evolves them toward better solutions through repeated genetic operations.

1. Initialize Population: Randomly generate a population of individuals (in this case, NN weights and biases).
2. Evaluate Fitness: Calculate the “fitness” of each individual, measuring how well it solves the problem. Here, lower error between NN output and training data means higher fitness.
3. Selection (Reproduction): Select individuals so that those with higher fitness have more opportunities to pass their genes to the next generation.
4. Crossover: Create new individuals (offspring) by exchanging parts of the genes between selected pairs. This combines promising elements from different solutions.
5. Mutation: With a certain probability, randomly alter parts of an individual’s genes. This promotes escape from local optima and maintains diversity.
6. Generational Replacement: Replace the current population with the newly generated individuals.
7. Termination Check: Stop if the maximum number of generations is reached or a satisfactory solution is found. Otherwise, return to step 2.

The general mathematics of selection, crossover, and mutation (the schema theorem, the building-block hypothesis, the trade-off between premature convergence and selection pressure, etc.) is covered in detail in the GA article — please refer to it. From here on, this article focuses specifically on how these operations behave in the concrete context of an NN’s weight space.

Python Implementation

A set of NN weights and biases is treated as one “gene,” and GA is used to optimize it. The code below is identical to what was actually run and verified for this article.

Key Parameters

import numpy as np
import math
import random
import matplotlib.pyplot as plt

# Parameter settings
GENERATIONS = 100       # Number of generations
POPULATION_SIZE = 1000  # Population size (number of NNs)
NUM_TEACHER_DATA = 1000 # Number of training data points

# NN structure
NUM_INPUT = 2
NUM_HIDDEN = 2
NUM_OUTPUT = 1

# GA parameters
CROSSOVER_RATE = 0.8    # Crossover rate
MUTATION_RATE = 0.05    # Mutation rate

# Target function
def target_function(x, y):
    # Add small value to avoid divergence when cos(y) is near 0
    cos_y = math.cos(y)
    if abs(cos_y) < 1e-6:
        cos_y = 1e-6
    return (math.sin(x*x) / cos_y + x*x - 5*y + 30) / 80

# Activation function
def sigmoid(x):
    return 1.0 / (1.0 + np.exp(-x))

Neural Network Class

Defines the NN corresponding to each individual.

class NeuralNetwork:
    def __init__(self):
        # Randomly initialize weights and biases
        self.w_ih = np.random.uniform(-1, 1, (NUM_INPUT, NUM_HIDDEN))
        self.b_h = np.random.uniform(-1, 1, NUM_HIDDEN)
        self.w_ho = np.random.uniform(-1, 1, (NUM_HIDDEN, NUM_OUTPUT))
        self.b_o = np.random.uniform(-1, 1, NUM_OUTPUT)

        self.fitness = 0.0 # Fitness

    def predict(self, x):
        # Forward propagation
        hidden_layer_input = np.dot(x, self.w_ih) + self.b_h
        hidden_layer_output = sigmoid(hidden_layer_input)
        output_layer_input = np.dot(hidden_layer_output, self.w_ho) + self.b_o
        # Output layer uses identity activation
        return output_layer_input[0]

    def calculate_fitness(self, teacher_inputs, teacher_outputs):
        # Calculate mean squared error over all training data
        error = 0.0
        for i in range(len(teacher_inputs)):
            prediction = self.predict(teacher_inputs[i])
            error += (prediction - teacher_outputs[i]) ** 2

        mean_squared_error = error / len(teacher_inputs)

        # Define fitness so lower error = higher fitness
        self.fitness = 1.0 / (mean_squared_error + 1e-9) # Avoid division by zero

GA Class

Implements GA operations (selection, crossover, mutation).

class GeneticAlgorithm:
    def __init__(self):
        self.population = [NeuralNetwork() for _ in range(POPULATION_SIZE)]

    def run_generation(self, teacher_inputs, teacher_outputs):
        # 1. Calculate fitness for all individuals
        for individual in self.population:
            individual.calculate_fitness(teacher_inputs, teacher_outputs)

        # 2. Generate new generation
        new_population = []

        # Elitism: Keep the best individual unchanged
        elite = max(self.population, key=lambda ind: ind.fitness)
        new_population.append(elite)

        while len(new_population) < POPULATION_SIZE:
            # 3. Selection (roulette wheel selection)
            parent1 = self._roulette_selection()
            parent2 = self._roulette_selection()

            # 4. Crossover
            child1, child2 = self._crossover(parent1, parent2)

            # 5. Mutation
            self._mutate(child1)
            self._mutate(child2)

            new_population.extend([child1, child2])

        self.population = new_population[:POPULATION_SIZE]

    def _roulette_selection(self):
        total_fitness = sum(ind.fitness for ind in self.population)
        pick = random.uniform(0, total_fitness)
        current = 0
        for individual in self.population:
            current += individual.fitness
            if current > pick:
                return individual
        return self.population[-1]

    def _crossover(self, parent1, parent2):
        child1 = NeuralNetwork()
        child2 = NeuralNetwork()

        if random.random() < CROSSOVER_RATE:
            # Randomly swap parameter sets (uniform crossover, simplified)
            child1.w_ih, child2.w_ih = (parent1.w_ih, parent2.w_ih) if random.random() < 0.5 else (parent2.w_ih, parent1.w_ih)
            child1.b_h, child2.b_h = (parent1.b_h, parent2.b_h) if random.random() < 0.5 else (parent2.b_h, parent1.b_h)
            child1.w_ho, child2.w_ho = (parent1.w_ho, parent2.w_ho) if random.random() < 0.5 else (parent2.w_ho, parent1.w_ho)
            child1.b_o, child2.b_o = (parent1.b_o, parent2.b_o) if random.random() < 0.5 else (parent2.b_o, parent1.b_o)
        else:
            child1, child2 = parent1, parent2 # No crossover

        return child1, child2

    def _mutate(self, individual):
        # Replace each weight/bias with probability MUTATION_RATE
        for w in [individual.w_ih, individual.b_h, individual.w_ho, individual.b_o]:
            if random.random() < MUTATION_RATE:
                w += np.random.uniform(-0.1, 0.1, w.shape)

Main Function

def main():
    # Generate training data
    teacher_inputs = np.random.uniform(-5, 5, (NUM_TEACHER_DATA, NUM_INPUT))
    teacher_outputs = np.array([target_function(x[0], x[1]) for x in teacher_inputs])

    # Generate test data
    test_inputs = np.random.uniform(-5, 5, (NUM_TEACHER_DATA, NUM_INPUT))
    test_outputs = np.array([target_function(x[0], x[1]) for x in test_inputs])

    ga = GeneticAlgorithm()

    elite_errors = []
    print("Training started...")
    for gen in range(GENERATIONS):
        ga.run_generation(teacher_inputs, teacher_outputs)

        # Find the best individual (elite)
        elite = max(ga.population, key=lambda ind: ind.fitness)

        # Evaluate elite on test data
        test_error = 0.0
        for i in range(len(test_inputs)):
            prediction = elite.predict(test_inputs[i])
            test_error += (prediction - test_outputs[i]) ** 2

        mean_squared_error = test_error / len(test_inputs)
        elite_errors.append(mean_squared_error)

        if (gen + 1) % 10 == 0:
            print(f"Generation: {gen + 1}, Test Error (MSE): {mean_squared_error:.6f}")

    # Plot results
    plt.plot(elite_errors)
    plt.title("Elite Individual's Error on Test Data")
    plt.xlabel("Generation")
    plt.ylabel("Mean Squared Error")
    plt.grid(True)
    plt.savefig("ga_nn_learning_curve.png")
    plt.show()

if __name__ == '__main__':
    main()

Execution Verification: Reproducing the Learning Curve Did Not Converge

We fixed numpy.random.seed(42) / random.seed(42) and actually ran the code above for 100 generations with a population size of 1000. The result, however, differed from this article’s earlier description (“as generations progress, the error monotonically decreases”).

Over the 100 generations, the test error ranged widely — minimum 0.057615, maximum 0.229795, mean 0.087463 — with no convergence at all, let alone monotonic decrease. The population’s average MSE (the reciprocal of the training-data fitness) only dropped from an initial value of 2.414 to 2.148 by the final generation; most of the 1000 individuals were not learning at all.

Root Cause: Crossover Shares References Instead of Copying Values

The cause lies in the implementation of _crossover and _mutate. In Python, assigning an object shares a reference, so child1.w_ih, child2.w_ih = (parent1.w_ih, parent2.w_ih) does not duplicate the array — it simply attaches the parent’s NumPy array object to the child as well. Furthermore, w += np.random.uniform(...) inside _mutate is an in-place operation for NumPy arrays (+=), so it overwrites the shared array itself.

It’s also worth noting that when crossover doesn’t happen (the remaining 20% governed by CROSSOVER_RATE), child1, child2 = parent1, parent2 makes the child and parent the exact same object.

This produces the following real-world damage:

• The elite individual that’s supposedly “protected” by new_population.append(elite) can still get overwritten mid-generation, the instant a different child individual sharing its reference is mutated
• A mutation applied to one individual can unintentionally alter the weights of an unrelated individual (a parent, or a different child)

We verified this concretely with a small-scale reproduction using a population size of 20 and a mutation rate of 0.5.

import numpy as np, random

np.random.seed(1); random.seed(1)
POPULATION_SIZE = 20

# (NeuralNetwork class, crossover, and mutate use the same logic as the original code)
pop = [NeuralNetwork() for _ in range(POPULATION_SIZE)]

elite = max(pop, key=lambda i: i.fitness)
elite_w_ih_snapshot = elite.w_ih.copy()
new_population = [elite]  # Appended as a reference, same as the original code

while len(new_population) < POPULATION_SIZE:
    p1, p2 = random.choice(pop), random.choice(pop)
    c1, c2 = crossover(p1, p2)
    mutate(c1); mutate(c2)
    new_population.extend([c1, c2])

changed = not np.array_equal(elite_w_ih_snapshot, new_population[0].w_ih)
print("new_population[0] is elite:", new_population[0] is elite)
print("Did the elite's w_ih change during generation processing?:", changed)
ids = [id(ind.w_ih) for ind in new_population]
print("Number of unique w_ih objects:", len(set(ids)), "/", POPULATION_SIZE)

Execution result:

new_population[0] is elite: True
Did the elite's w_ih change during generation processing?: True
  before: [-0.59341353 -0.49534851  0.48765171 -0.60914104]
  after:  [-0.73847006 -0.6086954   0.48052509 -0.71388218]
Number of unique w_ih objects: 12 / 20

The best individual — supposedly protected by “elitism” — gets overwritten by a mutation applied to a different individual before generational replacement even finishes. Out of 20 individuals, only 12 unique w_ih array objects exist; the weights of 8 individuals are implicitly shared with others (i.e., changing one changes the other too). At the scale of a population of 1000 over 100 generations, this compounds into weights that “random-walk” without bound. Indeed, the population’s maximum absolute weight kept growing uncontrolled, from an initial value of 1.056 to 3.217 by the final generation.

The Fix: Use copy.deepcopy in Crossover and Elitism

The fix is simple — copy values instead of sharing references.

import copy

# Elitism
new_population.append(copy.deepcopy(elite))

# Crossover (always deepcopy before assignment, whether or not crossover actually happens)
def _crossover(self, parent1, parent2):
    child1 = NeuralNetwork()
    child2 = NeuralNetwork()
    if random.random() < CROSSOVER_RATE:
        p1, p2 = copy.deepcopy(parent1), copy.deepcopy(parent2)
        child1.w_ih, child2.w_ih = (p1.w_ih, p2.w_ih) if random.random() < 0.5 else (p2.w_ih, p1.w_ih)
        child1.b_h, child2.b_h = (p1.b_h, p2.b_h) if random.random() < 0.5 else (p2.b_h, p1.b_h)
        child1.w_ho, child2.w_ho = (p1.w_ho, p2.w_ho) if random.random() < 0.5 else (p2.w_ho, p1.w_ho)
        child1.b_o, child2.b_o = (p1.b_o, p2.b_o) if random.random() < 0.5 else (p2.b_o, p1.b_o)
    else:
        child1, child2 = copy.deepcopy(parent1), copy.deepcopy(parent2)
    return child1, child2

Running this fixed version with the same seed, the same 100 generations, and the same population size of 1000 gives:

Over the 100 generations, the test error stays within a tight range — minimum 0.058539, maximum 0.062846, mean 0.060388 — in stark contrast to the buggy version, converging near 0.059 in a near-monotonic fashion. The population’s average MSE also steadily improved from 2.414 to 1.904, and the maximum absolute weight stayed within the range 1.056–1.292 (compared to growing to 3.217 in the buggy version). Figure 1 shows both learning curves and the weight-divergence trend.

The aliasing bug causes the error to oscillate divergently, while the fixed version converges smoothly and the weights don’t diverge — the difference is obvious at a glance. This kind of pitfall — “arrays are shared by reference, and += is an in-place operation” — is extremely easy to fall into when implementing evolutionary computation with NumPy. Wherever you intend to duplicate an individual — in crossover, mutation, or elitism — you must explicitly call copy.deepcopy (or .copy() on each array).

The Theoretical Limit of Crossover: Hidden-Unit Permutation Symmetry and the Competing Conventions Problem

Even after fixing the aliasing bug, this implementation’s crossover operator has one more fundamental problem left, one that stems from the NN’s symmetry under permutation of hidden units.

Proof of Permutation Symmetry

Consider the \(H\) hidden units in equation (1), and an arbitrary permutation \(\pi:\{1,\ldots,H\}\to\{1,\ldots,H\}\) . Define the parameters

Then

and since \(\pi\) is a bijection, re-indexing the sum with \(h'=\pi(h)\) shows \(f_{\theta'}(x) = f_\theta(x)\) holds identically for all \(x\) . In other words, \(\theta'\) , obtained by simply reordering the hidden units, computes exactly the same function as \(\theta\) . In this article’s setting with \(H=2\) , there are \(2!=2\) equivalent representations (the identity permutation and the transposition) — since the sigmoid is not an odd function, there is no sign-flip symmetry of the kind seen with \(\tanh\) .

This symmetry clashes badly with crossover because even two parents that are “identical as functions” occupy entirely different points in parameter space if their internal labeling (which column is “hidden unit 1”) differs. Crossing over at the block level can break the correspondence between \(w_{ih}\) and \(w_{ho}\) (which column and which row refer to the same hidden unit), producing functionally broken offspring. In the literature this is known as the Competing Conventions Problem.

Execution Verification: Crossing Over Two Functionally Identical Twins

To clearly demonstrate the theoretical effect, we hand-constructed a network where the hidden units’ roles are clearly distinct (i.e., not symmetric), and verified crossover between it and a “twin” with its hidden-unit order swapped.

import numpy as np

def make_network(w_ih, b_h, w_ho, b_o):
    # The original article's NeuralNetwork always initializes randomly, so we
    # overwrite the attributes after instantiation to inject explicit weights
    net = NeuralNetwork()
    net.w_ih, net.b_h, net.w_ho, net.b_o = w_ih, b_h, w_ho, b_o
    return net

def permute_hidden_units(net, perm):
    # Reordering columns (w_ih), elements (b_h), and rows (w_ho) with the same
    # perm leaves the function unchanged
    w_ih2 = net.w_ih[:, perm].copy()
    b_h2 = net.b_h[perm].copy()
    w_ho2 = net.w_ho[perm, :].copy()
    return make_network(w_ih2, b_h2, w_ho2, net.b_o.copy())

# Hand-constructed so the hidden units' roles are clearly distinct (opposite-sign,
# differently-scaled output weights)
w_ih = np.array([[0.6, -0.3], [-0.2, 0.5]])
b_h = np.array([0.4, -0.7])
w_ho = np.array([[1.4], [-1.1]])
b_o = np.array([0.1])
net = make_network(w_ih, b_h, w_ho, b_o)

twin = permute_hidden_units(net, [1, 0])  # Swap hidden units 0 and 1

# Cross net and twin using the same block-level crossover logic as the original _crossover
mismatch = make_network(twin.w_ih.copy(), net.b_h.copy(), net.w_ho.copy(), net.b_o.copy())

Execution result (500 test points, numpy.random.seed(3)):

Net and Twin have an exact difference of 0 in their predictions at every test point (complete functional equivalence, without even floating-point rounding error), yet simply crossing them over at the block level worsens the MSE from 0.42 to somewhere between 0.73 and 0.94 — on par with, or worse than, the mean of a purely randomly initialized population (0.8755). Figure 2 visualizes this.

Interestingly, running the same experiment with an elite individual actually evolved by the GA (trained on this article’s teacher data) produced only a slight degradation from crossover (elite MSE 0.3668, versus 0.363–0.367 for the offspring crossed with its twin). Investigating the cause, this elite’s output weights were \(w_{ho}\approx[0.380, 0.364]\) — nearly equal — meaning the two hidden units had evolved to play nearly redundant (near-symmetric) roles. This is a coincidental outcome and should not be expected in general. The more the hidden units’ roles diverge (the further apart the values of \(w_{ho}\) are), the worse the damage from the Competing Conventions Problem under crossover becomes.

As Pretorius & Pillay (2024) also point out, this problem underlies a persistent skepticism toward the effectiveness of crossover operators in evolving NN weights at all. Their paper proposes using genetic programming to automatically design the crossover operator itself — which, read the other way, starts from the premise that “hand-designed, simple crossover cannot be trusted.”

Roulette Selection and Fitness Scaling: The Two Faces of Weak vs. Runaway Selection Pressure

The fitness definition from equation (4), \(\mathrm{fitness}(\theta)=1/(L(\theta)+\varepsilon)\) , directly affects the GA’s search dynamics through the roulette-selection probability

Let’s plug in the actual execution results (fixed version, generation 100). The elite’s test MSE was 0.059338, and the population’s average MSE (training error) was 1.904. Using these as rough figures,

With \(N=1000\) , \(P(\text{elite}) \approx 16.9 / (16.9 + 999\times 0.53) \approx 3.1\%\) . Under uniform selection this would be \(0.1\%\) , so the single elite individual alone has roughly 31 times that selection probability. This is a “meaningful but not runaway” level of selection pressure, and it’s consistent with the fact that, even in the fixed version, the population’s average MSE only improved from 2.41 to 1.90 over 100 generations (i.e., no dramatic convergence occurred).

What’s dangerous about this \(1/(L+\varepsilon)\) design is that if even a single individual finds a solution that’s orders of magnitude better (say \(L\approx10^{-4}\) ), the selection probability instantly becomes nearly monopolized. With \(L=10^{-4}\) , \(\mathrm{fitness}\approx10^4\) — more than the sum of the remaining 999 individuals (each with \(\mathrm{fitness}\approx0.5\) or so) combined. In this case \(P(\text{elite})\) instantly exceeds 90%, causing premature convergence, where diversity is lost within a few generations. In other words, this fitness design has selection pressure that’s weak and slow to converge under normal conditions, yet swings to excessive selection pressure the moment an outlier individual appears — a design with two extreme modes baked in. More stable selection pressure can be obtained with alternative designs such as rank selection (basing selection probability on rank rather than the raw fitness value) or logarithmic fitness scaling. The general mechanics of premature convergence and the trade-off with selection pressure are covered in detail in the corresponding section of the GA article .

Summary of Edge Cases and Caveats

Here is a summary of the points to keep in mind when implementing and operating Neuroevolution:

• Aliasing bugs: If crossover or elitism assigns individuals (or their internal arrays) without duplicating them, in-place operations like += can unintentionally overwrite other individuals — including the elite, which is supposed to be protected. Always use copy.deepcopy or .copy().
• Competing Conventions Problem: Because of hidden-unit permutation symmetry, naive crossover can be destructive even between functionally identical parents. The larger the hidden layer, the more equivalent representations exist (\(H!\) of them), and the worse the problem becomes.
• The two faces of fitness scaling: Ratio-based scaling such as \(1/(L+\varepsilon)\) has weak, slow-converging selection pressure under normal conditions, but selection pressure spikes sharply — risking premature convergence — the moment an outlier high-quality individual appears.
• Local optima and premature convergence: If selection pressure is too strong, or the population size too small, diversity can be lost before the global optimum is reached. See the GA article for the general theory.
• When to use gradient methods instead: As the number of parameters grows (9 in this article, but thousands to millions in practical NNs), the GA’s poor sample efficiency becomes a real liability. Neuroevolution is practically useful mainly where gradients are undefined or unreliable (certain reinforcement-learning settings, non-differentiable rewards, architecture search, etc.).

Recent Research Trends (2023 Onward)

• Automatic design of crossover operators: Given the long-standing debate over whether crossover operators are effective at all for evolving NN weights, Pretorius, C. J., & Pillay, N. (2024) propose using genetic programming to automatically generate reusable or single-use crossover operators, and compare them against hand-designed crossover and crossover-free GAs. Given the severity of the Competing Conventions Problem confirmed in this article, this points to a real need for designs that don’t rely on naive crossover.
• Self-adaptive mutation: Hiraga, M., Komura, M., Miyamoto, A., Morimoto, D., & Ohkura, K. (2024) introduce a self-adaptive mechanism that automatically tunes step size in mutation-based evolving neural networks (MBEANN), reporting improved performance over a fixed mutation rate. The fixed mutation rate (0.05) used in this article is simple but hard to tune, making this direction practically important.
• Large-scale GPU parallelization: TensorNEAT, by Wang, L., Zhao, M., Liu, E., Sun, K., & Cheng, R. (2025), reformulates NEAT’s (NeuroEvolution of Augmenting Topologies) network structure into a unified tensor representation and evaluates the entire population in parallel on a GPU, reporting up to a 500x speedup over NEAT-Python. The evaluation cost in this article — several minutes even at a population of 1000 over 100 generations — is a major scalability bottleneck for Neuroevolution, and this kind of parallelization directly enables scaling to larger networks.

Related Articles

• Genetic Algorithms: Fundamentals and Python Implementation - Derives the general theory of GA in detail, including the schema theorem, premature convergence, and the crossover/mutation-rate trade-off.
• The Mathematics of Two-Swarm Cooperative PSO (TCPSO): Deriving Slave/Master Stability Conditions with Python - Another swarm intelligence optimization approach
• Supervised Learning with Neural Networks in Python - Comparison with gradient-based (backpropagation) NN training
• Cross-Entropy Method: A Practical Monte Carlo Optimization Technique - An alternative stochastic optimization approach

References

• Stanley, K. O., & Miikkulainen, R. (2002). “Evolving neural networks through augmenting topologies.” Evolutionary Computation, 10(2), 99-127.
• Pretorius, C. J., & Pillay, N. (2024). “Neural network crossover in genetic algorithms using genetic programming.” Genetic Programming and Evolvable Machines, 25, Article 7.
• Hiraga, M., Komura, M., Miyamoto, A., Morimoto, D., & Ohkura, K. (2024). “Improving the performance of mutation-based evolving artificial neural networks with self-adaptive mutations.” PLOS ONE, 19(7), e0307084.
• Wang, L., Zhao, M., Liu, E., Sun, K., & Cheng, R. (2025). “TensorNEAT: A GPU-accelerated Library for NeuroEvolution of Augmenting Topologies.” arXiv:2504.08339 (accepted at ACM Transactions on Evolutionary Learning and Optimization).