The word "learning" makes it sound as though something fundamentally different from an ordinary program happens inside a model. As though it understands examples, discovers patterns, and gradually becomes smarter.
It is better to begin with an algorithm whose entire process can be followed by hand.
Give a model the four rows of the AND operation:
0 AND 0 = 0
0 AND 1 = 0
1 AND 0 = 0
1 AND 1 = 1At first, the model does not know the correct rule. It has only two weights and one more number, the bias. It answers the first example, compares the result with the expected answer, and changes those three values slightly if it was wrong.
After several passes, the errors stop. The resulting weights now answer all four inputs correctly.
That is perceptron learning: an ordinary program changes its internal state after each error until it finds a suitable boundary between answers 0 and 1.
We will follow the entire process, from the first wrong answer to the discovered line. The complete source is included in the appendix.
Three Numbers Determine Every Answer
A perceptron receives two input values, multiplies each by its weight, adds a bias, and compares the sum with zero:
pub fn answer(&self, input: [f64; 2]) -> f64 {
let sum =
self.weights[0] * input[0]
+ self.weights[1] * input[1]
+ self.bias;
if sum > 0.0 {
1.0
} else {
0.0
}
}The entire model is described by this expression:
w₀·a + w₁·b + biasIf the result is greater than zero, the perceptron answers 1. Otherwise, it answers 0.
The weight w₀ determines how strongly the first input affects the answer, while w₁ does the same for the second. A positive weight pulls the sum upward; a negative weight pulls it downward.
The bias can raise or lower the whole sum independently of the inputs. Without it, the boundary between answers would always pass through the point (0, 0).
A bias can be viewed as an additional constant input that is always equal to one. It also has a trainable weight, allowing the model to move the boundary without changing the influence of its primary inputs.
The perceptron does not store the AND table inside itself. After training, it still contains only three numbers. Every answer is calculated again using the same formula.
An Error Becomes a Weight Update
Now the model needs a way to correct itself.
The perceptron rule looks like this:
pub fn correct(
&mut self,
input: [f64; 2],
target: f64,
rate: f64,
) -> bool {
let answer = self.answer(input);
let error = target - answer;
if error == 0.0 {
return false;
}
self.weights[0] += rate * error * input[0];
self.weights[1] += rate * error * input[1];
self.bias += rate * error;
true
}Here, target is the correct answer from the training example, and answer is the model's result.
The difference between them can have only three values:
0 — the model answered correctly
+1 — the model answered 0, but the target is 1
-1 — the model answered 1, but the target is 0The weights do not change after a correct answer.
If the answer was too low, the error is +1 and the weights move upward. If the model incorrectly returned one, the error is -1, so the weights move in the opposite direction.
The rate parameter sets the size of one step.
Notice that the error is multiplied by the input:
self.weights[0] += rate * error * input[0];If the first input is zero, w₀ does not change. That input did not contribute to the wrong answer, so its corresponding weight does not need correction.
There are no derivatives, matrices, or separate loss function here. The model answered incorrectly, so its weights move by one small step.
Following the Entire AND Training Run
Start with zero weights:
w = [0.0, 0.0]
bias = 0.0With these values, the sum is always zero. Because the model uses the strict condition sum > 0, its initial answer for every input is 0.
That is already correct for three rows of AND. An error occurs only for (1, 1), where the target is one.
Print every correction:
$ cargo run --release --quiet
learning AND from 4 examples, rate 0.1
start: w = [0.0, 0.0], bias = 0.0
epoch input target answered -> w = [ w0, w1 ], bias
1 (1, 1) 1 0 -> [ 0.1, 0.1], 0.1
2 (0, 0) 0 1 -> [ 0.1, 0.1], 0.0
2 (0, 1) 0 1 -> [ 0.1, 0.0], -0.1
2 (1, 1) 1 0 -> [ 0.2, 0.1], 0.0
3 (0, 1) 0 1 -> [ 0.2, 0.0], -0.1
3 (1, 0) 0 1 -> [ 0.1, 0.0], -0.2
3 (1, 1) 1 0 -> [ 0.2, 0.1], -0.1
4 (1, 0) 0 1 -> [ 0.1, 0.1], -0.2
4 (1, 1) 1 0 -> [ 0.2, 0.2], -0.1
5 (0, 1) 0 1 -> [ 0.2, 0.1], -0.2
10 corrections in total, converged: true
final: w = [0.2, 0.1], bias = -0.2 (4 of 4 correct)The first correction is straightforward.
For input (1, 1), the model answered 0 when the target was one. The error is +1, so both weights and the bias increase by 0.1:
[0.0, 0.0], 0.0
→
[0.1, 0.1], 0.1The sum is now positive even for (0, 0) because the bias of 0.1 alone is enough for the model to answer one. To correct this error, the perceptron reduces the bias back to zero.
The next example, (0, 1), also incorrectly receives one. The first input is zero, so w₀ does not change. Only the second input's weight and the bias decrease:
[0.1, 0.1], 0.0
→
[0.1, 0.0], -0.1Training continues in this way. The positive example (1, 1) pushes the weights upward, while negative examples pull individual weights and the bias downward.
The model does not know where it should end up. Each correction fixes only the current error and may create a new one on another example.
By the fifth epoch, the contradictions are gone. A complete pass through all four rows no longer changes any weight, so training stops.
The run required ten corrections in total.
What the Model Actually Learned
After training, the perceptron contains three values:
w₀ = 0.2
w₁ = 0.1
bias = -0.2This produces the rule:
0.2·a + 0.1·b - 0.2 > 0Check all four inputs.
For (0, 0), the sum is -0.2, so the answer is 0.
For (0, 1), the result is -0.1, again producing 0.
For (1, 0), the sum is zero. The condition sum > 0 is false, so the answer is also 0.
Only (1, 1) produces a positive sum:
0.2 + 0.1 - 0.2 = 0.1The model answers 1.
The perceptron did not save four separate rules. It found one formula that describes the entire AND table at once.
Moreover, the same formula determines an answer not only for zeros and ones but also for any intermediate values. The model can process (0.8, 0.7), for example, even though that input never appeared during training.
This is no longer a lookup table. It is a learned boundary.
Does Training Converge from Any Initial Weights?
One successful run proves little. Perhaps the zero initialization was unusually convenient.
Test the perceptron rule with ten thousand random initializations for several logical functions:
$ cargo run --release --quiet
10000 random starts per function, rate 0.1, cap 1000 epochs
function converged epochs (min/median/max) corrections (median/max)
AND 10000/10000 1 / 9 / 21 17 / 53
OR 10000/10000 1 / 8 / 23 12 / 37
NAND 10000/10000 1 / 9 / 20 16 / 51
IMPL 10000/10000 1 / 7 / 21 12 / 44
XOR 0/10000 never — no separating line existsAND, OR, NAND, and implication converged in all ten thousand runs.
The initial weights affect the path and number of corrections, but not whether training succeeds. For AND, the median run finished in nine epochs, while the worst required twenty-one.
This is more than an experimental result. The perceptron convergence theorem applies to linearly separable data: if a straight line can separate the examples, the algorithm finds suitable weights after a finite number of corrections.
The final row shows the condition without which that guarantee disappears.
XOR did not converge in any of the ten thousand runs.
The algorithm kept changing the weights, but no suitable state existed. Different initialization or more epochs cannot fix that problem.
The Weights Define a Straight Line
The perceptron's formula can be read not only as a calculation but also as geometry.
The boundary between answers is defined by:
w₀·a + w₁·b + bias = 0For two inputs, this is a straight line.
On one side, the sum is positive and the model answers 1. On the other, it is non-positive and the answer is 0.
Draw the trained perceptron's behavior across the entire unit square:
$ cargo run --release --quiet
the same boundary drawn over the unit square (# = answers 1):
b=1.0 ...........##########
b=0.9 ............#########
b=0.8 .............########
b=0.7 ..............#######
b=0.6 ...............######
b=0.5 ................#####
b=0.4 ................#####
b=0.3 ..................###
b=0.2 ...................##
b=0.1 ....................#
b=0.0 .....................
a=0.0 a=1.0Dots mark the region where the model answers 0, and hash signs mark the region where it answers 1.
The line cuts off the upper-right corner. That is where both inputs are large and where AND's only positive example, (1, 1), is located.
Training can now be understood even more simply.
The perceptron did not choose four answers independently. After each error, it moved the line slightly until every training point was on the correct side.
Many Suitable Lines May Exist
The learned boundary is not unique.
Run training from three different initial states:
start 0: w = [ 0.0, 0.0], bias 0.0 -> after 10 corrections: 0.20·a +0.10·b -0.20 = 0 (4 of 4 correct)
start 1: w = [ 0.8, -0.6], bias 0.3 -> after 21 corrections: 0.30·a +0.10·b -0.40 = 0 (4 of 4 correct)
start 2: w = [ -0.5, 0.9], bias -0.7 -> after 16 corrections: 0.20·a +0.70·b -0.90 = 0 (4 of 4 correct)The result is three different formulas, but each classifies all four examples correctly.
The perceptron stops training as soon as it passes through the entire dataset without an error. It does not try to find the most elegant boundary, place it exactly in the middle, or maximize the distance to the nearest points.
The result therefore depends on the initial weights and the order of examples.
For AND, that does not cause a problem: every learned line produces the correct truth table. Beyond the training points, however, different models may give different answers.
This is an important limit of the simple learning rule. It guarantees correct separation of the known examples, but it does not choose the best one among all possible boundaries.
What Happens to a Point on the Boundary?
Inspect the linear sums for the first learned model:
signed distance of every example from the learned line:
(0, 0) score -0.20 -> answers 0 (target 0)
(0, 1) score -0.10 -> answers 0 (target 0)
(1, 0) score +0.00 -> answers 0 (target 0)
(1, 1) score +0.10 -> answers 1 (target 1)The point (1, 0) lies exactly on the boundary: its sum is zero.
It receives answer 0 only because the code uses a strict comparison:
if sum > 0.0 {
1.0
} else {
0.0
}If we replace it with sum >= 0.0, the same point is classified as one, and the model no longer implements AND correctly.
The rule for handling zero may look like a minor detail, but it is part of the model's definition. The weights cannot be interpreted separately from the function that turns the sum into a final answer.
We can also see that the perceptron does not try to push points away from the boundary. It is enough for every point to land on the correct side, even if the distance is zero and the result depends on the tie-breaking rule.
Algorithms designed specifically to find a large margin between classes solve a different problem.
Why XOR Cannot Be Learned
For AND, the positive point is in the upper-right corner. One line can easily separate it from the other three.
OR, NAND, and implication also have a suitable boundary.
XOR arranges its answers differently:
0 XOR 0 = 0
0 XOR 1 = 1
1 XOR 0 = 1
1 XOR 1 = 0Class-one points occupy two opposite corners of the square, while class-zero points occupy the other two.
One straight line cannot separate them.
Therefore:
XOR 0/10000does not indicate poor training. It reveals the model's limit.
The correction rule keeps fixing the current error faithfully, but every boundary shift breaks another example. The algorithm cannot know that no solution exists, so without an epoch limit it will continue moving the weights forever.
Solving XOR requires more than changing the training procedure. The model itself must change by gaining an intermediate layer that transforms the inputs before drawing the separating line.
What "Learning" Actually Means Here
At the beginning, the perceptron had three zeros and four AND examples.
For each example, it performed an ordinary calculation. If the answer was wrong, the program changed the weights according to a fixed rule.
After ten corrections, it produced this expression:
0.2·a + 0.1·b - 0.2It separated all four inputs correctly, and the changes stopped.
This simple algorithm already contains the main parts of machine learning: a model with adjustable parameters, examples with correct answers, and a procedure that updates those parameters after an error.
But the perceptron also immediately demonstrates an important limit: training does not create capabilities absent from the model itself.
If a straight line can separate the data, the rule is guaranteed to find one suitable boundary. If no such line exists, no number of examples or epochs can help.
The simplest model that can learn shows both the power and the limit of learning.
The program really does discover a rule from examples. But it can discover only a rule that it is able to express.
Appendix: Full Source Files
perceptron/src/lib.rs — 110 lines
//! Scene 01: the whole of learning, in one rule.
//!
//! A perceptron holds two weights and a bias. It answers 1 when the
//! weighted sum clears zero and 0 otherwise. Training is one line:
//! when the answer is wrong, move each weight by the input times the
//! error. No derivatives, no loss function, no matrices.
/// Four labelled examples — the whole training set of a truth table.
pub type Examples = Vec<([f64; 2], f64)>;
pub fn truth_table(function: &str) -> Examples {
[[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]
.into_iter()
.map(|input| {
let (a, b) = (input[0] == 1.0, input[1] == 1.0);
let target = match function {
"AND" => a && b,
"OR" => a || b,
"NAND" => !(a && b),
"IMPL" => !a || b,
_ => a ^ b,
};
(input, if target { 1.0 } else { 0.0 })
})
.collect()
}
#[derive(Clone, Copy, Debug)]
pub struct Perceptron {
pub weights: [f64; 2],
pub bias: f64,
}
/// One recorded correction: the article shows these verbatim.
#[derive(Clone, Copy)]
pub struct Update {
pub epoch: usize,
pub input: [f64; 2],
pub target: f64,
pub answer: f64,
pub weights: [f64; 2],
pub bias: f64,
}
impl Perceptron {
pub fn new(weights: [f64; 2], bias: f64) -> Self {
Self { weights, bias }
}
/// The step decision: fire when the weighted sum clears zero.
pub fn answer(&self, input: [f64; 2]) -> f64 {
let sum = self.weights[0] * input[0] + self.weights[1] * input[1] + self.bias;
if sum > 0.0 { 1.0 } else { 0.0 }
}
/// The perceptron rule. `error` is target minus answer, so it is
/// -1, 0 or +1: nudge the weights toward the input when the answer
/// was too low, away from it when too high, leave them alone when
/// the answer was right.
pub fn correct(&mut self, input: [f64; 2], target: f64, rate: f64) -> bool {
let answer = self.answer(input);
let error = target - answer;
if error == 0.0 {
return false;
}
self.weights[0] += rate * error * input[0];
self.weights[1] += rate * error * input[1];
self.bias += rate * error;
true
}
/// Train until a whole pass makes no corrections, or give up.
/// Returns every correction that was made along the way.
pub fn train(
&mut self,
examples: &Examples,
rate: f64,
max_epochs: usize,
) -> (Vec<Update>, bool) {
let mut history = Vec::new();
for epoch in 1..=max_epochs {
let mut corrections = 0;
for (input, target) in examples {
let answer = self.answer(*input);
if self.correct(*input, *target, rate) {
corrections += 1;
history.push(Update {
epoch,
input: *input,
target: *target,
answer,
weights: self.weights,
bias: self.bias,
});
}
}
if corrections == 0 {
return (history, true);
}
}
(history, false)
}
pub fn correct_count(&self, examples: &Examples) -> usize {
examples
.iter()
.filter(|(input, target)| self.answer(*input) == *target)
.count()
}
}perceptron/src/main.rs — 59 lines
//! Trains AND from four examples and prints every correction — the
//! entire learning process, start to finish, on one screen.
use anyhow::Result;
use perceptron::{Perceptron, truth_table};
const RATE: f64 = 0.1;
fn main() -> Result<()> {
let examples = truth_table("AND");
let mut unit = Perceptron::new([0.0, 0.0], 0.0);
println!("learning AND from {} examples, rate {RATE}", examples.len());
println!(
"start: w = [{:.1}, {:.1}], bias = {:.1}\n",
unit.weights[0], unit.weights[1], unit.bias
);
let (history, converged) = unit.train(&examples, RATE, 100);
println!("epoch input target answered -> w = [ w0, w1 ], bias");
for update in &history {
println!(
"{:>5} ({:.0}, {:.0}) {:.0} {:.0} -> [{:>5.1}, {:>5.1}], {:>5.1}",
update.epoch,
update.input[0],
update.input[1],
update.target,
update.answer,
update.weights[0],
update.weights[1],
update.bias
);
}
println!(
"\n{} corrections in total, converged: {converged}",
history.len()
);
println!(
"final: w = [{:.1}, {:.1}], bias = {:.1} ({} of 4 correct)",
unit.weights[0],
unit.weights[1],
unit.bias,
unit.correct_count(&examples)
);
println!("\nwhat it answers now:");
for (input, target) in &examples {
println!(
" {} AND {} -> {} (target {})",
input[0] as u8,
input[1] as u8,
unit.answer(*input) as u8,
*target as u8
);
}
Ok(())
}convergence/src/main.rs — 62 lines
//! Scene 02: does it always work, or did the first run get lucky?
//!
//! The same rule is started from ten thousand random weight vectors on
//! four different functions. The perceptron convergence theorem says
//! training must halt whenever a separating line exists — this run
//! checks that claim by brute force, and records how much work it took.
//! XOR is included on purpose: it has no such line.
use anyhow::Result;
use perceptron::{Perceptron, truth_table};
const RUNS: usize = 10_000;
const RATE: f64 = 0.1;
const MAX_EPOCHS: usize = 1_000;
/// Deterministic weights in [-1, 1) from a counter.
fn seeded(run: usize) -> ([f64; 2], f64) {
let mut state = 0x14_00_2026_u64 ^ (run as u64).wrapping_mul(0x9E37_79B9_7F4A_7C15);
let mut next = || {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
(state >> 11) as f64 / (1_u64 << 53) as f64 * 2.0 - 1.0
};
([next(), next()], next())
}
fn main() -> Result<()> {
println!("{RUNS} random starts per function, rate {RATE}, cap {MAX_EPOCHS} epochs\n");
println!("function converged epochs (min/median/max) corrections (median/max)");
for function in ["AND", "OR", "NAND", "IMPL", "XOR"] {
let examples = truth_table(function);
let mut epochs = Vec::with_capacity(RUNS);
let mut corrections = Vec::with_capacity(RUNS);
let mut converged = 0;
for run in 0..RUNS {
let (weights, bias) = seeded(run);
let mut unit = Perceptron::new(weights, bias);
let (history, ok) = unit.train(&examples, RATE, MAX_EPOCHS);
if ok {
converged += 1;
epochs.push(history.last().map_or(1, |update| update.epoch));
corrections.push(history.len());
}
}
if converged == 0 {
println!("{function:<9} {converged:>4}/{RUNS} never — no separating line exists");
continue;
}
epochs.sort_unstable();
corrections.sort_unstable();
println!(
"{function:<9} {converged:>4}/{RUNS} {:>3} / {:>3} / {:<12} {:>3} / {}",
epochs[0],
epochs[epochs.len() / 2],
epochs[epochs.len() - 1],
corrections[corrections.len() / 2],
corrections[corrections.len() - 1]
);
}
Ok(())
}boundary/src/main.rs — 75 lines
//! Scene 03: what the two numbers mean.
//!
//! The trained weights are not a lookup table — they are a line. This
//! prints that line for several starting points, draws where it falls
//! between the four inputs, and shows that different runs settle on
//! different lines: the rule finds A solution, not THE solution.
use anyhow::Result;
use perceptron::{Perceptron, truth_table};
const RATE: f64 = 0.1;
/// Where the boundary crosses the unit square, as an ASCII picture.
/// Rows are b = 1 down to b = 0; a runs left to right.
fn picture(unit: &Perceptron) -> Vec<String> {
let mut rows = Vec::new();
for row in 0..11 {
let b = 1.0 - row as f64 / 10.0;
let mut line = String::new();
for column in 0..21 {
let a = column as f64 / 20.0;
line.push(if unit.answer([a, b]) == 1.0 { '#' } else { '.' });
}
rows.push(line);
}
rows
}
fn main() -> Result<()> {
let examples = truth_table("AND");
println!("three different starts, three different lines — all correct on AND\n");
for (index, start) in [([0.0, 0.0], 0.0), ([0.8, -0.6], 0.3), ([-0.5, 0.9], -0.7)]
.into_iter()
.enumerate()
{
let mut unit = Perceptron::new(start.0, start.1);
let (history, _) = unit.train(&examples, RATE, 100);
println!(
"start {index}: w = [{:>5.1}, {:>5.1}], bias {:>5.1} -> after {} corrections: \
{:.2}·a {:+.2}·b {:+.2} = 0 ({} of 4 correct)",
start.0[0],
start.0[1],
start.1,
history.len(),
unit.weights[0],
unit.weights[1],
unit.bias,
unit.correct_count(&examples)
);
}
// The margin: how far each example sits from the boundary.
let mut unit = Perceptron::new([0.0, 0.0], 0.0);
unit.train(&examples, RATE, 100);
println!("\nsigned distance of every example from the learned line:");
for (input, target) in &examples {
let score = unit.weights[0] * input[0] + unit.weights[1] * input[1] + unit.bias;
println!(
" ({:.0}, {:.0}) score {score:+.2} -> answers {} (target {})",
input[0],
input[1],
unit.answer(*input) as u8,
*target as u8
);
}
println!("\nthe same boundary drawn over the unit square (# = answers 1):");
for (row, line) in picture(&unit).into_iter().enumerate() {
let b = 1.0 - row as f64 / 10.0;
println!(" b={b:.1} {line}");
}
println!(" a=0.0 a=1.0");
Ok(())
}