Tag: Neural-Networks

Following up on A Fruit Fly Circuit as a Speech Emotion Reservoir: I pulled the real 499-neuron circuit out of MaleCNS v1.0 and ran it as a reservoir on MNIST, in PyTorch and in MLX. Code and data: gist.

Getting the real circuit

MaleCNS v1.0 is a public, CC-BY electron-microscopy reconstruction of a male fruit fly’s central nervous system.

I applied the selection rule stated in the original article: central-brain intrinsic neurons, traced status, ranked by strength, top 512, largest strongly connected component, edges with at least 5 synaptic contacts. This landed on exactly 499 neurons — the same count the paper reports — with 13,452 directed edges, against their reported 15,865. Their exact ranking formula for “strongest” isn’t published, so this is the closest match I could reproduce.

[…634 words]

We taught a fruit fly to hear human emotion uses a 499-neuron circuit from the male fruit fly brain and central nervous system, mapped by FlyEM at Janelia with Cambridge, the MRC Laboratory of Molecular Biology, and Google Research.

The circuit is fully connected: every neuron can reach every other neuron through a directed path, across 15,865 connections built from 867,344 synaptic contacts.

The team wired this circuit into a speech emotion classifier using reservoir computing and an identify human emotion like happy, angry from voice inputs.

I built micromlp: a single file of Python, no dependencies, no PyTorch. It downloads a real dataset, builds a 2-layer MLP, implements automatic differentiation from scratch, trains with gradient descent, and makes predictions. I use the California housing dataset from chapter 2 of Hands-On Machine Learning. The task: predict a district’s median house value from its census stats.

This is inspired by Karpathy’s microgpt. There’s a difference between knowing .backward() exists and knowing what it does when you call it. I found it worth closing that gap once, on a toy.

[…1482 words]

A partial derivative measures how much a function changes when you nudge one input, holding every other input fixed. For f(x, y) = x²y + y + 2, the partial derivative with respect to x asks: if y stays put, how fast does f move as x moves?

Manual differentiation

Mathematically, we know that ∂f/∂x = 2xy and ∂f/∂y = x² + 1, using a handful of rules:

• the derivative of a constant is 0
• the derivative of ax is a
• the derivative of x^a is a·x^(a-1)
• the derivative of a sum is the sum of the derivatives: (u + v)' = u' + v'
• the derivative of a product follows the product rule: (u·v)' = u'v + uv'

But how can a program know that?

[…913 words]

A neural network is just a math function: y = FNN(x).

FNN has a nested form. Think of it as a stack of layers. A 3-layer neural network that returns a scalar value looks like this:

y = FNN(x) = f₃(f₂(f₁(x)))

Each f — f₁, f₂, … fₙ — has the same form:

f(x) = g(Wx + b)

W (the weight matrix) and b (a bias vector) are the learned parameters, usually trained via gradient descent. g is the activation function, and it can be chosen differently for each layer.

Wx + b is linear — wrapping it in g is what makes each layer non-linear. Without g (or with g chosen to be linear), the whole FNN collapses into a single linear function: stack 100 such layers and the composition of linear maps is still just one linear map, no matter how deep the network looks. g is what lets a stack of layers approximate anything more than a straight line. Popular choices for g are sigmoid and ReLU.

There are many variants of neural networks — CNNs, RNNs, transformers, and more — each shaped by assumptions about the data they process. The example above, where every neuron in one layer connects to every neuron in the next, is the plainest of them: a multilayer perceptron (MLP), also called a vanilla neural network.