Digital Signal Processing & VOLK

When we say that a machine “knows” something, we often mean that it can use information to make a better prediction. That definition is modest, but it is more useful than it first appears.

Consider a small system that estimates the position of a moving object. At each time step, it receives a measurement, updates an internal estimate, and chooses what to do next. In pseudocode, the shape might be:

estimate = initial_state

for observation in observations:
    estimate = update(estimate, observation)
    action = choose_action(estimate)
    act(action)

The important object here is not the observation by itself. It is estimate: a compressed history of the world that the system carries forward. The estimate can be wrong and still be useful. It only has to preserve the distinctions that matter for the next action.

A representation is useful when it makes the right differences visible.

This is why the same raw data can support very different kinds of intelligence. A camera image might be represented as pixels, objects, trajectories, or affordances. Each representation throws information away. Each also makes some future decisions easier.

The cost of remembering

There is a temptation to treat more memory as automatically better. But remembering everything is not the same as understanding anything. A system with unlimited history still needs to decide what to use.

One way to make this precise is to imagine that the full history is $H_t = (y_0, y_1, \ldots, y_t)$, while the machine acts through a smaller state $z_t$. The update has the form

$$ z_{t+1} = g(z_t, y_{t+1}). $$

The design problem is to choose $g$ so that $z_t$ is small enough to work with but rich enough to predict what matters. In other words, the machine needs a sufficient summary, not a complete transcript.

A practical test

When designing a system, I would ask:

• What distinctions must the representation preserve?
• What can safely be forgotten?
• Which errors are reversible, and which are not?

These questions are more concrete than asking whether the machine is intelligent. They give us something to build, measure, and revise.