Steps: 0
Looking for Sugar
Heading Home
The ants start out just wandering around the board. If they find the sugar, they receive a reward. That reward is echoed back along the path they took, updating the value of those locations. By favoring locations with higher values, the ants will be more likely to move towards the goal (and so earn more rewards and reinforce the path).
The heat maps at the bottom of the screen display the currently value functions. Ants will be more likely to move towards the high value "green" locations.
Once the ants have converged on a path to the goal, try clicking on the board to move the Sugar (or the Home). The ant movement and their value maps should update as they "forget" the old location and learn the path to the new one.
Use the controls to play with the number of ants, their speed, the size of their reward, and the various TD(λ) learning factors. For example, by adjusting Exploration vs. Exploitation, you can control whether the ants are locked in to the best path they have found so far vs. how much they explore to see if there is a better way..
It's interesting to note how the path to the sugar and the path back are not the same. The paths are avoiding each other so that the ants don't bump into each other too often. There is no rule telling them to do this. It's just an emergent behavior.