So, what...it's a magnetic compass hidden inside of a figure, right? Nope. Ok...so it's a gyroscope of some kind then? That's not it either. These South Pointing Chariots use complex gear arrangements to adjust the figure's orientation depending on the motion of the wheels.
Here's a great web site with tons of info about South Facing Chariots. Of particular note is the functional models page showing working examples made from everything from brass, to wood, to LEGOs! So cool.