I am a new driver. I learned to drive at two driving schools. The first one taught me well, but the way they taught parking was horrendous. The second one was better. But not enough, sadly.
The parking spot in my society is very tight. Pillar on the left. Neighbour’s car on the right. And very little space in front. I had to practise for almost 5–6 hours, split across two Sundays, and only after that could I do it. After 15–20 attempts, it has almost become muscle memory.
Move the car forward while turning fully left towards the front pillar. Start reversing. The neighbour’s car is visible in the right mirror… You know. Regular parking drills. Sometimes I cut it too close. But 9 times out of 10, it is quite easy now.
Last week, I took the car to the gym because it was raining. The spot was twice as wide, and there were no other cars. It took me 3 attempts, and the car still ended up slightly crooked.
This bothered me more than it should have.
There is an obvious explanation for this: I am just used to my spot. But my spot is the hardest one I have come across. If I can do the hardest one, shouldn’t the easy ones be trivial? That’s how we often assume things work in real life: train on difficult things, so that when the easier thing comes, you can handle it easily.
But it didn’t work that way. And in my line of work, this has a particular name.
The kid who knows 2×3
Imagine a kid who is learning his multiplication tables. He knows what 2×2 is: 4. Even 2×3: 6. 2×8: 16. But ask him what 3×3 is, and he fumbles. He never learnt the logic of multiplication, only the answers.
In statistical modelling (or Machine Learning, as it is called for PR purposes), there is something called overfitting. It’s when a model gets very good at the examples it was trained on, but fails miserably on a new dataset. It basically “remembers” the outputs, but doesn’t “understand” the logic that produces them.
The neighbour’s car & the pillar
What I realised is that while parking at my society, I am not really judging the space. I am reading cues. I know what the pillar should look like before I start turning, and where my neighbour’s car should be before I start reversing in. That’s why I can do it with ease.
ML has a name for this too: shortcut learning. The famous example is a model trained to distinguish huskies from wolves1. As it turned out, it was just looking for a snowy background, because the pictures of the wolves almost always had snow in them. It learnt the background successfully, not the animal.
Similarly, I suspect that if my neighbour takes her car out of town, I might struggle in my relatively spacious parking spot.
Hard is not the same as varied
Difficulty is important for attaining excellence. But it needs some variation; otherwise, it fails.
Mix up your drills with different types of problems, and you will look sloppier in practice. But when the time comes, you will be better prepared. Doing the same relatively hard thing 1,000 times may not be as fruitful.
My society spot is that textbook drill. One very hard problem, which I have solved around a thousand times. It made me brilliant at that problem and roughly average at the category.
This analogy isn’t perfect. Overfit models fail completely on new data. But I could still park. The overfitting might be less in my driving and more in how I judge my driving.
My home spot is my training data, and I have been using it to estimate how good I am. Which is overfitting, of some sort.
I could fix this (in theory, at least). Keep parking in tight spots. Let some random security guard watch me take 4 attempts. That should eventually help. But I haven’t started.
For the time being, I am happy being a good parker of one very tight spot, rather than a good parker.
Observations, data, and the not-so-ordinary world. Subscribe if that's your kind of thing.
If you found this interesting, I would love to hear your thoughts.
Ribeiro, Singh and Guestrin, "Why Should I Trust You?" (2016). They trained the snow-detecting classifier badly on purpose, to see whether people would notice. Most didn't, which is its own blog post.