There's a tension between complexity of your model and accuracy.

The more complexity you add, the more you overfit.

You explain what you see well, but also over-explain noise.

You're now overfit, poorly fit to novel inputs.

If you update your model by n bits it had better give you more than n bits of accuracy or you're falling behind.