Is it better to maximize the complexity always to get to the bayes error with the data we have (if we can’t have more data) or not necessarily?
As mentioned in the Overfitting and underfitting video, a model that is too complex may capture noise. Complexity in general should be tuned along with the underfitting-overfitting trade-off.