You are looking for Solomonff Induction, the mathematical description of a perfect learning algorithm.
The TLDR is you do Bayesian Inference over the set of all programs, with prior probabilities as 2^(-K(p)) where K(p) is the length of a program p. You can prove that this method has a lower expected generalization error than all programs.
You are looking for Solomonff Induction, the mathematical description of a perfect learning algorithm.
The TLDR is you do Bayesian Inference over the set of all programs, with prior probabilities as 2^(-K(p)) where K(p) is the length of a program p. You can prove that this method has a lower expected generalization error than all programs.