Machine Learning: 2 Books in 1: Machine Learning for Beginners, Machine Learning Mathematics. An Introduction Guide to Understand Data Science Through the Business Application

h:F (h)≥c
 F
S
(h) − m→∞→ inf 
h:F (h)≥c
 F(h)”
The thought is that the empirical risk of “ERM” is essential for convergence
of the smallest potential risk, even once the “good” hypotheses with less
risk than the threshold have been removed. Vapnik succeeded in proving
that such “strict consistency” of the ERM has real equivalence to uniform
convergence, of the form in probability. The equivalence below is true for
every individual distribution and is independent of the universal consistency
of the “Empirical Risk Minimizer”.
“sup 
h
∈
H 
( F ( h ) − F
S
 ( h ) ) − m→∞→ 0”
Based on this research study, it can be implied that “up to trivial situations,
a uniform convergence property indeed characterizes learnability, at least
using the ERM learning rule”.

---

Learnability without Uniform Convergence
A “stochastic convex optimization” or learnability without uniform
convergence problem can be considered an exceptional case of the “General
Learning Setting” explained above, including additional limitations that
“the objective function f(h;z) is Lipschitz-continuous and convex in h for
every z, and that H is closed, convex and bounded”. The problems where
“H” is a subset of a “Hilbert space” will be addressed here. An exceptional
scenario is “the familiar linear prediction setting, where z = (x, y) is an
instance-label pair, each hypothesis h belongs to a subset H of a Hilbert
space, and f (h; x, y) = l(
⟨
h, φ(x)
⟩
, y) for some feature mapping φ and a
loss function l : R × Y → R, which is convex with respect to its first
argument”.
The scenario has been successfully established where the stochastic
dependence on “h” is linear, similar to the previous example, has been
established successfully. When the “domain H” and the “mapping φ” is
bounded, there is uniform convergence, meaning that “|F (h) − F
S
 (h)|” is
uniformly bounded overall “h 
∈
H”. This uniform convergence of “F
S
(h)
to F(h)” validates selection of the empirical minimizer “hˆS =arg min