3  Relevant Results

$$ \newcommand{\LetThereBe}[2]{\newcommand{#1}{#2}} \newcommand{\letThereBe}[3]{\newcommand{#1}[#2]{#3}} \newcommand{\ForceToBe}[2]{\renewcommand{#1}{#2}} \newcommand{\forceToBe}[3]{\renewcommand{#1}[#2]{#3}} \newcommand{\MayThereBe}[2]{\newcommand{#1}{#2}} \newcommand{\mayThereBe}[3]{\newcommand{#1}[#2]{#3}} % Declare mathematics (so they can be overwritten for PDF) \newcommand{\declareMathematics}[2]{\DeclareMathOperator{#1}{#2}} \newcommand{\declareMathematicsStar}[2]{\DeclareMathOperator*{#1}{#2}} % striked integral \newcommand{\avint}{\mathop{\mathchoice{\,\rlap{-}\!\!\int} {\rlap{\raise.15em{\scriptstyle -}}\kern-.2em\int} {\rlap{\raise.09em{\scriptscriptstyle -}}\!\int} {\rlap{-}\!\int}}\nolimits} % \d does not work well for PDFs \LetThereBe{\d}{\differential} \LetThereBe{\Im}{\IM} \LetThereBe{\Re}{\RE} \LetThereBe{\Cpls}{\mathrm{C{\scriptsize PL}}} \LetThereBe{\OptCpls}{\mathrm{O{\scriptsize pt}}} \letThereBe{\linefrac}{2}{#1/#2} \LetThereBe{\ExtProd}{\mathsf{\Lambda}} \letThereBe{\unicodeInt}{1}{\mathop{\vcenter{\mathchoice{\huge\unicode{#1}}{\unicode{#1}}{\unicode{#1}}{\unicode{#1}}}}\nolimits} \letThereBe{\Oiint}{1}{\underset{ #1 \;}{ {\rlap{\mspace{1mu} \boldsymbol{\bigcirc}}{\rlap{\int}{\;\int}}} }} \letThereBe{\sOiint}{1}{\unicodeInt{x222F}_{#1}} $$ $$ % Simply for testing \LetThereBe{\foo}{\textrm{FIXME: this is a test!}} % Font styles \letThereBe{\mcal}{1}{\mathcal{#1}} \letThereBe{\chem}{1}{\mathrm{#1}} % Sets \LetThereBe{\C}{\mathbb{C}} \LetThereBe{\R}{\mathbb{R}} \LetThereBe{\Z}{\mathbb{Z}} \LetThereBe{\N}{\mathbb{N}} \LetThereBe{\Q}{\mathbb{Q}} \LetThereBe{\K}{\mathbb{K}} \LetThereBe{\im}{\mathrm{i}} % Sets from PDEs \LetThereBe{\boundaryOf}{\partial} \letThereBe{\closureOf}{1}{\overline{#1}} \letThereBe{\Contf}{1}{\mcal C^{#1}} \letThereBe{\contf}{2}{\Contf{#2}(#1)} \letThereBe{\compactContf}{2}{\mcal C_c^{#2}(#1)} \letThereBe{\ball}{2}{B\brackets{#1, #2}} \letThereBe{\closedBall}{2}{B\parentheses{#1, #2}} \LetThereBe{\compactEmbed}{\subset\subset} \letThereBe{\inside}{1}{#1^o} \LetThereBe{\neighborhood}{\mcal O} \letThereBe{\neigh}{1}{\neighborhood \brackets{#1}} % Basic notation - vectors and random variables \letThereBe{\vi}{1}{\boldsymbol{#1}} %vector or matrix \letThereBe{\dvi}{1}{\vi{\dot{#1}}} %differentiated vector or matrix \letThereBe{\vii}{1}{\mathbf{#1}} %if \vi doesn't work \letThereBe{\dvii}{1}{\vii{\dot{#1}}} %if \dvi doesn't work \letThereBe{\rnd}{1}{\mathup{#1}} %random variable \letThereBe{\vr}{1}{\mathbf{#1}} %random vector or matrix \letThereBe{\vrr}{1}{\boldsymbol{#1}} %random vector if \vr doesn't work \letThereBe{\dvr}{1}{\vr{\dot{#1}}} %differentiated vector or matrix \letThereBe{\vb}{1}{\pmb{#1}} %#TODO \letThereBe{\dvb}{1}{\vb{\dot{#1}}} %#TODO \letThereBe{\oper}{1}{\mathsf{#1}} \letThereBe{\quotient}{2}{{^{\displaystyle #1}}/{_{\displaystyle #2}}} % Basic notation - general \letThereBe{\set}{1}{\left\{#1\right\}} \letThereBe{\seqnc}{4}{\set{#1_{#2}}_{#2 = #3}^{#4}} \letThereBe{\Seqnc}{3}{\set{#1}_{#2}^{#3}} \letThereBe{\brackets}{1}{\left( #1 \right)} \letThereBe{\parentheses}{1}{\left[ #1 \right]} \letThereBe{\dom}{1}{\mcal{D}\, \brackets{#1}} \letThereBe{\complexConj}{1}{\overline{#1}} \LetThereBe{\divider}{\; \vert \;} \LetThereBe{\gets}{\leftarrow} \letThereBe{\rcases}{1}{\left.\begin{aligned}#1\end{aligned}\right\}} \letThereBe{\rcasesAt}{2}{\left.\begin{alignedat}{#1}#2\end{alignedat}\right\}} \letThereBe{\lcases}{1}{\begin{cases}#1\end{cases}} \letThereBe{\lcasesAt}{2}{\left\{\begin{alignedat}{#1}#2\end{alignedat}\right.} \letThereBe{\evaluateAt}{2}{\left.#1\right|_{#2}} \LetThereBe{\Mod}{\;\mathrm{mod}\;} \LetThereBe{\bigO}{O} \letThereBe{\BigO}{1}{\bigO\brackets{#1}} % Special symbols \LetThereBe{\const}{\mathrm{const}} \LetThereBe{\konst}{\mathrm{konst.}} \LetThereBe{\vf}{\varphi} \LetThereBe{\ve}{\varepsilon} \LetThereBe{\tht}{\theta} \LetThereBe{\Tht}{\Theta} \LetThereBe{\after}{\circ} \LetThereBe{\lmbd}{\lambda} \LetThereBe{\Lmbd}{\Lambda} % Shorthands \LetThereBe{\xx}{\vi x} \LetThereBe{\yy}{\vi y} \LetThereBe{\AA}{\vi A} \LetThereBe{\bb}{\vi b} \LetThereBe{\vvf}{\vi \vf} \LetThereBe{\ff}{\vi f} \LetThereBe{\gg}{\vi g} % Basic functions \letThereBe{\absval}{1}{\left| #1 \right|} \LetThereBe{\id}{\mathrm{id}} \letThereBe{\floor}{1}{\left\lfloor #1 \right\rfloor} \letThereBe{\ceil}{1}{\left\lceil #1 \right\rceil} \declareMathematics{\image}{im} %image \declareMathematics{\domain}{dom} %image \declareMathematics{\tg}{tg} \declareMathematics{\sign}{sign} \declareMathematics{\card}{card} %cardinality \letThereBe{\setSize}{1}{\left| #1 \right|} \LetThereBe{\countElements}{\#} \declareMathematics{\exp}{exp} \letThereBe{\Exp}{1}{\exp\brackets{#1}} \LetThereBe{\ee}{\mathrm{e}} \letThereBe{\indicator}{1}{\mathbb{I}_{#1}} \declareMathematics{\arccot}{arccot} \declareMathematics{\gcd}{gcd} % Greatest Common Divisor \declareMathematics{\lcm}{lcm} % Least Common Multiple \declareMathematics{\epi}{epi} %epigraph of a function \letThereBe{\limInfty}{1}{\lim_{#1 \to \infty}} \letThereBe{\limInftyM}{1}{\lim_{#1 \to -\infty}} % Useful commands \letThereBe{\onTop}{2}{\mathrel{\overset{#2}{#1}}} \letThereBe{\onBottom}{2}{\mathrel{\underset{#2}{#1}}} \letThereBe{\tOnTop}{2}{\mathrel{\overset{\text{#2}}{#1}}} \letThereBe{\tOnBottom}{2}{\mathrel{\underset{\text{#2}}{#1}}} \LetThereBe{\EQ}{\onTop{=}{!}} \LetThereBe{\letDef}{:=} %#TODO: change the symbol \LetThereBe{\isPDef}{\onTop{\succ}{?}} \LetThereBe{\inductionStep}{\tOnTop{=}{induct. step}} \LetThereBe{\fromDef}{\triangleq} % Optimization \declareMathematicsStar{\argmin}{argmin} \declareMathematicsStar{\argmax}{argmax} \letThereBe{\maxOf}{1}{\max\set{#1}} \letThereBe{\minOf}{1}{\min\set{#1}} \declareMathematics{\prox}{prox} \declareMathematics{\loss}{loss} \declareMathematics{\supp}{supp} \letThereBe{\Supp}{1}{\supp\brackets{#1}} \LetThereBe{\constraint}{\text{s.t.}\;} $$ $$ % Operators - Analysis \LetThereBe{\hess}{\nabla^2} \LetThereBe{\lagr}{\mcal L} \LetThereBe{\lapl}{\Delta} \declareMathematics{\grad}{grad} \declareMathematics{\divergence}{div} \declareMathematics{\Dgrad}{D} \LetThereBe{\gradient}{\nabla} \LetThereBe{\jacobi}{\nabla} \LetThereBe{\Jacobi}{\vi{\mathrm J}} \letThereBe{\jacobian}{2}{\Dgrad_{#1}\brackets{#2}} \LetThereBe{\d}{\mathrm{d}} \LetThereBe{\dd}{\,\mathrm{d}} \letThereBe{\partialDeriv}{2}{\frac {\partial #1} {\partial #2}} \letThereBe{\npartialDeriv}{3}{\partialDeriv{^{#1} #2} {#3^{#1}}} \letThereBe{\partialOp}{1}{\frac {\partial} {\partial #1}} \letThereBe{\npartialOp}{2}{\frac {\partial^{#1}} {\partial #2^{#1}}} \letThereBe{\pDeriv}{2}{\partialDeriv{#1}{#2}} \letThereBe{\npDeriv}{3}{\npartialDeriv{#1}{#2}{#3}} \letThereBe{\deriv}{2}{\frac {\d #1} {\d #2}} \letThereBe{\nderiv}{3}{\frac {\d^{#1} #2} {\d #3^{#1}}} \letThereBe{\derivOp}{1}{\frac {\d} {\d #1}\,} \letThereBe{\nderivOp}{2}{\frac {\d^{#1}} {\d #2^{#1}}\,} % Convergence \LetThereBe{\pointwiseTo}{\to} \LetThereBe{\uniformlyTo}{\rightrightarrows} \LetThereBe{\normallyTo}{\tOnTop{\longrightarrow}{norm}} \LetThereBe{\compactlyTo}{\tOnTop{\longrightarrow}{comp.}} \LetThereBe{\locallyUnifTo}{\tOnTop{\longrightarrow}{l.u.}} % Curves \letThereBe{\graphOf}{1}{\parentheses{#1}} \declareMathematics{\interior}{Int} % complex \LetThereBe{\Cinfty}{\tilde{\C}} \declareMathematics{\residual}{res} \letThereBe{\resAt}{1}{\residual_{#1}} \declareMathematics{\complexarg}{arg} \declareMathematics{\complexArg}{Arg} \LetThereBe{\carg}{\complexarg} \LetThereBe{\cArg}{\complexArg} \LetThereBe{\IM}{\mathfrak{Im}} \LetThereBe{\RE}{\mathfrak{Re}} \letThereBe{\imOf}{1}{\IM\,#1} \letThereBe{\reOf}{1}{\RE\,#1} \letThereBe{\ImOf}{1}{\IM \brackets{#1}} \letThereBe{\ReOf}{1}{\RE \brackets{#1}} % Convex \letThereBe{\cconj}{1}{#1^*} $$ $$ % Linear algebra \letThereBe{\norm}{1}{\left\lVert #1 \right\rVert} \letThereBe{\seminorm}{1}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert #1 \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \letThereBe{\scal}{2}{\left\langle #1, #2 \right\rangle} \letThereBe{\avg}{1}{\overline{#1}} \letThereBe{\Avg}{1}{\bar{#1}} \letThereBe{\linspace}{1}{\mathrm{lin}\set{#1}} \letThereBe{\algMult}{1}{\mu_{\mathrm A} \brackets{#1}} \letThereBe{\geomMult}{1}{\mu_{\mathrm G} \brackets{#1}} \LetThereBe{\Nullity}{\mathrm{nullity}} \letThereBe{\nullity}{1}{\Nullity \brackets{#1}} \LetThereBe{\nulty}{\nu} \declareMathematics{\SpanOf}{span} \letThereBe{\Span}{1}{\SpanOf\set{#1}} \LetThereBe{\projection}{\Pi} % Linear algebra - Matrices \LetThereBe{\tr}{\top} \LetThereBe{\Tr}{^\tr} \LetThereBe{\pinv}{\dagger} \LetThereBe{\Pinv}{^\dagger} \LetThereBe{\Inv}{^{-1}} \LetThereBe{\ident}{\vi{I}} \letThereBe{\mtr}{1}{\begin{pmatrix}#1\end{pmatrix}} \letThereBe{\bmtr}{1}{\begin{bmatrix}#1\end{bmatrix}} \declareMathematics{\trace}{tr} \declareMathematics{\diagonal}{diag} \declareMathematics{\rank}{rank} % Multilinear algebra \LetThereBe{\tensorProd}{\otimes} \LetThereBe{\tprod}{\tensorProd} \LetThereBe{\extProd}{\wedge} \LetThereBe{\wdg}{\extProd} \LetThereBe{\wedges}{\wedge \dots \wedge} \declareMathematics{\altMap}{Alt} $$ $$ % Statistics \LetThereBe{\iid}{\overset{\text{i.i.d.}}{\sim}} \LetThereBe{\ind}{\overset{\text{ind}}{\sim}} \LetThereBe{\condp}{\,\vert\,} \letThereBe{\complementOf}{1}{{#1^{\mathrm{c}}}} \LetThereBe{\acov}{\gamma} \LetThereBe{\acf}{\rho} \LetThereBe{\stdev}{\sigma} \LetThereBe{\procMean}{\mu} \LetThereBe{\procVar}{\stdev^2} \declareMathematics{\variance}{var} \letThereBe{\Variance}{1}{\variance \brackets{#1}} \declareMathematics{\cov}{cov} \declareMathematics{\corr}{cor} \letThereBe{\sampleVar}{1}{\rnd S^2_{#1}} \letThereBe{\populationVar}{1}{V_{#1}} \declareMathematics{\expectedValue}{\mathbb{E}} \declareMathematics{\rndMode}{Mode} \letThereBe{\RndMode}{1}{\rndMode\brackets{#1}} \letThereBe{\expect}{1}{\expectedValue #1} \letThereBe{\Expect}{1}{\expectedValue \brackets{#1}} \letThereBe{\expectIn}{2}{\expectedValue_{#1} #2} \letThereBe{\ExpectIn}{2}{\expectedValue_{#1} \brackets{#2}} \LetThereBe{\betaF}{\mathrm B} \LetThereBe{\fisherMat}{J} \LetThereBe{\mutInfo}{I} \LetThereBe{\expectedGain}{I_e} \letThereBe{\KLDiv}{2}{D\brackets{#1 \parallel #2}} \LetThereBe{\entropy}{H} \LetThereBe{\diffEntropy}{h} \LetThereBe{\probF}{\pi} \LetThereBe{\densF}{\vf} \LetThereBe{\att}{_t} %at time \letThereBe{\estim}{1}{\hat{#1}} \letThereBe{\estimML}{1}{\hat{#1}_{\mathrm{ML}}} \letThereBe{\estimOLS}{1}{\hat{#1}_{\mathrm{OLS}}} \letThereBe{\estimMAP}{1}{\hat{#1}_{\mathrm{MAP}}} \letThereBe{\predict}{3}{\estim {\rnd #1}_{#2 | #3}} \letThereBe{\periodPart}{3}{#1+#2-\ceil{#2/#3}#3} \letThereBe{\infEstim}{1}{\tilde{#1}} \letThereBe{\predictDist}{1}{{#1}^*} \LetThereBe{\backs}{\oper B} \LetThereBe{\diff}{\oper \Delta} \LetThereBe{\BLP}{\oper P} \LetThereBe{\arPoly}{\Phi} \letThereBe{\ArPoly}{1}{\arPoly\brackets{#1}} \LetThereBe{\maPoly}{\Theta} \letThereBe{\MaPoly}{1}{\maPoly\brackets{#1}} \letThereBe{\ARmod}{1}{\mathrm{AR}\brackets{#1}} \letThereBe{\MAmod}{1}{\mathrm{MA}\brackets{#1}} \letThereBe{\ARMA}{2}{\mathrm{ARMA}\brackets{#1, #2}} \letThereBe{\sARMA}{3}{\mathrm{ARMA}\brackets{#1}\brackets{#2}_{#3}} \letThereBe{\SARIMA}{3}{\mathrm{ARIMA}\brackets{#1}\brackets{#2}_{#3}} \letThereBe{\ARIMA}{3}{\mathrm{ARIMA}\brackets{#1, #2, #3}} \LetThereBe{\pacf}{\alpha} \letThereBe{\parcorr}{3}{\rho_{#1 #2 | #3}} \LetThereBe{\noise}{\mathscr{N}} \LetThereBe{\jeffreys}{\mathcal J} \LetThereBe{\likely}{\mcal L} \letThereBe{\Likely}{1}{\likely\brackets{#1}} \LetThereBe{\loglikely}{\mcal l} \letThereBe{\Loglikely}{1}{\loglikely \brackets{#1}} \LetThereBe{\CovMat}{\Gamma} \LetThereBe{\covMat}{\vi \CovMat} \LetThereBe{\rcovMat}{\vrr \CovMat} \LetThereBe{\AIC}{\mathrm{AIC}} \LetThereBe{\BIC}{\mathrm{BIC}} \LetThereBe{\AICc}{\mathrm{AIC}_c} \LetThereBe{\nullHypo}{H_0} \LetThereBe{\altHypo}{H_1} \LetThereBe{\rve}{\rnd \ve} \LetThereBe{\rtht}{\rnd \theta} \LetThereBe{\rX}{\rnd X} \LetThereBe{\rY}{\rnd Y} \LetThereBe{\rZ}{\rnd Z} \LetThereBe{\rA}{\rnd A} \LetThereBe{\rB}{\rnd B} \LetThereBe{\rU}{\rnd U} \LetThereBe{\vrZ}{\vr Z} \LetThereBe{\vrY}{\vr Y} \LetThereBe{\vrX}{\vr X} \LetThereBe{\vrU}{\vr U} \LetThereBe{\rW}{\rnd W} \LetThereBe{\rS}{\rnd S} \LetThereBe{\rM}{\rnd M} \LetThereBe{\rtau}{\rnd \tau} % Bayesian inference \LetThereBe{\paramSet}{\mcal T} \LetThereBe{\sampleSet}{\mcal Y} \LetThereBe{\bayesSigmaAlg}{\mcal B} % Different types of convergence \LetThereBe{\inDist}{\onTop{\to}{d}} \letThereBe{\inDistWhen}{1}{\onBottom{\onTop{\longrightarrow}{d}}{#1}} \LetThereBe{\inProb}{\onTop{\to}{P}} \letThereBe{\inProbWhen}{1}{\onBottom{\onTop{\longrightarrow}{P}}{#1}} \LetThereBe{\inMeanSq}{\onTop{\to}{\ltwo}} \LetThereBe{\inltwo}{\onTop{\to}{\ltwo}} \letThereBe{\inMeanSqWhen}{1}{\onBottom{\onTop{\longrightarrow}{\ltwo}}{#1}} \LetThereBe{\convergeAS}{\tOnTop{\to}{a.s.}} \letThereBe{\convergeASWhen}{1}{\onBottom{\tOnTop{\longrightarrow}{a.s.}}{#1}} % Asymptotic qualities \LetThereBe{\simAsymp}{\tOnTop{\sim}{as.}} % Stochastic analysis \letThereBe{\diffOn}{2}{\diff #1_{[#2]}} % \LetThereBe{\timeSet}{\Theta} \LetThereBe{\eventSet}{\Omega} \LetThereBe{\filtration}{\mcal F} % TODO: Rename allFiltrations and the like \letThereBe{\allFiltrations}{1}{\set{\filtration_t}_{#1}} \letThereBe{\natFilter}{1}{\filtration_t^{#1}} \letThereBe{\NatFilter}{2}{\filtration_{#2}^{#1}} \letThereBe{\filterAll}{1}{\set{#1}_{t \geq 0}} \letThereBe{\FilterAll}{2}{\set{#1}_{#2}} \LetThereBe{\borelAlgebra}{\mcal B} \LetThereBe{\sAlgebra}{\mcal A} \LetThereBe{\quadVar}{Q} \LetThereBe{\totalVar}{V} \LetThereBe{\adaptIntProcs}{\mcal M} \letThereBe{\reflectProc}{2}{#1^{#2}} $$ $$ % Distributions \letThereBe{\WN}{2}{\mathrm{WN}\brackets{#1,#2}} \declareMathematics{\uniform}{Unif} \declareMathematics{\binomDist}{Bi} \declareMathematics{\negbinomDist}{NBi} \declareMathematics{\betaDist}{Beta} \declareMathematics{\betabinomDist}{BetaBin} \declareMathematics{\gammaDist}{Gamma} \declareMathematics{\igammaDist}{IGamma} \declareMathematics{\invgammaDist}{IGamma} \declareMathematics{\expDist}{Ex} \declareMathematics{\poisDist}{Po} \declareMathematics{\erlangDist}{Er} \declareMathematics{\altDist}{A} \declareMathematics{\geomDist}{Ge} \LetThereBe{\normalDist}{\mathcal N} %\declareMathematics{\normalDist}{N} \letThereBe{\normalD}{1}{\normalDist \brackets{#1}} \letThereBe{\mvnormalD}{2}{\normalDist_{#1} \brackets{#2}} \letThereBe{\NormalD}{2}{\normalDist \brackets{#1, #2}} \LetThereBe{\lognormalDist}{\log\normalDist} $$ $$ % Game Theory \LetThereBe{\doms}{\succ} \LetThereBe{\isdom}{\prec} \letThereBe{\OfOthers}{1}{_{-#1}} \LetThereBe{\ofOthers}{\OfOthers{i}} \LetThereBe{\pdist}{\sigma} \letThereBe{\domGame}{1}{G_{DS}^{#1}} \letThereBe{\ratGame}{1}{G_{Rat}^{#1}} \letThereBe{\bestRep}{2}{\mathrm{BR}_{#1}\brackets{#2}} \letThereBe{\perf}{1}{{#1}_{\mathrm{perf}}} \LetThereBe{\perfG}{\perf{G}} \letThereBe{\imperf}{1}{{#1}_{\mathrm{imp}}} \LetThereBe{\imperfG}{\imperf{G}} \letThereBe{\proper}{1}{{#1}_{\mathrm{proper}}} \letThereBe{\finrep}{2}{{#2}_{#1{\text -}\mathrm{rep}}} %T-stage game \letThereBe{\infrep}{1}{#1_{\mathrm{irep}}} \LetThereBe{\repstr}{\tau} %strategy in a repeated game \LetThereBe{\emptyhist}{\epsilon} \letThereBe{\extrep}{1}{{#1^{\mathrm{rep}}}} \letThereBe{\avgpay}{1}{#1^{\mathrm{avg}}} \LetThereBe{\succf}{\pi} %successor function \LetThereBe{\playf}{\rho} %player function \LetThereBe{\actf}{\chi} %action function $$ $$ \LetThereBe{\fourierOp}{\mcal{F}} \letThereBe{\fourier}{1}{\widehat{#1}} \letThereBe{\ifourier}{1}{\check{#1}} % Shortcuts \letThereBe{\FT}{1}{\fourier{#1}} \letThereBe{\iFT}{1}{\ifourier{#1}} \LetThereBe{\FTOp}{\fourierOp} \LetThereBe{\XX}{\mcal X} \LetThereBe{\YY}{\mcal Y} \LetThereBe{\cc}{\mathsf{c}} \LetThereBe{\lspace}{\mcal L} \LetThereBe{\lone}{\lspace^{1}} \letThereBe{\Lone}{1}{\lone\brackets{#1}} \LetThereBe{\ltwo}{\lspace^2} \letThereBe{\Ltwo}{1}{\ltwo\brackets{#1}} \letThereBe{\lp}{1}{\lspace^{#1}} \letThereBe{\Lp}{2}{\lp{#1}\brackets{#2}} \LetThereBe{\linfty}{\lspace^{\infty}} \letThereBe{\Linfty}{1}{\linfty\brackets{#1}} \LetThereBe{\ltwoEq}{\onTop{=}{\ltwo}} \letThereBe{\decayContf}{1}{\mcal C_0\brackets{#1}} \letThereBe{\cinftyContf}{1}{\mcal C^{\infty}_c\brackets{#1}} \LetThereBe{\bddMeasFs}{\mcal M_{\mathrm{b}}} \LetThereBe{\contBddFs}{\mcal C_{\mathrm{b}}} \LetThereBe{\contFs}{\mcal C} \LetThereBe{\Borel}{\mathscr{B}} \LetThereBe{\FinSignMeas}{\mathscr{M}} \LetThereBe{\Probs}{\mathscr{P}} \letThereBe{\probs}{1}{\Probs\brackets{#1}} \LetThereBe{\SCMProbs}{\Probs_2} \letThereBe{\scmProbs}{1}{\SCMProbs\brackets{#1}} \LetThereBe{\ACProbs}{\SCMProbs^{\mathrm{ac}}} \letThereBe{\acProbs}{1}{\ACProbs\brackets{#1}} \letThereBe{\pushforward}{2}{#1_{\sharp}#2} \letThereBe{\pf}{2}{\pushforward{#1}{#2}} \declareMathematics{\Law}{Law} \declareMathematics{\Project}{pr} \letThereBe{\project}{1}{\Project^{#1}} \letThereBe{\pr}{1}{\project{#1}} \declareMathematics{\TV}{TV} \letThereBe{\cpls}{1}{\Cpls\brackets{#1}} \letThereBe{\optCpls}{1}{\OptCpls\brackets{#1}} \LetThereBe{\OT}{\mathrm{OT}} \LetThereBe{\wasserstein}{\mcal W_2} \LetThereBe{\wsGrad}{\gradient_{\mcal W}} \LetThereBe{\csubseteq}{\onTop{\subseteq}{\mathrm{c}}} \LetThereBe{\bddsubseteq}{\onTop{\subseteq}{\mathrm{bdd}}} \letThereBe{\cTr}{1}{#1^{\cc}} \letThereBe{\cbTr}{1}{#1^{\overline{\cc}}} $$ $$ % ODEs \LetThereBe{\timeInt}{\mcal I} \LetThereBe{\stimeInt}{\mcal J} \LetThereBe{\Wronsk}{\mcal W} \letThereBe{\wronsk}{1}{\Wronsk \parentheses{#1}} \LetThereBe{\prufRadius}{\rho} \LetThereBe{\prufAngle}{\vf} \LetThereBe{\weyr}{\sigma} \LetThereBe{\linDifOp}{\mathsf{L}} \LetThereBe{\Hurwitz}{\vi H} \letThereBe{\hurwitz}{1}{\Hurwitz \brackets{#1}} % Cont. Models \LetThereBe{\dirac}{\delta} \LetThereBe{\torus}{\mathbb{T}} % PDEs % \avint -- defined in format-respective tex files \LetThereBe{\fundamental}{\Phi} \LetThereBe{\fund}{\fundamental} \letThereBe{\normaDeriv}{1}{\partialDeriv{#1}{\vec{n}}} \letThereBe{\volAvg}{2}{\avint_{\ball{#1}{#2}}} \LetThereBe{\VolAvg}{\volAvg{x}{\ve}} \letThereBe{\surfAvg}{2}{\avint_{\boundaryOf \ball{#1}{#2}}} \LetThereBe{\SurfAvg}{\surfAvg{x}{\ve}} \LetThereBe{\corrF}{\varphi^{\times}} \LetThereBe{\greenF}{G} \letThereBe{\reflect}{1}{\tilde{#1}} \LetThereBe{\conv}{*} \letThereBe{\dotP}{2}{#1 \cdot #2} \letThereBe{\translation}{1}{\tau_{#1}} \declareMathematics{\dist}{dist} \letThereBe{\regularizef}{1}{\eta_{#1}} \letThereBe{\fourier}{1}{\widehat{#1}} \letThereBe{\ifourier}{1}{\check{#1}} \LetThereBe{\fourierOp}{\mcal F} \LetThereBe{\ifourierOp}{\mcal F^{-1}} \letThereBe{\FourierOp}{1}{\fourierOp\set{#1}} \letThereBe{\iFourierOp}{1}{\ifourierOp\set{#1}} \LetThereBe{\laplaceOp}{\mcal L} \letThereBe{\LaplaceOp}{1}{\laplaceOp\set{#1}} \letThereBe{\Norm}{1}{\absval{#1}} % SINDy \LetThereBe{\Koop}{\mcal K} \letThereBe{\oneToN}{1}{\left[#1\right]} \LetThereBe{\meas}{\mathrm{m}} \LetThereBe{\stateLoss}{\mcal J} \LetThereBe{\lagrm}{p} % Stochastic analysis \LetThereBe{\RiemannInt}{(\mcal R)} \LetThereBe{\RiemannStieltjesInt}{(\mcal {R_S})} \LetThereBe{\LebesgueInt}{(\mcal L)} \LetThereBe{\ItoInt}{(\mcal I)} \LetThereBe{\Stratonovich}{\circ} \LetThereBe{\infMean}{\alpha} \LetThereBe{\infVar}{\beta} % Dynamical systems \LetThereBe{\nUnit}{\mathrm N} \LetThereBe{\timeUnit}{\mathrm T} % Masters thesis \LetThereBe{\evolOp}{\oper{\vf}} \letThereBe{\obj}{1}{\mathbb{#1}} \LetThereBe{\timeSet}{\obj T} \LetThereBe{\stateSpace}{\obj X} \LetThereBe{\contStateSpace}{\stateSpace_{C}} \LetThereBe{\orbit}{Or} \letThereBe{\Orbit}{1}{\orbit\brackets{#1}} \LetThereBe{\limitSet}{\obj \Lambda} \LetThereBe{\crossSection}{\obj \Sigma} \declareMathematics{\codim}{codim} % Left and right closed-or-open intervals \LetThereBe{\lco}{\langle} \LetThereBe{\rco}{\rangle} \letThereBe{\testInt}{1}{\mathrm{Int}_{#1}} \letThereBe{\evalOp}{1}{\oper{\eta}_{#1}} \LetThereBe{\nonzeroEl}{\bullet} \LetThereBe{\zeroEl}{\circ} \LetThereBe{\solOp}{\oper{S}} \LetThereBe{\infGen}{\oper{A}} \LetThereBe{\indexSet}{\mcal I} \letThereBe{\indicesOf}{1}{\indexSet\parentheses{#1}} \letThereBe{\IndicesOf}{2}{\indexSet_{#2}\parentheses{#1}} \LetThereBe{\meshGrid}{\obj M} \declareMathematics{\starter}{starter} \declareMathematics{\indexer}{indx} \declareMathematics{\enumerator}{enum} \LetThereBe{\inSS}{_{\infty}} \LetThereBe{\manifold}{\mcal M} \LetThereBe{\curve}{\mcal C} % Numerical methods \declareMathematics{\globErr}{err} \declareMathematics{\locErr}{le} \declareMathematics{\locTrErr}{lte} \declareMathematics{\estimErr}{est} \declareMathematics{\incrementFunc}{Inc} \letThereBe{\incrementF}{1}{\incrementFunc \brackets{#1}} \LetThereBe{\discreteNodes}{\mcal T} \LetThereBe{\stableFunc}{R} \letThereBe{\stableF}{1}{\stableFunc\brackets{#1}} \LetThereBe{\stableRegion}{\Omega} %Stochastic analysis \LetThereBe{\RiemannInt}{(\mcal R)} \LetThereBe{\RiemannStieltjesInt}{(\mcal {R_S})} \LetThereBe{\LebesgueInt}{(\mcal L)} \LetThereBe{\ItoInt}{(\mcal I)} \LetThereBe{\Stratonovich}{\circ} \LetThereBe{\infMean}{\alpha} \LetThereBe{\infVar}{\beta} %Optimization \LetThereBe{\goldRatio}{\tau} %Interpolation \LetThereBe{\lagrPoly}{l} $$

Definition 3.1 (Polish space) A Polish space is a separable completely metrizable topological space. In other words, complete¹ metric space that has a countable dense subset.

Theorem 3.1 (Monotone convergence [1]) Let \((S, \Sigma, \mu)\) be a measure space and \(X \in \Sigma\). If \((f_n)\) is a sequence of non-negative measurable functions on \(X\) such that \[ 0 \leq f_1(x) \leq f_2(x) \leq \dots \] for all \(x \in X\), i.e., \((f_n)\) is monotone increasing, then the point-wise supremum \(f \letDef \sup_{n} f_n\) is measurable and \[ \int_X f \dd \mu = \lim_{n \to \infty} \int_X f_n \dd \mu = \sup_{n} \int_X f_n \dd \mu. \]

Theorem 3.2 (Dominated convergence [2]) Let \((f_n)\) be a sequence of complex-valued measurable functions on a given measure space \((S, \Sigma, \mu)\). Suppose that \(f_n \to f\), i.e., \(f_n\) converges point-wise to \(f\), \[ \lim_{n \to \infty} f_n(x) = f(x) \] for every \(x \in S\). Assume that \(f_n\) is dominated by some integrable function \(g\) (called majorant) in the sense \[ \absval{f_n(x)} \leq g(x) \] for all points \(x \in S\) and indices \(n\). Then \(f_n\), \(f\) are integrable and \[ \lim_{n \to \infty} \int_S f_n \dd \mu = \int_S \lim_{n \to \infty} f_n \dd \mu = \int_S f \dd \mu. \]

Proposition 3.1 (Jensen’s inequality [3]) Let \((\Omega, \mcal A, \mu)\) be a probability space. Let \(f : \Omega \to \R\) be a \(\mu\)-measurable function and \(\vf : \R \to \R\) convex. Then \[ \vf\brackets{\int_{\Omega} f \dd \mu} \leq \int_{\Omega} \vf \after f \dd \mu, \] and, in particular for \(p \geq 1\), \[ \int_{\Omega} f(x) \dd \mu(x) \leq \brackets{\int_{\Omega} \absval{f(x)}^p \dd \mu(x)}^{\frac 1 p}. \]

Lemma 3.1 (Fatou’s [4]) Given a measure space \((\Omega, \mcal A, \mu)\) and a set \(X \in \mcal A\), let \((f_n)\) be a sequence of \(\mu\)-measurable non-negative functions on \(X\). Define \(f(x) = \liminf_{n \to \infty} f_n(x)\) for every \(x \in X\). Then \(f\) is \(\mu\)-measurable, and \[ \int_X f \dd \mu \leq \liminf_{n \to \infty} \int_X f_n \dd \mu. \]

Theorem 3.3 (Riesz-Fischer [5]) For \(1 \leq p < \infty\), \(\lp{p}(E)\) is a Banach² space. Furthermore, if \(f_n \onBottom{\to}{\lp{p}} f\), then \((f_n)_n\) has a subsequence that converges to \(f\) point-wise almost everywhere on \(E\).

Theorem 3.4 (Cauchy-Schwarz inequality [6]) For all vectors \(\vi u, \vi v\) of an inner product space, the following holds \[ \absval{\scal{\vi u}{\vi v}} \leq \norm{\vi u} \norm{\vi v}. \]

Theorem 3.5 (Riesz representation [7]) Let \(H\) be a Hilbert space[^hilbert] whose inner product \(\scal {x} {y}\) is linear in its first argument and anti-linear in its second argument. For every continuous linear functional \(\vf \in H^*\), there exists a unique vector \(f_{\vf} \in H\), called the Riesz representation of \(\vf\), such that \[ \vf(x) = \scal x {f_{\vf}} \quad \forall x \in H. \] Importantly, for complex HIlbert spaces, \(f_{\vf}\) is always located in the anti-linear coordinate of the inner product.

Theorem 3.6 (Stone-Weierstrass) Suppose \(X\) is a compact Hausdorff space³ and \(A\) is subalgebra of \(\contf{X, \R}{}\) which contains a non-zero constant function. Then \(A\) is dense in \(\contf{X, \R}{}\) if and only if it separates points.

Theorem 3.7 (Hahn-Banach) Let \(X\) be a vector space over the field \(\K = \R, \C\). Let further

• \(Y \subseteq X\) be a linear subspace,
• \(p \colon X \to \R\) be a sublinear⁴ function and
• \(f \colon Y \to \K\) be a linear functional so that \(\reOf{f(y)} \leq p(y)\) for all \(y \in Y\).

Then, there exists a linear functional \(F \colon X \to \K\) so that \[ F \vert_{Y} = f \quad \& \quad \forall x \in X: \; \reOf{F(x)} \leq p(x). \]

Theorem 3.8 (Geometric Hahn-Banach) Let \(A\) and \(B\) be non-empty convex subsets of a real locally convex topological vector space \(X\). If \(\interior A \neq \emptyset\) and \(B \cap \interior A = \emptyset\) then there exists a continuous linear functional \(f\) on \(X\) such that \(\sup f(A) \leq \inf f(B)\) and \(f(a) < \inf f(B)\) for all \(a \in \interior A\) (such an \(f\) is necessarily non-zero).

3.1 A Note on Weak Topology

See [8] and related pages for more information.

Suppose \((X, Y, b)\) is a pairing (or a dual system) of vector spaces over a topological field \(\K\), i.e., \(X\) and \(Y\) are vector spaces of \(\K\) and \(b \colon X \times Y \to \K\) is a bilinear map. For all \(x \in X\), let \(b(x, \cdot) \colon Y \to \K\) denote the linear function on \(Y\) defined by \(y \mapsto b(x,y)\), and similarly for fixed \(y \in Y\).

Definition 3.2 (Weak topology of pairing) The weak topology on \(X\) induced by \(Y\) (and \(b\)) is the weakest topology on \(X\), denoted by \(\sigma(X, Y, b)\) or simply \(\sigma(X,Y)\), making all maps \(b(\cdot, y) \colon X \to \K\) continuous as \(y\) ranges over \(Y\).

If the field \(\K\) has an absolute value \(\absval{\cdot}\), then the weak topology \(\sigma(X, Y, b)\) on \(X\) is induced by the family of seminorms, \(p_y \colon X \to \R\), defined by \(p_y(x) := \absval{b(x,y)}\) for all \(y \in Y\) and \(x \in X\). In particular, this shows that weak topologies are locally convex.

3.1.1 Canonical Duality

Consider now the space case where \(Y\) is a vector subspace of the algebraic dual space⁵. There is the so-called canonical pairing \((X, Y, \scal{\cdot}{\cdot})\) or simply \((X,Y)\) whose bilinear map \(\scal{\cdot}{\cdot}\) is the canonical evaluation map, defined by \(\scal x {x'} = x'(x)\) for all \(x \in X\) and \(x' \in Y\).

The topology \(\sigma(X, Y)\) is the initial topology of \(X\) with respect to \(Y\).

3.1.2 Weak* Topology

Let \(X\) be a topological vector space (TVS)⁶ over \(\K\). We call the topology \(X\) starts with the original (or given) topology. We may define a possibly different topology on \(X\) using the topological or continuous dual space \(X^*\), which consists of all linear functionals from \(X\) into the base field \(\K\) that are continuous with respect to the given topology.

Definition 3.3 (Weak topology) The weak topology on \(X\) is the weak topology on \(X\), see Definition 3.2, with respect to the canonical pairing \(\scal X {X^*}\).

Definition 3.4 (Weak* topology) The weak topology on \(X^*\) is the weak topology on \(X^*\) with respect to the canonical pairing \(\scal{X} {X^*}\). This topology is also called the weak* topology.

Definition 3.5 (Weak* convergence) Suppose that the normed space where we want to work, \(U\), itself is the dual of some Banach⁷ space, i.e., \(U = X^*\)⁸. A sequence \((u_n) \subset U\) converges in weak*-sense (or in the weak* topology) to \(u \in U\), if \(u_n(x) \to u(x)\) for any \(x \in X\).

Theorem 3.9 (Riesz-Markov) If \(K\) is compact Hausdorff⁹ topological space, the dual of \(\contFs(K)\) is isomorphic to the space \(\FinSignMeas(K)\) of measures on \(K\), equipped with the total variation norm.

Theorem 3.10 (Banach-Alaoglu) Let \(X\) be a Banach space. Then the closed unit ball \(B\) of \(X^*\) is compact in the weak* topology, see Definition 3.4.

Corollary 3.1 Let \(X\) be a normed vector space. Then the closed unit ball \(B = \set{x \in X \mid \norm{x} \leq 1}\) of the dual space is compact in weak* topology.

[1]

Contributors, W. (2026). Monotone convergence theorem.

[2]

Contributors, W. (2025). Dominated convergence theorem.

[3]

Contributors, W. (2026). Jensen’s inequality.

[4]

Contributors, W. (2025). Fatou’s lemma.

[5]

Meziani, H. (2022). Real analysis MAA 6616 lecture 25 convergence in lp spaces and completeness - Lecture25.

[6]

Contributors, W. (2026). Cauchy–Schwarz inequality.

[7]

Contributors, W. (2025). Riesz representation theorem.

[8]

Contributors, W. (2026). Weak topology.

---

1. Every Cauchy sequence of points in \(M\) has a limit also in \(M\)

2. Banach space is a complete normed space.

3. Hausdorff space is a topological space where distinct points have disjoint neighborhoods

4. A function \(p \colon X \to \R\) on a vector field \(X\) over \(\K\) is called sublinear if it is positively homogeneous, i.e., \(p(rx) = rp(x)\) for all \(r \geq 0\), and subadditive, \(p(x+y) \leq p(x) + p(y)\).

5. Algebraic dual space of \(X\) is the vector space of linear functionals on \(X\).

6. Topological vector space is a vector space equipped with a topology such that vector addition and scalar multiplication are continuous.

7. Banach space is a complete normed space.

8. Recall that not everything is a dual space.

9. Hausdorff topological space separates points, i.e., every pair of distinct points have disjoint neighborhoods.