JR's Yoyo Diary by Myself

Expand

JR's Machine Learning Notes by Myself

• KL Divergence (Too early to remeber)
• Perceptron (Too early to remeber)
• K-Nearest Neighbor (Too early to remeber)
• Entropy & Information Gain (Too early to remeber)
• Decision Tree (Too early to remeber)
• Naive Bayes (Too early to remeber)
• Chebyshev Polynomials(2019.10.1)
• Augmented Lagrangian(2021.1.17)
• Augmented Lagrangian Python Implementation (2021.1.17)
• Low-Rank Representation (2021.2.1) (Python Implementation)
• Subdifferential of Matrix Norm (2021.4.7, updated at 2024.9.4) (Original Paper)
• (Super Recommended) Multisite Adaption Algorithm Based on Low-Rank Representation:
  Criticizing a Paper and Developing My Own Idea, Which is Also Full of Flaws (2021.6.3) (Python Implementation)
• (Super Recommended) Low Rank Matrix Completion Made Unified with Convex Optimization Theory (2021.7.11)
  (Python Implementation, CVX, Tex)
• (Super Recommended) Conjugate Gradient Method: Steepest Descent with Orthogonal
  Directions under Change of Coordinate (2021.7.21) (Python Implementation, Tex)
• Detailed Derivation of Robust Covariate Shift Regression (2023.3.22) ( Original paper )
• A Lipschitz Parameterization of Neural Networks
• Implicit Model: Fixed Point Layer with Elegant Differentiation ( PyTorch Implementation )
• (Super Recommended) A Tour Through Policy Gradient and Its Variants (2023.11.3)
• (Super Recommended) Recurrent State-Space Model and Dreamer (2023.12.5) (Python Implementation)
• Sort Out Soft Q-Learning (2024.3.26) (Python Implementation)
• Proximal Algorithms, Operator Splitting and ADMM (2024.7.16)
• Diffusion Model (2024.11.11) (Python Implementation)
• OSQP (2024.11.28) (Python Implementation)
• TinyMPC (2024.11.28) (Python Implementation)

Good To Know

• Bellman-Gronwall Lemma
• Cayley Transform
• Delta Method
• Gershgorin's-Theorem
• Cross Product, Matrix Exponetial and Rotation
• Perron-Frobenius Theorem
• S-Lemma
• Sum-of-Square Optimization and S-Procedure
• Calculus of Variation

Reproducing Kernel Hilbert Space and Applications

• Functional Analysis Review and RKHS (2023.12.9)
• Kernel (2023.12.9)
• Kernel PCA (2023.12.10) (approach1.py, approach2.py)
• Functional Gradient (2024.3.3) (Regression Example) Backtracking line search also works in RKHS !!
• Kernelized Stein Discrepancy (2024.3.13)
• Stein Variational Gradient Descent (2024.3.16) (Mixture of Gaussian Example)
TODO: Stein Variational Gradient Descent and application in Soft Q-Learning

Toward Semidefinite Programming and Primal-dual path-following Algorithm Python Implementation

Expand

EE263 Linear Dynamical System by Stephen Boyd (2022.12 - )

Expand

Probabilistic Graphical Model

Quit 10708 (too obscure) for “Probabilistic Graphical Models: Principles and Techniques” book and Coursea courses by Daphne Koller (2021.8.7)

It's the first round going through this book, skip some content to quickly get a big picture of PGM,

Expand

EE364A Convex Optimization I by Stephen Boyd (Uploaded on 2020.11.30)

YouTube Link Expand

EE364B Convex Optimization II by Stephen Boyd

Expand

EE261 Fourier Transform and Its Applications by Brad Osgood

YouTube Link Expand

Deep Learning

Expand

C++

• std::forward()(Too early to remeber)
• std::enable_if<> (Too early to remeber)

18.085 by Gilbert Strang

YouTube Link Expand

18.03 Differential Equations by Arthur Mattuck (Review)

YouTube Link Expand

CS229 Machine Learning by Andrew Ng

YouTube Link

• Logistic Regression (Too early to remeber)
• Poisson Regression (Too early to remeber)
• Gaussian Discriminant Analysis (Too early to remeber)
• Locally Weighted Linear Regression (Too early to remeber)
• Naive Bayes Multinomial Event Model (Too early to remeber)
• MAP and MAP Estimation (Too early to remeber)