TY - JOUR
T1 - Free Energy Minimization
T2 - A Unified Framework for Modeling, Inference, Learning, and Optimization [Lecture Notes]
AU - Jose, Sharu Theresa
AU - Simeone, Osvaldo
N1 - Funding Information:
The authors have received funding from the European Research Council under the European Union’s Horizon 2020 Research and Innovation Programme (grant 725731).
Publisher Copyright:
© 1991-2012 IEEE.
Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.
PY - 2021/3
Y1 - 2021/3
N2 - The goal of this lecture note is to review the problem of free energy minimization as a unified framework underlying the definition of maximum entropy modeling, generalized Bayesian inference, learning with latent variables, the statistical learning analysis of generalization, and local optimization. Free energy minimization is first introduced, here and historically, as a thermodynamic principle. Then, it is described mathematically in the context of Fenchel duality. Finally, the applications to modeling, inference, learning, and optimization are covered, starting from basic principles.
AB - The goal of this lecture note is to review the problem of free energy minimization as a unified framework underlying the definition of maximum entropy modeling, generalized Bayesian inference, learning with latent variables, the statistical learning analysis of generalization, and local optimization. Free energy minimization is first introduced, here and historically, as a thermodynamic principle. Then, it is described mathematically in the context of Fenchel duality. Finally, the applications to modeling, inference, learning, and optimization are covered, starting from basic principles.
UR - http://www.scopus.com/inward/record.url?scp=85102029970&partnerID=8YFLogxK
U2 - 10.1109/MSP.2020.3041414
DO - 10.1109/MSP.2020.3041414
M3 - Article
AN - SCOPUS:85102029970
SN - 1053-5888
VL - 38
SP - 120
EP - 125
JO - IEEE SIGNAL PROCESSING MAGAZINE
JF - IEEE SIGNAL PROCESSING MAGAZINE
IS - 2
M1 - 9363495
ER -