SCI和EI收录∣中国化工学会会刊

中国化学工程学报 ›› 2024, Vol. 69 ›› Issue (5): 63-71.DOI: 10.1016/j.cjche.2023.12.005

• • 上一篇    下一篇

Combining reinforcement learning with mathematical programming: An approach for optimal design of heat exchanger networks

Hui Tan1, Xiaodong Hong1,2, Zuwei Liao1, Jingyuan Sun1, Yao Yang1, Jingdai Wang1, Yongrong Yang1   

  1. 1. State Key Laboratory of Chemical Engineering, College of Chemical and Biological Engineering, Zhejiang University, Hangzhou 310027, China;
    2. ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou 311215, China
  • 收稿日期:2023-11-11 修回日期:2023-12-14 出版日期:2024-05-28 发布日期:2024-07-01
  • 通讯作者: Zuwei Liao,E-mail:liaozw@zju.edu.cn
  • 基金资助:
    The financial support provided by the Project of National Natural Science Foundation of China (U22A20415, 21978256, 22308314) and “Pioneer” and “Leading Goose” Research & Development Program of Zhejiang (2022C01SA442617) are gratefully acknowledged. Besides, the authors would like to thank artificial intelligence + High Performance Computing Center of ZJU-ICI.

Combining reinforcement learning with mathematical programming: An approach for optimal design of heat exchanger networks

Hui Tan1, Xiaodong Hong1,2, Zuwei Liao1, Jingyuan Sun1, Yao Yang1, Jingdai Wang1, Yongrong Yang1   

  1. 1. State Key Laboratory of Chemical Engineering, College of Chemical and Biological Engineering, Zhejiang University, Hangzhou 310027, China;
    2. ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou 311215, China
  • Received:2023-11-11 Revised:2023-12-14 Online:2024-05-28 Published:2024-07-01
  • Contact: Zuwei Liao,E-mail:liaozw@zju.edu.cn
  • Supported by:
    The financial support provided by the Project of National Natural Science Foundation of China (U22A20415, 21978256, 22308314) and “Pioneer” and “Leading Goose” Research & Development Program of Zhejiang (2022C01SA442617) are gratefully acknowledged. Besides, the authors would like to thank artificial intelligence + High Performance Computing Center of ZJU-ICI.

摘要: Heat integration is important for energy-saving in the process industry. It is linked to the persistently challenging task of optimal design of heat exchanger networks (HEN). Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem, it is not easy to find solutions of high quality for large-scale problems. The reinforcement learning (RL) method, which learns strategies through ongoing exploration and exploitation, reveals advantages in such area. However, due to the complexity of the HEN design problem, the RL method for HEN should be dedicated and designed. A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods. An insightful state representation of the HEN structure as well as a customized reward function is introduced. A Q-learning algorithm is applied to update the HEN structure using the ϵ-greedy strategy. Better results are obtained from three literature cases of different scales.

关键词: Heat exchanger network, Reinforcement learning, Mathematical programming, Process design

Abstract: Heat integration is important for energy-saving in the process industry. It is linked to the persistently challenging task of optimal design of heat exchanger networks (HEN). Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem, it is not easy to find solutions of high quality for large-scale problems. The reinforcement learning (RL) method, which learns strategies through ongoing exploration and exploitation, reveals advantages in such area. However, due to the complexity of the HEN design problem, the RL method for HEN should be dedicated and designed. A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods. An insightful state representation of the HEN structure as well as a customized reward function is introduced. A Q-learning algorithm is applied to update the HEN structure using the ϵ-greedy strategy. Better results are obtained from three literature cases of different scales.

Key words: Heat exchanger network, Reinforcement learning, Mathematical programming, Process design