SCI和EI收录∣中国化工学会会刊

Chinese Journal of Chemical Engineering ›› 2024, Vol. 74 ›› Issue (10): 227-237.DOI: 10.1016/j.cjche.2024.04.029

Previous Articles     Next Articles

A local space transfer learning-based parallel Bayesian optimization with its application

Luhang Yang1, Xixiang Zhang1, Jingyi Lu1,3, Zhou Tian1, Wenli Du1,2   

  1. 1 Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China;
    2 Qingyuan Innovation Laboratory, Quanzhou 362801, China;
    3 Longmen Laboratory, Luoyang 471023, China
  • Received:2023-10-25 Revised:2024-03-17 Accepted:2024-04-28 Online:2024-07-24 Published:2024-10-28
  • Contact: Wenli DuLuhang Yang,E-mail:luhang.yang@mail.ecust.edu.cn;Jingyi Lu,E-mail:Jylu_cise@ecust.edu.cn;Wenli Du,E-mail:wldu@ecust.edu.cnwldu@ecust.edu.cn
  • Supported by:
    This work was supported by National Natural Science Foundation of China (62394343), Major Program of Qingyuan Innovation Laboratory (00122002), Major Science and Technology Projects of Longmen Laboratory (231100220600), Shanghai Committee of Science and Technology (23ZR1416000) and Shanghai AI Lab.

A local space transfer learning-based parallel Bayesian optimization with its application

Luhang Yang1, Xixiang Zhang1, Jingyi Lu1,3, Zhou Tian1, Wenli Du1,2   

  1. 1 Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China;
    2 Qingyuan Innovation Laboratory, Quanzhou 362801, China;
    3 Longmen Laboratory, Luoyang 471023, China
  • 通讯作者: Wenli DuLuhang Yang,E-mail:luhang.yang@mail.ecust.edu.cn;Jingyi Lu,E-mail:Jylu_cise@ecust.edu.cn;Wenli Du,E-mail:wldu@ecust.edu.cnwldu@ecust.edu.cn
  • 基金资助:
    This work was supported by National Natural Science Foundation of China (62394343), Major Program of Qingyuan Innovation Laboratory (00122002), Major Science and Technology Projects of Longmen Laboratory (231100220600), Shanghai Committee of Science and Technology (23ZR1416000) and Shanghai AI Lab.

Abstract: The optimization of process parameters in polyolefin production can bring significant economic benefits to the factory. However, due to small data sets, high costs associated with parameter verification cycles, and difficulty in establishing an optimization model, the optimization process is often restricted. To address this issue, we propose using a transfer learning Bayesian optimization strategy to improve the efficiency of parameter optimization while minimizing resource consumption. Specifically, we leverage Gaussian process (GP) regression models to establish an integrated model that incorporates both source and target grade production task data. We then measure the similarity weights of each model by comparing their predicted trends, and utilize these weights to accelerate the solution of optimal process parameters for producing target polyolefin grades. In order to enhance the accuracy of our approach, we acknowledge that measuring similarity in a global search space may not effectively capture local similarity characteristics. Therefore, we propose a novel method for transfer learning optimization that operates within a local space (LSTL-PBO). This method employs partial data acquired through random sampling from the target task data and utilizes Bayesian optimization techniques for model establishment. By focusing on a local search space, we aim to better discern and leverage the inherent similarities between source tasks and the target task. Additionally, we incorporate a parallel concept into our method to address multiple local search spaces simultaneously. By doing so, we can explore different regions of the parameter space in parallel, thereby increasing the chances of finding optimal process parameters. This localized approach allows us to improve the precision and effectiveness of our optimization process. The performance of our method is validated through experiments on benchmark problems, and we discuss the sensitivity of its hyperparameters. The results show that our proposed method can significantly improve the efficiency of process parameter optimization, reduce the dependence on source tasks, and enhance the method's robustness. This has great potential for optimizing processes in industrial environments.

Key words: Transfer learning, Bayesian optimization, Process parameters, Parallel framework, Local search space

摘要: The optimization of process parameters in polyolefin production can bring significant economic benefits to the factory. However, due to small data sets, high costs associated with parameter verification cycles, and difficulty in establishing an optimization model, the optimization process is often restricted. To address this issue, we propose using a transfer learning Bayesian optimization strategy to improve the efficiency of parameter optimization while minimizing resource consumption. Specifically, we leverage Gaussian process (GP) regression models to establish an integrated model that incorporates both source and target grade production task data. We then measure the similarity weights of each model by comparing their predicted trends, and utilize these weights to accelerate the solution of optimal process parameters for producing target polyolefin grades. In order to enhance the accuracy of our approach, we acknowledge that measuring similarity in a global search space may not effectively capture local similarity characteristics. Therefore, we propose a novel method for transfer learning optimization that operates within a local space (LSTL-PBO). This method employs partial data acquired through random sampling from the target task data and utilizes Bayesian optimization techniques for model establishment. By focusing on a local search space, we aim to better discern and leverage the inherent similarities between source tasks and the target task. Additionally, we incorporate a parallel concept into our method to address multiple local search spaces simultaneously. By doing so, we can explore different regions of the parameter space in parallel, thereby increasing the chances of finding optimal process parameters. This localized approach allows us to improve the precision and effectiveness of our optimization process. The performance of our method is validated through experiments on benchmark problems, and we discuss the sensitivity of its hyperparameters. The results show that our proposed method can significantly improve the efficiency of process parameter optimization, reduce the dependence on source tasks, and enhance the method's robustness. This has great potential for optimizing processes in industrial environments.

关键词: Transfer learning, Bayesian optimization, Process parameters, Parallel framework, Local search space