Turn off MathJax
Article Contents
Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055
Citation: Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055

Comparison of Multi-task Approaches on Molecular Property Prediction

doi: 10.1063/1674-0068/cjcp2203055
More Information
  • Corresponding author: E-mail: wgzhu@ustc.edu.cn
  • Received Date: 2022-03-29
  • Accepted Date: 2022-04-20
  • Available Online: 2022-07-22
  • With the bloom of deep learning algorithms, various models have been widely utilized in quantum chemistry calculation to design new molecules and explore molecular properties. However, limited studies focus on multi-task molecular property prediction, which offers more efficient ways to simultaneously learn different but related properties by leveraging the inter-task relationship. In this work, we apply the hard parameter sharing framework and advanced loss weighting methods to multi-task molecular property prediction. Based on the performance comparison between single-task baseline and multi-task models on several task sets, we find that the prediction accuracy largely depends on the inter-task relationship, and hard parameter sharing improves the performance when the correlation becomes complex. In addition, we show that proper loss weighting methods help achieve more balanced multi-task optimization and enhance the prediction accuracy. Our additional experiments on varying amount of training data further validate the multi-task advantages and show that multi-task models with proper loss weighting methods can achieve more accurate prediction of molecular properties with much less computational cost.

     

  • 1 As there are different versions of QM9 dataset, we use the dataset from DIG [32] package where the data of $ \omega_1 $ is not provided. Therefore, we keep the same with many other studies on QM9 and have not taken $ \omega_1 $ into consideration.
  • loading
  • [1]
    R. Ramakrishnan, M. Hartmann, E. Tapavicza, and O. A. Von Lilienfeld, J. Chem. Phys. 143, 084111 (2015). doi: 10.1063/1.4928757
    [2]
    R. Ramakrishnan, P. O. Dral, M. Rupp, and O. A. Von Lilienfeld, Sci. Data 1, 140022 (2014). doi: 10.1038/sdata.2014.22
    [3]
    L. Ruddigkeit, R. van Deursen, L. C. Blum, and J. L. Reymond, J. Chem. Inf. Model. 52, 2864 (2012). doi: 10.1021/ci300415d
    [4]
    G. Chen, P. Chen, C. Y. Hsieh, C. K. Lee, B. Liao, R. Liao, W. Liu, J. Qiu, Q. Sun, J. Tang, R. Zemel, and S. Zhang, arXiv: 1906.09427 (2019).
    [5]
    K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh, Nature 559, 547 (2018). doi: 10.1038/s41586-018-0337-2
    [6]
    J. Wei, X. Chu, X. Y. Sun, K. Xu, H. X. Deng, J. Chen, Z. Wei, and M. Lei, InfoMat 1, 338 (2019). doi: 10.1002/inf2.12028
    [7]
    B. Sanchez-Lengeling and A. Aspuru-Guzik, Science 361, 360 (2018). doi: 10.1126/science.aat2663
    [8]
    J. Wang, C. Y. Hsieh, M. Wang, X. Wang, Z. Wu, D. Jiang, B. Liao, X. Zhang, B. Yang, Q. He, D. Cao, X. Chen, and T. Hou, Nat. Mach. Intell. 3, 914 (2021). doi: 10.1038/s42256-021-00403-1
    [9]
    Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande, Chem. Sci. 9, 513 (2018). doi: 10.1039/C7SC02664A
    [10]
    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, in Proceedings of the 34th International Conference on Machine Learning, 1263 (2017).
    [11]
    K. T. Schütt, H. E. Sauceda, P. J. Kindermans, A. Tkatchenko, and K. R. Müller, J. Chem. Phys. 148, 241722 (2018). doi: 10.1063/1.5019779
    [12]
    C. Lu, Q. Liu, C. Wang, Z. Huang, P. Lin, and L. He, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 1052 (2019).
    [13]
    O. T. Unke and M. Meuwly, J. Chem. Theory Comput. 15, 3678 (2019). doi: 10.1021/acs.jctc.9b00181
    [14]
    C. Chen, W. Ye, Y. Zuo, C. Zheng, and S. P. Ong, Chem. Mater. 31, 3564 (2019). doi: 10.1021/acs.chemmater.9b01294
    [15]
    J. Klicpera, J. Groß, and S. Günnemann, in International Conference on Learning Representations, (2020).
    [16]
    X. Wang, S. Ye, W. Hu, E. Sharman, R. Liu, Y. Liu, Y. Luo, and J. Jiang, J. Am. Chem. Soc. 142, 7737 (2020). doi: 10.1021/jacs.0c01825
    [17]
    Z. Qiao, M. Welborn, A. Anandkumar, F. R. Manby, and T. F. Miller III, J. Chem. Phys. 153, 124111 (2020). doi: 10.1063/5.0021955
    [18]
    Z. Zhang, J. Guan, and S. Zhou, Bioinformatics 37, 2981 (2021). doi: 10.1093/bioinformatics/btab195
    [19]
    Z. Zhang, Q. Liu, H. Wang, C. Lu, and C. Lee, arXiv: 2112.00911 (2021).
    [20]
    J. Behler, Chem. Rev. 121, 10037 (2021). doi: 10.1021/acs.chemrev.0c00868
    [21]
    Y. Zhang, C. Hu, and B. Jiang, J. Phys. Chem. Lett. 10, 4962 (2019). doi: 10.1021/acs.jpclett.9b02037
    [22]
    H. Wang, L. Zhang, J. Han, and W. E, Comput. Phys. Commun. 228, 178 (2018). doi: 10.1016/j.cpc.2018.03.016
    [23]
    B. Jiang, J. Li, and H. Guo, J. Phys. Chem. Lett. 11, 5120 (2020). doi: 10.1021/acs.jpclett.0c00989
    [24]
    I. Misra, A. Shrivastava, A. Gupta, and M. Hebert, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3994 (2016).
    [25]
    V. Sanh, T. Wolf, and S. Ruder, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 6949 (2019).
    [26]
    S. Sosnin, M. Vashurina, M. Withnall, P. Karpov, M. Fedorov, and I. V. Tetko, Mol. Inf. 38, 1800108 (2019). doi: 10.1002/minf.201800108
    [27]
    M. Crawshaw, arXiv: 2009.09796 (2020).
    [28]
    Z. Tan, Y. Li, W. Shi, and S. Yang, J. Chem. Inf. Model. 61, 3824 (2021). doi: 10.1021/acs.jcim.1c00646
    [29]
    Z. Liu, L. Lin, Q. Jia, Z. Cheng, Y. Jiang, Y. Guo, and J. Ma, J. Chem. Inf. Model. 61, 1066 (2021). doi: 10.1021/acs.jcim.0c01224
    [30]
    Y. Liu, L. Wang, M. Liu, X. Zhang, B. Oztekin, and S. Ji, arXiv: 2102.05013 (2021).
    [31]
    S. Ruder, arXiv: 1706.05098 (2017).
    [32]
    M. Liu, Y. Luo, L. Wang, Y. Xie, H. Yuan, S. Gui, H. Yu, Z. Xu, J. Zhang, Y. Liu, K. Yan, H. Liu, C. Fu, B. M. Oztekin, X. Zhang, and S. Ji, J. Mach. Learn. Res. 22, 1 (2021).
    [33]
    D. Eigen and R. Fergus, in Proceedings of the IEEE International Conference on Computer Vision, 2650 (2015).
    [34]
    A. Kendall, Y. Gal, and R. Cipolla, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7482 (2018).
    [35]
    L. Liebel and M. Körner, arXiv: 1805.06334 (2018).
    [36]
    S. Liu, E. Johns, and A. J. Davison, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1871 (2019).
    [37]
    S. Jean, O. Firat, and M. Johnson, arXiv: 1909.06434 (2019).
    [38]
    T. Gong, T. Lee, C. Stephenson, V. Renduchintala, S. Padhy, A. Ndirango, G. Keskin, and O. H. Elibol, IEEE Access 7, 141627 (2019). doi: 10.1109/ACCESS.2019.2943604
    [39]
    Z. Chen, V. Badrinarayanan, C. Y. Lee, and A. Rabinovich, in International Conference on Machine Learning, 794 (2018).
    [40]
    K. Schutt, P. Kessel, M. Gastegger, K. Nicoli, A. Tkatchenko, and K. R. Muller, J. Chem. Theory Comput. 15, 448 (2019). doi: 10.1021/acs.jctc.8b00908
    [41]
    H. B. Lee, E. Yang, and S. J. Hwang, in International Conference on Machine Learning, 2956 (2018).
    [42]
    Y. Gao, J. Ma, M. Zhao, W. Liu, and A. L. Yuille, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3205 (2019).
    [43]
    D. Xu, W. Ouyang, X. Wang, and N. Sebe, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 675 (2018).
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(4)  / Tables(4)

    Article Metrics

    Article views (533) PDF downloads(48) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return