Comparison of Multi-task Approaches on Molecular Property Prediction

Chao Han Hao Wang Jianbao Zhu Qi Liu Wenguang Zhu

Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055
Citation: Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055

doi: 10.1063/1674-0068/cjcp2203055

Comparison of Multi-task Approaches on Molecular Property Prediction

More Information
    • 关键词:
    •  / 
    •  / 
    •  / 
    •  
  • Figure  1.  The architecture of single-task SphereNet. Taking molecular spatial information and atomic number as input, it predicts molecular property A as a single task.

    Figure  2.  Multi-task SphereNet with hard parameter sharing framework. With task-specific output modules, it can predict multiple properties simultaneously.

    Figure  3.  Inter-task Pearson correlation coefficients of properties in QM9 dataset. A higher absolute value indicates a stronger linear correlation, while the sign of coefficients indicates whether properties are correlated positively or negatively.

    Figure  4.  Model performance with different amounts of training data. We use 10%, 30%, and 50% samples in QM9 as training dataset respectively and show models' $ {\rm{std._{MAE} }}$ on test dataset to evaluate their overall performance.

    Table  I.   Performance comparison od differnt models on learning properties of atomization energies (in meV). Task-specific MAE and $ {\rm{std._{MAE} }}$ (in %) on test dataset are both shown in the table. The best results are shown in bold.

    Model$ U_0 $$ U $ $ G $ $ H $ $ {\rm{std._{MAE} }}$
    STL31.832.933.733.40.324
    Adapted STL31.031.331.031.30.306
    Uniform34.334.634.134.60.338
    Uncertainty34.134.334.034.30.336
    Revised uncertainty34.134.534.034.40.337
    DWA34.134.334.134.40.337
    下载: 导出CSV

    Table  II.   Performance comparison on 4 learning electronic properties ($ \langle R^2\rangle $ in $a_0^2 $, ε in meV, std.MAE in %). The bold fonts show the best results.

    Model$ \langle R^2\rangle $ $ \varepsilon_{\rm{HOMO}} $ $ \varepsilon_{\rm{LUMO}} $ $ \Delta\varepsilon $ $ {\rm{std._{MAE} }}$
    STL0.48695.267.61197.66
    Adapted STL0.51110498.21429.10
    Uniform0.47097.983.71308.29
    Uncertainty0.68689.871.61177.50
    Revised uncertainty0.58190.273.91187.56
    DWA0.46897.784.31308.30
    下载: 导出CSV

    Table  III.   Performance comparison on learning all 12 properties from QM9 (U0, U, G, H, ZPVE, ε in meV, Cv in cal·mol−1·K, α in $a_0^3 $, μ in D, $\langle R^2\rangle $ in $ a_0^2 $, std.MAE in %). The bold fonts show the best results.

    Model$ U_0 $
    $ U $
    $ G $
    $ H $
    $ C_{\rm{v}} $
    ZPVE $ \alpha $ $ \mu $ $ \langle R^2\rangle $ $ \varepsilon_{\rm{HOMO}} $
    $ \varepsilon_{\rm{LUMO}} $
    $ \Delta\varepsilon $
    $ {\rm{std._{MAE} }}$
    STL31.832.933.733.40.06622.940.1600.06000.48695.267.61193.32
    Adapted STL39.639.738.940.00.067311.10.1610.08070.48496.488.21303.76
    Uniform35.034.434.234.30.06483.230.1570.07430.42487.979.01203.38
    Uncertainty32.231.431.231.80.06112.580.1620.06940.52485.073.21143.22
    Revised uncertainty33.032.632.534.60.06182.970.1550.07000.48285.874.01143.25
    DWA36.034.635.634.80.06382.920.1570.07570.42988.578.21193.38
    下载: 导出CSV

    Table  IV.   Efficiency comparison in terms of number of parameters and training time cost per epoch.

    Model Number of parameter Time/s
    STL $2.28\times 10^7 $ 660
    Adapted STL $1.91\times 10^6 $ 64
    Uniform $1.34\times 10^7 $ 123
    Uncertainty $1.34\times 10^7 $ 122
    下载: 导出CSV
  • [1] R. Ramakrishnan, M. Hartmann, E. Tapavicza, and O. A. Von Lilienfeld, J. Chem. Phys. 143, 084111 (2015). doi: 10.1063/1.4928757
    [2] R. Ramakrishnan, P. O. Dral, M. Rupp, and O. A. Von Lilienfeld, Sci. Data 1, 140022 (2014). doi: 10.1038/sdata.2014.22
    [3] L. Ruddigkeit, R. van Deursen, L. C. Blum, and J. L. Reymond, J. Chem. Inf. Model. 52, 2864 (2012). doi: 10.1021/ci300415d
    [4] G. Chen, P. Chen, C. Y. Hsieh, C. K. Lee, B. Liao, R. Liao, W. Liu, J. Qiu, Q. Sun, J. Tang, R. Zemel, and S. Zhang, arXiv: 1906.09427 (2019).
    [5] K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh, Nature 559, 547 (2018). doi: 10.1038/s41586-018-0337-2
    [6] J. Wei, X. Chu, X. Y. Sun, K. Xu, H. X. Deng, J. Chen, Z. Wei, and M. Lei, InfoMat 1, 338 (2019). doi: 10.1002/inf2.12028
    [7] B. Sanchez-Lengeling and A. Aspuru-Guzik, Science 361, 360 (2018). doi: 10.1126/science.aat2663
    [8] J. Wang, C. Y. Hsieh, M. Wang, X. Wang, Z. Wu, D. Jiang, B. Liao, X. Zhang, B. Yang, Q. He, D. Cao, X. Chen, and T. Hou, Nat. Mach. Intell. 3, 914 (2021). doi: 10.1038/s42256-021-00403-1
    [9] Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande, Chem. Sci. 9, 513 (2018). doi: 10.1039/C7SC02664A
    [10] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, in Proceedings of the 34th International Conference on Machine Learning, 1263 (2017).
    [11] K. T. Schütt, H. E. Sauceda, P. J. Kindermans, A. Tkatchenko, and K. R. Müller, J. Chem. Phys. 148, 241722 (2018). doi: 10.1063/1.5019779
    [12] C. Lu, Q. Liu, C. Wang, Z. Huang, P. Lin, and L. He, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 1052 (2019).
    [13] O. T. Unke and M. Meuwly, J. Chem. Theory Comput. 15, 3678 (2019). doi: 10.1021/acs.jctc.9b00181
    [14] C. Chen, W. Ye, Y. Zuo, C. Zheng, and S. P. Ong, Chem. Mater. 31, 3564 (2019). doi: 10.1021/acs.chemmater.9b01294
    [15] J. Klicpera, J. Groß, and S. Günnemann, in International Conference on Learning Representations, (2020).
    [16] X. Wang, S. Ye, W. Hu, E. Sharman, R. Liu, Y. Liu, Y. Luo, and J. Jiang, J. Am. Chem. Soc. 142, 7737 (2020). doi: 10.1021/jacs.0c01825
    [17] Z. Qiao, M. Welborn, A. Anandkumar, F. R. Manby, and T. F. Miller III, J. Chem. Phys. 153, 124111 (2020). doi: 10.1063/5.0021955
    [18] Z. Zhang, J. Guan, and S. Zhou, Bioinformatics 37, 2981 (2021). doi: 10.1093/bioinformatics/btab195
    [19] Z. Zhang, Q. Liu, H. Wang, C. Lu, and C. Lee, arXiv: 2112.00911 (2021).
    [20] J. Behler, Chem. Rev. 121, 10037 (2021). doi: 10.1021/acs.chemrev.0c00868
    [21] Y. Zhang, C. Hu, and B. Jiang, J. Phys. Chem. Lett. 10, 4962 (2019). doi: 10.1021/acs.jpclett.9b02037
    [22] H. Wang, L. Zhang, J. Han, and W. E, Comput. Phys. Commun. 228, 178 (2018). doi: 10.1016/j.cpc.2018.03.016
    [23] B. Jiang, J. Li, and H. Guo, J. Phys. Chem. Lett. 11, 5120 (2020). doi: 10.1021/acs.jpclett.0c00989
    [24] I. Misra, A. Shrivastava, A. Gupta, and M. Hebert, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3994 (2016).
    [25] V. Sanh, T. Wolf, and S. Ruder, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 6949 (2019).
    [26] S. Sosnin, M. Vashurina, M. Withnall, P. Karpov, M. Fedorov, and I. V. Tetko, Mol. Inf. 38, 1800108 (2019). doi: 10.1002/minf.201800108
    [27] M. Crawshaw, arXiv: 2009.09796 (2020).
    [28] Z. Tan, Y. Li, W. Shi, and S. Yang, J. Chem. Inf. Model. 61, 3824 (2021). doi: 10.1021/acs.jcim.1c00646
    [29] Z. Liu, L. Lin, Q. Jia, Z. Cheng, Y. Jiang, Y. Guo, and J. Ma, J. Chem. Inf. Model. 61, 1066 (2021). doi: 10.1021/acs.jcim.0c01224
    [30] Y. Liu, L. Wang, M. Liu, X. Zhang, B. Oztekin, and S. Ji, arXiv: 2102.05013 (2021).
    [31] S. Ruder, arXiv: 1706.05098 (2017).
    [32] M. Liu, Y. Luo, L. Wang, Y. Xie, H. Yuan, S. Gui, H. Yu, Z. Xu, J. Zhang, Y. Liu, K. Yan, H. Liu, C. Fu, B. M. Oztekin, X. Zhang, and S. Ji, J. Mach. Learn. Res. 22, 1 (2021).
    [33] D. Eigen and R. Fergus, in Proceedings of the IEEE International Conference on Computer Vision, 2650 (2015).
    [34] A. Kendall, Y. Gal, and R. Cipolla, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7482 (2018).
    [35] L. Liebel and M. Körner, arXiv: 1805.06334 (2018).
    [36] S. Liu, E. Johns, and A. J. Davison, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1871 (2019).
    [37] S. Jean, O. Firat, and M. Johnson, arXiv: 1909.06434 (2019).
    [38] T. Gong, T. Lee, C. Stephenson, V. Renduchintala, S. Padhy, A. Ndirango, G. Keskin, and O. H. Elibol, IEEE Access 7, 141627 (2019). doi: 10.1109/ACCESS.2019.2943604
    [39] Z. Chen, V. Badrinarayanan, C. Y. Lee, and A. Rabinovich, in International Conference on Machine Learning, 794 (2018).
    [40] K. Schutt, P. Kessel, M. Gastegger, K. Nicoli, A. Tkatchenko, and K. R. Muller, J. Chem. Theory Comput. 15, 448 (2019). doi: 10.1021/acs.jctc.8b00908
    [41] H. B. Lee, E. Yang, and S. J. Hwang, in International Conference on Machine Learning, 2956 (2018).
    [42] Y. Gao, J. Ma, M. Zhao, W. Liu, and A. L. Yuille, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3205 (2019).
    [43] D. Xu, W. Ouyang, X. Wang, and N. Sebe, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 675 (2018).
  • 加载中
图(4) / 表(4)
计量
  • 文章访问数:  533
  • HTML全文浏览量:  216
  • PDF下载量:  48
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-03-29
  • 录用日期:  2022-04-20
  • 网络出版日期:  2022-07-22

目录

    /

    返回文章
    返回