Comparison of Multi-task Approaches on Molecular Property Prediction

Chao Han; Hao Wang; Jianbao Zhu; Qi Liu; Wenguang Zhu

doi:10.1063/1674-0068/cjcp2203055

Article Contents

Article Navigation > Chinese Journal of Chemical Physics > 2022 > Online First

Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055

Citation:

Chao Han, Hao Wang, Jianbao Zhu, Qi Liu, Wenguang Zhu. Comparison of Multi-task Approaches on Molecular Property Prediction[J]. Chinese Journal of Chemical Physics . doi: 10.1063/1674-0068/cjcp2203055

Citation:

PDF( 806 KB)

Comparison of Multi-task Approaches on Molecular Property Prediction

doi: 10.1063/1674-0068/cjcp2203055

1.
School of Physical Sciences, University of Science and Technology of China, Hefei 230026, China
2.
Anhui Province Key Lab. of Big Data Analysis and Application, School of Computer Science and Technology, University of Science and Technology of China, Hefei 230026, China

More Information

Corresponding author: E-mail: wgzhu@ustc.edu.cn
Received Date: 2022-03-29
Accepted Date: 2022-04-20

Available Online: 2022-07-22

Abstract

Abstract

With the bloom of deep learning algorithms, various models have been widely utilized in quantum chemistry calculation to design new molecules and explore molecular properties. However, limited studies focus on multi-task molecular property prediction, which offers more efficient ways to simultaneously learn different but related properties by leveraging the inter-task relationship. In this work, we apply the hard parameter sharing framework and advanced loss weighting methods to multi-task molecular property prediction. Based on the performance comparison between single-task baseline and multi-task models on several task sets, we find that the prediction accuracy largely depends on the inter-task relationship, and hard parameter sharing improves the performance when the correlation becomes complex. In addition, we show that proper loss weighting methods help achieve more balanced multi-task optimization and enhance the prediction accuracy. Our additional experiments on varying amount of training data further validate the multi-task advantages and show that multi-task models with proper loss weighting methods can achieve more accurate prediction of molecular properties with much less computational cost.
- Deep learning,
- Multi-task learning,
- Molecular property prediction,
- Uncertainty weighting

¹ As there are different versions of QM9 dataset, we use the dataset from DIG [32] package where the data of $ \omega_1 $ is not provided. Therefore, we keep the same with many other studies on QM9 and have not taken $ \omega_1 $ into consideration.

FullText(HTML)

References(43)

References

[1]	R. Ramakrishnan, M. Hartmann, E. Tapavicza, and O. A. Von Lilienfeld, J. Chem. Phys. 143, 084111 (2015). doi: 10.1063/1.4928757
[2]	R. Ramakrishnan, P. O. Dral, M. Rupp, and O. A. Von Lilienfeld, Sci. Data 1, 140022 (2014). doi: 10.1038/sdata.2014.22
[3]	L. Ruddigkeit, R. van Deursen, L. C. Blum, and J. L. Reymond, J. Chem. Inf. Model. 52, 2864 (2012). doi: 10.1021/ci300415d
[4]	G. Chen, P. Chen, C. Y. Hsieh, C. K. Lee, B. Liao, R. Liao, W. Liu, J. Qiu, Q. Sun, J. Tang, R. Zemel, and S. Zhang, arXiv: 1906.09427 (2019).
[5]	K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, and A. Walsh, Nature 559, 547 (2018). doi: 10.1038/s41586-018-0337-2
[6]	J. Wei, X. Chu, X. Y. Sun, K. Xu, H. X. Deng, J. Chen, Z. Wei, and M. Lei, InfoMat 1, 338 (2019). doi: 10.1002/inf2.12028
[7]	B. Sanchez-Lengeling and A. Aspuru-Guzik, Science 361, 360 (2018). doi: 10.1126/science.aat2663
[8]	J. Wang, C. Y. Hsieh, M. Wang, X. Wang, Z. Wu, D. Jiang, B. Liao, X. Zhang, B. Yang, Q. He, D. Cao, X. Chen, and T. Hou, Nat. Mach. Intell. 3, 914 (2021). doi: 10.1038/s42256-021-00403-1
[9]	Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V. Pande, Chem. Sci. 9, 513 (2018). doi: 10.1039/C7SC02664A
[10]	J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, in Proceedings of the 34th International Conference on Machine Learning, 1263 (2017).
[11]	K. T. Schütt, H. E. Sauceda, P. J. Kindermans, A. Tkatchenko, and K. R. Müller, J. Chem. Phys. 148, 241722 (2018). doi: 10.1063/1.5019779
[12]	C. Lu, Q. Liu, C. Wang, Z. Huang, P. Lin, and L. He, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 1052 (2019).
[13]	O. T. Unke and M. Meuwly, J. Chem. Theory Comput. 15, 3678 (2019). doi: 10.1021/acs.jctc.9b00181
[14]	C. Chen, W. Ye, Y. Zuo, C. Zheng, and S. P. Ong, Chem. Mater. 31, 3564 (2019). doi: 10.1021/acs.chemmater.9b01294
[15]	J. Klicpera, J. Groß, and S. Günnemann, in International Conference on Learning Representations, (2020).
[16]	X. Wang, S. Ye, W. Hu, E. Sharman, R. Liu, Y. Liu, Y. Luo, and J. Jiang, J. Am. Chem. Soc. 142, 7737 (2020). doi: 10.1021/jacs.0c01825
[17]	Z. Qiao, M. Welborn, A. Anandkumar, F. R. Manby, and T. F. Miller III, J. Chem. Phys. 153, 124111 (2020). doi: 10.1063/5.0021955
[18]	Z. Zhang, J. Guan, and S. Zhou, Bioinformatics 37, 2981 (2021). doi: 10.1093/bioinformatics/btab195
[19]	Z. Zhang, Q. Liu, H. Wang, C. Lu, and C. Lee, arXiv: 2112.00911 (2021).
[20]	J. Behler, Chem. Rev. 121, 10037 (2021). doi: 10.1021/acs.chemrev.0c00868
[21]	Y. Zhang, C. Hu, and B. Jiang, J. Phys. Chem. Lett. 10, 4962 (2019). doi: 10.1021/acs.jpclett.9b02037
[22]	H. Wang, L. Zhang, J. Han, and W. E, Comput. Phys. Commun. 228, 178 (2018). doi: 10.1016/j.cpc.2018.03.016
[23]	B. Jiang, J. Li, and H. Guo, J. Phys. Chem. Lett. 11, 5120 (2020). doi: 10.1021/acs.jpclett.0c00989
[24]	I. Misra, A. Shrivastava, A. Gupta, and M. Hebert, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3994 (2016).
[25]	V. Sanh, T. Wolf, and S. Ruder, in Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 6949 (2019).
[26]	S. Sosnin, M. Vashurina, M. Withnall, P. Karpov, M. Fedorov, and I. V. Tetko, Mol. Inf. 38, 1800108 (2019). doi: 10.1002/minf.201800108
[27]	M. Crawshaw, arXiv: 2009.09796 (2020).
[28]	Z. Tan, Y. Li, W. Shi, and S. Yang, J. Chem. Inf. Model. 61, 3824 (2021). doi: 10.1021/acs.jcim.1c00646
[29]	Z. Liu, L. Lin, Q. Jia, Z. Cheng, Y. Jiang, Y. Guo, and J. Ma, J. Chem. Inf. Model. 61, 1066 (2021). doi: 10.1021/acs.jcim.0c01224
[30]	Y. Liu, L. Wang, M. Liu, X. Zhang, B. Oztekin, and S. Ji, arXiv: 2102.05013 (2021).
[31]	S. Ruder, arXiv: 1706.05098 (2017).
[32]	M. Liu, Y. Luo, L. Wang, Y. Xie, H. Yuan, S. Gui, H. Yu, Z. Xu, J. Zhang, Y. Liu, K. Yan, H. Liu, C. Fu, B. M. Oztekin, X. Zhang, and S. Ji, J. Mach. Learn. Res. 22, 1 (2021).
[33]	D. Eigen and R. Fergus, in Proceedings of the IEEE International Conference on Computer Vision, 2650 (2015).
[34]	A. Kendall, Y. Gal, and R. Cipolla, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7482 (2018).
[35]	L. Liebel and M. Körner, arXiv: 1805.06334 (2018).
[36]	S. Liu, E. Johns, and A. J. Davison, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 1871 (2019).
[37]	S. Jean, O. Firat, and M. Johnson, arXiv: 1909.06434 (2019).
[38]	T. Gong, T. Lee, C. Stephenson, V. Renduchintala, S. Padhy, A. Ndirango, G. Keskin, and O. H. Elibol, IEEE Access 7, 141627 (2019). doi: 10.1109/ACCESS.2019.2943604
[39]	Z. Chen, V. Badrinarayanan, C. Y. Lee, and A. Rabinovich, in International Conference on Machine Learning, 794 (2018).
[40]	K. Schutt, P. Kessel, M. Gastegger, K. Nicoli, A. Tkatchenko, and K. R. Muller, J. Chem. Theory Comput. 15, 448 (2019). doi: 10.1021/acs.jctc.8b00908
[41]	H. B. Lee, E. Yang, and S. J. Hwang, in International Conference on Machine Learning, 2956 (2018).
[42]	Y. Gao, J. Ma, M. Zhao, W. Liu, and A. L. Yuille, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3205 (2019).
[43]	D. Xu, W. Ouyang, X. Wang, and N. Sebe, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 675 (2018).