Yao-long Zhang, Xue-yao Zhou, Bin Jiang. Accelerating the Construction of Neural Network Potential Energy Surfaces: A Fast Hybrid Training Algorithm[J]. Chinese Journal of Chemical Physics , 2017, 30(6): 727-734. doi: 10.1063/1674-0068/30/cjcp1711212
Citation: Yao-long Zhang, Xue-yao Zhou, Bin Jiang. Accelerating the Construction of Neural Network Potential Energy Surfaces: A Fast Hybrid Training Algorithm[J]. Chinese Journal of Chemical Physics , 2017, 30(6): 727-734. doi: 10.1063/1674-0068/30/cjcp1711212

Accelerating the Construction of Neural Network Potential Energy Surfaces: A Fast Hybrid Training Algorithm

doi: 10.1063/1674-0068/30/cjcp1711212
  • Received Date: 2017-11-14
  • Rev Recd Date: 2017-12-12
  • Machine learning approaches have been promising in constructing high-dimensional potential energy surfaces (PESs) for molecules and materials. Neural networks (NNs) are one of the most popular such tools because of its simplicity and efficiency. The training algorithm for NNs becomes essential to achieve a fast and accurate fit with numerous data. The Levenberg-Marquardt (LM) algorithm has been recognized as one of the fastest and robust algorithms to train medium sized NNs and widely applied in recent NN based high quality PESs. However, when the number of ab initio data becomes large, the efficiency of LM is limited, making the training time consuming. Extreme learning machine (ELM) is a recently proposed algorithm which determines the weights and biases of a single hidden layer NN by a linear solution and is thus extremely fast. It, however, does not produce sufficiently small fitting error because of its random nature. Taking advantages of both algorithms, we report a generalized hybrid algorithm in training multilayer NNs. Tests on H+H2 and CH4+Ni(111) systems demonstrate the much higher efficiency of this hybrid algorithm (ELM-LM) over the original LM. We expect that ELM-LM will find its widespread applications in building up high-dimensional NN based PESs.

     

  • loading
  • [1]
    J. N. Murrell, S. Carter, S. C. Farantos, P. Huxley, and A. J. C. Varandas, Molecular Potential Energy Functions, Chichester:Wiley, (1984).
    [2]
    P. J. K. H. J. Werner, G. Knizia, F. R. Manby, M. Schütz, P. Celani, W. Györffy, D. Kats, T. Korona, R. Lindh, A. Mitrushenkov, G. Rauhut, K. R. Shamasundar, T. B. Adler, R. D. Amos, A. Bernhardsson, A. Berning, D. L. Cooper, M. J. O. Deegan, A. J. Dobbyn, F. Eckert, E. Goll, C. Hampel, A. Hesselmann, G. Hetzer, T. Hrenar, G. Jansen, C. Köppl, Y. Liu, A. W. Lloyd, R. A. Mata, A. J. May, S. J. McNicholas, W. Meyer, M. E. Mura, A. Nicklaß, D. P. O'Neill, P. Palmieri, D. Peng, K. Pflüger, R. Pitzer, M. Reiher, T. Shiozaki, H. Stoll, A. J. Stone, R. Tarroni, T. Thorsteinsson, and M. Wang, see http://www.molpro.net., (2015).
    [3]
    G. Kresse and J. Furthmuller, Comp. Mater. Sci. 6, 15(1996).
    [4]
    B. Jiang, J. Li, and H. Guo, Int. Rev. Phys. Chem. 35, 479(2016).
    [5]
    M. I. Jordan and T. M. Mitchell, Science 349, 255(2015).
    [6]
    J. Behler and M. Parrinello, Phys. Rev. Lett. 98, 146401(2007).
    [7]
    A. P. Bartók, M. C. Payne, R. Kondor, and G. Csányi, Phys. Rev. Lett. 104, 136403(2010).
    [8]
    J. C. Snyder, M. Rupp, K. Hansen, K. R. Müller, and K. Burke, Phys. Rev. Lett. 108, 253002(2012).
    [9]
    J. Cui and R. V. Krems, Phys. Rev. Lett. 115, 073202(2015).
    [10]
    A. P. Bartók and G. Csányi, Int. J. Quant. Chem. 115, 1051(2015).
    [11]
    J. Behler, J. Chem. Phys. 145, 170901(2016).
    [12]
    T. B. Blank, S. D. Brown, A. W. Calhoun, and D. J. Doren, J. Chem. Phys. 103, 4129(1995).
    [13]
    J. Chen, X. Xu, and D. H. Zhang, J. Chem. Phys. 138, 154301(2013).
    [14]
    J. Chen, X. Xu, X. Xu, and D. H. Zhang, J. Chem. Phys. 138, 221104(2013).
    [15]
    J. Li and H. Guo, Phys. Chem. Chem. Phys. 16, 6753(2014).
    [16]
    J. Li and H. Guo, J. Chem. Phys. 143, 221103(2015).
    [17]
    D. D. Lu and J. Li, J. Chem. Phys. 145, 014303(2016).
    [18]
    J. Li, R. Dawes, and H. Guo, Phys. Chem. Chem. Phys. 18, 29825(2016).
    [19]
    B. Jiang and H. Guo, Phys. Chem. Chem. Phys. 16, 24704(2014).
    [20]
    B. Kolb, X. Luo, X. Zhou, B. Jiang, and H. Guo, J. Phys. Chem. Lett. 8, 666(2017).
    [21]
    B. Jiang and H. Guo, J. Chem. Phys. 144, 091101(2016).
    [22]
    B. Jiang and H. Guo, Phys. Rev. Lett. 114, 166101(2015).
    [23]
    T. H. Liu, B. N. Fu, and D. H. Zhang, Sci. China:Chem. 57, 147(2014).
    [24]
    T. Liu, Z. Zhang, B. Fu, X. Yang, and D. H. Zhang, Chem. Sci. 7, 1840(2016).
    [25]
    X. J. Shen, J. Chen, Z. J. Zhang, K. J. Shao, and D. H. Zhang, J. Chem. Phys. 143, 144701(2015).
    [26]
    X. Shen, Z. Zhang, and D. H. Zhang, Phys. Chem. Chem. Phys. 17, 25499(2015).
    [27]
    S. Lorenz, A. Groß, and M. Scheffler, Chem. Phys. Lett. 395, 210(2004).
    [28]
    K. Shakouri, J. Behler, J. Meyer, and G. J. Kroes, J. Phys. Chem. Lett. 8, 2131(2017).
    [29]
    S. Kondati Natarajan, T. Morawietz, and J. Behler, Phys. Chem. Chem. Phys. 17, 8356(2015).
    [30]
    T. Morawietz, V. Sharma, and J. Behler, J. Chem. Phys. 136, 064103(2012).
    [31]
    J. Behler and M. Parrinello, Phys. Rev. Lett. 98, 146401(2007).
    [32]
    S. K. Natarajan and J. Behler, Phys. Chem. Chem. Phys. 18, 28704(2016).
    [33]
    J. Behler, S. Lorenz, and K. Reuter, J. Chem. Phys. 127, 014705(2007).
    [34]
    B. Jiang and H. Guo, J. Chem. Phys. 139, 054112(2013).
    [35]
    K. Shao, J. Chen, Z. Zhao, and D. H. Zhang, J. Chem. Phys. 145, 071101(2016).
    [36]
    L. M. Raff, M. Malshe, M. Hagan, D. I. Doughan, M. G. Rockley, and R. Komanduri, J. Chem. Phys. 122, 084104(2005).
    [37]
    S. Manzhos, X. G. Wang, R. Dawes, and T. Carrington, J. Phys. Chem. A 110, 5295(2006).
    [38]
    J. Behler, J. Chem. Phys. 134, 074106(2011).
    [39]
    S. Manzhos, R. Dawes, and T. Carrington, Int. J. Quant. Chem. 115, 1012(2015).
    [40]
    S. Manzhos and T. Carrington Jr., J. Chem. Phys. 125, 194105(2006).
    [41]
    L. M. Raff, R. Komanduri, M. Hagan, and S. T. S. Bukkapatnam, Neural Networks in Chemical Reaction Dynamics, Oxford:Oxford University Press, (2012).
    [42]
    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Nature 323, 533(1986).
    [43]
    C. Igel and M. Hsken, in Proceedings of the Second International ICSC Symposium on Neural Computation, NC'2000, Berlin, Germany:ICSC Academic Press, 115(2000).
    [44]
    M. T. Hagan and M. B. Menhaj, IEEE Trans. Neural Networks 5, 989(1994).
    [45]
    T. B. Blank and S. D. Brown, J. Chemom. 8, 391(1994).
    [46]
    G. B. Huang, Q. Y. Zhu, and C. K. Siew, presented at the 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), (2004).
    [47]
    J. Tang, C. Deng, and G. B. Huang, IEEE Trans. Neural Netw. Learn. Syst. 27, 809(2016).
    [48]
    H. Chen and F. Jin, Advances in Neural Networks-ISNN 2006, J. Wang, Z. Yi, J. M. Zurada, B. L. Lu and H. Yin, Eds, Berlin, Heidelberg:Springer 509(2006).
    [49]
    B. Jiang and H. Guo, J. Chem. Phys. 141, 034109(2014).
    [50]
    J. Li, B. Jiang, and H. Guo, J. Chem. Phys. 139, 204103(2013).
    [51]
    Z. Xie and J. M. Bowman, J. Chem. Theo. Comp. 6, 26(2010).
    [52]
    K. Levenberg, Q. J. Math. 2, 164(1944).
    [53]
    D. W. Marquardt, J. Soc. Ind. Appl. Math. 11, 431(1963).
    [54]
    A. I. Boothroyd, W. J. Keogh, P. G. Martin, and M. R. Peterson, J. Chem. Phys. 104, 7139(1996).
    [55]
    X. Zhou, F. Nattino, Y. Zhang, J. Chen, G. J. Kroes, H. Guo, and B. Jiang, Phys. Chem. Chem. Phys. 19, 30540(2017).
    [56]
    D. Nguyen and B. Widrow, 1990 IJCNN International Joint Conference on Neural Networks, 21(1990).
    [57]
    X. Hu, W. L. Hase, and T. Pirraglia, J. Comp. Chem. 12, 1014(1991).
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (1124) PDF downloads(526) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return