Welcome to Journal of Beijing Institute of Technology
Advance Search
Volume 29Issue 4
Dec. 2020
Turn off MathJax
Article Contents
Article Navigation> JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY> 2020> 29(4): 526-530
Jihua Lu, Youcheng Zhang. Novel Representations of Word Embedding Based on the Zolu Function[J]. JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY, 2020, 29(4): 526-530. doi: 10.15918/j.jbit1004-0579.20076
Citation: Jihua Lu, Youcheng Zhang. Novel Representations of Word Embedding Based on the Zolu Function[J].JOURNAL OF BEIJING INSTITUTE OF TECHNOLOGY, 2020, 29(4): 526-530.doi:10.15918/j.jbit1004-0579.20076
shu

Novel Representations of Word Embedding Based on the Zolu Function

doi:10.15918/j.jbit1004-0579.20076
  • 1.

    School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China

  • 2.

    Department of Electronic Engineering, Tsinghua University, Beijing 100091, China

Funds:the National Natural Science Foundation of China(61771051; 61675025)
More Information
  • Corresponding author:lecturer, Ph.D. E-mail:lujihua@bit.edu.cn
  • Received Date:2020-07-06
  • Publish Date:2020-12-30
    Abstract
  • Two learning models, Zolu-continuous bags of words (ZL-CBOW) and Zolu-skip-grams (ZL-SG), based on the Zolu function are proposed. The slope of Relu in word2vec has been changed by the Zolu function. The proposed models can process extremely large data sets as well as word2vec without increasing the complexity. Also, the models outperform several word embedding methods both in word similarity and syntactic accuracy. The method of ZL-CBOW outperforms CBOW in accuracy by 8.43% on the training set of capital-world, and by 1.24% on the training set of plural-verbs. Moreover, experimental simulations on word similarity and syntactic accuracy show that ZL-CBOW and ZL-SG are superior to LL-CBOW and LL-SG, respectively.
    • FullText(HTML)
  • loading
    • References (13)
  • [1]
    Ji S, Satish N, Li S, et al. Parallelizing word2vec in shared and distributed memory [J]. IEEE Transactions on Parallel and Distributed Systems, 2019, 30(9): 2090−2100. doi:10.1109/TPDS.2019.2904058
    [2]
    Goodman E, Zimmerman C, Hudson C. Packet2Vec: Utilizing Word2Vec for feature extraction in packet data [J]. Computer Science, 2020(4): 1−16.
    [3]
    Gennaro G D, Buonanno A, Girolamo A D, et al. An analysis of Word2Vec for the Italian language[M]// Progresses in Artificial Intelligence and Neural Systems, Springer, 2021: 137-146.
    [4]
    Church K W. Word2Vec [J]. Natural Language Engineering, 2017, 23(1): 155−162. doi:10.1017/S1351324916000334
    [5]
    Mikolov T, Chen K, Corrado G, et al. Efficient estimation of word representations in vector space [J]. Computer Science, 2013, 7: 1−12.
    [6]
    Mikolov T, Sutskever I, Chen Kai, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems, December 5-10, 2013, Lake Tahoe, Nevada, USA, 2013: 3136-3144.
    [7]
    Yoshua B, Ducharme R, Vincent P, et al. A neural probabilistic language model [J]. Journal of Machine Learning Research (JMLR), 2003(3): 1137−1155.
    [8]
    Fuchida Y, Taniguchi T, Takano T, et al. Driving Word2vec: Distributed semantic vector representation for symbolized naturalistic driving data[C]//2016 IEEE Intelligent Vehicles Symposium (IV), Gotenburg, Sweden, 2016: 1313-1320.
    [9]
    Xu Y, Liu J. Implicitly Incorporating morphological information into Word embedding [J]. Computer Science, 2017, 5: 1−7.
    [10]
    Angulo C, Hernández-Vela A, Bautista M A, et al. BoVDW: Bag-of-visual-and-depth-words for gesture recognition[C]//Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), Tsukuba, Japan, 2012: 449-452.
    [11]
    Killian J, Morchid M, Dufour R, et al. A log-linear weighting approach in the Word2vec space for spoken language understanding[C]//2016 IEEE Spoken Language Technology Workshop (SLT), San Diego,USA, 2016: 356 -361.
    [12]
    Lu Jihua, Zhang Youcheng, Li Hongyu. A novel word-embedding model and training method based on Zolu function: CN201710421488.9 [P]. 2017-10-20. (in Chinese)
    [13]
    Rosenfeld R. Two decades of statistical language modeling: Where do we go from here? [J]. Proceedings of the IEEE, 2002, 88(8): 1270−1278.
    • Relative Articles
  • Supplements (0)
  • Cited By
  • 加载中

Catalog

    通讯作者:陈斌, bchen63@163.com
    • 1.

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(1)/Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (242) PDF downloads(42) Cited by()
    Proportional views
    Related

    Address:5 Zhongguancun South Street, Haidian District, Beijing

    Tel:86-10-68914374

    Email:blgywb@bit.edu.cn

    Copy Right © Journal of Beijing Institute of Technology

    Supported by:Beijing Renhe Information Technology Co. Ltd

    /

    Return
    Return
      Baidu
      map