[1]邓万宇,李力,牛慧娟.基于Spark的并行极速神经网络[J].郑州大学学报(工学版),2016,37(05):47-56.[doi:10.3969/ j.issn.1671 -6833.2016.05.010]
　Deng Wanyu,Li Li,Niu Huijuan.Sparked-based Parallel Extreme Learning Machine[J].Journal of Zhengzhou University (Engineering Science),2016,37(05):47-56.[doi:10.3969/ j.issn.1671 -6833.2016.05.010]
点击复制

基于Spark的并行极速神经网络()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
37
期数:
2016年05期
页码:
47-56
栏目:
出版日期:
2016-11-25

文章信息/Info

Title:
Sparked-based Parallel Extreme Learning Machine
作者:
邓万宇; 李力; 牛慧娟
西安邮电大学计算机学院,陕西西安,710121
Author(s):
Deng Wanyu; Li Li; Niu Huijuan
School of Computer Science, Xi’an University of Posts and Telecommunications, Xi’an, Shaanxi 710121
关键词:
极速学习机; 神经网络; 并行化ELM算法; Spark
Keywords:
extreme learning machine; neural network; parallelization ELM algorithm; Spark
分类号:
TP389.1
DOI:
10.3969/ j.issn.1671 -6833.2016.05.010
文献标志码:
A
摘要:
随着数据规模的快速膨胀,基于单机的串行神经网络结构面临着巨大的计算挑战,难以满足现实应用中的扩展需求.在极速学习机(extreme learning machine,ELM)基础上,基于Spark并行框架提出一种并行的极速神经网络学习方法,以Spark平台特有的RDD高效数据集管理机制对其进行封装,并将大规模数据中的高复杂度矩阵计算进行并行化,实现ELM加速求解,仅需一组Map和Reduce操作即可完成算法的训练.在大量真实数据集上的实验结果表明,基于Spark的并行ELM算法相较于串行ELM获得了显著的性能提升.
Abstract:
With the rapid expansion of data scale, the serial neural network structure based on a single machine is facing huge computing challenges, and it is difficult to meet the expansion requirements in real applications. Based on the extreme learning machine (extreme learning machine, ELM), based on the Spark parallel framework A parallel extremely fast neural network learning method is proposed, which is encapsulated with the unique RDD efficient data set management mechanism of the Spark platform, and the high-complexity matrix calculation in large-scale data is parallelized to achieve ELM accelerated solution. A set of Map and Reduce operations can complete the training of the algorithm. The experimental results on a large number of real data sets show that the parallel ELM algorithm based on Spark has achieved significant performance improvement compared with the serial ELM.

相似文献/References:

[1]蒋建东,张豪杰,王静.基于HHT的电力负荷组合预测应用[J].郑州大学学报(工学版),2015,36(04):1.[doi:10.3969/ j. issn.1671 - 6833.2015.04.001]
　JIANG Jian-dong,ZHANG Hao-jie,WANG Jing.Research and Application of HHT-Based Power Load Combination Forecasting[J].Journal of Zhengzhou University (Engineering Science),2015,36(05):1.[doi:10.3969/ j. issn.1671 - 6833.2015.04.001]
[2]肖斌,张恒宾,刘宏伟.改进PSO-BPNN算法在管道腐蚀预测中的应用[J].郑州大学学报(工学版),2022,43(01):27.[doi:10.13705/j.issn.1671-6833.2022.01.008]
　XIAO Bin,ZHANG Hengbin,LIU Hongwei.Application of Improved PSO-BPNN Algorithm in Corroded Pipelines Prediction[J].Journal of Zhengzhou University (Engineering Science),2022,43(05):27.[doi:10.13705/j.issn.1671-6833.2022.01.008]
[3]杨华芬,杨有,尚晋..一种改进的进化神经网络优化设计方法[J].郑州大学学报(工学版),2010,31(05):116.[doi:10.3969/j.issn.1671-6833.2010.05.028]
[4]周洪煜,陈晓煜,徐春霞..预测控制在中央空调净化系统中的应用[J].郑州大学学报(工学版),2008,29(03):73.[doi:10.3969/j.issn.1671-6833.2008.03.019]
　ZHOU Hongyu,CHEN Xiaoyu,Xu Chunxia.Application of predictive control in central air conditioning purification system[J].Journal of Zhengzhou University (Engineering Science),2008,29(05):73.[doi:10.3969/j.issn.1671-6833.2008.03.019]
[5]郭克希,谭佩莲,唐进元..基于人工神经网络的螺旋锥齿轮磨削加工表面粗糙度预测[J].郑州大学学报(工学版),2009,30(03):65.
　GUO Kexi,TAN Peilian,TANG Jinyuan.Surface Roughness Forecasting of Spiral Bevel Gear Based on Artificial Neural Network[J].Journal of Zhengzhou University (Engineering Science),2009,30(05):65.
[6]刘伟,刘赞,王玲玲..神经网络与结构编码法预测直馏汽油色谱保留指数[J].郑州大学学报(工学版),2004,25(03):26.[doi:10.3969/j.issn.1671-6833.2004.03.007]
　LIU Wei,LIU Zan,Wang Lingling.Neural network and structural coding method to predict the chromatographic retention index of straight-run gasoline[J].Journal of Zhengzhou University (Engineering Science),2004,25(05):26.[doi:10.3969/j.issn.1671-6833.2004.03.007]
[7]樊亚军,曲仕茹..利用BP神经网络实现三维飞机目标识别[J].郑州大学学报(工学版),2004,25(04):56.[doi:10.3969/j.issn.1671-6833.2004.04.015]
　Fan Yajun,Qu Shiru.BP neural network is used to realize three-dimensional aircraft target recognition[J].Journal of Zhengzhou University (Engineering Science),2004,25(05):56.[doi:10.3969/j.issn.1671-6833.2004.04.015]
[8]王少波,柴艳丽,梁醒培..神经网络学习样本点的选取方法比较[J].郑州大学学报(工学版),2003,24(01):63.[doi:10.3969/j.issn.1671-6833.2003.01.014]
　Wang Shaobo,Chai Yanli,Liang Xingpei.Comparison of the selection methods of neural network learning sample points[J].Journal of Zhengzhou University (Engineering Science),2003,24(05):63.[doi:10.3969/j.issn.1671-6833.2003.01.014]
[9]刘应梅,杨宛辉,章健,等.基于人工神经网络的变电站故障诊断[J].郑州大学学报(工学版),1999,20(04):86.[doi:10.3969/j.issn.1671-6833.1999.04.027]
　LIU Yingmei,Yang Wanhui,ZHANG Jian,et al.Substation fault diagnosis based on artificial neural network[J].Journal of Zhengzhou University (Engineering Science),1999,20(05):86.[doi:10.3969/j.issn.1671-6833.1999.04.027]
[10]高金峰,宁彦卿.谐波分析神经网络的开关电容实现[J].郑州大学学报(工学版),1997,18(04):1.
　Gao Jinfeng,Ning Yanqing.Harmonic analysis neural network switch capacitance realization[J].Journal of Zhengzhou University (Engineering Science),1997,18(05):1.

更新日期/Last Update: