CDH514部署手册.docx

CDH514部署手册.docx

《CDH514部署手册.docx》由会员分享，可在线阅读，更多相关《CDH514部署手册.docx（71页珍藏版）》请在冰豆网上搜索。

CDH514部署手册部署手册CDH大数据集群部署手册cdh5.14版本一、概述针对海尔大数据应用的需求，本文完成了CDH5.12大数据集群的部署，并记录了详细步骤。

二、集群部署准备1.集群硬件信息1.1集群各主机IP与主机名映射表1.2主机qdcdh00-01内存情况：

主机qdcdh02-09内存情况：

1.3主机逻辑cpu个数：

$cat/proc/cpuinfo|grepprocessor|wc-l主机物理cpu个数：

$cat/proc/cpuinfo|grepphysicalid|sort-u|wc-l主机单个cpu核心数：

$cat/proc/cpuinfo|grepcpucores|sort-u主机单个cpucore线程数：

$lscpu|grepThread主机qdcdh00-01的cpu信息：

主机qdcdh02-09的cpu信息：

1.4主机硬盘分区情况查看命令：

$df-h主机qdcdh00-04硬盘分区情况：

主机qdcdh05-09硬盘分区情况：

2.集群服务部署规划2.1集群各主机角色分为四类：

边缘节点一台（qdcdh00）、管理节点一台（qdcdh01）、主节点三台（qdcdh02-04）和数据节点五台（qdcdh05-09）;2.2各服务在集群主机上的分配：

服务主机qdcdh00qdcdh01qdcdh02qdcdh03qdcdh04qdcdh05-9CM服务ServerServer（未启用）AgenAgenAgenAgenDatabaseMysql（M）Mysql（S）kerberosKDC（M）KDC（S）openldapServer（M）Server（M）HAProxyServerHDFSGatewayNamenodeNamenodeDatanodeYARNGatewayHistoryResourcemanagerResourcemanagerNodeManagerserverHivehiveserver2hiveserver2metastoreImpalacatalogstatestoreimpaladHBaseGatewaymastermastermasterRegionSparkgatewayhistoryservergatewayOozieserverSqoopGatewayserverZookeeperserverserverserverKafkaKudumastermastermastertabletSentryGatewayserverHueserver/LBserver3.管理节点到其他节点免密钥配置3.1配置qdcdh01到其他主机的免密钥登录：

$ssh-keygen-trsa$ssh-copy-idqdcdh00-94.系统优化4.1调整swap分区的使用：

$echovm.swappiness=1/etc/sysctl.conf$sysctl-p4.2关闭透明大页：

$echonever/sys/kernel/mm/transparent_hugepage/defrag$echonever/sys/kernel/mm/transparent_hugepage/enabled将这两句添加到/etc/rc.local中。

5.时钟同步5.1设置qdcdh01为ntpserver,修改/etc/ntp.conf文件，将红色部分内容注释，添加蓝色内容：

rootqdcdh01cdh_install#cat/etc/ntp.conf#Formoreinformationaboutthisfile,seethemanpages#ntp.conf（5）,ntp_acc（5）,ntp_auth（5）,ntp_clock（5）,ntp_misc（5）,ntp_mon（5）.driftfile/var/lib/ntp/drift#Permittimesynchronizationwithourtimesource,butdonot#permitthesourcetoqueryormodifytheserviceonthissystem.restrictdefaultkodnomodifynotrapnopeernoqueryrestrict-6defaultkodnomodifynotrapnopeernoquery#Permitallaccessovertheloopbackinterface.Thiscould#betightenedaswell,buttodosowouldeffectsomeof#theadministrativefunctions.restrict127.0.0.1restrict-6:

1#Hostsonlocalnetworkarelessrestricted.#restrict192.168.1.0mask255.255.255.0nomodifynotrap#Usepublicserversfromthepool.ntp.orgproject.#Pleaseconsiderjoiningthepool（http:

/www.pool.ntp.org/join.html）.#server0.centos.pool.ntp.orgiburst#server1.centos.pool.ntp.orgiburst#server2.centos.pool.ntp.orgiburst#server3.centos.pool.ntp.orgiburstserver127.127.1.0fudge127.127.1.0stratum8restrict10.138.225.0mask255.255.255.0nomodifynotrap#broadcast192.168.1.255autokey#broadcastserver#broadcastclient#broadcastclient#broadcast224.0.1.1autokey#multicastserver#multicastclient224.0.1.1#multicastclient#manycastserver239.255.254.254#manycastserver#manycastclient239.255.254.254autokey#manycastclient#Enablepublickeycryptography.#cryptoincludefile/etc/ntp/crypto/pw#Keyfilecontainingthekeysandkeyidentifiersusedwhenoperating#withsymmetrickeycryptography.keys/etc/ntp/keys#Specifythekeyidentifierswhicharetrusted.#trustedkey4842#Specifythekeyidentifiertousewiththentpdcutility.#requestkey8#Specifythekeyidentifiertousewiththentpqutility.#controlkey8#Enablewritingofstatisticsrecords.#statisticsclockstatscryptostatsloopstatspeerstats5.2启动qdcdh01上的ntpd服务，并设置开机启动：

$servicentpdstart$chkconfigntpdon5.3在集群的其他主机上，修改/etc/ntp.conf文件，修改server配置字段内容问qdcdh01主机，注释红色部分，添加蓝色部分：

rootqdcdh00#cat/etc/ntp.conf#Formoreinformationaboutthisfile,seethemanpages#ntp.conf（5）,ntp_acc（5）,ntp_auth（5）,ntp_clock（5）,ntp_misc（5）,ntp_mon（5）.driftfile/var/lib/ntp/drift#Permittimesynchronizationwithourtimesource,butdonot#permitthesourcetoqueryormodifytheserviceonthissystem.restrictdefaultkodnomodifynotrapnopeernoqueryrestrict-6defaultkodnomodifynotrapnopeernoquery#Permitallaccessovertheloopbackinterface.Thiscould#betightenedaswell,buttodosowouldeffectsomeof#theadministrativefunctions.restrict127.0.0.1restrict-6:

1#Hostsonlocalnetworkarelessrestricted.#restrict192.168.1.0mask255.255.255.0nomodifynotrap#Usepublicserversfromthepool.ntp.orgproject.#Pleaseconsiderjoiningthepool（http:

/www.pool.ntp.org/join.html）.#server0.centos.pool.ntp.orgiburst#server1.centos.pool.ntp.orgiburst#server2.centos.pool.ntp.orgiburst#server3.centos.pool.ntp.orgiburstserver#broadcast192.168.1.255autokey#broadcastserver#broadcastclient#broadcastclient#broadcast224.0.1.1autokey#multicastserver#multicastclient224.0.1.1#multicastclient#manycastserver239.255.254.254#manycastserver#manycastclient239.255.254.254autokey#manycastclient#Enablepublickeycryptography.#cryptoincludefile/etc/ntp/crypto/pw#Keyfilecontainingthekeysandkeyidentifiersusedwhenoperating#withsymmetrickeycryptography.keys/etc/ntp/keys#Specifythekeyidentifierswhicharetrusted.#trustedkey4842#Specifythekeyidentifiertousewiththentpdcutility.#requestkey8#Specifythekeyidentifiertousewiththentpqutility.#controlkey8#Enablewritingofstatisticsrecords.#statisticsclockstatscryptostatsloopstatspeerstats5.4如5.2操作，重启各主机的ntpd服务，并添加开机启动。

6.CDH本地yum源搭建6.1在qdcdh01主机上安装httpd服务，并将本次安装的cdh5.12相关安装文件放到/var/www/html下：

$yuminstallhttpd6.2在集群的各主机/etc/yum.repos.d下添加自建的cdh_5.12.repo文件：

cm_5.12name=clouderamanager5.12.1baseurl=http:

/qdcdh01/cdh_5.12_files/cm_5.12.1gpgcheck=0mysql_5.7name=mysql5.7baseurl=http:

/qdcdh01/cdh_5.12_files/other_rpms/mysql_5.7gpgcheck=0cdh_other_paksname=cdhotherpackagesbaseurl=http:

/qdcdh01/cdh_5.12_files/other_rpms/other_depsgpgcheck=06.3检查yum源中是否存在cdh5.12的相关包：

7.MySQL主从安装配置7.1在qdcdh01和qdcdh00上安装MySQL，其中qdcdh01为master，qdcdh00为slave；7.2卸载系统中原先安装的mysql-libs：

$rpm-emysql-libs-nodeps7.3在qdcdh01上安装MySQLserver：

$yuminstallmysqlmysql-servermysql-community-libs-compat7.4初始化MySQL数据库：

$mysqld-user=mysql-datadir=/var/lib/mysql-initialize7.5创建MySQL日志目录：

$mkdir/var/log/mysql$chownmysql:

mysql-R/var/log/mysql7.6修改配置文件/etc/f:

rootqdcdh01yum.repos.d#cat/etc/f#Foradviceonhowtochangesettingspleasesee#http:

/mysqld#Removeleading#andsettotheamountofRAMforthemostimportantdata#cacheinMySQL.Startat70%oftotalRAMfordedicatedserver,else10%.#innodb_buffer_pool_size=128M#Removeleading#toturnonaveryimportantdataintegrityoption:

logging#changestothebinarylogbetweenbackups.#log_bin#Removeleading#tosetoptionsmainlyusefulforreportingservers.#TheserverdefaultsarefasterfortransactionsandfastSELECTs.#Adjustsizesasneeded,experimenttofindtheoptimalvalues.#join_buffer_size=128M#sort_buffer_size=2M#read_rnd_buffer_size=2Mdatadir=/var/lib/mysqlsocket=/var/lib/mysql/mysql.sockserver-id=38transaction-isolation=READ-COMMITTEDkey_buffer_size=32Mmax_allowed_packet=32Mthread_stack=256Kthread_cache_size=64query_cache_limit=8Mquery_cache_size=64Mquery_cache_type=1max_connections=550log_bin=/var/lib/mysql/mysql_binary_logbinlog_format=mixedrelay-log=/var/lib/mysql/mysql-relay-binrelay-log-index=/var/lib/mysql/mysql-relay-bin.indexmaster-info-file=/var/lib/mysql/mysql-master.inforelay-log-info-file=/var/lib/mysql/mysql-relay-log.infolog-bin=/var/lib/mysql/mysql-binread_buffer_size=2Mread_rnd_buffer_size=16Msort_buffer_size=8Mjoin_buffer_size=8Minnodb_file_per_table=1innodb_flush_log_at_trx_commit=2innodb_log_buffer_size=64Minnodb_buffer_pool_size=4Ginnodb_thread_concurrency=8innodb_flush_method=O_DIRECTinnodb_log_file_size=512Mbind-address=0.0.0.0default-storage-engine=innodbsql_mode=STRICT_ALL_TABLES#Disablingsymbolic-linksisrecommendedtopreventassortedsecurityriskssymbolic-links=0log-error=/var/log/mysqld.logpid-file=/var/run/mysqld/mysqld.pid7.7启动MySQL服务，并设置开机启动：

$servicemysqldstart$chkconfigmysqldon7.8通过mysql_secure_installation命令设置mysql，需要输入的mysql初始化密码位于/var/log/mysqld.log中：

$mysql_secure_installation设置mysql密码为Pass_haier,并设置不允许root用户远程登录，然后用新密码登录测试：

$mysql-uroot-hlocalhost-p7.9将mysql-connector-java.jar放到Java共享目录下：

$cpmysql-connector-java-5.1.38-bin.jar/usr/share/java/mysql-connector-java.jar7.10以上步骤同样在qdcdh00上操作一遍；7.11在qdcdh00和qdcdh01两台机器上同步执行如下命令：

$mysql-uroot-p7.12在master（qdcdh01）上设置用来同步数据的用户rep、slave的host（qdcdh00）及密码repmysql:

mysqlusemysql;mysqlGRANTALLPRIVILEGESON*.*TOroot%IDENTIFIEDBYmysqlWITHGRANTOPTION;mysqlGRANTREPLICATIONSLAVEON*.*TOIDENTIFIEDBYrepmysql;mysqlFLUSHTABLESWITHREADLOCK;mysqlSHOWMASTERSTATUS;+-+-+-+-+-+|File|Position|Binlog_Do_DB|Binlog_Ignore_DB|Executed_Gtid_Set|+-+-+-+-+-+|mysql-bin.000001|1393|+-+-+-+-+-+1rowinset（0.00sec）此时master上不要做任何操作；7.13在slave（qdcdh00）上设置进行设置master的主机名，master的同步用户名及同步用户密码，以及在master上执行的SHOWMASTERSTATUS的属性：

mysqlusemysql;mysqlstopslave;mysqlCHANGEMASTERTOMASTER_HOST=,MASTER_USER=rep,MASTER_PASSWORD=repmysql,mysqlMASTER_LOG_FILE=mysql-bin.000001,MASTER_LOG_POS=1393;mysqlstartslave;7.14在masterMySQL上进行测试：

mysqlUNLOCKTABLES;mysqlcreatedatabasetest1;mysqlusetest1;mysqlcreatetabletb1（idINT（4）,text（varchar（10）;7.15在slavemysql上查看数据库是否同步过去，同步则MySQL主从配置完成。

8.创建集群各服务的数据库createdatabasescmdefaultcharactersetutf8;grantallonscm.*toscm%identifiedbyPass_scm;createdatabaseamondefaultcharactersetutf8;grantallonamon.*toamon%identifiedbyPass_amon;createdatabasermandefaultcharactersetutf8;grantallonrman.*torman%identifiedbyPass_rman;createdatabasemetastoredefaultcharactersetutf8;grantallonmetastore.*tohive%identifiedbyPass_hive;createdatabasesentrydefaultcharactersetutf8;grantallonsentry.*tosentry%identifiedbyPass_sentry;createdatabasenavdefaultcharactersetutf8;grantallonnav.*tonav%identifiedbyPass_nav;createdatabasenavmsdefaultcharactersetutf8;grantallonnavms.*tonavms%identifiedbyPass_navms;createdatabasehuedefaultcharactersetutf8defaultcollateu