Greenplum的安装.docx
《Greenplum的安装.docx》由会员分享，可在线阅读，更多相关《Greenplum的安装.docx（26页珍藏版）》请在冰豆网上搜索。
Greenplum的安装
Greenplum的安装
Geenplum是一款基于postgresql开发的mpp数据库，适合于数据仓库数据存储，是对hadoop无法有效处理结构化数据的一个很好的补充。
五年前曾经使用过greenplum，那时的版本还是3.6，现在已经更新到4.3.5了，有消息称今年将开源。
和以前的相比，安装方式也有了一些小变化，简记如下，如要更详细的说明，还是得看官方文档。
一、测试环境CentOS6.58台虚机Greenplum4.3.5二、环境准备1.修改系统参数
编辑/etc/sysctl.conf，以下是最小配置
kernel.shmmax=500000000
kernel.shmmni=4096
kernel.shmall=4000000000
kernel.sem=2505120001002048
kernel.sysrq=1
kernel.core_uses_pid=1
kernel.msgmnb=65536
kernel.msgmax=65536
kernel.msgmni=2048
net.ipv4.tcp_syncookies=1
net.ipv4.ip_forward=0
net.ipv4.conf.default.accept_source_route=0
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_max_syn_backlog=4096
net.ipv4.conf.all.arp_filter=1
net.ipv4.ip_local_port_range=102565535
dev_max_backlog=10000
net.core.rmem_max=2097152
net.core.wmem_max=2097152
vm.overcommit_memory=22.修改文件打开数等限制
编辑/etc/security/limits.conf
*softnofile65536
*hardnofile65536
*softnproc131072
*hardnproc131072编辑/etc/security/limits.d/90-nproc.conf
*softnofile65536
*hardnofile65536
*softnproc131072
*hardnproc1310723.修改scheduler
编辑scheduler，原先默认是cfq
echodeadline>/sys/block/vda/queue/scheduler
[root@bigdata03~]#more/sys/block/vda/queue/scheduler
noopanticipatory[deadline]cfq4.编辑/boot/grub/grub.conf
新增elevator=deadline
[root@bigdata03~]#more/boot/grub/grub.conf
...elevator=deadlinecrashkernel=auto...5.编辑设备块大小
[root@bigdata03~]#/sbin/blockdev--setra16384/dev/vda
[root@bigdata03~]#/sbin/blockdev--getra/dev/vda
163846.配置hosts文件
编辑/etc/hosts，添加，选中203作为master，其他为segment,偷懒没改，直接加了层别名
172.30.13.203bigdata03.rdchzmdw
172.30.13.204bigdata04.rdchzsdw1
172.30.13.205bigdata05.rdchzsdw2
172.30.13.206bigdata06.rdchzsdw3
172.30.13.207bigdata07.rdchzsdw4
172.30.13.208bigdata08.rdchzsdw5
172.30.13.209bigdata09.rdchzsdw6
172.30.13.210bigdata10.rdchzsdw77.分别创建各数据节点文件（一个含master，一个不含）
[root@bigdata03~]#more/tmp/hostfile_exkeys
mdw
sdw1
sdw2
sdw3
sdw4
sdw5
sdw6
sdw7[root@bigdata03data]#more/tmp/hostfile_segments
sdw1
sdw2
sdw3
sdw4
sdw5
sdw6
sdw7三、集群安装1.软件下载，上传、解压缩
下载地址：
https:
//network.pivotal.io/products/pivotal-gpdb
unzipgreenplum-db-4.3.5.2-build-1-RHEL5-x86_64.zip
/bin/bashgreenplum-db-4.3.5.2-build-1-RHEL5-x86_64.bin输入yes同意aggrement
输入安装路径，比如/usr/local/gpmaster最后安装完后，结果是这样的：
[root@bigdata03gpmaster]#ll
total284
drwxr-xr-x3gpadmingpadmin4096Jun2412:
29bin
drwxr-xr-x2gpadmingoadmin4096Jun2412:
09demo
drwxr-xr-x5gpadmingpadmin4096Jun2412:
09docs
drwxr-xr-x2gpadmingpadmin4096Jun2412:
09etc
drwxr-xr-x3gpadmingpadmin4096Jun2412:
09ext
-rw-r--r--1gpadmingpadmin43025Jun2412:
29GPDB-LICENSE.txt
-rw-r--r--1gpadmingpadmin664Jul2814:
51greenplum_path.sh
drwxr-xr-x6gpadmingpadmin4096Jun2412:
09include
drwxr-xr-x9gpadmingpadmin12288Jun2412:
09lib
-rw-r--r--1gpadmingpadmin193083Jun2412:
29LICENSE.thirdparty
drwxr-xr-x2gpadmingpadmin4096Jun2412:
29sbin
drwxr-xr-x3gpadmingpadmin4096Jun2411:
57share2.建立master与各节点的信任关系
[root@bigdata03~]#source/usr/local/greenplum-db/greenplum_path.sh
[root@bigdata03~]#gpssh-exkeys-f/tmp/hostfile_exkeys
[STEP1of5]createlocalIDandauthorizeonlocalhost
[STEP2of5]keyscanallhostsandupdateknown_hostsfile
[STEP3of5]authorizecurrentuseronremotehosts
...sendtosdw1
***
***Enterpasswordforsdw1:
...sendtosdw2
...sendtosdw3
...sendtosdw4
...sendtosdw5
...sendtosdw6
...sendtosdw7
[STEP4of5]determinecommonauthenticationfilecontent
[STEP5of5]copyauthenticationfilestoallremotehosts
...finishedkeyexchangewithsdw1
...finishedkeyexchangewithsdw2
...finishedkeyexchangewithsdw3
...finishedkeyexchangewithsdw4
...finishedkeyexchangewithsdw5
...finishedkeyexchangewithsdw6
...finishedkeyexchangewithsdw7
[INFO]completedsuccessfully实质就是master与各个节点创建了个ssh的信任关系，信任的key存放在/root/.ssh/authorized_keys里
--测试是否无需密码登陆，ok
[root@bigdata03~]#sshroot@172.30.13.204
Lastlogin:
WedJul2908:
45:
582015from172.30.11.31
[root@bigdata04~]#exit
logout
Connectionto172.30.13.204closed.
[root@bigdata03~]#
[root@bigdata03~]#sshroot@sdw7
Lastlogin:
WedJul2908:
49:
202015from172.30.11.31
[root@bigdata10~]#exit
logout
Connectiontosdw7closed.
[root@bigdata03~]#3.安装各个节点准备
首先把包传送到各个节点上，并解压缩以及创建用户，组等
[root@bigdata03~]#gpseginstall-f/tmp/hostfile_exkeys-ugpadmin-p123456
20150729:
09:
20:
20:
029326gpseginstall:
bigdata03:
root-[INFO]:
-InstallationInfo:
link_namegreenplum-db
binary_path/usr/local/gpmaster
binary_dir_location/usr/local
binary_dir_namegpmaster
20150729:
09:
20:
20:
029326gpseginstall:
bigdata03:
root-[INFO]:
-checkclusterpasswordaccess
20150729:
09:
20:
22:
029326gpseginstall:
bigdata03:
root-[INFO]:
-de-duplicatehostnames
20150729:
09:
20:
22:
029326gpseginstall:
bigdata03:
root-[INFO]:
-masterhostname:
bigdata03.rdchz
20150729:
09:
20:
23:
029326gpseginstall:
bigdata03:
root-[INFO]:
-checkforusergpadminoncluster
20150729:
09:
20:
24:
029326gpseginstall:
bigdata03:
root-[INFO]:
-addusergpadminonmaster
20150729:
09:
20:
24:
029326gpseginstall:
bigdata03:
root-[INFO]:
-addusergpadminoncluster
20150729:
09:
20:
24:
029326gpseginstall:
bigdata03:
root-[INFO]:
-chown-Rgpadmin:
gpadmin/usr/local/greenplum-db
20150729:
09:
20:
24:
029326gpseginstall:
bigdata03:
root-[INFO]:
-chown-Rgpadmin:
gpadmin/usr/local/gpmaster
20150729:
09:
20:
25:
029326gpseginstall:
bigdata03:
root-[INFO]:
-rm-f/usr/local/gpmaster.tar;rm-f/usr/local/gpmaster.tar.gz
20150729:
09:
20:
25:
029326gpseginstall:
bigdata03:
root-[INFO]:
-cd/usr/local;tarcfgpmaster.targpmaster
20150729:
09:
20:
27:
029326gpseginstall:
bigdata03:
root-[INFO]:
-gzip/usr/local/gpmaster.tar
20150729:
09:
20:
54:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
mkdir-p/usr/local
20150729:
09:
20:
55:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
rm-rf/usr/local/gpmaster
20150729:
09:
20:
56:
029326gpseginstall:
bigdata03:
root-[INFO]:
-scpsoftwaretoremotelocation
20150729:
09:
21:
00:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
gzip-f-d/usr/local/gpmaster.tar.gz
20150729:
09:
21:
06:
029326gpseginstall:
bigdata03:
root-[INFO]:
-md5checkonremotelocation
20150729:
09:
21:
09:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
cd/usr/local;tarxfgpmaster.tar
20150729:
09:
21:
11:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
rm-f/usr/local/gpmaster.tar
20150729:
09:
21:
12:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
cd/usr/local;rm-fgreenplum-db;ln-fsgpmastergreenplum-db
20150729:
09:
21:
13:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
chown-Rgpadmin:
gpadmin/usr/local/greenplum-db
20150729:
09:
21:
14:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
chown-Rgpadmin:
gpadmin/usr/local/gpmaster
20150729:
09:
21:
15:
029326gpseginstall:
bigdata03:
root-[INFO]:
-rm-f/usr/local/gpmaster.tar.gz
20150729:
09:
21:
15:
029326gpseginstall:
bigdata03:
root-[INFO]:
-Changingsystempasswords...
20150729:
09:
21:
17:
029326gpseginstall:
bigdata03:
root-[INFO]:
-exchangesshkeysforuserroot
20150729:
09:
21:
22:
029326gpseginstall:
bigdata03:
root-[INFO]:
-exchangesshkeysforusergpadmin
20150729:
09:
21:
30:
029326gpseginstall:
bigdata03:
root-[INFO]:
-/usr/local/greenplum-db/./sbin/gpfixuserlimts-f/etc/security/limits.conf-ugpadmin
20150729:
09:
21:
30:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
./usr/local/greenplum-db/./greenplum_path.sh;/usr/local/greenplum-db/./sbin/gpfixuserlimts-f/etc/security/limits.conf-ugpadmin
20150729:
09:
21:
31:
029326gpseginstall:
bigdata03:
root-[INFO]:
-versionstringonmaster:
gpsshversion4.3.5.2build1
20150729:
09:
21:
31:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
./usr/local/greenplum-db/./greenplum_path.sh;/usr/local/greenplum-db/./bin/gpssh--version
20150729:
09:
21:
32:
029326gpseginstall:
bigdata03:
root-[INFO]:
-remotecommand:
./usr/local/gpmaster/greenplum_path.sh;/usr/local/gpmaster/bin/gpssh--version
20150729:
09:
21:
38:
029326gpseginstall:
bigdata03:
root-[INFO]:
-SUCCESS--Requestedcommandscompleted
[root@bigdata03~]#
--或者用gp自带的工具包测试能否无密码登陆已经安装包是否已经被解压到各节点上面，正常情况下会显示出各个节点的gphome信息
[root@bigdata03~]#gpssh-fhostfile_exkeys-els-l$GPHOME4.创建各个节点的数据文件夹，并授权，保证数据文件夹是干净的
[root@bigdata03data]#gpssh-f/tmp/hostfile_segments-e'mkdir-p/data/primary'
[sdw4]mkdir-p/data/primary
[sdw3]mkdir-p/data/primary
[sdw5]mkdir-p/data/primary
[sdw1]mkdir-p/data/primary
[sdw6]mkdir-p/data/primary
[sdw2]mkdir-p/data/primary
[sdw7]mkdir-p/data/primary
[root@bigdata03data]#gpssh-f/tmp/hostfile_segments-e'chowngpadmin/data/primary'
[sdw4]chowngpadmin/data/primary
[sdw3]chowngpadmin/data/primary
[sdw5]chowngpadmin/data/primary
[sdw1]chowngpadmin/data/primary
[sdw6]chowngpadmin/data/primary
[sdw2]chowngpadmin/data/primary
[sdw7]chowngpadmin/data/primary【
注意：
这里只是在segs上创建了数据文件夹，需要在master上也要创建文件夹并授权
mkdir-p/data/master
chowngpadmin/data/master
】
5.时钟同步
[root@bigdata03~]#vi/etc/ntp.conf
server172.30.11.8
【
在每个节点上都要安装ntp服务：
yuminstallntp
servicentpdstart
chkconfigntpdon
】
各个节点上的时钟以master为准，如04节点，需要改在第一行
[root@bigdata04~]#vi/etc/ntp.conf
servermdw[root@bigdata03~]#gpssh-f/tmp/hostfile_exkeys-v-e'ntpd'
[Reset...]
[INFO]loginsdw4
[INFO]loginsdw3
[INFO]loginmdw
[INFO]loginsdw5
[INFO]loginsdw1
[INFO]loginsdw6
[INFO]loginsdw2
[INFO]loginsdw7
[sdw4]ntpd
[sdw3]ntpd
[mdw]ntpd
[sdw5]ntpd
[sdw1]ntpd
[sdw6]ntpd
[sdw2]ntpd
[sdw7]ntpd
[INFO]completedsuccessfully
[Cleanup...]
[root@bigdata03~]#6.检查参数设置
检查参数设置，正常情况是这样的
[root@bigdata03~]#gpcheck-f/tmp/hostfile_exkeys-mmdw-ssdw1
20150729:
11:
18:
20:
001805gpcheck:
bigdata03:
root-[INFO]:
-dedupehostnames
20150729:
11:
18:
20:
00180
升级会员 