手把手教你搭建mesos/marathon集群(一)——环境准备、zk(etcd)和docker安装


以mesos框架做资源调度,marathon作为容器编排的mesos/marathon架构体系在很多企业的中都有应用,基础概念和架构体系请读者自行百度,本文就不再赘述。本文主要讲一下mesos/marathon集群的搭建。
在接触mesos/marathon的初期,笔者也在网上查阅很多mesos/marathon集群搭建的文章,有直接在物理机上搭建的,也有使用docker三剑客(swarm docker-compose docker -machine)搭建的容器化集群。下面介绍一下如何在物理机上部署mesos/marathon集群,至于第二种方法,有兴趣的读者可以尝试一下。
首先介绍一下测试环境,一共6台服务器,如下

机器列表.png


这里说明一下为什么集群中用到了zookeeper和etcd两种kv存储,这是因为使用了calico网络查件,因此docker的--cluster-store必须使用etcd,而mesos又必须使用zk,所以才会出现这种情况。
言归正传,下面开始讲述集群的搭建
## 一、环境准备 ##
操作系统:Red Hat Enterprise Linux 7.2;
关闭防火墙;
修改主机hostname,并把集群每个节点的hostname与ip的对应关系添加到每个节点的hosts文件;
为docker创建lv,用作dm.thinpooldev。
## 二、搭建etcd集群 ##
1、解压etcd-v2.2.1-linux-amd64.tar.gz
tar zxvf etcd-v2.2.1-linux-amd64.tar.gz
cp etcd-v2.2.1-linux-amd64 /usr/local/
2、编辑etcd配置文件,/etc/etcd/etcd.conf 内容如下 ,加粗部分每个节点不一样
ETCD_NAME=etcd01
ETCD_DATA_DIR="/var/lib/etcd/etcd"
ETCD_LISTEN_PEER_URLS="http://10.124.152.129:2380"
ETCD_LISTEN_CLIENT_URLS="http://10.124.152.129:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.124.152.129:2380"
ETCD_INITIAL_CLUSTER="etcd01=http://10.124.152.129:2380,etcd02=http://10.124.152.130:2380,etcd03=http://10.124.152.131:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-00"
ETCD_ADVERTISE_CLIENT_URLS="http://10.124.152.129:2379"
3、编辑 /usr/lib/systemd/system/etcd.service 内容如下
[Unit]

Description=EtcdServer

After=network.target

After=network-online.target

Wants=network-online.target

[Service]

Type=notify

WorkingDirectory=/usr/local/etcd-v2.2.1-linux-amd64/

EnvironmentFile=-/etc/etcd/etcd.conf

User=root

setGOMAXPROCS to number of processors

ExecStart=/usr/local/etcd-v2.2.1-linux-amd64/etcd

Restart=on-failure

LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
4、systemctl启动etcd服务并设置为开机启动
5、使用etcdctl检查集群状态信息
cd /usr/local/etcd-v2.2.1-linux-amd64/
./etcdctl cluster-health
## 三、搭建zk集群 ##
1、解压zookeeper
tar zxvf zookeeper-3.4.10.tar.gz
cp -r zookeeper-3.4.10 /usr/local
2、修改zookeeper配置文件
cd /usr/local/zookeeper-3.4.10/conf
cp zoo_sample.cfg zoo.cfg,编辑zoo.cfg,内容如下(三个节点内容都一样)

The number of milliseconds of each tick

tickTime=2000

The number of ticks that the initial

synchronization phase can take

initLimit=10

The number of ticks that can pass between

sending a request and getting an acknowledgement

syncLimit=5

the directory where the snapshot is stored.

do not use /tmp for storage, /tmp here is just

example sakes.

dataDir=/usr/local/zookeeper-3.4.10/data
dataLogDir=/usr/local/zookeeper-3.4.10/log

the port at which the clients will connect

clientPort=2181

the maximum number of client connections.

increase this if you need to handle more clients

maxClientCnxns=60

#

Be sure to read the maintenance section of the

administrator guide before turning on autopurge.

#

http://zookeeper.apache.org/do ... nance

#

The number of snapshots to retain in dataDir

autopurge.snapRetainCount=3

Purge task interval in hours

Set to "0" to disable auto purge feature

autopurge.purgeInterval=1

server.1=mesos-1:2888:3888
server.2=mesos-2:2888:3888
server.3=mesos-3:2888:3888
3、zookeeper注册为系统服务,编辑/etc/systemd/system/zookeeper.service,内容如下(三个节点内容一样)
[Unit]
Description=zookeeper
After=syslog.target network.target

[Service]
Type=forking
Environment=ZOO_LOG_DIR=/usr/local/zookeeper-3.4.10/log
ExecStart=/usr/local/zookeeper-3.4.10/bin/zkServer.sh start
ExecStop=/usr/local/zookeeper-3.4.10/bin/zkServer.sh stop
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target
4、创建myid文件
三个节点的/usr/local/zookeeper-3.4.10/data路径下都创建myid文件,内容分别为1,2,3
5、注册服务,启动zookeeper并设置为开机启动
6、检查各节点的状态是否正常
## 四、安装docker ##
笔者的测试环境由于不能连外网,因此使用的是rpm包离线安装docker。docker离线安装也很简单,只需要手动安装docker-engine-1.12.3-1.el7.centos.x86_64.rpm和docker-engine-selinux-1.12.3-1.el7.centos.noarch.rpm这两个rpm包即可。
docker配置文件/etc/systemd/system/docker.service 内容如下:
[Unit]
Description=Docker Application Container Engine
After=network.target

[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver overlay --insecure-registry 10.124.152.135 --cluster-store=etcd://10.124.152.129:2379,10.124.152.130:2379,10.124.152.131:2379 --cluster-advertise=bond_mgmt:2376 --storage-opt dm.fs=ext4 --storage-opt dm.thinpooldev=/dev/mapper/vg_docker-lv_docker_01 --storage-opt dm.use_deferred_removal=true --storage-opt dm.use_deferred_deletion=true
ExecReload=/bin/kill -s HUP
MountFlags=slave
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Environment=

[Install]
WantedBy=multi-user.target
启动docker服务并配置开机启动,至此就完成了docker的安装。

1 个评论

对于Red Hat,推荐storage.driver为devicemapper以及direct-lvm模式,基于块设备存储的devicemapper性能上要优于基于文件系统存储的overlay

要回复文章请先登录注册