Rootop 服务器运维与web架构

Mongodb集群之Replica Set

Mongodb集群之Replica Set

介绍转自网络
Mongodb有三种集群方式:Replica Set / Sharding / Master-Slaver

Mongodb(M)表示主节点,Mongodb(S)表示备节点,Mongodb(A)表示仲裁节点。主备节点存储数据,仲裁节点不存储数据。客户端同时连接主节点与备节点,不连接仲裁节点。
默认设置下,主节点提供所有增删查改服务,备节点不提供任何服务。但是可以通过设置使备节点提供查询服务,这样就可以减少主节点的压力,当客户端进行数据查询时,请求自动转到备节点上。这个设置叫做Read Preference Modes,同时Java客户端提供了简单的配置方式,可以不必直接对数据库进行操作。
仲裁节点是一种特殊的节点,它本身并不存储数据,主要的作用是决定哪一个备节点在主节点挂掉之后提升为主节点,所以客户端不需要连接此节点。这里虽然只有一个备节点,但是仍然需要一个仲裁节点来提升备节点级别。

配置过程:
环境信息:centos6.5 x64
软件版本:mongodb-linux-x86_64-2.6.1

master :192.168.0.175
slave :192.168.0.176
arbitrate:192.168.0.159

master配置:

[root@master ~]# wget -c http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.6.1.tgz
[root@master ~]# tar zxvf mongodb-linux-x86_64-2.6.1.tgz
[root@master ~]# cp -R mongodb-linux-x86_64-2.6.1 /usr/local/mongodb #mongodb不需要编译,解压后是可执行文件。
[root@master ~]# mkdir /usr/local/mongodb/data #创建数据存放目录
[root@master ~]# mkdir /usr/local/mongodb/logs #日志存放目录
[root@master ~]# mkdir /usr/local/mongodb/etc #配置文件存放目录
[root@master ~]# vi /usr/local/mongodb/etc/mongodb.conf

#bind_ip = IPADDRESS #绑定接口IP,可不写。
port = 27017 #端口号默认27017
fork = true #守护进程方式运行
pidfilepath = /usr/local/mongodb/mongodb.pid #pid路径
logpath = /usr/local/mongodb/logs/mongodb.log #日志
dbpath = /usr/local/mongodb/data #数据存放目录
journal = true #存储模式
nohttpinterface = true #关闭http接口,默认关闭27018端口访问
directoryperdb = true #每个数据库将被保存在一个单独的目录
logappend = true #日志输出方式
replSet = reptest #副本集名称
oplogSize = 1000 #复制日志大小MB,默认为硬盘剩余空间的5%

保存退出。

slave配置:

[root@slave ~]# wget -c http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.6.1.tgz
[root@slave ~]# tar zxvf mongodb-linux-x86_64-2.6.1.tgz
[root@slave ~]# cp -R mongodb-linux-x86_64-2.6.1 /usr/local/mongodb
[root@slave ~]# mkdir /usr/local/mongodb/data
[root@slave ~]# mkdir /usr/local/mongodb/logs
[root@slave ~]# mkdir /usr/local/mongodb/etc
[root@slave ~]# vi /usr/local/mongodb/etc/mongodb.conf

port = 27017
fork = true
pidfilepath = /usr/local/mongodb/mongodb.pid
logpath = /usr/local/mongodb/logs/mongodb.log
dbpath = /usr/local/mongodb/data
journal = true
nohttpinterface = true
directoryperdb = true
logappend = true
replSet = reptest
oplogSize = 1000

保存退出。

arbitrate配置:

[root@arbitrate ~]# wget -c http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.6.1.tgz
[root@arbitrate ~]# tar zxvf mongodb-linux-x86_64-2.6.1.tgz
[root@arbitrate ~]# cp -R mongodb-linux-x86_64-2.6.1 /usr/local/mongodb
[root@arbitrate ~]# mkdir /usr/local/mongodb/data
[root@arbitrate ~]# mkdir /usr/local/mongodb/logs
[root@arbitrate ~]# mkdir /usr/local/mongodb/etc
[root@arbitrate ~]# vi /usr/local/mongodb/etc/mongodb.conf

port = 27017
fork = true
pidfilepath = /usr/local/mongodb/mongodb.pid
logpath = /usr/local/mongodb/logs/mongodb.log
dbpath = /usr/local/mongodb/data
journal = true
nohttpinterface = true
directoryperdb = true
logappend = true
replSet = reptest
oplogSize = 1000

保存退出。

启动主、备、仲裁节点:

[root@master ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf
[root@slave ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf
[root@arbitrate ~]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf

可以通过客户端连接mongodb,也可以直接在三个节点中选择一个连接mongodb。
这里在master执行:

[root@master ~]# /usr/local/mongodb/bin/mongo 192.168.0.175:27017 #默认端口号的话可以不加
MongoDB shell version: 2.6.1
connecting to: 192.168.0.175/test
> use admin #切换数据库
switched to db admin
> cfg={ _id:"reptest", members:[ {_id:0,host:'192.168.0.175:27017',priority:2}, {_id:1,host:'192.168.0.176:27017',priority:1},{_id:2,host:'192.168.0.159:27017',arbiterOnly:true}] }; #配置集群
{
 "_id" : "reptest ",
 "members" : [
 {
 "_id" : 0,
 "host" : "192.168.0.175:27017",
 "priority" : 2
 },
 {
 "_id" : 1,
 "host" : "192.168.0.176:27017",
 "priority" : 1
 },
 {
 "_id" : 2,
 "host" : "192.168.0.159:27017",
 "arbiterOnly" : true
 }
 ]
}
> rs.initiate(cfg) #初始化
{
 "info" : "Config now saved locally. Should come online in about a minute.",
 "ok" : 1
}
>

cfg可以是任意的名字,最好不要是mongodb的关键字,conf,config都可以。最外层的_id表示replica set的名字,members里包含的是所有节点的地址以及优先级。优先级最高的即成为主节点,即这里的10.10.148.130:27017。特别注意的是,对于仲裁节点,需要有个特别的配置——arbiterOnly:true。这个千万不能少了,不然主备模式就不能生效。
配置的生效时间根据不同的机器配置会有长有短,配置不错的话基本上十几秒内就能生效,有的配置需要一两分钟。如果生效了,执行rs.status()命令会看到如下信息:

> rs.status()
{
 "set" : "reptest",
 "date" : ISODate("2015-03-20T07:52:31Z"),
 "myState" : 1,
 "members" : [
 {
 "_id" : 0,
 "name" : "192.168.0.175:27017",
 "health" : 1,
 "state" : 1,
 "stateStr" : "PRIMARY",
 "uptime" : 71,
 "optime" : Timestamp(1426837925, 1),
 "optimeDate" : ISODate("2015-03-20T07:52:05Z"),
 "electionTime" : Timestamp(1426837934, 1),
 "electionDate" : ISODate("2015-03-20T07:52:14Z"),
 "self" : true
 },
 {
 "_id" : 1,
 "name" : "192.168.0.176:27017",
 "health" : 1,
 "state" : 2,
 "stateStr" : "SECONDARY",
 "uptime" : 25,
 "optime" : Timestamp(1426837925, 1),
 "optimeDate" : ISODate("2015-03-20T07:52:05Z"),
 "lastHeartbeat" : ISODate("2015-03-20T07:52:30Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T07:52:30Z"),
 "pingMs" : 0,
 "syncingTo" : "192.168.0.175:27017"
 },
 {
 "_id" : 2,
 "name" : "192.168.0.159:27017",
 "health" : 1,
 "state" : 7,
 "stateStr" : "ARBITER",
 "uptime" : 25,
 "lastHeartbeat" : ISODate("2015-03-20T07:52:30Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T07:52:30Z"),
 "pingMs" : 0
 }
 ],
 "ok" : 1
}
reptest:PRIMARY>

同时可以查看对应节点的日志,发现正在等待别的节点生效或者正在分配数据文件。

测试:

在主节点,写入数据,去备节点查看。

[root@master mongodb]# /usr/local/mongodb/bin/mongo 192.168.0.175:27017
MongoDB shell version: 2.6.1
connecting to: 192.168.0.175:27017/test
reptest:PRIMARY> db.test.insert({'name':'111','phone':'12345678'});
WriteResult({ "nInserted" : 1 })
reptest:PRIMARY> db.test.find();
{ "_id" : ObjectId("550bd2228c9d92e7d9ef3b4c"), "name" : "111", "phone" : "12345678" }
reptest:PRIMARY>

备节点查看:

[root@slave mongodb]# /usr/local/mongodb/bin/mongo 192.168.0.176
MongoDB shell version: 2.6.1
connecting to: 192.168.0.176/test
reptest:SECONDARY> db.test.find();
error: { "$err" : "not master and slaveOk=false", "code" : 13435 } #没有权限
reptest:SECONDARY> rs.slaveOk(); #设置从库可查询
reptest:SECONDARY> db.test.find();
{ "_id" : ObjectId("550bd320da20623f56ebbc46"), "name" : "111", "phone" : "12345678" }
reptest:SECONDARY>

数据已同步。

关闭master(杀掉进程),看备机能否变为主:
备机执行rs.status();命令

reptest:PRIMARY> rs.status();
{
 "set" : "reptest",
 "date" : ISODate("2015-03-20T08:01:59Z"),
 "myState" : 1,
 "members" : [
 {
 "_id" : 0,
 "name" : "192.168.0.175:27017",
 "health" : 0,
 "state" : 8,
 "stateStr" : "(not reachable/healthy)", #不可达
 "uptime" : 0,
 "optime" : Timestamp(1426838434, 1),
 "optimeDate" : ISODate("2015-03-20T08:00:34Z"),
 "lastHeartbeat" : ISODate("2015-03-20T08:01:56Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:01:34Z"),
 "pingMs" : 0
 },
 {
 "_id" : 1,
 "name" : "192.168.0.176:27017",
 "health" : 1,
 "state" : 1,
 "stateStr" : "PRIMARY", #变为master
 "uptime" : 638,
 "optime" : Timestamp(1426838434, 1),
 "optimeDate" : ISODate("2015-03-20T08:00:34Z"),
 "electionTime" : Timestamp(1426838498, 1),
 "electionDate" : ISODate("2015-03-20T08:01:38Z"),
 "self" : true
 },
 {
 "_id" : 2,
 "name" : "192.168.0.159:27017",
 "health" : 1,
 "state" : 7,
 "stateStr" : "ARBITER",
 "uptime" : 595,
 "lastHeartbeat" : ISODate("2015-03-20T08:01:58Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:01:59Z"),
 "pingMs" : 0
 }
 ],
 "ok" : 1
}
reptest:PRIMARY>

往备机(176)插入数据:

reptest:PRIMARY> db.test.insert({'name':'333','phone':'33333333'});
WriteResult({ "nInserted" : 1 })
reptest:PRIMARY> db.test.find();
{ "_id" : ObjectId("550bd320da20623f56ebbc46"), "name" : "111", "phone" : "12345678" }
{ "_id" : ObjectId("550bd3a2bdbf370988e2e1be"), "name" : "222", "phone" : "87654321" }
{ "_id" : ObjectId("550bd484fe01fa0c130acc2f"), "name" : "333", "phone" : "33333333" }
reptest:PRIMARY>

启动主机(176)查看身份及数据:

reptest:SECONDARY> rs.status()
{
 "set" : "reptest",
 "date" : ISODate("2015-03-20T08:05:29Z"),
 "myState" : 1,
 "members" : [
 {
 "_id" : 0,
 "name" : "192.168.0.175:27017",
 "health" : 1,
 "state" : 1,
 "stateStr" : "PRIMARY",
 "uptime" : 16,
 "optime" : Timestamp(1426838660, 1),
 "optimeDate" : ISODate("2015-03-20T08:04:20Z"),
 "electionTime" : Timestamp(1426838721, 1),
 "electionDate" : ISODate("2015-03-20T08:05:21Z"),
 "self" : true
 },
 {
 "_id" : 1,
 "name" : "192.168.0.176:27017",
 "health" : 1,
 "state" : 2,
 "stateStr" : "SECONDARY",
 "uptime" : 16,
 "optime" : Timestamp(1426838660, 1),
 "optimeDate" : ISODate("2015-03-20T08:04:20Z"),
 "lastHeartbeat" : ISODate("2015-03-20T08:05:29Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:05:28Z"),
 "pingMs" : 0,
 "lastHeartbeatMessage" : "syncing to: 192.168.0.175:27017",
 "syncingTo" : "192.168.0.175:27017"
 },
 {
 "_id" : 2,
 "name" : "192.168.0.159:27017",
 "health" : 1,
 "state" : 7,
 "stateStr" : "ARBITER",
 "uptime" : 16,
 "lastHeartbeat" : ISODate("2015-03-20T08:05:29Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:05:29Z"),
 "pingMs" : 1
 }
 ],
 "ok" : 1
}
reptest:PRIMARY> db.test.find();
{ "_id" : ObjectId("550bd320da20623f56ebbc46"), "name" : "111", "phone" : "12345678" }
{ "_id" : ObjectId("550bd3a2bdbf370988e2e1be"), "name" : "222", "phone" : "87654321" }
{ "_id" : ObjectId("550bd484fe01fa0c130acc2f"), "name" : "333", "phone" : "33333333" }
reptest:PRIMARY>

身份变为master,数据也同步过来了。

现在模拟仲裁机宕机且完全不能恢复(重装后状态):

[root@arbitrate mongodb]# kill -9 $(cat mongodb.pid)
[root@arbitrate mongodb]# rm -rf data/*
[root@arbitrate mongodb]# rm -rf logs/*

master上通过rs.status();查看状态:

{
 "_id" : 2,
 "name" : "192.168.0.159:27017",
 "health" : 0,
 "state" : 8,
 "stateStr" : "(not reachable/healthy)",
 "uptime" : 0,
 "lastHeartbeat" : ISODate("2015-03-20T08:07:59Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:07:38Z"),
 "pingMs" : 0
 }

提示159这台机器不可达。

在仲裁机启动服务:

[root@arbitrate mongodb]# /usr/local/mongodb/bin/mongod -f /usr/local/mongodb/etc/mongodb.conf

在master上再次查看状态:

{
 "_id" : 2,
 "name" : "192.168.0.159:27017",
 "health" : 1,
 "state" : 7,
 "stateStr" : "ARBITER",
 "uptime" : 51,
 "lastHeartbeat" : ISODate("2015-03-20T08:11:05Z"),
 "lastHeartbeatRecv" : ISODate("2015-03-20T08:11:06Z"),
 "pingMs" : 0
 }

变为仲裁状态。
关闭master(175),查看slave状态变为PRIMARY,表明仲裁机重装以后依旧生效。

Replica Set集群搭建测试完成。

原创文章,转载请注明。本文链接地址: https://www.rootop.org/pages/3318.html

作者:Venus

专注于 服务器运维与web架构 E-mail:venus#rootop.org

评论已关闭。