使用OBD部署一个OceanBase 三副本集群并使用Promethues监控OB(在不同节点)
Tags: grafanaOceanBase分布式数据库环境搭建监控高可用
- OceanBase Docker安装体验:https://www.xmmup.com/oceanbase-dockeranzhuangtiyan.html
- 手动部署 OceanBase 单副本集群:https://www.xmmup.com/shoudongbushu-oceanbase-danfubenjiqun.html
- 手动部署 OceanBase 三副本集群(在同一个节点):https://www.xmmup.com/shoudongbushu-oceanbase-sanfubenjiqunzaitongyigejiedian.html
说明
相关连接:
https://open.oceanbase.com/blog/10900159
https://open.oceanbase.com/answer/detail?id=13700844
练习目的
本次练习目的掌握 OceanBase 集群的自动化部署(OBD工具)技术,能自动部署OB集群,包含 OBProxy,以及部署 OBAgent 等。
练习条件
- 有 服务器,内存资源至少
12G*3
台 或者32G*1
台
练习内容
请记录并分享下列内容:
- (必选)部署 OBD 软件,离线或者在线都可以。
- (必选)使用 OBD 部署一套三副本集群(单机3节点或者3节点都可以),部署至少1个 OBProxy。
- (必选)部署至少一个 OBAgent 。
- (可选)部署 Promethues ,能查看采集的 OB 性能或状态数据。
- (可选)部署 Grafana,能使用 OB 的性能模板查看 OB性能数据。
环境规划
环境是这样的,只有一台虚拟机,配置16c80G,准备使用docker来模拟出4台CentOS主机,搭建OceanBase 3.1.1社区版的集群。OBServer每台至少需要8G内存,否则不能启动。
具体配置如下所示:
主机名 | IP | 端口 | 主机映射端口 | Zone | 作用 |
---|---|---|---|---|---|
lhrob1 | 172.72.8.11 | 2881 | 28811 | zone1 | OB Server1、OBAgent |
lhrob2 | 172.72.8.12 | 2881 | 28812 | zone2 | OB Server2、OBAgent |
lhrob3 | 172.72.8.13 | 2881 | 28813 | zone3 | OB Server3、OBAgent |
lhrobproxy | 172.72.8.14 | 2883 | 28814 | 用于OBD、OBProxy、OBClient、mysql、Promethues、Grafana、OB客户端、时间服务器等 |
OBAgent是OceanBase社区版数据库服务监控组件。
OBD 全称是 OceanBase Deployer,是 OceanBase 社区版的命令行下自动化部署软件。
ODP:OceanBase 数据库代理,简称为 ODP。ODP 是 OceanBase 数据库专用的连接代理软件。ODP 的核心功能包括:保证最佳路由,避免分布式事务;保护 OceanBase 数据库的高可用能力,单台服务器故障不影响应用。
初始化准备
申请环境
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | docker network create --subnet=172.72.8.0/24 lhrob-network docker network inspect lhrob-network docker rm -f lhrob1 lhrob2 lhrob3 lhrobproxy docker run -d --name lhrob1 -h lhrob1 \ --net=lhrob-network --ip 172.72.8.11 \ -p 28811:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrob2 -h lhrob2 \ --net=lhrob-network --ip 172.72.8.12 \ -p 28812:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrob3 -h lhrob3 \ --net=lhrob-network --ip 172.72.8.13 \ -p 28813:2881 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker run -d --name lhrobproxy -h lhrobproxy \ --net=lhrob-network --ip 172.72.8.14 \ -p 28814:2883 -p 23000:3000 -p 29090:9090 \ -v /sys/fs/cgroup:/sys/fs/cgroup \ --privileged=true lhrbest/lhrcentos76:8.5 \ /usr/sbin/init docker exec -it lhrobproxy bash [root@docker35 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 494cb2a2cafe lhrbest/lhrcentos76:8.5 "/usr/sbin/init" 26 minutes ago Up 26 minutes 0.0.0.0:28814->2883/tcp, :::28814->2883/tcp lhrobproxy 06d2587bbd61 lhrbest/lhrcentos76:8.5 "/usr/sbin/init" 26 minutes ago Up 26 minutes 0.0.0.0:28813->2881/tcp, :::28813->2881/tcp lhrob3 969ca85b3bea lhrbest/lhrcentos76:8.5 "/usr/sbin/init" 26 minutes ago Up 26 minutes 0.0.0.0:28812->2881/tcp, :::28812->2881/tcp lhrob2 f86e1423e13a lhrbest/lhrcentos76:8.5 "/usr/sbin/init" 26 minutes ago Up 26 minutes 0.0.0.0:28811->2881/tcp, :::28811->2881/tcp lhrob1 |
配置时钟源
如果您使用集群安装 OceanBase,则需要保证集群内各机器的时间同步。否则集群无法启动,服务在运行时也会出现异常。如果您已配置 NTP 时钟同步,则无需重新配置。
OceanBase 集群中的服务器时间必须保持一致,否则会导致 OceanBase 集群无法启动,运行时也会出现故障。物理机与时钟服务器的误差在 50ms 以下可认为时钟是同步状态,OceanBase 集群最大容忍误差不能超过 100ms。当超过 100ms 时,会出现无主情况。恢复时钟同步后。重启 OceanBase 集群, 可以恢复正常。
部署 OceanBase 集群时,各个 OBServer 的 RPC 允许的时钟偏差最大是100ms。
这里以“172.72.8.14”为时间服务器,其它3台OBServer同步该机器的时间:
1 2 3 4 | yum install ntp ntpdate -y ntpq -4p ntpstat timedatectl |
1、修改“172.72.8.14”为时间服务器/etc/ntp.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | # For more information about this file, see the man pages # ntp.conf(5), ntp_acc(5), ntp_auth(5), ntp_clock(5), ntp_misc(5), ntp_mon(5). driftfile /var/lib/ntp/drift #新增:日志目录 logfile /var/log/ntpd.log # Permit time synchronization with our time source, but do not # permit the source to query or modify the service on this system. restrict default nomodify notrap nopeer noquery # Permit all access over the loopback interface. This could # be tightened as well, but to do so would effect some of # the administrative functions. restrict 127.0.0.1 restrict ::1 #新增:这一行的含义是授权172.72.8.0网段上的所有机器可以从这台机器上查询和同步时间. restrict 172.72.8.0 mask 255.255.255.0 nomodify notrap # Hosts on local network are less restricted. #restrict 192.168.1.0 mask 255.255.255.0 nomodify notrap # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst #新增:时间服务器列表. server 0.cn.pool.ntp.org iburst server 1.cn.pool.ntp.org iburst server 2.cn.pool.ntp.org iburst server 3.cn.pool.ntp.org iburst #新增:当外部时间不可用时,使用本地时间 server 127.0.0.1 iburst fudge 127.0.0.1 stratum 10 #broadcast 192.168.1.255 autokey # broadcast server #broadcastclient # broadcast client #broadcast 224.0.1.1 autokey # multicast server #multicastclient 224.0.1.1 # multicast client #manycastserver 239.255.254.254 # manycast server #manycastclient 239.255.254.254 autokey # manycast client # Enable public key cryptography. #crypto includefile /etc/ntp/crypto/pw # Key file containing the keys and key identifiers used when operating # with symmetric key cryptography. keys /etc/ntp/keys # Specify the key identifiers which are trusted. #trustedkey 4 8 42 # Specify the key identifier to use with the ntpdc utility. #requestkey 8 # Specify the key identifier to use with the ntpq utility. #controlkey 8 # Enable writing of statistics records. #statistics clockstats cryptostats loopstats peerstats # Disable the monitoring facility to prevent amplification attacks using ntpdc # monlist command when default restrict does not include the noquery flag. See # CVE-2013-5211 for more details. # Note: Monitoring will not be disabled with the limited restriction flag. disable monitor |
配置开机启动:
1 2 3 4 5 6 7 8 9 10 | systemctl enable ntpd systemctl is-enabled ntpd ntpdate -u 1.cn.pool.ntp.org systemctl restart ntpd [root@lhrobproxy /]# ntpstat synchronised to NTP server (84.16.73.33) at stratum 2 time correct to within 98 ms polling server every 64 s |
其它客户端,修改“/etc/ntp.conf”,注释server开头的行,并添加如下行:
1 2 3 4 5 6 7 | server 172.72.8.14 restrict 172.72.8.14 nomodify notrap noquery server 127.0.0.1 fudge 127.0.0.1 stratum 10 |
配置开机启动:
1 2 | systemctl enable ntpd systemctl restart ntpd |
客户端配置自动同步:
1 2 | crontab -e * * * * * /usr/sbin/ntpdate -u 172.72.8.14 & > /dev/null |
检查:
- 机器三节点之间时间同步检查,检查本机和目标节点时间误差常用命令是:
clockdiff
。
1 2 3 4 5 6 7 8 9 | [root@lhrobproxy soft]# clockdiff lhrob1 .. host=lhrob1 rtt=562(280)ms/0ms delta=0ms/0ms Sun Jan 9 10:39:43 2022 [root@lhrobproxy soft]# clockdiff lhrob2 . host=lhrob2 rtt=750(187)ms/0ms delta=0ms/0ms Sun Jan 9 10:39:57 2022 [root@lhrobproxy soft]# clockdiff lhrob3 . host=lhrob3 rtt=750(187)ms/0ms delta=0ms/0ms Sun Jan 9 10:40:00 2022 |
delta = 目标主机减当前主机时间 ,时间单位是毫秒。
三节点时间同步误差如果超过 50ms,则后面初始化集群一定会失败。 这里还要留意节点的时间误差可能有个缓慢递增的特点,也许当前集群还能正常工作,一天后由于节点时间误差扩大到 50ms 以外,该节点就掉线了。
配置内核参数
4个节点都运行:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | cat >> /etc/security/limits.conf <<"EOF" root soft nofile 655350 root hard nofile 655350 * soft nofile 655350 * hard nofile 655350 * soft stack 20480 * hard stack 20480 * soft nproc 655360 * hard nproc 655360 * soft core unlimited * hard core unlimited EOF echo "fs.aio-max-nr=1048576" >> /etc/sysctl.conf sysctl -p |
如果只是测试,您可以只设置 fs.aio-max-nr=1048576
。
创建用户
1 2 3 4 5 6 7 | useradd -U admin -d /home/admin -s /bin/bash echo "admin:lhr" | chpasswd chown -R admin:admin /home/admin echo "admin ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers |
设置无密码SSH登陆
可以使用rac上的sshUserSetup.sh快速配置,只在lhrobproxy上运行:
1 | sh sshUserSetup.sh -user admin -hosts "lhrob1 lhrob2 lhrob3 lhrobproxy" -advanced exverify -confirm |
安装集群
在lhrobproxy操作:
安装OBD
1 2 3 | yum install -y yum-utils yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo yum install -y ob-deploy |
设置yaml的配置文件
OBD 针对不同的部署场景提供不同的配置文件。这些配置文件示例在 OceanBase 开源项目地址里,具体是:https://github.com/oceanbase/obdeploy/tree/master/example 。
OBD根据这个yaml文件即可自动创建集群。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | cat > /tmp/obd_observer_obproxy.yaml <<"EOF" ## Only need to configure when remote login is required user: username: admin #用户名,前提三个节点保持一致 password: lhr #密码 ,前提三个节点保持一致 key_file: #密钥,可省略 #################### 下面是 observer搭建参数 ############################## oceanbase-ce: servers: - name: observer01 # zone名 # Please don't use hostname, only IP can be supported ip: 172.72.8.11 # OB1 地址 - name: observer02 ip: 172.72.8.12 # OB2 地址 - name: observer03 ip: 172.72.8.13 # OB3 地址 global: mysql_port: 2881 # 数据库端口 rpc_port: 2882 # 远程访问的协议端口号 home_path: /home/admin/oceanbase # 软件目录 #data_dir: /data # 数据目录 #redo_dir: /redo # redo目录 devname: eth0 # 设置要部署节点的网卡 memory_limit: 8G system_memory: 2G # 系统剩余保留内存2G lower_case_table_names: 1 # 数据库不区分大小写 foreign_key_checks: 0 # DML 语句不检查外建约束,DDL 操作不受影响 sys_bkgd_migration_retry_num: 5 # 副本迁移失败时最多重试次数。 stack_size: 512K # 设置程序函数调用栈的大小。 磁盘要512k 对齐,如果不是可能会启动失败 cpu_count: 16 # cpu 16核 cache_wash_threshold: 1G # 设置触发缓存清理的容量阈值。如果内存空间小于指定值时,内存空间将被清理。 __min_full_resource_pool_memory: 1073741824 # 默认普通租户的内存最小规格必须大于等于 5 GB,这里设置成1G,就说明我可以最低设置租户内存为1G workers_per_cpu_quota: 10 # 用于设置分配给每个 CPU 配额的工作线程数量。 schema_history_expire_time: 1d # 元数据历史数据过期时间。 net_thread_count: 4 # 设置网络 I/O 线程数,The value of net_thread_count had better be same as cpu's core number. major_freeze_duty_time: Disable minor_freeze_times: 10 # 多少次小合并触发一次全局合并。 enable_separate_sys_clog: True # 是否把系统事务日志与用户事务日志分开存储。 enable_merge_by_turn: FALSE datafile_size: 5G #datafile_disk_percentage: 0.1 # 数据库系统初始化用于存储数据,例如这里设置了40,表示百分之四十,如果我单节点是1TB的,约400多G空间将会被立即占用 syslog_level: ERROR # 日志警报级别 enable_syslog_recycle: True # 开启回收系统日志的功能 max_syslog_file_count: 4 # 日志文件数量 log_dir_size_threshold: 1G cluster_id: 1 # 集群ID # observer cluster name, consistent with obproxy's cluster_name appname: lhrob312cluster # 集群名,要与下面obproxy的对应一致 ###下面设置节点信息 observer01: zone: zone1 # 设置节点所在的 Zone 的名字 observer02: zone: zone2 observer03: zone: zone3 ########################## 以下是obrpoxy的 搭建参数 ######################3 obproxy: servers: - 127.0.0.1 depends: - oceanbase-ce global: listen_port: 2883 prometheus_listen_port: 2884 home_path: /home/admin/obproxy # oceanbase root server list # format: ip:mysql_port,ip:mysql_port # rs_list: 172.72.8.11:2881;172.72.8.12:2881;172.72.8.13:2881 enable_cluster_checkout: false skip_proxy_sys_private_check: true # cluster_name: lhrob312cluster EOF |
使用OBD安装集群
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | sh /etc/profile.d/obd.sh obd cluster deploy lhrob312cluster -c /tmp/obd_observer_obproxy.yaml -f obd cluster list obd cluster display lhrob312cluster -- 集群初始化 obd cluster start lhrob312cluster -- 修改参数 obd cluster edit-config lhrob312cluster -- 安装客户端 yum install -y obclient mariadb mariadb-libs mariadb-devel |
执行过程:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | [root@lhrobproxy ~]# obd cluster deploy lhrob312cluster -c /tmp/obd_observer_obproxy.yaml -f Update OceanBase-community-stable-el7 ok Update OceanBase-development-kit-el7 ok Download oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm (46.45 M): 100% [###############################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################] Time: 0:00:18 2.65 MB/s Package oceanbase-ce-3.1.2 is available. Download obproxy-3.2.0-1.el7.x86_64.rpm (7.80 M): 100% [#####################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################] Time: 0:00:03 2.50 MB/s Package obproxy-3.2.0 is available. install oceanbase-ce-3.1.2 for local ok install obproxy-3.2.0 for local ok +-------------------------------------------------------------------------------------------+ | Packages | +--------------+---------+-----------------------+------------------------------------------+ | Repository | Version | Release | Md5 | +--------------+---------+-----------------------+------------------------------------------+ | oceanbase-ce | 3.1.2 | 10000392021123010.el7 | 7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed | | obproxy | 3.2.0 | 1.el7 | 8d5c6978f988935dc3da1dbec208914668dcf3b2 | +--------------+---------+-----------------------+------------------------------------------+ Repository integrity check ok Parameter check ok Open ssh connection ok Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository install ok Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository lib check !! [WARN] z1(172.72.8.11) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3 [WARN] z2(172.72.8.12) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3 [WARN] z3(172.72.8.13) oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed require: libmariadb.so.3 Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository install ok Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository lib check ok Try to get lib-repository Download oceanbase-ce-libs-3.1.2-10000392021123010.el7.x86_64.rpm (155.22 K): 100% [#########################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################] Time: 0:00:00 1.71 MB/s Package oceanbase-ce-libs-3.1.2 is available. install oceanbase-ce-libs-3.1.2 for local ok Use oceanbase-ce-libs-3.1.2-94fff0ab31de053051dba66039e3185fa390cad5 for oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed Remote oceanbase-ce-libs-3.1.2-94fff0ab31de053051dba66039e3185fa390cad5 repository install ok Remote oceanbase-ce-3.1.2-7fafba0fac1e90cbd1b5b7ae5fa129b64dc63aed repository lib check ok Cluster status check ok Initializes observer work home ok Initializes obproxy work home ok lhrob312cluster deployed [root@lhrobproxy ~]# obd cluster list +------------------------------------------------------------------------+ | Cluster List | +-----------------+------------------------------------+-----------------+ | Name | Configuration Path | Status (Cached) | +-----------------+------------------------------------+-----------------+ | lhrob312cluster | /root/.obd/cluster/lhrob312cluster | deployed | +-----------------+------------------------------------+-----------------+ [root@lhrobproxy ~]# obd cluster start lhrob312cluster Get local repositories and plugins ok Open ssh connection ok Load cluster param plugin ok Check before start observer ok [WARN] (172.72.8.11) clog and data use the same disk (/) [WARN] (172.72.8.12) clog and data use the same disk (/) [WARN] (172.72.8.13) clog and data use the same disk (/) Check before start obproxy ok Start observer ok observer program health check ok Connect to observer ok Initialize cluster Cluster bootstrap ok Wait for observer init ok +-----------------------------------------------+ | observer | +-------------+---------+------+-------+--------+ | ip | version | port | zone | status | +-------------+---------+------+-------+--------+ | 172.72.8.11 | 3.1.2 | 2881 | zone1 | active | | 172.72.8.12 | 3.1.2 | 2881 | zone2 | active | | 172.72.8.13 | 3.1.2 | 2881 | zone3 | active | +-------------+---------+------+-------+--------+ Start obproxy ok obproxy program health check ok Connect to obproxy ok +---------------------------------------------+ | obproxy | +-----------+------+-----------------+--------+ | ip | port | prometheus_port | status | +-----------+------+-----------------+--------+ | 127.0.0.1 | 2883 | 2884 | active | +-----------+------+-----------------+--------+ lhrob312cluster running [root@lhrobproxy soft]# obd cluster list +------------------------------------------------------------------------+ | Cluster List | +-----------------+------------------------------------+-----------------+ | Name | Configuration Path | Status (Cached) | +-----------------+------------------------------------+-----------------+ | lhrob312cluster | /root/.obd/cluster/lhrob312cluster | running | +-----------------+------------------------------------+-----------------+ [root@lhrobproxy soft]# [root@lhrobproxy ~]# netstat -tulnp| grep 88 tcp 0 0 0.0.0.0:2883 0.0.0.0:* LISTEN 6251/obproxy tcp 0 0 0.0.0.0:2884 0.0.0.0:* LISTEN 6251/obproxy [root@lhrobproxy ~]# ps -ef|grep ob admin 6220 1 0 11:22 ? 00:00:00 bash /home/admin/obproxy/obproxyd.sh /home/admin/obproxy 127.0.0.1 2883 daemon admin 6251 1 58 11:22 ? 00:00:33 /home/admin/obproxy/bin/obproxy --listen_port 2883 root 6544 188 0 11:23 pts/0 00:00:00 grep --color=auto ob -- 剩余节点类似 [root@lhrob1 /]# netstat -tulnp | grep 88 tcp 0 0 0.0.0.0:2881 0.0.0.0:* LISTEN 3255/observer tcp 0 0 0.0.0.0:2882 0.0.0.0:* LISTEN 3255/observer [root@lhrob1 /]# ps -ef|grep ob admin 3255 1 99 10:51 ? 00:06:09 /home/admin/oceanbase/bin/observer -r 172.72.8.11:2882:2881;172.72.8.12:2882:2881;172.72.8.13:2882:2881 -o __min_full_resource_pool_memory=1073741824,memory_limit=8G,system_memory=2G,lower_case_table_names=1,foreign_key_checks=0,sys_bkgd_migration_retry_num=5,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=True,enable_merge_by_turn=False,datafile_size=5G,enable_syslog_recycle=True,max_syslog_file_count=4,log_dir_size_threshold=1G -z zone1 -p 2881 -P 2882 -n lhrob312cluster -c 1 -d /home/admin/oceanbase/store -i eth0 -l INFO root 3974 402 0 10:53 pts/0 00:00:00 grep --color=auto ob [root@lhrob2 /]# netstat -tulnp | grep 88 tcp 0 0 0.0.0.0:2881 0.0.0.0:* LISTEN 3157/observer tcp 0 0 0.0.0.0:2882 0.0.0.0:* LISTEN 3157/observer [root@lhrob2 /]# ps -ef|grep ob admin 3157 1 99 10:51 ? 00:08:22 /home/admin/oceanbase/bin/observer -r 172.72.8.11:2882:2881;172.72.8.12:2882:2881;172.72.8.13:2882:2881 -o __min_full_resource_pool_memory=1073741824,memory_limit=8G,system_memory=2G,lower_case_table_names=1,foreign_key_checks=0,sys_bkgd_migration_retry_num=5,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=True,enable_merge_by_turn=False,datafile_size=5G,enable_syslog_recycle=True,max_syslog_file_count=4,log_dir_size_threshold=1G -z zone2 -p 2881 -P 2882 -n lhrob312cluster -c 1 -d /home/admin/oceanbase/store -i eth0 -l INFO root 3927 391 0 10:54 pts/0 00:00:00 grep --color=auto ob [root@lhrob2 /]# [root@lhrob3 /]# netstat -tulnp | grep 88 tcp 0 0 0.0.0.0:2881 0.0.0.0:* LISTEN 3139/observer tcp 0 0 0.0.0.0:2882 0.0.0.0:* LISTEN 3139/observer [root@lhrob3 /]# ps -ef|grep ob admin 3139 1 99 10:51 ? 00:07:55 /home/admin/oceanbase/bin/observer -r 172.72.8.11:2882:2881;172.72.8.12:2882:2881;172.72.8.13:2882:2881 -o __min_full_resource_pool_memory=1073741824,memory_limit=8G,system_memory=2G,lower_case_table_names=1,foreign_key_checks=0,sys_bkgd_migration_retry_num=5,stack_size=512K,cpu_count=16,cache_wash_threshold=1G,workers_per_cpu_quota=10,schema_history_expire_time=1d,net_thread_count=4,major_freeze_duty_time=Disable,minor_freeze_times=10,enable_separate_sys_clog=True,enable_merge_by_turn=False,datafile_size=5G,enable_syslog_recycle=True,max_syslog_file_count=4,log_dir_size_threshold=1G -z zone3 -p 2881 -P 2882 -n lhrob312cluster -c 1 -d /home/admin/oceanbase/store -i eth0 -l INFO root 3911 378 0 10:54 pts/0 00:00:00 grep --color=auto ob |