DorisDB小规模集群部署最佳实践(2/2)

  上期使用ansible进行安装前配置的详细过程,本期完成DorisDB安装。本文部分采用了ansible ad-hoc方法,因时间关系,更高端的ansible playbook未进行探索,旨在进行参考借鉴。

角色回顾

  • doris1上部署FE FOLLOWER使其成为Leader,同时部署BE;
  • doris2上部署FE令其作为OBSERVERr,同时部署BE;
  • doris3部署一个Broker方便后面学习数据导入,同时部署BE。

1. doris1部署操作

1) 解压部署包

cd ~
ansible others -m copy -a "src=/soft/DorisDB-SE-1.14.5.tar.gz dest=/soft"
ansible all -a "tar -zxvf /soft/DorisDB-SE-1.14.5.tar.gz"
ansible all -a "ls -lrt"

2) 根目录下新建dorisdb及fe、be文件夹

cd ~
ansible all[0:1] -a "mkdir -p dorisdb/fe"
ansible all -a "mkdir -p dorisdb/be"

注:

  • all[0:1]表示all主机组的第1台和第2台,即doris1、doris2;
  • 在doris1、doris2上创建fe目录;
  • 在所有主机上创建be目录。

3) 移动DorisDB-SE-1.14.5文件夹下fe文件夹和be文件夹到dorisdb对应目录

ansible all[0:1] -m shell -a "mv DorisDB-SE-1.14.5/fe/* dorisdb/fe"
ansible all -m shell -a "mv DorisDB-SE-1.14.5/be/* dorisdb/be"

4) 在fe.conf中绑定ip

因为虚拟机网卡通常有多个ip,在fe.conf中绑定ip:

vim dorisdb/fe/conf/fe.conf
...
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.80.31/24

5) 在be.conf中绑定ip

vim dorisdb/be/conf/be.conf
...
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.80.31/24

6) 在fe目录中创建元数据目录

mkdir -p dorisdb/fe/doris-meta

7) 启动fe,首个启动的fe自动成为leader

[root@dorisdb1 ~]# ./dorisdb/fe/bin/start_fe.sh --daemon
./dorisdb/fe/bin/start_fe.sh: line 82: /root/dorisdb/fe/log/fe.out: No such file or directory
./dorisdb/fe/bin/start_fe.sh: line 83: /root/dorisdb/fe/log/fe.out: No such file or directory

8) 查看角色是否成功

使用doris1上安装好的mysql-client访问doris:

mysql -h 127.0.0.1 -P9030 -uroot

输入sql指令,查看FE状态:

mysql> SHOW PROC '/frontends';
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+
| Name                            | IP           | HostName | EditLogPort | HttpPort | QueryPort | RpcPort | Role     | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg |
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+
| 192.168.80.31_9010_1618466985274 | 192.168.80.31 | doris1   | 9010        | 8030     | 9030      | 9020    | FOLLOWER | true     | 436696162 | true | true  | 34                | 2021-04-15 14:11:35 | true     |        |
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+

1 row in set (0.03 sec)
  • IsMaster和Alive均为true

  • IsMaster:判断是否为Leader

  • Alive:判断节点是否添加成功。

2. doris2部署操作

–1) 解压部署包
–tar -zxvf /soft/DorisDB-SE-1.14.5.tar.gz
–ansible已经解压,该步骤跳过。

–2) 根目录下新建dorisdb及fe、be文件夹
–cd ~
–mkdir -p dorisdb/fe
–mkdir -p dorisdb/be
–ansible已创建目录,该步骤跳过。

–3) 移动DorisDB-SE-1.14.5文件夹下fe文件夹和be文件夹到dorisdb对应目录
–mv DorisDB-SE-1.14.5/fe/* dorisdb/fe
–mv DorisDB-SE-1.14.5/be/* dorisdb/be
–ansible已经mv,该步骤跳过。

4) 在fe.conf中绑定ip
因为虚拟机网卡通常有多个ip,在fe.conf中绑定ip:

vim dorisdb/fe/conf/fe.conf
...
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.80.32/24

也可以使用ansible进行操作[doris1]

scp dorisdb/fe/conf/fe.conf doris2:/root/dorisdb/fe/conf/fe.conf
ansible all[1] -a "sed -i 's/192.168.80.31/192.168.80.32/g' /root/dorisdb/fe/conf/fe.conf"
ansible all[1]  -a "grep priority_networks /root/dorisdb/fe/conf/fe.conf"

5) 在be.conf中绑定ip

vim dorisdb/be/conf/be.conf
...
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.80.32/24

6) 在fe目录中创建元数据目录

mkdir -p dorisdb/fe/doris-meta

7) 在doris2启动
  因为doris1上已有启动的fe,doris2的fe是作为observer。doris2的fe首次启动时需要指定doris1中的doris1作为helper(仅首次需要):

[root@doris2 ~]# ./dorisdb/fe/bin/start_fe.sh --helper 192.168.80.31:9010 --daemon
./dorisdb/fe/bin/start_fe.sh: line 82: /root/dorisdb/fe/log/fe.out: No such file or directory
./dorisdb/fe/bin/start_fe.sh: line 83: /root/dorisdb/fe/log/fe.out: No such file or directory

8) 在doris1主机将doris2中fe的节点至集群

   回到doris1服务器,使用sql命令添加doris2中fe的节点至集群,按照咱们的规划,将doris2的fe作为observer,sql为:

ALTER SYSTEM ADD OBSERVER "192.168.80.32:9010";

执行完毕后再次执行下面语句来查看fe集群状态:

mysql> SHOW PROC '/frontends';
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+
| Name                            | IP           | HostName | EditLogPort | HttpPort | QueryPort | RpcPort | Role     | IsMaster | ClusterId | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg |
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+
| 192.168.80.32_9010_1618469978601 | 192.168.80.32 | doris2   | 9010        | 8030     | 9030      | 9020    | OBSERVER | false    | 436696162 | true | true  | 917               | 2021-04-15 15:00:52 | false    |        |
| 192.168.80.31_9010_1618466985274 | 192.168.80.31 | doris1   | 9010        | 8030     | 9030      | 9020    | FOLLOWER | true     | 436696162 | true | true  | 918               | 2021-04-15 15:00:52 | true     |        |
+---------------------------------+--------------+----------+-------------+----------+-----------+---------+----------+----------+-----------+------+-------+-------------------+---------------------+----------+--------+

2 rows in set (0.02 sec)

  查看信息,Role: OBSERVER ,Alive:ture,证明doris2中的节点作为OBSERVER已添加成功。

3. 对doris3部署操作

–1) 解压部署包
–tar -zxvf /soft/DorisDB-SE-1.14.5.tar.gz
–ansible已经解压,该步骤跳过。

2) 根目录下新建dorisdb及be、apache_hdfs_broker文件夹

cd ~
--mkdir -p dorisdb/be      #该步骤跳过,前面ansible已经创建
mkdir -p dorisdb/apache_hdfs_broker

3) 移动DorisDB-SE-1.14.5文件夹下be文件夹和apache_hdfs_broker文件夹到dorisdb对应目录

--mv DorisDB-SE-1.14.5/be/* dorisdb/be     #该步骤跳过,前面ansible已经创建
mv DorisDB-SE-1.14.5/apache_hdfs_broker/* dorisdb/apache_hdfs_broker

4) 分别绑定be和broker的IP

在be.conf中绑定ip

vim dorisdb/be/conf/be.conf

配置为:

...
# priority_networks = 10.10.10.0/24;192.168.0.0/16
priority_networks = 192.168.80.33/24

5) broker配置文件apache_hdfs_broker.conf中绑定IP

vim dorisdb/apache_hdfs_broker/conf/apache_hdfs_broker.conf  #新增网络配置
...
priority_networks = 192.168.80.33/24

4. 启动doris1主机be并并加入集群

  回到doris1,先创建数据存放目录:

mkdir -p dorisdb/be/storage

启动be:

./dorisdb/be/bin/start_be.sh --daemon

切换到访问doris1的mysql-client终端,输入sql,将该be加入集群:

mysql> ALTER SYSTEM ADD BACKEND "192.168.80.31:9050";
Query OK, 0 rows affected (0.01 sec)

查看be状态,确认添加成功:

mysql> SHOW PROC '/backends'\G

*************************** 1. row ***************************
            BackendId: 11001
              Cluster: default_cluster
                   IP: 192.168.80.31
             HostName: doris1
        HeartbeatPort: 9050
               BePort: 9060
             HttpPort: 8040
             BrpcPort: 8060
        LastStartTime: 2021-04-15 15:14:58
        LastHeartbeat: 2021-04-15 15:15:28
                Alive: true
 SystemDecommissioned: false
ClusterDecommissioned: false
            TabletNum: 0
     DataUsedCapacity: .000 
        AvailCapacity: 582.498 GB
        TotalCapacity: 591.093 GB
              UsedPct: 1.45 %
       MaxDiskUsedPct: 1.45 %
               ErrMsg: 
              Version: 1.14.5-8c4257e
               Status: {"lastSuccessReportTabletsTime":"2021-04-15 15:14:58"}
1 row in set (0.00 sec)

5. 启动doris2主机be并加入集群

  在doris2先创建数据存放目录

mkdir -p dorisdb/be/storage

启动be:

./dorisdb/be/bin/start_be.sh --daemon

回到doris1,使用sql,将doris2的be加入集群:

mysql> ALTER SYSTEM ADD BACKEND "192.168.80.32:9050";
Query OK, 0 rows affected (0.01 sec)

同样,用命令查询状态:

mysql> SHOW PROC '/backends'\G
*************************** 1. row ***************************
            BackendId: 11001
              Cluster: default_cluster
                   IP: 192.168.80.31
             HostName: doris1
        HeartbeatPort: 9050
               BePort: 9060
             HttpPort: 8040
             BrpcPort: 8060
        LastStartTime: 2021-04-15 15:14:58
        LastHeartbeat: 2021-04-15 15:21:18
                Alive: true
 SystemDecommissioned: false
ClusterDecommissioned: false
            TabletNum: 0
     DataUsedCapacity: .000 
        AvailCapacity: 582.498 GB
        TotalCapacity: 591.093 GB
              UsedPct: 1.45 %
       MaxDiskUsedPct: 1.45 %
               ErrMsg: 
              Version: 1.14.5-8c4257e
               Status: {"lastSuccessReportTabletsTime":"2021-04-15 15:20:58"}
*************************** 2. row ***************************
            BackendId: 11002
              Cluster: default_cluster
                   IP: 192.168.80.32
             HostName: doris2
        HeartbeatPort: 9050
               BePort: 9060
             HttpPort: 8040
             BrpcPort: 8060
        LastStartTime: 2021-04-15 15:20:53
        LastHeartbeat: 2021-04-15 15:21:18
                Alive: true
 SystemDecommissioned: false
ClusterDecommissioned: false
            TabletNum: 0
     DataUsedCapacity: .000 
        AvailCapacity: 585.818 GB
        TotalCapacity: 591.093 GB
              UsedPct: 0.89 %
       MaxDiskUsedPct: 0.89 %
               ErrMsg: 
              Version: 1.14.5-8c4257e
               Status: {"lastSuccessReportTabletsTime":"2021-04-15 15:20:53"}
2 rows in set (0.00 sec)

6. 启动doris3主机be并并加入集群

  转到doris3服务器,先创建数据存放目录:

mkdir -p dorisdb/be/storage

启动be:

./dorisdb/be/bin/start_be.sh --daemon

回到doris1,使用sql,将doris3的be加入集群:

mysql> ALTER SYSTEM ADD BACKEND "192.168.80.33:9050";
Query OK, 0 rows affected (0.00 sec)

继续,用命令查询状态:

mysql> SHOW PROC '/backends'\G
...

alive:true即为成功加入集群。

7. 启动doris3主机broker

  再转到doris3,启动broker。

./dorisdb/apache_hdfs_broker/bin/start_broker.sh --daemon

回到doris1,输入sql将broker加入集群,假设doris3服务器的broker随意命名为broker3

mysql> ALTER SYSTEM ADD BROKER broker3 "192.168.80.33:8000";
Query OK, 0 rows affected (0.01 sec)

输入sql查看broker状态:

mysql> SHOW PROC "/brokers"\G
*************************** 1. row ***************************
          Name: broker3
            IP: 192.168.80.33
          Port: 8000
         Alive: true
 LastStartTime: 2021-04-15 15:28:19
LastUpdateTime: 2021-04-15 15:28:39
        ErrMsg: 
1 row in set (0.00 sec)

当前brokers3,Alive: true说明正常。

8. 小结

  至此,使用ansible进行一个小规模DorisDB标准版集群部署完成,在大规模部署时,能有效节省时间。当前企业版会提供DorisManager图形化自动部署,标准版未提供该工具。