OLAP之Doris的docker镜像和集群搭建

发表于 2022-03-13 更新于 2024-01-13 阅读次数：阅读次数：

制作Doris镜像

我们在OLAP之Doris编译的基础上，开始制作docker镜像

WxlWez

我们为我们的目录增加Dockerfile_fe、Dockerfile_be两个Dockerfile文件

fe镜像Dockerfile(cd到编译好的output目录)

Dockerfile_fe

FROM primetoninc/jdk:1.8
# RUN yum install net-tools -y
COPY fe /opt/fe
WORKDIR /opt/fe
EXPOSE 8030 9030
ENTRYPOINT ["/opt/fe/bin/start_fe.sh"]

构建fe镜像，创建并配置镜像映射文件doris-meta和conf，启动容器

1	docker build -t doris-fe:0.15.0 -f Dockerfile_fe .

be镜像Dockerfile

Dockerfile_be

FROM primetoninc/jdk:1.8
# RUN yum install net-tools -y
COPY be /opt/be
WORKDIR /opt/be
EXPOSE 9050
ENTRYPOINT ["/opt/be/bin/start_be.sh"]

构建be镜像，配置be镜像映射文件storage，启动3个be容器组成集群。Doris默认至少安装3个be实例。

1	docker build -t doris-be:0.15.0 -f Dockerfile_be .

需要使用镜像可从官方pull，已经上传最新

1 2	docker pull bulolo/doris-fe:0.15.0 docker pull bulolo/doris-be:0.15.0

docker运行

docker run

不推荐使用docker run，因为还要进入容器内查看IP，再添加对应ip的backends

FE 运行

1	docker run -itd --name fe_1 -p 8030:8030 -p 9030:9030 -v <LOCAL_PATH>/fe_1/conf:/opt/fe/conf -v <LOCAL_PATH>/fe_1/log:/opt/fe/log -v <LOCAL_PATH>/fe_1/doris-meta:/opt/fe/doris-meta doris-fe:0.15.0

BE 运行

1
2
3

docker run -itd --name be_1 -p 9150:9050 -v <LOCAL_PATH>/be_1/conf:/opt/be/conf -v <LOCAL_PATH>/be_1/storage:/opt/be/storage doris-be:0.15.0
docker run -itd --name be_2 -p 9250:9050 -v <LOCAL_PATH>/be_2/conf:/opt/be/conf -v <LOCAL_PATH>/be_2/storage:/opt/be/storage doris-be:0.15.0
docker run -itd --name be_3 -p 9350:9050 -v <LOCAL_PATH>/be_3/conf:/opt/be/conf -v <LOCAL_PATH>/be_3/storage:/opt/be/storage doris-be:0.15.0

docker-compose

推荐使用docker-compose

docker-compose.yml

version: '3.7'

services:
  doris-fe1:
    image: doris-fe:0.15.0
    container_name: doris-fe1
    ports:
        - 8030:8030
        - 9030:9030
    ulimits:
      nofile:
        soft: "65536"
        hard: "65536"
    volumes:
        - ./fe_1/conf:/opt/fe/conf
        - ./fe_1/log:/opt/fe/log
        - ./fe_1/doris-meta:/opt/fe/doris-meta
    networks:
      doris-network :
        ipv4_address: 172.66.0.100
  doris-be1:
    image: doris-be:0.15.0
    container_name: doris-be1
    ports:
        - 9150:9050
    ulimits:
      nofile:
        soft: "65536"
        hard: "65536"
    volumes:
        - ./be_1/conf:/opt/be/conf
        - ./be_1/log:/opt/be/log
        - ./be_1/storage:/opt/be/storage
    networks:
      doris-network :
        ipv4_address: 172.66.0.101
  doris-be2:
    image: doris-be:0.15.0
    container_name: doris-be2
    ports:
        - 9250:9050
    ulimits:
      nofile:
        soft: "65536"
        hard: "65536"
    volumes:
        - ./be_2/conf:/opt/be/conf
        - ./be_2/log:/opt/be/log
        - ./be_2/storage:/opt/be/storage
    networks:
      doris-network :
        ipv4_address: 172.66.0.102
  doris-be3:
    image: doris-be:0.15.0
    container_name: doris-be3
    ports:
        - 9350:9050
    ulimits:
      nofile:
        soft: "65536"
        hard: "65536"
    volumes:
        - ./be_3/conf:/opt/be/conf
        - ./be_3/log:/opt/be/log
        - ./be_3/storage:/opt/be/storage
    networks:
      doris-network :
        ipv4_address: 172.66.0.103

networks:
  doris-network:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.66.0.0/16
          gateway: 172.66.0.1

说明：

ipv4_address:容器绑定固定ip
ulimits:设置系统最大打开文件句柄数,就是:

1
2
3

vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536

fe.conf
根据ip修改priority_networks

1	priority_networks = 172.66.0.100/16

be.conf
根据ip修改priority_networks

1	priority_networks = 172.66.0.101/16

执行docker-compose up -d

在 FE 中添加所有 BE 节点。本地需要安装mysql，Doris实现mysql协议，使用mysql客户端登录fe，默认用root密码为空。

mysql -P9030 -uroot -p
ALTER SYSTEM ADD BACKEND "172.66.0.101:9050";
ALTER SYSTEM ADD BACKEND "172.66.0.102:9050";
ALTER SYSTEM ADD BACKEND "172.66.0.103:9050";

修改密码

1	SET PASSWORD FOR 'root' = PASSWORD('123456');

使用 mysql-client 连接到 FE，并执行 SHOW PROC '/backends'; 查看 BE 运行情况。如一切正常，isAlive 列应为 true。
1d9Van
查看 Follower 或 Observer 运行状态。使用 mysql-client 连接到任一已启动的 FE，并执行：SHOW PROC '/frontends'; 可以查看当前已加入集群的 FE 及其对应角色。
vinOjZ
至此Doris安装完成，portal页面地址：http://localhost:8030/

打开后输入root和密码进入，如果没有修改过密码，则不填写密码

b03x7C

查看FE

mqdZ1G

查看BE

Is2zh8

基础使用

添加删除查看FE

ALTER SYSTEM ADD FOLLOWER "hostname:9050";
ALTER SYSTEM DROPP FOLLOWER "hostname:9050";  
SHOW PROC '/frontends';
show backends \G

增加删除查看BE

ALTER SYSTEM ADD BACKEND "hostname:9050";
ALTER SYSTEM DROPP BACKEND "hostname:9050"; // 不推荐
ALTER SYSTEM DECOMMISSION BACKEND "hostname:9050"; 推荐
SHOW PROC '/backends';
SHOW PROC '/backends'\G

创建数据库
1
create database doris;

创建用户

1	create user 'doris' identified by 'password';

赋权
1
grant all on doris to doris;

创建表

create table table1(
  id int default '0',
  name varchar(32) default '',
  city_code smallint,
  pv bigint sum default '0'
)
aggregate key(id, name, city_code)
distributed by hash(id) buckets 10
properties('replication_num' = '3');

create table table2(
  id int default '0',
  name varchar(32) default '',
  city_code smallint,
  event_day date,
  pv bigint sum default '0'
)
aggregate key(id, name, city_code, event_day)
partition by range(event_day)
(
  partition p202107 values less than ('2021-08-01'),
  partition p202108 values less than ('2021-09-01'),
  partition p202109 values less than ('2021-10-01')
)
distributed by hash(id) buckets 10
properties('replication_num' = '3');

插入表

1	insert into table1(id, name, city_code, pv) values(2, 'grace', 1, 2),(5, 'helen', 3, 3),(3, 'tom', 2, 2);

制作Doris镜像

docker运行

docker run

docker-compose

基础使用

相关链接