Elastic Stack 部署

发表于： 更新于： 分类于：
字数： 6890 阅读：≈ 14分钟 浏览： 评论：

I. 前提条件

参考文档： Installing the Elastic Stack

安装 Elastic Stack 时，必须在整个堆栈中使用相同的版本。例如，如果使用 Elasticsearch 8.12.0，则需要安装 Beats 8.12.0、APM Server 8.12.0、Elasticsearch Hadoop 8.12.0、Kibana 8.12.0 和 Logstash 8.12.0。

如果要升级现有安装，请参阅升级 Elastic Stack ，了解如何确保与 8.12.0 兼容。

1.1 网络要求

要在内部安装 Elastic Stack，需要为每个组件打开以下端口。

默认端口组件
3002Enterprise Search
5044Elastic Agent → Logstash
Beats → Logstash
5601Kibana
Elastic Agent → Fleet
Fleet Server → Fleet
8220Elastic Agent → Fleet Server
APM Server
9200-9300Elasticsearch REST API
9300-9400Elasticsearch node transport and communication
9600-9700Logstash REST API

1.2 安装顺序

按以下顺序安装要使用的 Elastic Stack 产品：

  1. Elasticsearch ( install instructions )
  2. Kibana ( install )
  3. Logstash ( install )
  4. Elastic Agent ( install instructions ) or Beats ( install instructions )
  5. APM ( install instructions )
  6. Elasticsearch Hadoop ( install instructions )

按此顺序安装可确保每个产品所依赖的组件都已就位。

1.3 节点规划

参数VM1VM2VM3VM4
Hostnameapp1app2app3app4
Serviceselasticsearch, kibanaelasticsearch, logstashelasticsearchelasticsearch
IP Address192.168.31.61192.168.31.62192.168.31.63192.168.31.64
Node Rolesmaster,mastermaster, data, voting_onlymaster, data
OSCentOS 7.9CentOS 7.9CentOS 7.9CentOS 7.9

1.4 服务器配置

RHEL 8 初始化

II. Elasticsearch 安装

2.1 登录 sudo 用户 

1

su user

2.2 Hosts 修改

修改各节点服务器 hosts

1

sudo vim /etc/hosts

添加内容如下：

1
2
3
4

192.168.31.61  app1
192.168.31.62  app2
192.168.31.63  app3
192.168.31.64  app4

2.3 创建工作目录 

1
2

mkdir elastic-install-files
cd elastic-install-files

2.4 导入 Elasticsearch GPG 密钥 

1

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2.5 下载并手动安装 RPM

最新的 Elasticsearch 稳定版可在 Download Elasticsearch 页面找到。其他版本可在 Past Releases page 页面找到。

  1. 安装 perl-Digest-SHA
1

sudo yum install perl-Digest-SHA
  1. 安装 Elasticsearch
1
2
3
4

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.12.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.12.0-x86_64.rpm.sha512
shasum -a 512 -c elasticsearch-8.12.0-x86_64.rpm.sha512 
sudo rpm --install elasticsearch-8.12.0-x86_64.rpm
  1. 导出安装后的输出信息。
  2. 将 elasticsearch 服务配置为使用 systemd 自动启动。
1
2

sudo systemctl daemon-reload
sudo systemctl enable elasticsearch

⚠️ 重要：先不要启动 Elasticsearch 服务！在重新启动之前，还有几个配置步骤要做。

2.6 创建数据目录

  1. 创建数据、日志、临时目录
1

sudo mkdir -p /mnt/sdc/elastic/elasticsearch/{data,log,tmp}
  1. 更改目录所属用户/组和权限
1
2
3

cd /mnt/sdc/elastic/
sudo chown -R elasticsearch:elasticsearch ./elasticsearch/
sudo chmod -R 2750 ./elasticsearch/

2.7 在节点之间配置 TLS

2.7.1 创建证书存放目录 

1
2

sudo mkdir /etc/elasticsearch/certs
sudo chmod 750 /etc/elasticsearch/certs

2.7.2 生成证书授权

  1. 启动 Elasticsearch 之前，请在任何单节点上使用 [elasticsearch-certutil](https://www.elastic.co/guide/en/elasticsearch/reference/current/certutil.html) 工具为群集生成 CA。

创建一个CA证书，并指定有效期3650天，默认1095天(三年)，密码为 capasswd （使用默认路径直接回车，生产环境请修改密码，密码不要有符号）。

1

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil ca --days 3650 --pass "capasswd" --silent
  1. 在任何单个节点上，为群集中的节点生成证书和私钥。其中包括上一步生成的 elastic-stack-ca.p12 输出文件（使用默认路径直接回车）。
1

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --ca-pass "capasswd" --pass "transpasswd" --silent
  1. 在群集的每个节点上，将 elastic-certificates.p12 文件复制到 $ES_PATH_CONF 目录（如果使用软件包发行版（Debian 或 RPM），变量默认为 /etc/elasticsearch）。
1

sudo scp /usr/share/elasticsearch/elastic-certificates.p12 root@appX:/etc/elasticsearch/certs/

2.7.3 将密码存储到密钥存储中

  1. 如果在创建节点证书时输入了密码，在集群中的每个节点上，运行以下命令将传输密码（transpasswd）存储到 Elasticsearch 密钥存储中：
1

sudo /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
1

sudo /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
  1. 为集群中的每个节点完成前面的步骤。
  2. 在集群中的每个节点上，启动 Elasticsearch。 启动 和 停止 Elasticsearch 的方法因安装方式而异。

📢 注意：
您必须重新启动整个群集。配置为使用 TLS 传输的节点无法与使用未加密传输连接的节点通信（反之亦然）。

2.8 在 HTTP 层启用 TLS

2.8.1 生成 HTTP 证书

  1. 前提条件：完成前面 2.7 中的步骤。
  2. 在集群中的每个节点上，停止 Elasticsearch 和 Kibana（如果它们正在运行）。
  3. 运行 Elasticsearch HTTP 证书工具以生成证书签名请求 (CSR)。
1

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil http --silent

💡说明：
此命令生成一个 .zip 文件，其中包含用于 Elasticsearch 和 Kibana 的证书和密钥。每个文件夹都包含一个解释如何使用这些文件的 README.txt。HTTPS 私钥密码： <font style="color:#E8323C;">httppasswd</font> （生产环境按需修改，密码不要有符号）

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131

[root@es1 security]# sudo /usr/share/elasticsearch/bin/elasticsearch-certutil http --silent
## Elasticsearch HTTP Certificate Utility
## Do you wish to generate a Certificate Signing Request (CSR)?
## 是否生成 CSR -> n(否)
Generate a CSR? [y/N]n
## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
## 是否使用已存在的 CA -> y(是)
Use an existing CA? [y/N]y
## What is the path to your CA?
## 粘贴 CA 绝对路径
CA Path: /usr/share/elasticsearch/elastic-stack-ca.p12
## 输入 CA 密码
Password for elastic-stack-ca.p12:
## How long should your certificates be valid?
## 输入证书有效时间（这里写 10 年）
For how long should your certificate be valid? [5y] 10y
## Do you wish to generate one certificate per node?
## 是否为每个节点生成证书 -> y(是)
Generate a certificate per node? [y/N]y
## What is the name of node #1?
## 配置节点1信息
node #1 name: app1
## 哪些主机名将用于连接到 app1
## Which hostnames will be used to connect to app1?
app1
## 回车继续
You entered the following hostnames.

 - app1
## 是否正确？
Is this correct [Y/n]y
## Which IP addresses will be used to connect to app1?
192.168.31.61
## 回车继续
You entered the following IP addresses.

 - 192.168.31.61
Is this correct [Y/n]y
## Other certificate options
Key Name: app1
Subject DN: CN=app1
Key Size: 2048

## 您想更改这些选项中的任何一个吗? -> n(否)
Do you wish to change any of these options? [y/N]n
## 生成其他证书? -> y(是)
Generate additional certificates? [Y/n]y

## 配置节点2信息
## What is the name of node #2?
node #2 name: app2
## Which hostnames will be used to connect to app2?
app2

You entered the following hostnames.

 - app2
Is this correct [Y/n]y
## Which IP addresses will be used to connect to app2?
192.168.31.62

You entered the following IP addresses.

 - 192.168.31.62
Is this correct [Y/n]y
## Other certificate options
Key Name: app2
Subject DN: CN=app2
Key Size: 2048

Do you wish to change any of these options? [y/N]n
Generate additional certificates? [Y/n]y

## 配置节点3信息
## What is the name of node #3?
node #3 name: app3
## Which hostnames will be used to connect to app3?
app3

You entered the following hostnames.

 - app3
Is this correct [Y/n]y
## Which IP addresses will be used to connect to app3?
192.168.31.63

You entered the following IP addresses.

 - 192.168.31.63
Is this correct [Y/n]y
## Other certificate options
Key Name: app3
Subject DN: CN=app3
Key Size: 2048

## 配置节点4信息
## What is the name of node #4?
node #4 name: app4
## Which hostnames will be used to connect to app4?
app4

You entered the following hostnames.

 - app4
Is this correct [Y/n]y
## Which IP addresses will be used to connect to app4?
192.168.31.64

You entered the following IP addresses.

 - 192.168.31.64
Is this correct [Y/n]y
## Other certificate options
Key Name: app4
Subject DN: CN=app4
Key Size: 2048

Do you wish to change any of these options? [y/N]n
Generate additional certificates? [Y/n]y

Do you wish to change any of these options? [y/N]n
## 生成额外的证书？ -> n(否)
Generate additional certificates? [Y/n]n
## What password do you want for your private key(s)?
## 输入 HTTPS 证书密码，此处使用 httppasswd 
Provide a password for the "http.p12" file:  [<ENTER> for none]
## 重复密码以确认
Repeat password to confirm:
## Where should we save the generated files?
## 粘贴生成 HTTPS 证书的绝对路径 (使用默认路径直接回车)
What filename should be used for the output zip file? [/usr/share/elasticsearch/elasticsearch-ssl-http.zip]
  1. 解压生成的 elasticsearch-ssl-http.zip 文件。这个压缩文件包含一个用于 Elasticsearch 和 Kibana 的目录。
1

sudo unzip /usr/share/elasticsearch/elasticsearch-ssl-http.zip
1

sudo tree ~/elasticsearch
1

sudo tree ~/kibana
  1. 查看证书详情（验证配置密码 httppasswd）
1

sudo openssl pkcs12 -in ~/elasticsearch/app1/http.p12 -info

2.8.2 拷贝证书至各节点 

1

sudo scp ~/elasticsearch/appX/http.p12 root@appX:/etc/elasticsearch/certs/

2.8.3 修改各节点证书权限 

1

sudo chmod -R 660 /etc/elasticsearch/certs/*

2.8.4 将密码存储到密钥存储中

  1. 将您的 HTTPS 私钥密码（httppasswd）添加到 Elasticsearch 的安全设置中。
1

sudo /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
  1. 为集群中的每个节点完成前面的步骤。
  2. 在集群中的每个节点上，启动 Elasticsearch。 启动 和 停止 Elasticsearch 的方法因安装方式而异。

2.9 Elasticsearch 配置

2.9.1 备份默认置文件 

1

cp -avc /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.bak

2.9.2 节点1配置参考 

1

sudo vim /etc/elasticsearch/elasticsearch.yml

内容如下：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

cluster.name: elk-cluster
node.name: app1

path.data: /mnt/sdc/elastic/elasticsearch/data
path.logs: /mnt/sdc/elastic/elasticsearch/log

bootstrap.memory_lock: true
network.host: 192.168.31.61
discovery.seed_hosts: ["192.168.31.61", "192.168.31.62", "192.168.31.63", "192.168.31.64"]
cluster.initial_master_nodes: ["app1", "app2", "app3"]

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  client_authentication: required
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
transport.host: 192.168.31.61
# Manual additional configuration
node.roles: [ master ]

查看配置

1

sudo grep -vxE '[[:blank:]]*([#;].*)?' /etc/elasticsearch/elasticsearch.yml

或

1

sudo awk '$1 ~ /^[^;#]/' /etc/elasticsearch/elasticsearch.yml

2.9.3 jvm.options 配置 

1

sudo vim /etc/elasticsearch/jvm.options.d/jvm-heap.options

内容如下：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

## JVM heap size
-Xms2g
-Xmx2g

## Expert settings
# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=/mnt/sdc/elastic/elasticsearch/data
 
# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=/mnt/sdc/elastic/elasticsearch/log/hs_err_pid%p.log
 
## GC logging
-Xlog:gc*,gc+age=trace,safepoint:file=/mnt/sdc/elastic/elasticsearch/log/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m

⚠️注意：
Xms 和 Xmx 设置为不超过总内存的 50%

2.10 重要系统配置

2.10.1 Swapping & Temporary directory 

1

sudo systemctl edit elasticsearch

添加如下内容：

1
2
3

[Service]
LimitMEMLOCK=infinity
Environment=ES_TMPDIR=/mnt/sdc/elastic/elasticsearch/tmp

2.10.2 TCP retransmission timeout

  1. 在 /etc/sysctl.d 中创建一个新文件
1

sudo vim /etc/sysctl.d/99-custom.conf
  1. 按以下格式，每行包含一个要设置的变量
1

net.ipv4.tcp_retries2=5
  1. 不重启系统应用更改
1

sudo sysctl -p /etc/sysctl.d/99-custom.conf
  1. 验证更改
1

sudo sysctl net.ipv4.tcp_retries2

2.11 Elasticsearch 启动

2.11.1 启动 Elasticsearch

  1. 启动 Elasticsearch 服务
1

sudo systemctl start elasticsearch
  1. 分离根证书用于测试
1

sudo openssl pkcs12 -in /usr/share/elasticsearch/elastic-stack-ca.p12 -out /usr/share/elasticsearch/elastic-stack-ca.crt.pem -clcerts -nokeys
  1. 确保 Elasticsearch 正常运行
1

sudo curl --cacert /usr/share/elasticsearch/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://192.168.31.61:9200
  1. 查看 Elasticsearch 服务状态
1

sudo systemctl status elasticsearch
  1. 重置 elastic 密码（最少6位）
1

sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i

2.11.2 无法启动 Elasticsearch 解决

  1. 持续跟踪 elasticsearch 输出日志
1

sudo journalctl -f -u elasticsearch
  1. 列出 Elasticsearch 日志条目
1

sudo journalctl - elasticsearch
  1. 查看集群持续输出日志：
1

tail -f /mnt/sdc/elastic/elasticsearch/log/elk-cluster.log
  1. 查看集群最后 N 行输出日志
1

tail -n 30 /mnt/sdc/elastic/elasticsearch/log/elk-cluster.log
  1. 重置节点（使用 [elasticsearch-node](https://www.elastic.co/guide/en/elasticsearch/reference/current/node-tool.html) 命令，可以在节点上执行某些不安全的操作，这些操作只有在节点关闭时才能进行）
1

sudo /usr/share/elasticsearch/bin/elasticsearch-node repurpose

2.12 设置第二个节点

2.12.1 节点2配置参考 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

cluster.name: elk-cluster
node.name: app2

path.data: /mnt/sdc/elastic/elasticsearch/data
path.logs: /mnt/sdc/elastic/elasticsearch/log

bootstrap.memory_lock: true
network.host: 192.168.31.62
discovery.seed_hosts: ["192.168.31.61", "192.168.31.62", "192.168.31.63", "192.168.31.64"]
cluster.initial_master_nodes: ["app1", "app2", "app3"]

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  client_authentication: required
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
transport.host: 192.168.31.62
# Manual additional configuration
node.roles: [ master ]

2.12.2 启动节点2

  1. 在第二个节点上启动 Elasticsearch：
1

sudo systemctl start elasticsearch
  1. 查看节点2 连接节点1 进度
1

sudo tail -f /mnt/sdc/elasticsearch/log/elk.log
  1. 查看节点2 运行状态
1

sudo curl --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app2:9200

2.12.3 查看节点状态

  1. 查看当前所有节点状态
1

sudo curl --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app1:9200/_cat/nodes?v
  1. 查看集群健康状态
1

sudo curl --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app1:9200/_cluster/health?pretty

也可浏览器直接访问👇

1

https://192.168.31.61:9200/_cluster/health

2.13 设置其它节点

2.13.1 节点3配置参考 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

cluster.name: elk-cluster
node.name: app3

path.data: /mnt/sdc/elastic/elasticsearch/data
path.logs: /mnt/sdc/elastic/elasticsearch/log

bootstrap.memory_lock: true
network.host: 192.168.31.63
discovery.seed_hosts: ["192.168.31.61", "192.168.31.62", "192.168.31.63", "192.168.31.64"]
cluster.initial_master_nodes: ["app1", "app2", "app3"]

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  client_authentication: required
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
transport.host: 192.168.31.63
# Manual additional configuration
node.roles: [ master, data, voting_only ]

2.13.2 节点4配置参考 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

cluster.name: elk-cluster
node.name: app4

path.data: /mnt/sdc/elastic/elasticsearch/data
path.logs: /mnt/sdc/elastic/elasticsearch/log

bootstrap.memory_lock: true
network.host: 192.168.31.64
discovery.seed_hosts: ["192.168.31.61", "192.168.31.62", "192.168.31.63", "192.168.31.64"]

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  client_authentication: required
  keystore.path: certs/elastic-certificates.p12
  truststore.path: certs/elastic-certificates.p12

http.host: 0.0.0.0
transport.host: 192.168.31.64
# Manual additional configuration
node.roles: [ data ]

2.14 后续配置

  1. 停止 Elasticsearch
1

sudo systemctl stop elasticsearch
  1. 注释掉所有master 节点以下配置
1

#cluster.initial_master_nodes: ["app1", "app2", "app3"]
  1. 重新启动 Elasticsearch
1

sudo systemctl start elasticsearch
  1. 查看 Elasticsearch 服务状态
1

sudo systemctl status elasticsearch

III. Kibana 安装

3.1 登录 sudo 用户 

1

su user

3.2 创建工作目录 

1
2

mkdir kibana-install-files
cd kibana-install-files

3.3 下载并手动安装 Kibana

Kibana 的最新稳定版本可在 Download Kibana 页面找到。其他版本可在 Past Releases page 页面找到。

1
2
3
4

wget https://artifacts.elastic.co/downloads/kibana/kibana-8.12.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/kibana/kibana-8.12.0-x86_64.rpm.sha512
shasum -a 512 -c kibana-8.12.0-x86_64.rpm.sha512 
sudo rpm --install kibana-8.12.0-x86_64.rpm

3.4 创建数据目录

  1. 创建数据、日志、临时目录
1

sudo mkdir -p /mnt/sdc/elastic/kibana/{data,log}
  1. 更改目录所属用户/组和权限
1
2
3

cd /mnt/sdc/elastic/
sudo chown -R kibana:kibana ./kibana/
sudo chmod -R 2750 ./kibana/

3.5 创建证书存放目录 

1
2

sudo mkdir /etc/kibana/certs
sudo chmod 750 /etc/kibana/certs

3.6 Kibana 安全配置

参考文档： Configure security in Kibana

  1. 在 kibana.yml 配置文件中设置 xpack.security.encryptionKey 属性。
1

xpack.security.encryptionKey: "something_at_least_32_characters"
  1. 可选：配置 Kibana 的会话过期设置（参考文档： Session management ）。
1
2
3
4
5
6

# 会话空闲超时
xpack.security.session.idleTimeout: "30m"
# 会话寿命
xpack.security.session.lifespan: "1h"
# 会话清理间隔
xpack.security.session.cleanupInterval: "30m"

3.7 在 Kibana 和 Elasticsearch 之间设置 TLS 加密

3.7.1 加密 Kibana 和 Elasticsearch 之间的流量 

1

elasticsearch.ssl.certificateAuthorities: $KBN_PATH_CONF/elasticsearch-ca.pem

3.7.2 加密浏览器和 Kibana 之间的流量

参考文档1： Mutual TLS with Elasticsearch

参考文档2： elasticsearch-certutil

  1. 为 Kibana 生成服务器证书和私钥（参考文档： Elastic Discuss ）。
1
2
3
4
5

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem \
--ca /usr/share/elasticsearch/elastic-stack-ca.p12 --ca-pass "capasswd" \
--name kibana-server \
--dns app1 \
--silent

📢 参数说明：
–name 指定生成证书的名称。
–dns 指定以逗号分隔的 DNS 名称列表（域名或主机名）。
–ip 指定以逗号分隔的 IP 地址列表（域名或主机名解析后的 IP）。

  1. 解压缩 certificate-bundle.zip 文件，复制 kibana-server.crt 和 kibana-server.key 到证书目录
1
2
3
4

sudo cp ~/kibana/elasticsearch-ca.pem /etc/kibana/certs/
sudo cp ~/kibana-server/kibana-server.crt /etc/kibana/certs/
sudo cp ~/kibana-server/kibana-server.key /etc/kibana/certs/
sudo chmod -R 660 /etc/kibana/certs/*
  1. 打开 kibana.yml，添加以下几行，配置 Kibana 访问服务器证书和加密私钥。
1
2

server.ssl.certificate: $KBN_PATH_CONF/kibana-server.crt
server.ssl.key: $KBN_PATH_CONF/kibana-server.key
  1. 在 kibana.yml 中添加以下一行，为入站连接启用 TLS。
1

server.ssl.enabled: true

3.8 服务账户令牌

3.8.1 创建服务账户令牌

以下命令通过 REST API 为 elastic/kibana 服务帐户（ Service accounts ）创建一个名为 kibana_token 的服务帐户令牌

1

sudo curl -X POST --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app1:9200/_security/service/elastic/kibana/credential/token/kibana_token

ℹ️ 输出是不记名令牌，它是 Base64 编码的字符串：
{"created":true,"token":{"name":"kibana_token","value":"AAEAAWV.....RDJycHlR"}}

3.8.2 将 token 值添加到 kibana-keystore 

1

sudo /usr/share/kibana/bin/kibana-keystore add elasticsearch.serviceAccountToken

查看服务账户

1

sudo /usr/share/kibana/bin/kibana-keystore list

3.8.3 REST API 令牌管理

  1. 获取服务帐户凭据 API（ Get service account credentials ）
1

sudo curl -X GET --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app1:9200/_security/service/elastic/kibana/credential
  1. 删除服务帐户令牌（ Delete service account token ）
1

sudo curl -X DELETE --cacert /etc/elasticsearch/certs/elastic-stack-ca.crt.pem -u elastic:$ELASTIC_PASSWORD https://app1:9200/_security/service/elastic/kibana/credential/token/<token_name>

3.9 Kibana 配置参考 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

server.host: "192.168.31.61"
server.publicBaseUrl: "https://192.168.31.61:5601"

server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/certs/kibana-server.crt
server.ssl.key: /etc/kibana/certs/kibana-server.key

elasticsearch.hosts: ["https://192.168.31.61:9200", "https://192.168.31.62:9200", "https://192.168.31.63:9200", "https://192.168.31.64:9200"]

elasticsearch.serviceAccountToken: "AAEAAWV.....RDJycHlR"

elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/elasticsearch-ca.pem" ]
elasticsearch.ssl.verificationMode: full

# Default configuration
logging:
  appenders:
    file:
      type: file
      fileName: /kibana/log/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

path.data: /kibana/data
pid.file: /run/kibana/kibana.pid

i18n.locale: "zh-CN"
monitoring.ui.ccs.enabled: false

# Manual additional configuration
xpack.security.encryptionKey: "uyN6Y4D3xfPrZMhAwRBTcnK9p7eG5EJC"
xpack.security.session.idleTimeout: "30m"
xpack.security.session.lifespan: "1h"
xpack.security.session.cleanupInterval: "30m"
# Optional configuration
xpack.reporting.roles.enabled: false

IV. Logstash 安装

4.1 登录 sudo 用户 

1

su user

4.2 创建工作目录 

1
2

mkdir logstash-install-files
cd logstash-install-files

4.3 下载并手动安装 Logstash

Logstash 的最新稳定版本可在 Download Logstash 页面找到。其他版本可在 Past Releases page 页面找到。

1
2
3
4

wget https://artifacts.elastic.co/downloads/logstash/logstash-8.12.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/logstash/logstash-8.12.0-x86_64.rpm.sha512
shasum -a 512 -c logstash-8.12.0-x86_64.rpm.sha512 
sudo rpm --install logstash-8.12.0-x86_64.rpm

4.4 创建数据目录

  1. 创建数据、日志目录
1

sudo mkdir -p /mnt/sdc/elastic/logstash/{data,log}
  1. 更改目录所属用户/组和权限
1
2
3

cd /mnt/sdc/elastic/
sudo chown -R logstash:logstash ./logstash/
sudo chmod -R 755 ./logstash/

4.5 创建证书存放目录 

1

sudo mkdir /etc/logstash/certs

4.6 Logstash 配置参考 

1
2
3
4

path.data: /mnt/sdc/elastic/logstash/data
path.logs: /mnt/sdc/elastic/logstash/log

path.config: /etc/logstash/conf.d/*.conf

4.7 保护与 Elasticsearch 连接安全

4.7.1 配置 Logstash 以使用 TLS/SSL 加密

参考文档： Configuring Logstash to use TLS/SSL encryption

  1. 复制在 2.8.1 步骤中生成的 CA 证书到 Logstash 证书目录
1

sudo scp ~/kibana/elasticsearch-ca.pem root@app2:/etc/logstash/certs/
  1. 在 logstash.conf 文件中配置 ssl 和 cacert 选项
1
2
3
4
5
6
7

output {
  elasticsearch {
    ...
    ssl => true
    cacert => '/etc/logstash/certs/elasticsearch-ca.pem' 
  }
}

4.7.2 使用 API Key 授予访问权限

您可以使用 API 密钥来授予对 Elasticsearch 资源的访问权限，而不是使用用户名和密码。

参考文档： Grant access using API keys

  1. 创建用于发布的 API Key
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

POST /_security/api_key
{
  "name": "logstash_host001", 
  "role_descriptors": {
    "logstash_writer": { 
      "cluster": ["manage_index_templates", "monitor", "manage_ilm"],
      "index": [
        {
          "names": ["*"],
          "privileges": ["write","create","create_index","manage","manage_ilm"]
        }
      ]
    }
  }
}

返回值类似如下：

1
2
3
4
5
6

{
  "id": "4ffGWocBFIhKxP1f5xaL",
  "name": "logstash_host001",
  "api_key": "TG-S2SvdTOqa5w6OvKIZEg",
  "encoded": "NGZmR1dvY0JGSWhLeFAxZjV4YUw6VEctUzJTdmRUT3FhNXc2T3ZLSVpFZw=="
}
  1. Elasticsearch 插件配置示例
1
2
3
4
5
6

output {
  elasticsearch {
    ...
    api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg" 
  }
}

💡Tips:
api_key 格式： id:api_key

4.8 Logstash pipeline 配置

  1. 示例管道配置
1

sudo vim /etc/logstash/conf.d/logstash-example.conf

参考配置内容：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

input {
    file {
        path => "/tmp/logs/example1.log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        add_field => {
            "log_type" => "logstash-example1"
        }
        stat_interval => "2"
    }
}

output {
  if [log_type] == "logstash-example1" {
    elasticsearch {
        hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
        index => "logstash-example1@-%{+YYYY.MM.dd}--000001"
        manage_template => false
        action => "create"
        ssl => true
        cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
        api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
    }
  }
}
  1. 检查配置文件格式
1

sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash-example.conf -t

V. Filebeat 安装

5.1 登录 sudo 用户 

1

su user

5.2 创建工作目录 

1
2

mkdir filebeat-install-files
cd filebeat-install-files

5.3 下载并手动安装 Filebeat

Filebeat 的最新稳定版本可在 Download Filebeat 页面找到。其他版本可在 Past Releases page 页面找到。

1
2
3
4

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.12.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.12.0-x86_64.rpm.sha512
shasum -a 512 -c filebeat-8.12.0-x86_64.rpm.sha512 
sudo rpm --install filebeat-8.12.0-x86_64.rpm

5.4 创建数据目录

  1. 创建数据、日志目录
1

sudo mkdir -p /mnt/sdc/elastic/filebeat/{data,log}
  1. 更改目录所属用户/组和权限
1
2

cd /mnt/sdc/elastic/
sudo chmod -R 750 ./filebeat/

5.5 创建证书存放目录 

1
2

sudo mkdir /etc/filebeat/certs
sudo chmod 750 /etc/filebeat/certs

5.6 与 Logstash 安全通信

参考文档1： Secure communication with Logstash

参考文档2： Filebeat与Logstash配置ssl加密通信

5.6.1 为 Logstash 生成证书

  1. 为 Logstash 生成证书和私钥
1
2
3
4
5
6

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem \
--ca elastic-stack-ca.p12 \
--name logstash \
--dns app2 \
--out logstash-certificate-bundle.zip \
--silent

此命令生成一个 logstash-certificate-bundle.zip 文件，解压该文件：

1

sudo unzip /usr/share/elasticsearch/logstash-certificate-bundle.zip

内容如下：

1
2
3

/logstash
|_ logstash.crt
|_ logstash.key
  1. logstash 与 beats 启动 ssl 加密通信时，仅支持使用 PKCS8 的密钥格式 ，将logstash.key 转换 为 pkcs#8 格式：
1

sudo openssl pkcs8 -inform PEM -in ~/logstash/logstash.key -topk8 -nocrypt -outform PEM -out ~/logstash/logstash.pkcs8.key
  1. 将 logstash.crt 和 logstash.pkcs8.key 到 logstash 证书目录。
1
2
3
4

sudo scp ~/kibana/elasticsearch-ca.pem root@appX:/etc/logstash/certs
sudo scp ~/logstash/logstash.crt root@appX:/etc/logstash/certs
sudo scp ~/logstash/logstash.pkcs8.key root@appX:/etc/logstash/certs
sudo chmod -R 660 /etc/logstash/certs/*

5.6.2 为 Filebeat 生成证书

  1. 创建实例文件
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

instances:
  - name: "app1" 
    dns: 
      - "app1"
  - name: "app2" 
    dns: 
      - "app2"
  - name: "app3" 
    dns: 
      - "app3"
  - name: "app4" 
    dns: 
      - "app4"
  1. 为 Filebeat 生成证书和私钥
1
2
3
4
5

sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem \
--ca elastic-stack-ca.p12 \
--in ~/instances-filebeat.yml \
--out filebeat-certificate-bundle.zip \
--silent

此命令生成一个 filebeat-certificate-bundle.zip 文件，解压该文件：

1

sudo unzip /usr/share/elasticsearch/filebeat-certificate-bundle.zip -d ~/filebeat-certificate/

内容如下：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

filebeat-certificate/
├── app1
│   ├── app1.crt
│   └── app1.key
├── app2
│   ├── app2.crt
│   └── app2.key
├── app3
│   ├── app3.crt
│   └── app3.key
└── app4
    ├── app4.crt
    └── app4.key
  1. 复制证书文件到各 Filebeat 节点目录。
1
2
3

sudo scp ~/kibana/elasticsearch-ca.pem root@appX:/etc/filebeat/certs
sudo scp ~/filebeat-certificate/appX/appX.crt root@appX:/etc/filebeat/certs
sudo scp ~/filebeat-certificate/appX/appX.key root@appX:/etc/filebeat/certs

5.7 使用 SSL 相互认证:

5.7.1 配置 Filebeat 以使用 SSL 

1
2
3
4
5

output.logstash:
  hosts: ["app2:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/elasticsearch-ca.pem"]
  ssl.certificate: "/etcfilebeat/certs/appX.crt"
  ssl.key: "/etcfilebeat/certs/appX.key"

5.7.2 配置 Logstash pipeline 以使用 SSL 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

5.8 验证测试

参考文档： Validate the Logstash server’s certificate

5.8.1 验证 Logstash 服务器的证书 

1

sudo curl --cacert /etc/filebeat/certs/elasticsearch-ca.pem --cert /etc/filebeat/certs/app1.crt --key /etc/filebeat/certs/app1.key -v https://app2:5044

如果测试成功，您将收到一个空响应错误：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

* About to connect() to app2 port 5044 (#0)
*   Trying 192.168.31.62...
* Connected to app2 (192.168.31.62) port 5044 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/filebeat/certs/elasticsearch-ca.pem
  CApath: none
* NSS: client certificate from file
* 	subject: CN=app1
* 	start date: Feb 24 14:25:00 2024 GMT
* 	expire date: Feb 23 14:25:00 2027 GMT
* 	common name: app1
* 	issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* 	subject: CN=logstash
* 	start date: Feb 24 13:56:23 2024 GMT
* 	expire date: Feb 23 13:56:23 2027 GMT
* 	common name: logstash
* 	issuer: CN=Elastic Certificate Tool Autogenerated CA
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: app2:5044
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host app2 left intact
curl: (52) Empty reply from server

5.8.2 测试 Filebeat 与 Logstash 的连接

  1. Logstash pipeline 配置参考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

input {
  beats {
  port => 5044
  ssl => true
  ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
  ssl_certificate => "/etc/logstash/certs/logstash.crt"
  ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
  ssl_verify_mode => "force_peer"
}
}

output {
  if [log_type] == "logstash-example1" {
  elasticsearch {
  hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
  index => "logstash-example1@-%{+YYYY.MM.dd}-000001"
  manage_template => false
  action => "create"
  ssl => true
  cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
  api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
}
} else if [log_type] == "logstash-example2" {
  elasticsearch {
  hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
  index => "logstash-example2@-%{+YYYY.MM.dd}-000001"
  manage_template => false
  action => "create"
  ssl => true
  cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
  api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
}
}
}
  1. filebeat.yml 配置参考（官方文档： filebeat.reference.yml ）
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

path.data: /mnt/sdc/elastic/filebeat/data
path.logs: /mnt/sdc/elastic/filebeat/log

filebeat.inputs:
- type: filestream
  id: demo-logstash-example2
  enabled: true
  backoff.init: 1s
  backoff.max: 10s
  prospector.scanner.check_interval: 10s
  close.on_state_change.inactive: 5m
  paths:
    - /tmp/logs/example2.log
  fields:
    log_type: logstash-example2
  fields_under_root: true
  parsers:
    - multiline:
        type: pattern
        pattern: '^\[?(?:\d\d){1,2}[-\/](?:0?[1-9]|1[0-2])[-\/](?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T ](?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)\s*(?:[0-9]+.[0-9]+)?\]?'
        negate: true
        match: after

output.logstash:
  hosts: ["appX:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/elasticsearch-ca.pem"]
  ssl.certificate: "/etc/filebeat/certs/appY.crt"
  ssl.key: "/etc/filebeat/certs/appY.key"
  1. 测试配置（参考文档： Filebeat command reference ）
1

sudo /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml test config
  1. 测试输出
1

sudo /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml test output
  1. 如果 Filebeat 作为服务运行，请先停止服务。然后在前台运行 Filebeat 来测试设置，这样就可以快速查看发生的任何错误：
1

sudo /usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -e -v

5.9 应用索引生命周期管理

5.9.1 创建 Logstash ILM 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

PUT _ilm/policy/logstash_ilm_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "50gb",
            "max_age": "30d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "60d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1,
            "index_codec": "best_compression"
          },
          "readonly": {},
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "120d",
        "actions": {
          "set_priority": {
            "priority": 0
          }
        }
      },
      "delete": {
        "min_age": "180d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

5.9.2 应用 ILM 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  if [log_type] == "logstash-example1" {
    elasticsearch {
      hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
      index => "logstash-example1@-%{+YYYY.MM.dd}-000001"
      manage_template => false
      action => "create"
      ssl => true
      cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
      api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
    }
  } else if [log_type] == "logstash-example2" {
    elasticsearch {
      hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
      index => "logstash-example2@-%{+YYYY.MM.dd}-000001"
      manage_template => false
      action => "create"
      ssl => true
      cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
      api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
    }
  } else if [log_type] == "logstash-example3" {
    elasticsearch {
      hosts => ["https://app1:9200","https://app2:9200","https://app3:9200", "https://app4:9200"]
      data_stream => "false"
      ilm_rollover_alias => "logstash-example3@"
      ilm_pattern => "{now/d}-000001"
      ilm_policy => "logstash_ilm_policy"
      ssl => true
      cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
      api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
    }
  }
}

VI. Metricbeat 安装

6.1 登录 sudo 用户 

1

su user

6.2 创建工作目录 

1
2

mkdir metricbeat-install-files
cd metricbeat-install-files

6.3 下载并手动安装 Metricbeat

Metricbeat 的最新稳定版本可在 Download Metricbeat 页面找到。其他版本可在 Past Releases page 页面找到。

1
2
3
4

wget https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.12.0-x86_64.rpm
wget https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.12.0-x86_64.rpm.sha512
shasum -a 512 -c metricbeat-8.12.0-x86_64.rpm.sha512
sudo rpm --install metricbeat-8.12.0-x86_64.rpm

6.4 创建数据目录

  1. 创建数据、日志目录
1

sudo mkdir -p /mnt/sdc/elastic/metricbeat/{data,log}
  1. 更改目录所属用户/组和权限
1
2

cd /mnt/sdc/elastic/
sudo chmod -R 750 ./metricbeat/

6.5 创建证书存放目录 

1
2

sudo mkdir /etc/metricbeat/certs
sudo chmod 750 /etc/metricbeat/certs

6.6 保护与 Logstash 连接安全

6.6.1 配置 Logstash Pipeline 以使用 TLS/SSL 加密

参考文档： Configuring Logstash to use TLS/SSL encryption

  1. 复制在 2.8.1 步骤中生成的 CA 证书到 Logstash 证书目录
1

sudo scp ~/kibana/elasticsearch-ca.pem root@app2:/etc/logstash/certs/
  1. 在 logstash-example.conf 文件中配置 ssl 和 cacert 选项
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  elasticsearch {
    ...
    ssl => true
    cacert => '/etc/logstash/certs/elasticsearch-ca.pem' 
    api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg" 
  }
}

6.6.2 配置 Metricbeat 以使用 SSL 

1
2
3
4
5

output.logstash:
  hosts: ["app2:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/elasticsearch-ca.pem"]
  ssl.certificate: "/etcfilebeat/certs/appX.crt"
  ssl.key: "/etcfilebeat/certs/appX.key"

6.7 创建 Metricbeat ILM 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

PUT _ilm/policy/metricbeat_ilm_policy
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_primary_shard_size": "50gb",
            "max_age": "30d"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "warm": {
        "min_age": "60d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1,
            "index_codec": "best_compression"
          },
          "readonly": {},
          "set_priority": {
            "priority": 50
          }
        }
      },
      "cold": {
        "min_age": "120d",
        "actions": {
          "set_priority": {
            "priority": 0
          }
        }
      },
      "delete": {
        "min_age": "180d",
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

6.8 验证测试

参考文档： Validate the Logstash server’s certificate

6.8.1 验证 Logstash 服务器的证书 

1

sudo curl --cacert /etc/filebeat/certs/elasticsearch-ca.pem --cert /etc/filebeat/certs/app1.crt --key /etc/filebeat/certs/app1.key -v https://app2:5044

6.8.2 测试 Metricbeat 与 Logstash 的连接

  1. Logstash pipeline 配置参考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/certs/elasticsearch-ca.pem"]
    ssl_certificate => "/etc/logstash/certs/logstash.crt"
    ssl_key => "/etc/logstash/certs/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  if [log_type] == "metricbeat" {
    elasticsearch {
      hosts => ["https://app1:9200", "https://app2:9200", "https://app3:9200", "https://app4:9200"]
      data_stream => "false"
      ilm_rollover_alias => "metricbeat"
      ilm_pattern => "{now/d}-000001"
      ilm_policy => "logstash_ilm_policy"
      ssl => true
      cacert => "/etc/logstash/certs/elasticsearch-ca.pem"
      api_key => "4ffGWocBFIhKxP1f5xaL:TG-S2SvdTOqa5w6OvKIZEg"
    }
  }
}
  1. metricbeat.yml 配置参考（官方文档： metricbeat.reference.yml ）
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

path.data: /metricbeat/data
path.logs: /metricbeat/log

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression

fields:
    log_type: metricbeat
fields_under_root: true

output.logstash:
  hosts: ["app2:5044"]
  ssl.certificate_authorities: ["/etc/filebeat/certs/elasticsearch-ca.pem"]
  ssl.certificate: "/etc/filebeat/certs/app1.crt"
  ssl.key: "/etc/filebeat/certs/app1.key"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~
  1. 测试配置（参考文档： Metricbeat command reference ）
1

sudo /usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml test config
  1. 测试输出
1

sudo /usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml test output
  1. 测试所有已配置的模块设置
1
2

sudo /usr/share/metricbeat/bin/metricbeat test modules
sudo /usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml test modules

ℹ️ 测试指定模块设置
语法：modules [MODULE_NAME] [METRICSET_NAME]
测试：sudo /usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml test modules system cpu

  1. 如果 Metricbeat 作为服务运行，请先停止服务。然后在前台运行 Metricbeat 来测试设置，这样就可以快速查看发生的任何错误：
1

sudo /usr/share/metricbeat/bin/metricbeat -c /etc/metricbeat/metricbeat.yml -e -v