etcd 入门

etcd（读作 et-see-dee）是一种开源的分布式统一键值存储，用于分布式系统或计算机集群的共享配置、服务发现和的调度协调。etcd 有助于促进更加安全的自动更新，协调向主机调度的工作，并帮助设置容器的覆盖网络。

etcd 是许多其他项目的核心组件。最值得注意的是，它是 Kubernetes 的首要数据存储，也是容器编排的实际标准系统。使用 etcd，云原生应用可以保持更为一致的运行时间，而且在个别服务器发生故障时也能正常工作。应用从 etcd 读取数据并写入到其中；通过分散配置数据，为节点配置提供冗余和弹性。

一种键值对分布式数据库。

通过 raft 共识机制，使得 etcd 具备高可用性。

raft visualization

安装并启动

etcd 作为 k8s 的常见组建，使用 Go 开发，可以方便的构建并安装到大多数的系统中。在 Linux 中只需要一个二进制程序即可启动。

cd /tmp
wget https://github.com/etcd-io/etcd/releases/download/v3.5.2/etcd-v3.5.2-linux-amd64.tar.gz
tar -zxvf etcd-v3.5.2-linux-amd64.tar.gz -C etcd
cd etcd/
./etcd
## 下面就是启动日志

对于 etcd 集群的启动，通过环境变量的方式进行参数配置并启动。

官方也提供了单机集群的方案，需要用到 goreman，然后修改 Procfile 文件，把 etcd 执行路径修改为真实路径以及网络配置的变动，通过 goreman -f Procfile start 即可启动一个 etcd 集群。

使用

常见两种方式：

命令行通过 etcdctl 进行使用
程序包调用

增删改查

# put
etcdctl --endpoints=$ENDPOINTS put foo "Hello World!"

# get
etcdctl --endpoints=$ENDPOINTS get foo
etcdctl --endpoints=$ENDPOINTS --write-out="json" get foo

# get keys by prefix
etcdctl --endpoints=$ENDPOINTS put web1 value1
etcdctl --endpoints=$ENDPOINTS put web2 value2
etcdctl --endpoints=$ENDPOINTS put web3 value3
etcdctl --endpoints=$ENDPOINTS get web --prefix

# delete keys
etcdctl --endpoints=$ENDPOINTS put key myvalue
etcdctl --endpoints=$ENDPOINTS del key

# delete keys by prefix
etcdctl --endpoints=$ENDPOINTS put k1 value1
etcdctl --endpoints=$ENDPOINTS put k2 value2
etcdctl --endpoints=$ENDPOINTS del k --prefix

事务

# use transaction
# etcd transaction mode
# if compare
# then op
# else op
# commit

etcdctl --endpoints=$ENDPOINTS put user1 bad
etcdctl --endpoints=$ENDPOINTS txn --interactive

compares:
value("user1") = "bad"

success requests (get, put, delete):
del user1

failure requests (get, put, delete):
put user1 good

SUCCESS

1

# in this case, key:user1 could be delete

监测 key

# watch keys
# in this case,you should open two terminal
# terminal 1, create a watcher, wait key:stock1 appear
etcdctl --endpoints=$ENDPOINTS watch stock1
# after terminal2 put stock1,could happen:
PUT
stock1
1000

# terminal 2, put stock1
etcdctl --endpoints=$ENDPOINTS put stock1 1000

# watch keys by prefix
# terminal 1,wait
etcdctl --endpoints=$ENDPOINTS watch stock --prefix
# after terminal2 put keys,could happen:
PUT
stock1
10
PUT
stock2
20

# terminal 2,put keys
etcdctl --endpoints=$ENDPOINTS put stock1 10
etcdctl --endpoints=$ENDPOINTS put stock2 20

# or watch keys history
etcdctl --endpoints=$ENDPOINTS watch --rev=1 foo

租约

# lease
## create a lease
etcdctl --endpoints=$ENDPOINTS lease grant 300
lease 49527f598bbfe014 granted with TTL(300s)

## key binding to lease
etcdctl --endpoints=$ENDPOINTS put sample value --lease=49527f598bbfe014
etcdctl --endpoints=$ENDPOINTS get sample
sample
value

## if set keep-alive, than the lease could never timeout
etcdctl --endpoints=$ENDPOINTS lease keep-alive 49527f598bbfe014

## revoke lease manual
etcdctl --endpoints=$ENDPOINTS lease revoke 49527f598bbfe014
lease 49527f598bbfe014 revoked

# or after 300 seconds,could get null data
etcdctl --endpoints=$ENDPOINTS get sample

分布式锁

# create lock
# in this case,you should open two terminal
# terminal 1,
etcdctl --endpoints=$ENDPOINTS lock mutex1
mutex1/49527f598bbfe018
# interrupt this lock
^C

# terminal 2, wait terminal 1 release lock
etcdctl --endpoints=$ENDPOINTS lock mutex1
# will get lock after terminal 1 interrupt
mutex1/49527f598bbfe01b

领导选举

# How to conduct leader election in etcd cluster
# but i can't unserstand
# tips: need think more

查看集群状态

# check cluster status
─ ~ ──────────────────────────────────────────────────────── INT ✘
╰─ etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint status
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|      ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|  172.16.242.3:2379 | 45d97e7a24a2c952 |   3.5.2 |   20 kB |     false |      false |         2 |         31 |                 31 |        |
| 172.16.242.3:22379 | 1803770bd57fe7f5 |   3.5.2 |   20 kB |     false |      false |         2 |         31 |                 31 |        |
| 172.16.242.3:32379 | 180470dceeda2be0 |   3.5.2 |   20 kB |      true |      false |         2 |         31 |                 31 |        |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

╭─ ~ ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── ✔
╰─ etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint health
+--------------------+--------+-------------+-------+
|      ENDPOINT      | HEALTH |    TOOK     | ERROR |
+--------------------+--------+-------------+-------+
|  172.16.242.3:2379 |   true | 10.182483ms |       |
| 172.16.242.3:32379 |   true | 11.098445ms |       |
| 172.16.242.3:22379 |   true | 13.308361ms |       |
+--------------------+--------+-------------+-------+

快照

# save the database
# must set endpoint to one host
ENDPOINTS=$HOST_1:2379

etcdctl --endpoints=$ENDPOINTS snapshot save my.db
{"level":"info","ts":1646487341.2083359,"caller":"snapshot/v3_snapshot.go:119","msg":"created temporary db file","path":"my.db.part"}
{"level":"info","ts":"2022-03-05T21:35:41.210+0800","caller":"clientv3/maintenance.go:200","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":1646487341.211041,"caller":"snapshot/v3_snapshot.go:127","msg":"fetching snapshot","endpoint":"172.16.242.3:2379"}
{"level":"info","ts":"2022-03-05T21:35:42.438+0800","caller":"clientv3/maintenance.go:208","msg":"completed snapshot read; closing"}
{"level":"info","ts":1646487342.465489,"caller":"snapshot/v3_snapshot.go:142","msg":"fetched snapshot","endpoint":"172.16.242.3:2379","size":"44 MB","took":1.257066939}
{"level":"info","ts":1646487342.4674938,"caller":"snapshot/v3_snapshot.go:152","msg":"saved","path":"my.db"}
Snapshot saved at my.db

etcdctl --write-out=table --endpoints=$ENDPOINTS snapshot status my.db
+----------+----------+------------+------------+
|   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+----------+----------+------------+------------+
| 79ec4d35 |    18024 |      36026 |      44 MB |
+----------+----------+------------+------------+
# in this case, I found keys is too much and revision too high. Because I use `ectdctl --endpoints=$ENDPOINTS check perf`.Every perfomance check could raise too much keys and revision.

合并v2到v3

# how to migrate etcd from v2 to v3
# let it pass
# write key in etcd version 2 store
export ETCDCTL_API=2
etcdctl --endpoints=http://$ENDPOINT set foo bar

# read key in etcd v2
etcdctl --endpoints=$ENDPOINTS --output="json" get foo

# stop etcd node to migrate, one by one

# migrate v2 data
export ETCDCTL_API=3
etcdctl --endpoints=$ENDPOINT migrate --data-dir="default.etcd" --wal-dir="default.etcd/member/wal"

# restart etcd node after migrate, one by one

# confirm that the key got migrated
etcdctl --endpoints=$ENDPOINTS get /foo

增减集群节点

# deal with membership
https://etcd.io/docs/v3.5/tutorials/how-to-deal-with-membership/

权限、用户与角色

# create role
etcdctl --endpoints=${ENDPOINTS} role add root
Role root created

## can be grant to every key,need use / to replace foo
etcdctl --endpoints=${ENDPOINTS} role grant-permission root readwrite foo
Role root updated

etcdctl --endpoints=${ENDPOINTS} role get root
Role root
KV Read:
	foo
KV Write:
	foo


# create user and grant role
## set password is 123
etcdctl --endpoints=${ENDPOINTS} user add root
Password of root:
Type password of root again for confirmation:
User root created

etcdctl --endpoints=${ENDPOINTS} user grant-role root root
Role root is granted to user root

etcdctl --endpoints=${ENDPOINTS} user get root
User: root
Roles: root

# enable auth
etcdctl --endpoints=${ENDPOINTS} auth enable
Authentication Enabled
# now all client requests go through auth

etcdctl --endpoints=${ENDPOINTS} --user=root:123 put foo bar
OK

etcdctl --endpoints=${ENDPOINTS} get foo
{"level":"warn","ts":"2022-03-05T22:19:54.108+0800","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-7b0e89ed-5f03-4f6d-b44c-ce59520a4c07/172.16.242.3:2379","attempt":0,"error":"rpc error: code = InvalidArgument desc = etcdserver: user name is empty"}
Error: etcdserver: user name is empty

etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo
foo
bar

etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo1

tips: root 在内部可能有独立定义（或者uid很高），除非先关闭auth，否则无法直接删除该用户

A/B

Etcd入门