[容器]用 Docker Machine 批量安装和配置 docker host

一、实验环境准备

  1. 准备VMware安装三台CentOS 8的虚拟机，启动且相互ping通网络，ip分别为192.168.50.12.208、192.168.50.12.127、192.168.50.126.后续各个host简称208、126、127；
  2. 以208为主安装Docker Machine对127和126进行Docker的安装和部署，需要利用ssh-copy-id 实现在127和126的免密码登录，免密登录参考之前的文章；
  3. (可选)在127 126上，国内为yum添加阿里云镜像避免安装下载docker的失败。

二、Docker Machine

在208上安装Docker Machine，不同平台有不同的安装命令参考官方文档

[woods@localhost ~]$ base=https://github.com/docker/machine/releases/download/v0.16.0 &&
  curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker-machine &&
  sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
  chmod +x /usr/local/bin/docker-machine

验证是否安装完成

[woods@localhost /]$ docker-machine -v
docker-machine version 0.16.0, build 702c267f

三、使用Docker Machine安装Docker

在208 host上安装Docker Machine，不同平台有不同的安装命令参考官方文档，这里是普通的Linux系统所以--drive选项用generic 而--generic-ip-address是指定安装host的ip地址

[woods@localhost ~]$ docker-machine create --driver generic --generic-ip-address=192.168.50.127 host2
Running pre-create checks...
Creating machine...
(host2) No SSH key specified. Assuming an existing key at the default location.
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with centos...

发现卡住了（也可能显示install docker 失败等信息），此时用docker-machine ls发现host2已经存在，先用docker-machine rm host2删除了，再执行docker-machine -D create安装，-D是显示日志。然后

1.从日志里面Cannot download Packages/docker-ce-cli-19.03.9-3.el7.x86_64.rpm: All mirrors were tried下载失败，备好梯子，docker-machine rm host2后从新执行docker-machine -D create安装

……
if ! type docker; then curl -sSL https://get.docker.com | sh -; fi
SSH cmd err, output: exit status 1: bash: 第 0 行:type: docker: 未找到
# Executing docker install script, commit: 26ff363bcf3b3f5a00498ac43694bf1c7d9ce16c
+ sh -c 'yum install -y -q yum-utils'
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
添加仓库自：https://download.docker.com/linux/centos/docker-ce.repo
+ '[' stable '!=' stable ']'
+ sh -c 'yum makecache'
CentOS-8 - AppStream                            8.1 kB/s | 4.3 kB     00:00    
CentOS-8 - Base                                 2.2 kB/s | 3.9 kB     00:01    
CentOS-8 - Extras                               540  B/s | 1.5 kB     00:02    
Docker CE Stable - x86_64                       1.1 kB/s | 3.5 kB     00:03    
元数据缓存已建立。
+ '[' -n '' ']'
+ sh -c 'yum install -y -q docker-ce'

错误：下载软件包出错
  Cannot download Packages/docker-ce-cli-19.03.9-3.el7.x86_64.rpm: All mirrors were tried

Error creating machine: Error running provisioning: error installing docker: 
notifying bugsnag: [Error creating machine: Error running provisioning: error installing docker: ]
……
  1. 从日志里面发现package docker-ce-3:19.03.9-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed或者错误：事务检查错误： file /usr/share/man/man1/docker-attach.1.gz from install of docker-ce-cli-1:19.03.8-3.el7.x86_64 conflicts with file from package podman-manpages-1.4.2-5.module_el8.1.0+237+63e26edc.noarch，用ssh登陆host2参考文章修复问题再安装 
……
(host2) Calling .GetSSHHostname
(host2) Calling .GetSSHPort
(host2) Calling .GetSSHKeyPath
(host2) Calling .GetSSHUsername
Using SSH client type: external
&{[-F /dev/null -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none -o LogLevel=quiet -o PasswordAuthentication=no -o ServerAliveInterval=60 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@192.168.50.127 -p 22] /usr/bin/ssh <nil>}
About to run SSH command:
if ! type docker; then curl -sSL https://get.docker.com | sh -; fi
SSH cmd err, output: exit status 1: bash: 第 0 行:type: docker: 未找到
# Executing docker install script, commit: 26ff363bcf3b3f5a00498ac43694bf1c7d9ce16c
+ sh -c 'yum install -y -q yum-utils'
+ sh -c 'yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo'
添加仓库自：https://download.docker.com/linux/centos/docker-ce.repo
+ '[' stable '!=' stable ']'
+ sh -c 'yum makecache'
CentOS-8 - AppStream                            2.6 kB/s | 4.3 kB     00:01    
CentOS-8 - Base                                 1.9 kB/s | 3.9 kB     00:02    
CentOS-8 - Extras                               2.6 kB/s | 1.5 kB     00:00    
Docker CE Stable - x86_64                       1.6 kB/s | 3.5 kB     00:02    
元数据缓存已建立。
+ '[' -n '' ']'
+ sh -c 'yum install -y -q docker-ce'
错误：
 问题: package docker-ce-3:19.03.9-3.el7.x86_64 requires containerd.io >= 1.2.2-3, but none of the providers can be installed
  - cannot install the best candidate for the job
  - package containerd.io-1.2.10-3.2.el7.x86_64 is excluded
  - package containerd.io-1.2.13-3.1.el7.x86_64 is excluded
  - package containerd.io-1.2.13-3.2.el7.x86_64 is excluded
  - package containerd.io-1.2.2-3.3.el7.x86_64 is excluded
  - package containerd.io-1.2.2-3.el7.x86_64 is excluded
  - package containerd.io-1.2.4-3.1.el7.x86_64 is excluded
  - package containerd.io-1.2.5-3.1.el7.x86_64 is excluded
  - package containerd.io-1.2.6-3.3.el7.x86_64 is excluded

Error creating machine: Error running provisioning: error installing docker: 
notifying bugsnag: [Error creating machine: Error running provisioning: error installing docker: ]
  1. TLS认证失败,Error checking and/or regenerating the certs，但此时host上已经成功安装docker及配置好远程连接，结果排查很久发现是host上的端口没有放通导致，参考文章放通后正常
……
SSH cmd err, output: <nil>: Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:5355            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN     
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN     
tcp6       0      0 :::5355                 :::*                    LISTEN     
tcp6       0      0 :::111                  :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
tcp6       0      0 ::1:631                 :::*                    LISTEN     
tcp6       0      0 :::2376                 :::*                    LISTEN     

Checking connection to Docker...
(node3) Calling .GetURL
Reading CA certificate from /home/woods/.docker/machine/certs/ca.pem
Reading client certificate from /home/woods/.docker/machine/certs/cert.pem
Reading client key from /home/woods/.docker/machine/certs/key.pem
Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.137.3:2376": dial tcp 192.168.137.3:2376: connect: no route to host
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.

notifying bugsnag: [Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "192.168.137.3:2376": dial tcp 192.168.137.3:2376: connect: no route to host
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.
]

host上的端口放通前后，docker-machine ls 或出现connect: no route to host

#放通前
[woods@localhost node3]$ docker-machine ls
NAME    ACTIVE   DRIVER    STATE     URL                        SWARM   DOCKER     ERRORS
node1   -        generic   Running   tcp://192.168.137.2:2376           Unknown    Unable to query docker version: Get https://192.168.137.2:2376/v1.15/version: dial tcp 192.168.137.2:2376: connect: no route to host
node3   -        generic   Running   tcp://192.168.137.3:2376           v19.03.9   
#放通后
[woods@localhost node3]$ docker-machine ls
NAME    ACTIVE   DRIVER    STATE     URL                        SWARM   DOCKER     ERRORS
node1   -        generic   Running   tcp://192.168.137.2:2376           v19.03.9   
node3   -        generic   Running   tcp://192.168.137.3:2376           v19.03.9

四、题外话

1. 安装后host上的docker配置在/etc/systemd/system/docker.service.d/10-machine.conf 

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: dis>
  Drop-In: /etc/systemd/system/docker.service.d
           └─10-machine.conf
   Active: active (running) since Thu 2020-05-21 12:10:31 EDT; 6min ago
     Docs: https://docs.docker.com
 Main PID: 11243 (dockerd)
    Tasks: 8
   Memory: 45.8M
   CGroup: /system.slice/docker.service
           └─11243 /usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.so>

$ cat 10-machine.conf 
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --storage-driver overlay2 --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=generic 
Environment=
  1. -H tcp://0.0.0.0:2376 使 docker daemon 接受远程连接。
  2. --tls* 对远程连接启用安全认证和加密。

2. machine上对host的配置在~/.docker/machine/machines/目录下其中node1是hostname，里面是证书及配置文件

[woods@localhost /]$ ls ~/.docker/machine/machines/
node1  node3
[woods@localhost machines]$ cd node3/
[woods@localhost node3]$ ls
ca.pem  cert.pem  config.json  id_rsa  id_rsa.pub  key.pem  server-key.pem  server.pem
[woods@localhost node3]$ cat config.json 
{
    "ConfigVersion": 3,
    "Driver": {
        "IPAddress": "192.168.137.3",
        "MachineName": "node3",
        "SSHUser": "root",
        "SSHPort": 22,
        "SSHKeyPath": "/home/woods/.docker/machine/machines/node3/id_rsa",
        "StorePath": "/home/woods/.docker/machine",
        "SwarmMaster": false,
        "SwarmHost": "",
        "SwarmDiscovery": "",
        "EnginePort": 2376,
        "SSHKey": "/home/woods/.ssh/id_rsa"
    },
    "DriverName": "generic",
    "HostOptions": {
        "Driver": "",
        "Memory": 0,
        "Disk": 0,
        "EngineOptions": {
            "ArbitraryFlags": [],
            "Dns": null,
            "GraphDir": "",
            "Env": [],
            "Ipv6": false,
            "InsecureRegistry": [],
            "Labels": [],
            "LogLevel": "",
            "StorageDriver": "",
            "SelinuxEnabled": false,
            "TlsVerify": true,
            "RegistryMirror": [],
            "InstallURL": "https://get.docker.com"
        },
        "SwarmOptions": {
            "IsSwarm": false,
            "Address": "",
            "Discovery": "",
            "Agent": false,
            "Master": false,
            "Host": "tcp://0.0.0.0:3376",
            "Image": "swarm:latest",
            "Strategy": "spread",
            "Heartbeat": 0,
            "Overcommit": 0,
            "ArbitraryFlags": [],
            "ArbitraryJoinFlags": [],
            "Env": null,
            "IsExperimental": false
        },
        "AuthOptions": {
            "CertDir": "/home/woods/.docker/machine/certs",
            "CaCertPath": "/home/woods/.docker/machine/certs/ca.pem",
            "CaPrivateKeyPath": "/home/woods/.docker/machine/certs/ca-key.pem",
            "CaCertRemotePath": "",
            "ServerCertPath": "/home/woods/.docker/machine/machines/node3/server.pem",
            "ServerKeyPath": "/home/woods/.docker/machine/machines/node3/server-key.pem",
            "ClientKeyPath": "/home/woods/.docker/machine/certs/key.pem",
            "ServerCertRemotePath": "",
            "ServerKeyRemotePath": "",
            "ClientCertPath": "/home/woods/.docker/machine/certs/cert.pem",
            "ServerCertSANs": [],
            "StorePath": "/home/woods/.docker/machine/machines/node3"
        }
    },
    "Name": "node3"
}
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 202,802评论 5赞 476
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 85,109评论 2赞 379
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 149,683评论 0赞 335
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,458评论 1赞 273
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,452评论 5赞 364
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,505评论 1赞 281
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 37,901评论 3赞 395
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,550评论 0赞 256
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 40,763评论 1赞 296
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,556评论 2赞 319
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,629评论 1赞 329
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,330评论 4赞 318
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 38,898评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,897评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,140评论 1赞 259
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 42,807评论 2赞 349
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,339评论 2赞 342