[BUG] multiple k3s cluster creation using k3d is not working on mac/colima/docker
bug
## What did you do
- How was the cluster created?
- gcp-infra.yaml:
`apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
name: gcp-infra
servers: 3
agents: 2
image: rancher/k3s:v1.27.6-k3s1
network: gcp-infra-net
kubeAPI:
hostPort: "6443"
options:
k3d:
wait: true
timeout: "120s"
disableLoadbalancer: false
k3s:
extraArgs:
- arg: "--cluster-cidr=10.50.0.0/16"
nodeFilters:
- server:*
- arg: "--service-cidr=10.51.0.0/16"
nodeFilters:
- server:*
nodeLabels:
- label: cloud=gcp
nodeFilters:
- server:*
- agent:*
kubeconfig:
updateDefaultKubeconfig: true
switchCurrentContext: false
`
- azure-infra.yaml:
`apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
name: azure-infra
servers: 3
agents: 2
kubeAPI:
hostPort: "7443"
image: rancher/k3s:v1.27.6-k3s1
network: azure-infra-net
options:
k3d:
wait: true
timeout: "180s"
disableLoadbalancer: false
k3s:
extraArgs:
- arg: "--cluster-cidr=10.52.0.0/16"
nodeFilters:
- server:*
- arg: "--service-cidr=10.53.0.0/16"
nodeFilters:
- server:*
nodeLabels:
- label: "infra=azure"
nodeFilters:
- server:*
- agent:*
kubeconfig:
updateDefaultKubeconfig: true
switchCurrentContext: false`
- k3d cluster create --config k3d/gcp-infra.yaml --trace
- k3d cluster create --config k3d/azure-infra.yaml --trace
- What did you do afterwards?
- first cluster create command completes successfully, second fails (irrespective to the order, means yaml structure is right, but 2 cluster creation on same colima-vm/docker-instance combination is conflicting.
## What did you expect to happen
i am working on a poc, trying to create multiple k3s clusters (that mimic hybrid cloud on local laptop), later use cilium to wire them up. but i am stuck at the first step, looks like k3d is not allowing to create multiple k3s clusters on same colima-vm/docker-instance combination. Not sure if it is supported or not.
## Screenshots or terminal output
- error (i am not sure why second cluster is still pointing to 6443, even i specified 7443 in yaml. i already cleaned everything network, volume, cache etc. so leftover is not a problem):
TRAC[0024] Starting node 'k3d-azure-infra-server-2'
INFO[0024] Starting node 'k3d-azure-infra-server-2'
DEBU[0024] Truncated 2025-11-16 11:50:13.862551993 +0000 UTC to 2025-11-16 11:50:13 +0000 UTC
DEBU[0024] Waiting for node k3d-azure-infra-server-2 to get ready (Log: 'k3s is up and running')
TRAC[0024] NodeWaitForLogMessage: Node 'k3d-azure-infra-server-2' waiting for log message 'k3s is up and running' since '2025-11-16 11:50:13 +0000 UTC'
TRAC[0182] Non-fatal last log line in node k3d-azure-infra-server-2: bThe connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?
ERRO[0182] Failed Cluster Start: Failed to start server k3d-azure-infra-server-2: Node k3d-azure-infra-server-2 failed to get ready: error waiting for log line `k3s is up and running` from node 'k3d-azure-infra-server-2': stopped returning log lines: context deadline exceeded
ERRO[0182] Failed to create cluster >>> Rolling Back
INFO[0182] Deleting cluster 'azure-infra'
## Which OS & Architecture
- mac silicon os + colima linux vm + docker
## Which version of `k3d`
- k3d --version
k3d version v5.8.3
k3s version v1.31.5-k3s1 (default)
## Which version of docker
- docker --version
Docker version 28.4.0, build d8eb465
0 条评论