Controller Manager
作为集群的管理控制中心,维护集群中的所有控制器,对维持集群的稳定和自我修复,实现高可用,副本控制等起关键作用。
内部结构图
关键性调用链
源码分析过程
组件启动的入口
1 | func main() { |
读取配置文件,进行配置读取和初始化默认配置
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go ->NewControllerManagerCommand
- 初始化Controller-manager的配置选项结构:NewKubeControllerManagerOptions()
- 创建执行命令结构包括Use,Long,和Run:cmd := &cobra.Command{
- 解析配置文件: s.AddFlags
1.KnownControllers()获取所有controller
2.将配置文件中的配置选项注入到配置对象中
3.同时将controller需要的参数写入.
1 | // NewControllerManagerCommand creates a *cobra.Command object with default parameters |
组件启动执行
从main中的command.Execute()到4.2中构造的Run
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
//加载所有控制器,并将对应参数注入到控制器中
1 | c, err := s.Config(KnownControllers(), ControllersDisabledByDefault.List()) |
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
KnownControllers()中的NewControllerInitializers初始化所有的控制器
1 | // NewControllerInitializers is a public map of named controller groups (you can start more than one in an init func) |
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
真正进入执行
- 启动controller-manager的http服务和对应处理器,包括安全和非安全:BuildHandlerChain
- 构造run的执行体
- 需要选主的情况,选主完执行run;不需要选主的直接执行run,然后panic
1 | // Run runs the KubeControllerManagerOptions. This should never exit. |
转到run内部核心的三个动作 :CreateControllerContext 、 StartControllers和ctx.InformerFactory.Start
CreateControllerContext
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
- 拿到对kube-APIserver中资源的操作句柄
- 确认Kube-APIServer的健康(最多等待10s),然后拿获取连接
- 创建控制器上下文
1 | // CreateControllerContext creates a context struct containing references to resources needed by the |
StartControllers
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/controllermanager.go
启动初始化的所有控制器
1 | // StartControllers starts a set of controllers with a specified ControllerContext |
ctx.InformerFactory.Start
controller-manager中的informer开始启动监听资源的事件,将事件放到自己的队列中(具有限流特性)。处理进程从队列总获取事件开始进行任务处理。
将新建的ReplicaSet,放入队列
1 | // obj could be an *apps.ReplicaSet, or a DeletionFinalStateUnknown marker item. |
从队列中获取对象进行处理(具体过程见下方)
1 | func (rsc *ReplicaSetController) processNextWorkItem() bool { |
以startReplicaSetController为例
在StartControllers中initFn方法是NewControllerInitializers中初始化Controller是定义,以下主要看下startReplicaSetController。
位置: k8s.io/kubernetes/cmd/kube-controller-manager/app/apps.go
其中NewReplicaSetController主要是初始化ReplicaSetController的结构,包括apiserver的客户端,informer的回调函数等等。NewReplicaSetController->NewBaseController
1 | func startReplicaSetController(ctx ControllerContext) (http.Handler, bool, error) { |
关键函数run:k8s.io/kubernetes/pkg/controller/replicaset/replica_set.go
run中执行rsc.worker。
1 | // Run begins watching and syncing. |
rsc.worker即为rsc.syncHandler,而syncHandler在创建时来源于rsc.syncReplicaSet(见NewBaseController方法)
那么我们转到syncReplicaSet
位置:k8s.io/kubernetes/pkg/controller/replicaset/replica_set.go
updateReplicaSetStatus:在pod死亡或者新建时更新
1 | // syncReplicaSet will sync the ReplicaSet with the given key if it has had its expectations fulfilled, |
转到updateReplicaSetStatus:k8s.io/kubernetes/pkg/controller/replicaset/replica_set_utils.go
调用UpdateStatus,通过apiserver更新
1 | // updateReplicaSetStatus attempts to update the Status.Replicas of the given ReplicaSet, with a single GET/PUT retry. |
PodGCController
1.gc掉超过阈值限制的pod,按时间排序gc
1 | func (gcc *PodGCController) gcTerminated(pods []*v1.Pod) { |
2.gc掉孤儿pod:pod上的node信息不在当前可调度的节点上,即没有和有效node绑定
1 | // gcOrphaned deletes pods that are bound to nodes that don't exist. |
3.gc掉没有调度成功的pod:表现在pod的NodeName为空,主要由于资源等条件不满足
1 | // gcUnscheduledTerminating deletes pods that are terminating and haven't been scheduled to a particular node. |