Kubernetes技术分析之安全


【编者的话】Docker的流行激活了一直不温不火的PaaS,随着而来的是各类Micro-PaaS的出现,Kubernetes是其中最具代表性的一员,它是Google多年大规模容器管理技术的开源版本。本系列文章将逐一分析Kubernetes,本文说明Kubernetes的安全方案和考虑。

Kubernetes安全

安全永远是一个重大的话题,特别是云计算平台,更需要设计出一套完善的安全方案,以应对复杂的场景。 Kubernetes主要使用Docker作为应用承载环境,Kubernetes首先设计出一套API和敏感信息处理方案,当然也基于Docker提供容器安全控制。以下是Kubernetes的安全设计原则:
1. 保证容器与其运行的宿主机之间有明确的隔离
2. 限制容器对基础设施或者其它容器造成不良影响的能力
3. 最小特权原则——限定每个组件只被赋予了执行操作所必需的最小特权,由此确保可能产生的损失达到最小
4. 允许系统用户明确区别于管理员
5. 允许赋予管理权限给用户
6. 允许应用能够从公开数据中提取敏感信息(keys, certs, passwords)

Authentication & Authorization

Kubernetes API Server是对外暴露的API访问地址,API server提供了认证 (Authentication)和授权 (Authorization)机制进行安全控制。

  • Authentication
    支持Client certificate authentication 、Token authentication 、Basic authentication集中方式。

  • Authorization
    在Authentication的基础上,Authorization可以对HTTP请求设置AlwaysDeny、AlwaysAllow、ABAC三种模式模式,其中ABAC可以设置不同用户的访问权限。


现在使用Basic authentication + ABAC model设置API server,
首先配置Basic authentication,设置用户密码,格式为每行password, user name, user id,
basic_auth.csv:
admin_passwd,admin,admin
test_passwd,test,test                       


然后配置ABAC访问策略, 设置admin具有任何权限,test用户只能访问pods,
policy_file.jsonl:
{"user":"admin"}
{"user":"test", "resource": "pods", "readonly": true}



访问策略配置详情参考:
https://github.com/kubernetes/ ... rithm
然后启动API Server:
$kube-apiserver
...
--basic-auth-file=basic_auth.csv \
--authorization-mode=ABAC --authorization-policy-file=policy_file.jsonl


访问API,可以看到test用户无法访问Pod之外的资源:
$ curl --basic -u admin:admin_passwd https://192.168.3.146:6443/api/v1/pods -k
[
...
]

$ curl --basic -u test:test_passwd https://192.168.3.146:6443/api/v1/pods -k
[
...
]

$ curl --basic -u test:test_passwd https://192.168.3.146:6443/api/v1/nodes -k
Forbidden: "/api/v1/nodes"


Admission Controllers

在认证和授权之外,Admission Controller也可以对Kubernetes API Server的访问控制,任何请求在访问API Server时需要经过一系列的验证,任何一环拒绝了请求,则会返回错误。
实际上Admission Controller是作为Kubernetes API Serve的一部分,并以插件代码的形式存在,在API Server启动的时候,可以配置需要哪些Admission Controller,以及它们的顺序,如:
--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota


Admission Controller支持的插件如下:
  • AlwaysAdmit

    Use this plugin by itself to pass-through all requests.
  • AlwaysDeny

    Rejects all requests. Used for testing.
  • DenyExecOnPrivileged
    This plug-in will intercept all requests to exec a command in a pod if that pod has a privileged container.If your cluster supports privileged containers, and you want to restrict the ability of end-users to exec commands in those containers, we strongly encourage enabling this plug-in.
  • ServiceAccount
    This plug-in implements automation for serviceAccounts. We strongly recommend using this plug-in if you intend to make use of Kubernetes ServiceAccount objects.
  • SecurityContextDeny
    This plug-in will deny any pod with a SecurityContext that defines options that were not available on the Container.
  • ResourceQuota
    This plug-in will observe the incoming request and ensure that it does not violate any of the constraints enumerated in theResourceQuota object in a Namespace. If you are using ResourceQuota objects in your Kubernetes deployment, you MUST use this plug-in to enforce quota constraints.
  • LimitRanger
    This plug-in will observe the incoming request and ensure that it does not violate any of the constraints enumerated in theLimitRange object in a Namespace. If you are using LimitRange objects in your Kubernetes deployment, you MUST use this plug-in to enforce those constraints.
  • NamespaceExists

    This plug-in will observe all incoming requests that attempt to create a resource in a Kubernetes Namespace and reject the request if the Namespace was not previously created. We strongly recommend running this plug-in to ensure integrity of your data.
    NamespaceAutoProvision (deprecated) This plug-in will observe all incoming requests that attempt to create a resource in a Kubernetes Namespace and create a new Namespace if one did not already exist previously.
  • NamespaceLifecycle
    This plug-in enforces that a Namespace that is undergoing termination cannot have new content created in it.A Namespace deletion kicks off a sequence of operations that remove all content (pods, services, etc.) in that namespace. In order to enforce integrity of that process, we strongly recommend running this plug-in.Once NamespaceAutoProvision is deprecated, we anticipate NamespaceLifecycle and NamespaceExists will be merged into a single plug-in that enforces the life-cycle of a Namespace in Kubernetes.


ServiceAccount

Service Account概念的引入是基于这样的使用场景:运行在pod里的进程需要调用Kubernetes API以及非Kubernetes API的其它服务(如image repository/被mount到pod上的NFS volumes中的file等)。我们使用Service Account来为pod提供id。
Service Account和User account可能会带来一定程度上的混淆,User account可以认为是与Kubernetes交互的个体,通常可以认为是human, 目前并不作为一个代码中的类型单独出现,比如第一节中配置的用户,它们的区别如下:
  • user account通常是为human设计的,而service account则是为跑在pod里的process。
  • user account是global的,即跨namespace使用;而service account是namespaced的,即仅在所属的namespace下使用。
  • 创建一个新的user account通常需要较高的特权并且需要经过比较复杂的business process(即对于集群的访问权限的创建),而service account则不然。


要使用ServiceAccount,在启动Kubernetes的时候,首先生成key:
$ openssl genrsa -out /tmp/kube-serviceaccount.key 2048


启动API Server增加service_account_key_file:
$ kube-apiserver ... --service_account_key_file=/tmp/kube-serviceaccount.key ...


并且API Server设置admission_control包含ServiceAccount:
--admission_control=...,ServiceAccount,...


启动Controller Manager增加service_account_private_key_file:
$ kube-controller-manager ... --service_account_private_key_file=/tmp/kube-serviceaccount.key...


每个namespace都会默认创建一个ServiceAccount:
$ kubectl get serviceaccount --all-namespaces
NAMESPACE       NAME      SECRETS
default         default   1
kube-system     default   3


可以创建ServiceAccount,
serviceaccount.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: build-robot


$ kubectl create -f serviceaccount.yaml
serviceaccounts/build-robot

$ kubectl get serviceaccounts/build-robot -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: 2015-08-14T09:59:39Z
name: build-robot
namespace: default
resourceVersion: "538168"
selfLink: /api/v1/namespaces/default/serviceaccounts/build-robot
uid: 2d55527f-426b-11e5-91cd-005056817c3e
secrets:
- name: build-robot-token-3uazg

可以看到ServiceAccount默认创建一个secret,也可以手动创建secret然后添加到ServiceAccount。

创建Pod使用ServiceAccount,
busybox-pod.yaml:
apiVersion: v1
kind: Pod
metadata:
name: busybox
spec:
containers:
- image: busybox
command:
  - sleep
  - "3600"
imagePullPolicy: IfNotPresent
name: busybox
serviceAccountName: build-robot


Pod创建成功后,可以查询Pod的容器挂载/var/run/secrets/kubernetes.io/serviceaccount:
"Volumes": {
"/var/run/secrets/kubernetes.io/serviceaccount": "/var/lib/kubelet/pods/05174b7d-426d-11e5-aacb-005056817c3e/volumes/kubernetes.io~secret/build-robot-token-3uazg"
},


实际上这个目录是ServiceAccount的Secret,里面包含了一个token,应用通过使用这个token便可以去访问Kubernetes API:
$ kubectl exec busybox ls /var/run/secrets/kubernetes.io/serviceaccount
token
$ kubectl exec busybox cat /var/run/secrets/kubernetes.io/serviceaccount/token
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImJ1aWxkLXJvYm90LXRva2VuLTN1YXpnIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImJ1aWxkLXJvYm90Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMmQ1NTUyN2YtNDI2Yi0xMWU1LTkxY2QtMDA1MDU2ODE3YzNlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YnVpbGQtcm9ib3QifQ.CjZwP83IqkMjQzGJfLFg4QxXVLmq_wLSIVXV3yrdrzSYT4GGVjgcaRz2HDdy3TFvX2jB0rn3f6X1W_qthyQUkEHoZQkhup05rZSjsvRBQaj2FdhDpZCfaiTm5JrLeQjPEnMWYiilPo9HNk8lTqrZ32fJMHoJzxwnJ7jJDpTme2i7UPix1fW0NhRxuQ2im9iiQuZk49drNFl1oT6CjYBfK0JpyOkeOVBLCX2hDp64Jpiwa3s9Zux0AgDgLkL_shttzreQmx4hzIC22pKlae_-jRR-2EH523ej6w_7YoLCxhmTMnIRkIn5vBRjZ3kFwelnmSiyi8RbfD_oBtgTTWkYsQ


Secrets

Kubernetes提供了Secret来处理敏感信息,目前Secret的类型有3种:
  • Opaque(default): 任意字符串
  • kubernetes.io/service-account-token: 作用于ServiceAccount
  • kubernetes.io/dockercfg: 作用于Docker registry


开发者可以任意定义Secret的格式和内容,现在创建一个假设Opaque Secret,比如应用要使用账号密码:
username: value-1
password: value-2


首先创建Secret,
secret.yaml:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
password: dmFsdWUtMg0K
username: dmFsdWUtMQ0K

注意:其中password和username的值是通过base64 加密。
$ kubectl create -f secret.yaml
$ kubectl describe secrets mysecret
Name:       mysecret
Namespace:  default
Labels:     <none>
Annotations:    <none>
Type:   Opaque

Data

password:   9 bytes
username:   9 bytes


现在创建一个Pod使用该Secret
red-pod.json:
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"name": "mypod"
},
"spec": {
"containers": [{
  "name": "mypod",
  "image": "redis",
  "volumeMounts": [{
    "name": "foo",
    "mountPath": "/etc/foo",
    "readOnly": true
  }]
}],
"volumes": [{
  "name": "foo",
  "secret": {
    "secretName": "mysecret"
  }
}]
}
}

这里将Secret作为一个Volume挂载到Po中容器的/etc/foo下,实际上Secret中的值都会以文件生成到/etc/foo下(文件名是key,文件内容是value),待Pod运行后查看:
$ kubectl exec mypod ls /etc/foo
password
username
$ kubectl exec mypod cat /etc/foo/password
value-2
$ kubectl exec mypod cat /etc/foo/username
value-1



Secret用于 Docker Registry的安全认证,参考:
https://github.com/GoogleCloud ... a-pod

Security context

Security context是用以对容器进行限制,使得不同的运行容器之前能够实现较为明晰的隔离,以及降低其影响宿主机和其它容器的可能性。通俗而言,容器中的security context用于表征在创建及运行容器时,它能够使用及访问的资源参数。
securityContext目前只实现了capabilities和privileged ,
etcd-discovery-controller.yaml:
kind: ReplicationController
apiVersion: v1
metadata:
name: etcd-discovery
creationTimestamp: 
spec:
strategy:
type: Recreate
resources: {}
triggers:
- type: ConfigChange
replicas: 1
selector:
name: etcd-discovery
template:
metadata:
  creationTimestamp: 
  labels:
    name: etcd-discovery
spec:
  containers:
  - name: discovery
    image: openshift/etcd-20-centos7
    args:
    - etcd-discovery.sh
    ports:
    - containerPort: 2379
      protocol: TCP
    resources: {}
    terminationMessagePath: "/dev/termination-log"
    imagePullPolicy: IfNotPresent
    capabilities: {}
    securityContext:
      capabilities: {}
      privileged: false
  restartPolicy: Always
  dnsPolicy: ClusterFirst
  serviceAccount: ''
status: {}


参考



==========================================================
作者简介
吴龙辉,现任网宿科技高级运营工程师,致力于云计算PaaS的研究和实践,活跃于CloudFoundry,Docker,Kubernetes等开源社区,贡献代码和撰写技术文档。
邮箱:wulh@chinanetcenter.com/wlh6666@qq.com

0 个评论

要回复文章请先登录注册