In the previous post you created a new user – employee. Kubernetes was able to authenticate the user (as the user’s name appeared in the error logs). Unfortunately, this employee doesn’t have any permissions yet.
Kubernetes uses Roles, ClusterRoles and RoleBindings, ClusterRoleBindings to configure Authorization for users.
Roles vs ClusterRoles
When you want access to a specific namespace you use Roles. If you wish to provide cluster wide access which spans across namespaces, you use ClusterRoles.
To assign a Role to a user you use a kubernetes object called a RoleBinding. Similarly to assign a ClusterRole to a user you use a ClusterRoleBinding. Simple.
A RoleBinding may reference a ClusterRole for permissions but as a RoleBinding is only for a namespace, the permissions granted to the user/group/service account are limited to the namespace mentioned in the RoleBinding
Alice recruits Ann
Alice being the Tech Lead should have ClusterRole access to all pods. Ann the new recruit should have access to all pods in the namespace alices-team.
Create the contexts
Follow the instructions in the previous post to create the contexts alice and ann.
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
alice minikube alice
ann minikube ann
* minikube minikube minikube Create the role
Here is an example role:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: alices-team
name: alices-engg
rules:
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]Lets look at each of the toplevel names
apiVersion : unlike the usual “v1” we use rbac.authorization.k8s.io/v1this has everything to do with rbac.
kind: we are creating a Role
Metadata: specifies the namespace this Role is applicable to and gives the Role a name.
rules: specify the rules that a user with this role is allowed to have. We will discuss more about this in the next section.
Role rules
The rule specification consists of a list of dictionaries. Each dictionary can be one of two types
- resources: defines rules for access rules for resources such as pods, roles etc.
- nonResourceURLs: defines access to non-resource urls. An example would be an api for authorization.
Resource Rule
apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch", "create", "update", "patch" ]
resourceNames: []apiGroups: which apiGroups you want to provide access to. It is a list of strings, each string corresponds to a group of APIs. The “v1” api is part of core which is represented by the empty string.
resources: the kind of resource you want to allow access to.
resourceNames: instead of specifying resources you can specify their names (optional)
verbs: What can the role do with these objects ?
NonResourceRules
nonResourceUrls: ["/healthz", "/healthz/*"]
verbs: ["get", "post"] As you can see a nonResourceUrl is a URL or partial URL and allows the role to access the said URL and perform said verbs on it.
Equality or Hierarchy?
Alice looks at Ann and sees a young go-getter, naive in her knowledge and lacking the experience to dot the necessary I’s and cross the necessary T’s. Nope, she would not give the keys to her castle to this young’un. Visitor rights maybe.
This, obviously, cannot be forever. Ann will grow and learn and slowly become more circumspect. At that point of time Alice doesn’t want to be the bottleneck.
Finally, once Ann has dropped enough production DBs by accident and pushed enough painful bugs to production, she would be ready to take over much of what Alice is doing. i.e thinking this way about new hires. Then, hopefully, she would be ready to have carte-blanche over her cluster while Alice is, hopefully, sitting on a beach, counting her millions.
So when does a person progress up these levels? If a person regularly needs the permission of a particular level to perform her work, it maybe time for her to get the next level of access. Subject to Alice’s subjective approval, of course.
The keys to wonderland
Alice decides that she needs 4 roles
Reader: One who can read, list etc anything about the cluster within a namespace, but cannot make any changes or create new resources
Writer: One who can do all that the Reader can do but in addition, can update/modify existing resources and create new ones with the namespace.
Admin: One who can do all that the Writer can do and also delete resources.
Superuser: Can do anything on any namespacv we12e1e. period.
Implementation
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: alices-team
name: reader
rules:
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch"]apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: alices-team
name: writer
rules:
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["get", "list", "watch", "create", "update", "patch"]apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: alices-team
name: admin
rules:
- apiGroups: [""]
resources: ["pods", "pods/log"]
verbs: ["*"]Look at the * in the admin role. This means the role can do all verbs. The * can also be used in resources to include all resources. In a Role it cannot be used in the metadata block. As a result a Role is always attached to a particular namespace.
One RoleBinding to bind them all…
Alice now wishes to assign the role to Ann. To do this she needs a RoleBinding. A RoleBinding, as the name suggests, binds the role to a user.
The user can be an individual or a group.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: anns-role
namespace: alices-team
subjects:
# You can specify more than one "subject"
- kind: User
name: ann # "name" is case sensitive
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role #this must be Role or ClusterRole
name: reader # this must match the name of the Role
apiGroup: rbac.authorization.k8s.io
This would give Ann reader rights for the namespace alices-team. Lets check those permissions.
$ kubectl config use-context ann
Switched to context "ann".
$ kubectl get pods -n alices-team
NAME READY STATUS RESTARTS AGE
alice 1/1 Running 6 9d
job1-847956d49f-9gfwq 1/1 Running 0 47s
superduperinfra 1/1 Running 6 9d
$ kubectl get pods -n bobs-team
Error from server (Forbidden): pods is forbidden: User "ann" cannot list resource "pods" in API group "" in the namespace "bobs-team"
$ kubectl get pods --all-namespaces
Error from server (Forbidden): pods is forbidden: User "ann" cannot list resource "pods" in API group "" at the cluster scopeCool, so Ann has only reader access to the namespace alices-team. Lets see if she can create pods in alices-team.
$ kubectl run job1 --image=nginx -n alices-team
Error from server (Forbidden): deployments.apps is forbidden: User "ann" cannot create resource "deployments" in API group "apps" in the namespace "alices-team"Groups
The problem with this approach is that every user needs a RoleBinding. A slightly better approach uses groups.
You may remember this from the previous post on RBAC
sudo openssl req -new -key employee.key -out employee.csr -subj "/CN=employee/O=alice"That /O is the group name. By decoding the CSR of ann we see the group alice
$ openssl req -text -noout -verify -in ../creds/ann.csr
verify OK
Certificate Request:
Data:
Version: 1 (0x0)
Subject: CN = ann, O = aliceEvery user has a group.
By creating a RoleBinding for a group, then as long as we create the certificates with the appropriate group, they automatically have required access. Kubernetes does not really understand Groups, so it relies on the certificate. This sucks a bit as we need to provide a new certificate to a user to give him/her more privileges. Secure but inconvenient.
RoleBinding for a group
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: anns-role
namespace: alices-team
subjects:
- kind: Group
name: alice # Name is case sensitive
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role #this must be Role or ClusterRole
name: reader # this must match the name of the Role
apiGroup: rbac.authorization.k8s.ioLets test this stuff
# switch back to minikube for the wider priveleges
$ kubectl config use-context minikube
Switched to context "minikube".
# delete the RoleBinding
$ kubectl delete RoleBinding anns-role -n alices-team
rolebinding.rbac.authorization.k8s.io "anns-role" deleted
#apply the Group RoleBinding
$ kubectl apply -f GroupRoleBinding.yml
rolebinding.rbac.authorization.k8s.io/anns-role created
# switch back to ann's context
$ kubectl config use-context ann
Switched to context "ann".
# finally test if I can still see pods in alices-team
$ kubectl get pods -n alices-team
NAME READY STATUS RESTARTS AGE
alice 1/1 Running 6 9d
job1-847956d49f-9gfwq 1/1 Running 0 61m
superduperinfra 1/1 Running 6 9d
Scale
This is good for Alice, but Bob has to create the same Roles and name them slightly differently. Is there a way to Share Roles across teams ? It turns out you can. We can use ClusterRoles with RoleBindings!
This post is already too long, we will see that in the next post.
In real life
In real life much of this is handled by your service provider. Google Kubernetes Engine (a part of Google Cloud) attaches access to emails and email-groups. A person can be given access simply by adding them to an email-group. You can then attach Roles to email-groups and give members fine-grained accesses.
Learnings
A little upfront thought can avoid maintaining a lot of Resources!
Try not to repeat yourself if you can. 100 is a LOT less than 300 when each is chore!
Access is a gift, give it to people you trust.
Access is power and hence is also a prize. What contest are you running ?
Conclusion
Kubernetes provides you very fine-grained control over accesses. Every API call to the master node can be restricted and policed. The design of your security system is going to help you scale and avoid a lot of chaos.
