Managing PSPs and RBAC can be complex.
If you don’t have everything set just correctly, you can prevent properly configured pods from deploying at all. A policy that is too restrictive may present challenges as some containers that are pulled from public registries are using the root user. In these cases, modification and upkeep will need to be maintained by you. If the upstream image changes, you will need to merge those changes into yours. The same challenges exist for Helm based deployments.
Another potential pitfall with PSPs is that testing your policies requires deploying them somewhere and validating them. To do this correctly, you would need to apply the same configurations from your live cluster to your test one. If you are managing different clusters for different teams, it might require additional cluster building automation and configuration management.
One last pitfall is managing multiple policies in one cluster. One obvious reason you might have multiple policies is that one type of pod needs more privileges than the others. A way to achieve mixing-and-matching is by crafting your RBAC policies, service accounts, and PSPs with specific workloads. Since PSPs are cluster wide, you’ll have to come up with your own strategy that best suits your requirements.
For the same reasons you don’t give out root access, you also need your containers to run without unnecessary privileges. Prevent your teams from unknowingly running containers with elevated permissions and configurations to help safeguard your Kubernetes cluster from within. The last thing you need right now is a rogue container deleting all the files on an NFS mount so enable PSPs today, so you won’t be restoring services tomorrow.