Tag: PKS

Note: Most of this was written as vSphere Cloud Provider in K8s was transitioned to the Cloud Native Storage CSI driver. Use CNS and CSI when you can, the more stable versions of PKS don’t support CSI by default as the K8s version is older. Use PSO for ReadWriteMany on Pure FlashBlade and CNS for block on FlashArray with VMFS Datastores (vVols coming).

The Pure Service Orchestrator Team is excited to announce that PSO is now validated with PKS Enterprise 1.4. PSO is PKS Partner Ready.

You can find more information on the VMware Marketplace https://marketplace.vmware.com/vsx/solutions/pure-service-orchestrator-2-5-2?ref=search

Learn more about PKS and Pure Storage with these posts:

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS

Why PSO and PKS?

I think it is crucial to understand the options for Storage in Kubernetes first. If you have seen me present in the last 12 months you may have already seen this graphic. When I talk about Hypervisor options I am referring to the vSphere Cloud Provider (and now Cloud Native Storage). At the time you deploy the cluster you provide credentials to contact vCenter and create/manage/destroy persistent volumes on a vSphere Datastore. This option is built into PKS Enterprise. This allows you to create a custom Storage Class per datastore or even use SPBM for provisioning. Yes, that means VMFS and vVols are supported today. There is no validation or certification needed to use this plugin (VCP or CNS) with Pure Storage. Customers of PKS are already doing this today. VCP works very well with vVols. I have advocated at VMworld during my session that this unlocks data mobility options between DIY K8s clusters and PKS (or even between PKS clusters). 

How does PSO fit in with PKS?
If you require “ReadWriteMany” aka RWX. This means I want many scalable containers to all attach to and use a single Persistent Volume Claim. The Pure Storage FlashBlade handles this use case. If you need simplicity in storage management across multiple devices both file and block PSO can consolidate this into a single orchestration layer deployed to any cluster with single command. PSO will also scale to new devices with a single command. Simplifying what was traditionally a very complex portion of a K8s environment.

Use vSphere Cloud Provider and Pure Service Orchestrator Side-by-Side

Due to the nature of the Storage Classes made by PSO and ones you manually create for VCP/CNS you can provide choice to the end users of PKS with only an initial install effort for the Ops team and nearly zero effort Day 2 onward. For more information please take a look at my post on “Getting Started with PKS” and the “How to install PSO in a PKS Cluster” posts.

The thing that happened was…

So if you follow k8s development at all, you know that CSI became GA at the beginning of 2019. The vSphere Cloud Provider “driver” that is in-tree is now deprecated. VMware is has released Cloud Native Storage, the CSI driver for Kubernetes clusters running on vSphere. This does not change the need for Pure Service Orchestrator for RWX volumes or even possible with in guest iSCSI too. Officially for RWO (read write once) you should be using CNS.

Learn more about PKS and Pure Storage with these posts:
Getting started with Persistent Storage and PKS

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS

Learn more about PKS and Pure Storage with these posts:
Getting started with Persistent Storage and PKS

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS

Remember to have the K8s cluster created within PKS and remember to think about how those PKS vm’s can communicate with the FlashArray and FlashBlade.

More information and detail:
https://github.com/purestorage/helm-charts/tree/master/operator-k8s-plugin
First we must download the git repo with the installer for the Operator.

$ git clone --branch <version> https://github.com/purestorage/helm-charts.git
$ cd helm-charts/operator-k8s-plugin
$./install.sh --namespace=pso --orchestrator=k8s -f values.yaml
$ kubectl get all -n pso
NAME                                    READY   STATUS    RESTARTS   AGE
pod/pso-operator-b96cfcfbb-zbwwd        1/1     Running   0          27s
pod/pure-flex-dzpwm                     1/1     Running   0          17s
pod/pure-flex-ln6fh                     1/1     Running   0          17s
pod/pure-flex-qgb46                     1/1     Running   0          17s
pod/pure-flex-s947c                     1/1     Running   0          17s
pod/pure-flex-tzfn7                     1/1     Running   0          17s
pod/pure-provisioner-6c9f69dcdc-829zq   1/1     Running   0          17s
NAME                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/pure-flex   5         5         5       5            5           <none>          17s
NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/pso-operator       1/1     1            1           27s
deployment.apps/pure-provisioner   1/1     1            1           17s
NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/pso-operator-b96cfcfbb        1         1         1       27s
replicaset.apps/pure-provisioner-6c9f69dcdc   1         1         1       17s
 

Sample deployment you can copy this all to a file called deployment.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-pv-claim-rwx
  labels:
    app: minio
spec:
  storageClassName: pure-file
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 101Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  # This name uniquely identifies the Deployment
  name: minio-deployment
spec:
  selector:
    matchLabels:
      app: minio
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        # Label is used as selector in the service.
        app: minio
    spec:
      # Refer to the PVC created earlier
      volumes:
      - name: storage
        persistentVolumeClaim:
          # Name of the PVC created earlier
          claimName: minio-pv-claim-rwx
      containers:
      - name: minio
        # Pulls the default Minio image from Docker Hub
        image: minio/minio:latest
        args:
        - server
        - /storage
        env:
        # Minio access key and secret key
        - name: MINIO_ACCESS_KEY
          value: "minio"
        - name: MINIO_SECRET_KEY
          value: "minio123"
        ports:
        - containerPort: 9000
          hostPort: 9000
        # Mount the volume into the pod
        volumeMounts:
        - name: storage
          mountPath: "/storage"
---
apiVersion: v1
kind: Service
metadata:
  name: minio-service
spec:
  type: LoadBalancer
  ports:
    - port: 9000
      targetPort: 9000
      protocol: TCP
  selector:
    app: minio

Now apply the file to the cluster

# kubectl apply -f deployment.yaml

Check the pod status

$ kubectl get pod

NAME                               READY   STATUS    RESTARTS   AGE
minio-deployment-95b9d8474-xmtk2   1/1     Running   0          4h19m
pure-flex-9hbfj                    1/1     Running   2          3d4h
pure-flex-w4fvq                    1/1     Running   1          3d23hpure-flex-zbqvz                    1/1    Running   1          3d23h 
pure-provisioner-dd4c4ccb7-dp76c   1/1     Running   7          3d23h 

Check the PVC status

$ kubectl get pvc
NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
minio-pv-claim-rwx   Bound    pvc-04817b75-f98b-11e9-8402-005056a975c2   101Gi      RWX            pure-file      4h19m

Learn more about PKS and Pure Storage with these posts:
Getting started with Persistent Storage and PKS

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS

Learn more about PKS and Pure Storage with these posts:
Getting started with Persistent Storage and PKS

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS

To get started installing PSO with your PKS cluster using helm follow these instructions.
Before installing PSO the Plan in Enterprise PKS must have the “allow privileged” box checked. This setting allows the access to mount storage.

Scroll way down…

Apply the settings in the Installation Dashboard and wait for them to finish applying.

Create a cluster. Go get a Chick-fil-a Biscuit. 

# pks create-cluster testcluster -e test.domain.local -p small

Quick install for FlashBlade and NFS

Install Helm more info here and https://helm.sh/docs/using_helm/#role-based-access-control

1. Setup the rbac role for tiller.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tiller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: tiller
    namespace: kube-system

1. # helm init  –service-account tiller

Install PSO

1. # helm install -n pso pure/pure-k8s-plugin -f values.yaml

This is the quickest method to getting PSO up and running. We are not adding any packages to the PKS Stem. NFS is built in therefore supported out of the box by PKS.

Installing PSO for FlashArray

Before deploying the PKS Cluster you must tell Bosh director to install a few things at runtime.

Details and the packages are on my github page:

https://github.com/2vcps/pso_prereqs

This is the same method used by other vendors to add agents and drivers to PKS or CloudFoundry. 

Once you finish with the intructions you will have PSO able to mount both FlashArray and FlashBlade using their respective StorageClass, pure-block or pure-file.

Please pay attention to networking

PKS does not allow for the deployment to add another NIC to the vm’s that are deployed. With PKS and NSX-T this is also all kept behind logical routers. Please be sure that VM’s have access. I would prefer no firewall and no routing from a VM to the storage, this may not be possible. You may be able to use VLANS to reduce this routing to a minimum. Just be sure to document your full network path from VM to Storage for future reference.

Using PSO

Sample deployment you can copy this all to a file called deployment.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-pv-claim-rwx
  labels:
    app: minio
spec:
  storageClassName: pure-file
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 101Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  # This name uniquely identifies the Deployment
  name: minio-deployment
spec:
  selector:
    matchLabels:
      app: minio
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        # Label is used as selector in the service.
        app: minio
    spec:
      # Refer to the PVC created earlier
      volumes:
      - name: storage
        persistentVolumeClaim:
          # Name of the PVC created earlier
          claimName: minio-pv-claim-rwx
      containers:
      - name: minio
        # Pulls the default Minio image from Docker Hub
        image: minio/minio:latest
        args:
        - server
        - /storage
        env:
        # Minio access key and secret key
        - name: MINIO_ACCESS_KEY
          value: "minio"
        - name: MINIO_SECRET_KEY
          value: "minio123"
        ports:
        - containerPort: 9000
          hostPort: 9000
        # Mount the volume into the pod
        volumeMounts:
        - name: storage
          mountPath: "/storage"
---
apiVersion: v1
kind: Service
metadata:
  name: minio-service
spec:
  type: LoadBalancer
  ports:
    - port: 9000
      targetPort: 9000
      protocol: TCP
  selector:
    app: minio

Now apply the file to the cluster

# kubectl apply -f deployment.yaml

Check the pod status

$ kubectl get pod

NAME                               READY   STATUS    RESTARTS   AGE
minio-deployment-95b9d8474-xmtk2   1/1     Running   0          4h19m
pure-flex-9hbfj                    1/1     Running   2          3d4h
pure-flex-w4fvq                    1/1     Running   1          3d23h
pure-flex-zbqvz                    1/1     Running   1          3d23h
pure-provisioner-dd4c4ccb7-dp76c   1/1     Running   7          3d23h

Check the PVC status

$ kubectl get pvc

NAME                 STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
minio-pv-claim-rwx   Bound    pvc-04817b75-f98b-11e9-8402-005056a975c2   101Gi      RWX            pure-file      4h19m

Learn more about PKS and Pure Storage with these posts:
Getting started with Persistent Storage and PKS

Installing PSO in PKS with Helm
Installing PSO in PKS with the Operator
Use PKS + VMware SDDC + Pure Storage
Migrating PSO Volumes into vVols and PKS