CI/CD example with Python, Django, Kubernetes and Okteto

calvarado04 CI/CD, DevOps, Django, Groovy, Kubernetes, Okteto, Python November 23, 2020

As a sysadmin with experience providing tech support for enterprise applications for around a decade, all this DevOps stuff happened suddenly and little bit silently to be honest, mostly because when your main concerns are to keep the daily operations working properly and the IT infrastructure doing well, there is no much time to look for new and amazing technologies. I arrived little bit late to this wave but in Mexico nowadays (late 2020), many companies are not even understanding what's going on with all this stuff.

But fortunately for me, in 2017, my colleague César Olea told me on a nice conversation that all this DevOps, CI/CD and remarkably, Containers and Kubernetes were being a great success on the IT Industry; so, after that conversation, I started to look into all this new world on my own, first with Docker and after that with Rancher 1.6 directly, which was simply great for me because that version of Rancher was a glorified Docker-Compose application, that I felt like having my own data center working inside my laptop, with each container performing pretty much like a server. And that was pretty important for me, to be able to understand and adopt all this new technology and this new approach.

Sooner than later, I realized that all this containers are not just another fancy way to deal with applications, nope, is much more than that, is a complete set of practices to enhance and improve the entire IT department, even sometimes called as Digital Transformation, term that if your company is really being involved into that, could be fairly appropriate to use.

Everything working together makes a ton of sense

Now in 2020, everything is still moving forward quite fast, even this basic example will become irrelevant in few months, but anyway, this blog is mine and is pretty much an attempt to demonstrate for myself and for others that actually I have all these skills.

The list of technologies that I'm using is just the basic for a minimal CI/CD architecture:

Programming IDE, Visual Studio Code with all the relevant plugins installed.
Python with Django as programming language, I'm learning right now Python and Django/Flask
Github account and a repository to push all my code.
A Kubernetes cluster to deploy all my stuff and deploy Jenkins, I choose Okteto because is cheap and is nice.
Okteto internal registry for the container images.
Okteto ingress controllers to publish the application.
Jenkins as Continuous Integration/Continuous Deployment engine, this approach actually could be different on many companies, because many organizations just set a trigger on every new image pushed to the container registry.

Backend

PostgreSQL database managed by Django using Django Models.
Python using Django.
I'm generating an API Rest to be consumed by the Frontend, however, there are some parts that actually comes directly from Django because this framework can act as frontend as well as backend.
Nginx Load Balancer, this would not be needed on this very basic application, but I decided to include it just for the challenge and because on more complex applications Nginx is widely used.
Repository URL: https://github.com/calvarado2004/django-api-rest
Backend URL: https://nginx-api-rest.calvarado04.com/api/element

Frontend

Javascript with Vue app, which consumes the API Rest served by Django but not directly, is consuming the API through Nginx. I'm pretty new with Vue and Javascript, by the way.
Repository URL: https://github.com/calvarado2004/vuedjango
Frontend URL: https://vue-api-rest.calvarado04.com

Infrastructure

Kubernetes is the professional way to deploy and use containers on enterprise graded environments, change my mind hahaha.

So, that is the natural step that you must take if you are being involved with containers.

My K8s namespace looks like:

CI/CD Pipelines!

As you should know, Jenkins works with Groovy to build its pipelines, Groovy it is not my favorite language but is still usable and Jenkins have some useful help on the application itself as well as on its documentation.

Database deployment:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.


node {
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    
    stage('Deploy the PostgreSQL Database'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/db-k8s
            ${OKTETO_DIR}/okteto namespace
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest-db-deployment
            '''
            println output
        }
    }
}

Backend deployment pipeline, Django:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/backend-django'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    stage ('Build and Push Image with Okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            ${OKTETO_DIR}/okteto build -t ${CONTAINER_IMAGE}:${BUILD_TAG} .
            '''
            println output
        }
    }
    
    stage('Deploy the new image to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/backend-k8s
            ${OKTETO_DIR}/okteto namespace
            cat kubernetes.j2 | sed "s#{{ CONTAINER_IMAGE }}:{{ TAG_USED }}#${CONTAINER_IMAGE}:${BUILD_TAG}#g" > kubernetes.yaml
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest
            '''
            println output
        }
    }
}

Nginx pipeline

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/backend-django'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    
    stage('Deploy Nginx to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/nginx-k8s
            ${OKTETO_DIR}/okteto namespace
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/nginx-api-rest
            '''
            println output
        }
    }
}

Vue pipeline:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/frontend-vue'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/vuedjango.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            def output = sh returnStdout: true, script: '''git clone "${GIT_PROJECT}"'''
            println output
    }
    
    stage ('Build and Push Image with Okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/vuedjango
            ${OKTETO_DIR}/okteto build -t ${CONTAINER_IMAGE}:${BUILD_TAG} .
            '''
            println output
        }
    }
    
    stage('Deploy the new image to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/vuedjango/frontend-k8s
            ${OKTETO_DIR}/okteto namespace
            cat kubernetes.j2 | sed "s#{{ CONTAINER_IMAGE }}:{{ TAG_USED }}#${CONTAINER_IMAGE}:${BUILD_TAG}#g" > kubernetes.yaml
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest
            '''
            println output
        }
    }
}

Dockerfiles

Docker is just a company that works with containers, but its Dockerfiles became the standard way to define almost all of them.

Here is the Dockerfile for the backend:

FROM python:3.8.5

COPY djangovue /djangovue
COPY requirements.txt /djangovue/requirements.txt
WORKDIR /djangovue
RUN pip install -r requirements.txt && chmod 755 /djangovue/manage.py
CMD python manage.py runserver 0.0.0.0:8000

And the frontend one:

# build environment
FROM node:12.2.0-alpine as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json /app/package.json
RUN npm install --silent
RUN npm install @vue/cli@3.7.0 -g
COPY . /app
RUN npm run build

# production environment
FROM nginx:1.16.0-alpine
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Working with more than one environment

If you check this frontend, you will see two environment files, concretely a file called .env.development with the following content:

VUE_APP_DJANGO_HOST=localhost
VUE_APP_DJANGO_PORT=8000
VUE_APP_DJANGO_PROTOCOL=http

and a file called .env.production with something more interesting:

VUE_APP_DJANGO_HOST=nginx-api-rest.calvarado04.com
VUE_APP_DJANGO_PORT=443
VUE_APP_DJANGO_PROTOCOL=https

So, yes, this is the proper way to deal with more than one environment, define inside your code variables that you can check and modify later. Nevermore the developers mantra: but it works on my machine!...

Kubernetes definitions

Database:

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: django-api-rest-pvc
  namespace: calvarado2004
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: django-api-rest-credentials
  namespace: calvarado2004
type: Opaque
data:
  user: UG9zdGdyZXM=
  password: UG9zdGdyZXNrOHMk 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-api-rest-db-deployment
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: django-api-rest-db-container
  template:
    metadata:
      labels:
        app: django-api-rest-db-container
        tier: backend
    spec:
      containers:
        - name: django-api-rest-db-container
          image: postgres:12.4
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user

            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password

            - name: POSTGRES_DB
              value: djangovuedb

            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata

          ports:
            - containerPort: 5432
          volumeMounts:
            - name: django-api-rest-volume-mount
              mountPath: "/var/lib/postgresql/data"

      volumes:
        - name: django-api-rest-volume-mount
          persistentVolumeClaim:
            claimName: django-api-rest-pvc
---
kind: Service
apiVersion: v1
metadata:
  name: django-api-rest-db-service
  namespace: calvarado2004
spec:
  selector:
    app: django-api-rest-db-container
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432

Backend:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: django-api-rest-container
  template:
    metadata:
      labels:
        app: django-api-rest-container
    spec:
      containers:
        - name: django-api-rest-container
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          ports:
            - containerPort: 8000
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password
            - name: POSTGRES_HOST
              value: django-api-rest-db-service
      initContainers:
        - name: django-api-rest-init
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          command: ['python', 'manage.py', 'migrate']
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password
            - name: POSTGRES_HOST
              value: django-api-rest-db-service
---
kind: Service
apiVersion: v1
metadata:
  name: django-api-rest
  namespace: calvarado2004
spec:
  selector:
    app: django-api-rest-container
  type: ClusterIP
  ports:
  - name: django-http
    protocol: TCP
    port: 8000
    targetPort: 8000

Nginx:

Note that I'm consuming the Django application making reference to the internal DNS that is the standard way to do it on Kubernetes {{application}}.{{namespace}}.svc.cluster.local when you want to consume a service with another service internally. This approach will not work for Vue because that application is effectively consuming the API on client side (literally is doing its duty on your browser) and because of that, it needs to be referenced to the API published to Internet (or Intranet if is an internal app).

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config-map
data:
  nginx.conf: |-
    events {
      
    }
    http {
      include /etc/nginx/conf.d/*.conf;
      upstream backend_server {
          server django-api-rest.calvarado2004.svc.cluster.local:8000;
      }
      server {
          listen 80 default_server;
          server_name nginx-api-rest.calvarado04.com;
          location / {
              proxy_pass http://backend_server;
              proxy_set_header Host $http_host;
              proxy_redirect off;
              proxy_set_header X-Real-IP $remote_addr;
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header X-Forwarded-Proto $https;
              proxy_connect_timeout 360s;
              proxy_read_timeout 360s;
              proxy_hide_header Access-Control-Allow-Origin;
              proxy_hide_header Access-Control-Allow-Credentials;
              set $CORS_CREDS true;
              set $CORS_ORIGIN $http_origin;
              set $CORS_METHODS 'GET, POST, PUT, DELETE, OPTIONS';
              set $CORS_HEADERS 'Authentication-Token, Cache-Control, Cookie, If-Modified-Since, Range, User-Agent, X-Requested-With';
              set $CORS_EXPOSE_HEADERS 'Content-Disposition, Content-Length, Content-Range, Set-Cookie';
              set $CORS_PREFLIGHT_CACHE_AGE 600;
              set $X_FRAME_OPTIONS '';
              if ($request_method = 'OPTIONS') {
                add_header Access-Control-Allow-Origin $CORS_ORIGIN;
                add_header Access-Control-Allow-Methods $CORS_METHODS;
                add_header Access-Control-Allow-Headers $CORS_HEADERS;
                add_header Access-Control-Allow-Credentials $CORS_CREDS;
                add_header Access-Control-Max-Age $CORS_PREFLIGHT_CACHE_AGE;
                add_header Content-Type 'text/plain; charset=utf-8';
                add_header Content-Length 0;
                return 204;
              }
              if ($request_method != 'OPTIONS') {
                add_header Access-Control-Allow-Origin $CORS_ORIGIN;
                add_header Access-Control-Allow-Methods $CORS_METHODS;
                add_header Access-Control-Allow-Headers $CORS_HEADERS;
                add_header Access-Control-Allow-Credentials $CORS_CREDS;
                add_header Access-Control-Expose-Headers $CORS_EXPOSE_HEADERS;
                add_header X-Frame-Options $X_FRAME_OPTIONS;
              }
          }
      }
 
    }  
---      
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-api-rest-container
  template:
    metadata:
      labels:
        app: nginx-api-rest-container
    spec:
      containers:
        - name: nginx-api-rest-container
          image: nginx:latest
          volumeMounts: 
            - name: nginx-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
          ports:
            - containerPort: 443
          command: ["/bin/sh"]
          args: ["-c", "while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\""]
      volumes:
        - name: nginx-config
          configMap:
            name: nginx-config-map
---
kind: Service
apiVersion: v1
metadata:
  name: nginx-api-rest
  namespace: calvarado2004
  annotations:
    dev.okteto.com/auto-ingress: "true"
spec:
  selector:
    app: nginx-api-rest-container
  type: ClusterIP
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

Frontend:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vue-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vue-api-rest-container
  template:
    metadata:
      labels:
        app: vue-api-rest-container
    spec:
      containers:
        - name: vue-api-rest-container
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          ports:
            - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  name: vue-api-rest
  namespace: calvarado2004
  annotations:
    dev.okteto.com/auto-ingress: "true"
spec:
  selector:
    app: vue-api-rest-container
  type: ClusterIP
  ports:
    - name: "vue-api-rest"
      protocol: TCP
      port: 80
      targetPort: 80

Last thoughts

As you can realize, DevOps adoption is not easy at all because implies to understand and know how to make it work together a huge range of technologies, that used to be very specialized and kind of isolated ones from each others. Developers needs to know more in deep about infrastructure, and Sysadmins, DBA's, Testers and Security teams needs to understand and make some effort to achieve a confortable way to deploy easily to production but warranting the best levels of quality at the same time.

This is the deal, but at the end of the day, it's not rocket science... 😉