Category: Kubernetes

Posted on

Image Policy Webhooks on Kubernetes (image scanner admission controller)

Adding Trivy Scanner as custom Admission Controller

We will include an Image Policy Webhook on our kubeadm Kubernetes cluster in order to enhance its security, not allowing containers with more than 3 CRITICAL vulnerabilities from getting scheduled on our cluster.

To accomplish this, the first step involves deploying a Scanner. In this instance, I have utilized a custom trivy scanner that I developed in Go, which utilizes the Trivy scanner in its operation. You can review the project here: go-trivy-scanner

Changes required to kube-api

Add the option --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml

Append the plugin ImagePolicyWebhook to the option --enable-admission-plugins

Add the volume

  volumes:
  - hostPath:
      path: /etc/kubernetes/admission-control
      type: DirectoryOrCreate
    name: etc-kubernetes-admission-control

And the volume-mounts to kube-api

    volumeMounts:
    - mountPath: /etc/kubernetes/admission-control
      name: etc-kubernetes-admission-control
      readOnly: true

Configuration files

Proceeding with our custom Image Policy Webhook.

file /etc/kubernetes/admission-control/image-policy-webhook-conf.yaml

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: ImagePolicyWebhook
    path: /etc/kubernetes/admission-control/imagepolicyconfig.yaml

We define the Image Policy Config here:

file /etc/kubernetes/admission-control/imagepolicyconfig.yaml

imagePolicy:
  kubeConfigFile: /etc/kubernetes/admission-control/trivy-scanner.kubeconfig
  allowTTL: 50
  denyTTL: 50
  retryBackoff: 500
  defaultAllow: true

And we define the kubeconfig file, this is the minimal supported configuration to make it work:

file /etc/kubernetes/admission-control/trivy-scanner.kubeconfig

apiVersion: v1
kind: Config
clusters:
- cluster:
    server: https://trivy-scanner<my-domain>/scan
  name: okd
users:
- name: admin
  user: {}
preferences: {}
contexts:
- context:
    cluster: okd
    user: admin
  name: admin
current-context: admin

Reviewing our kube-api static pod

After that, on our master node, we will configure the static Pod kube-api, located on /etc/kubernetes/manifests/kube-apiserver.yaml mounting an admission-controller directory, where we will place our config files.

apiVersion: v1
kind: Pod
metadata:
  annotations:
    kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.124.20:6443
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --advertise-address=192.168.124.20
    - --allow-privileged=true
    - --authorization-mode=Node,RBAC
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook
    - --admission-control-config-file=/etc/kubernetes/admission-control/image-policy-webhook-conf.yaml
    [...]
    volumeMounts:
    [...]
    - mountPath: /etc/kubernetes/admission-control
      name: etc-kubernetes-admission-control
      readOnly: true
    [...]
  volumes:
  [...]
  - hostPath:
      path: /etc/kubernetes/admission-control
      type: DirectoryOrCreate
    name: etc-kubernetes-admission-control

This will restart our kube-api container

we can validate with

crictl ps -a
crictl logs <container>

Test the Image Policy Webhook

Once kube-api is back online, we can try to deploy a faulty pod with lot of vulnerabilities, this should fail:

file faulty-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: imagepolicy-nginx-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
kubectl create -f faulty-pod.yaml
Error from server (Forbidden): error when creating "faulty-pod.yaml": pods "imagepolicy-nginx-pod" is forbidden: image policy webhook backend denied one or more images: More than 3 CRITICAL vulnerabilities, rejected: [nginx:1.14.2]

Posted on

Deploy an Elasticsearch cluster for Kubernetes (ECK) on Google Compute Platform (GCP on GKE) with Terraform – Part I

This will be a very technical post but I think that is gonna be also quite interesting if you are working with cloud technologies.

Elasticsearch is a pretty nice technology widely used on big data stuff, analysis and so on. However, this tool is heavy and little bit difficult to deploy and maintain on healthy status.

I'm working a lot with Google Compute Platform (GCP) that's why I decided to include this part as well.

First things first

If you don't have a GCP account, is pretty straightforward to get one, even with some free usage, Google will give you 300 dollars to spend on it... by previous registration with your credit card 😉 go ahead and do it: https://console.cloud.google.com

Also download the gcloud CLI: https://cloud.google.com/sdk/docs/install

We will be using the project called GKE Terraform project as you can check below:

Get access to your gcloud project on the CLI and perform the browser steps needed to achieve it:

$ gcloud auth login

Get access to your project:

Let's create an empty VPC to simulate one environment with previous stuff deployed on it, like other instances and so on.

Well, at this point we have the very basic infrastructure to start using Terraform.

Infrastructure as Code, what does that mean?

Terraform is the leading tool to deploy infrastructure on this way, you can define a very complex set of infrastructure with code functions and treating them like objects and variables.

The GKE Terraform project is available here:

https://github.com/calvarado2004/terraform-gke

Please note that the size of the nodes is huge, you can go ahead and delete some of those pools of nodes and customize the CPU's and memory according to your needs and budget, I will do that, of course. You can check here another branch with smaller nodes: https://github.com/calvarado2004/terraform-gke/tree/resize-to-small

ECK can be deployed on a single node, but the minimal enterprise configuration should have:

  • One Kibana node
  • One Coordinator node
  • One Master node
  • Two Data nodes

This deployment is creating a pool of nodes for each type of node, in order to enable the autoresizing on further moments of the infrastructure lifecycle. That could give you an idea of the complexity that you can handle easily with Terraform.

Kubernetes have two internal layers of networking. We will be using the following three CIDRs:

  • 170.35.0.0/24 for our GCP VPC, the most external face.
  • 10.99.240.0/20 for our Kubernetes services.
  • 10.96.0.0/14 for our Kubernetes Pods.

You can install Terraform if you have Ubuntu using this way:

$ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
$ sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
$ sudo apt-get update && sudo apt-get install terraform

Otherwise, check how to install it on your machine:

https://www.terraform.io/downloads.html

This the content of the file gke.tf

variable "gke_username" {
  default     = ""
  description = "gke username"
}

variable "gke_password" {
  default     = ""
  description = "gke password"
}

variable "cluster_name" {
  default = "gke-cluster"
  description = "cluster name"
}

variable "zone" {
  default = "us-east1-b"
  description = "cluster zone"
}

#Your pods will have an IP address from this CIDR
variable "cluster_ipv4_cidr" {
  default = "10.96.0.0/14"
  description = "internal cidr for pods"
}

#Your Kubernetes services will have an IP from this range
variable "services_ipv4_cidr_block" {
  default = "10.99.240.0/20"
  description = "nternal range for the kubernetes services"
}

# GKE cluster
resource "google_container_cluster" "primary" {
  name     = var.cluster_name
  location = var.zone

  remove_default_node_pool = true
  initial_node_count       = 1

  network                  = google_compute_network.vpc-gke.name
  subnetwork               = google_compute_subnetwork.subnet.name
  cluster_ipv4_cidr        = var.cluster_ipv4_cidr
  services_ipv4_cidr_block = var.services_ipv4_cidr_block

  min_master_version = "1.17.13-gke.2001"	

  master_auth {
    username = var.gke_username
    password = var.gke_password

    client_certificate_config {
      issue_client_certificate = false
    }
  }

  cluster_autoscaling {
    enabled = false
  }

}

# Separately Managed Master Pool
resource "google_container_node_pool" "master-pool" {
  name       = "master-pool"
  location   = var.zone
  cluster    = google_container_cluster.primary.name
  node_count = 1

  autoscaling {
    min_node_count = 1
    max_node_count = 2
  }

  management {
    auto_repair  = true
    auto_upgrade = false
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
    ]

    labels = {
      es_type = "master_nodes"
    }
    # 6 CPUs, 12GB of RAM
    preemptible  = false
    image_type   = "ubuntu_containerd"
    machine_type = "custom-6-12288"
    local_ssd_count = 0
    disk_size_gb    = 50
    disk_type       = "pd-standard"
    tags         = ["gke-node", "${var.cluster_name}-master"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

# Separately Managed Data Pool
resource "google_container_node_pool" "data-pool" {
  name       = "data-pool"
  location   = var.zone
  cluster    = google_container_cluster.primary.name
  node_count = 2

  autoscaling {
    min_node_count = 2
    max_node_count = 4
  }

  management {
    auto_repair = true
    auto_upgrade = false
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
    ]

    labels = {
      es_type = "data_nodes"
    }

    # 14 CPUs, 41GB of RAM
    preemptible  = false
    image_type   = "ubuntu_containerd"
    machine_type = "custom-14-41984"
    local_ssd_count = 0
    disk_size_gb    = 50
    disk_type       = "pd-standard"

    tags         = ["gke-node", "${var.cluster_name}-data"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

# Separately Managed Coordinator Pool
resource "google_container_node_pool" "coord-pool" {
  name       = "coord-pool"
  location   = var.zone
  cluster    = google_container_cluster.primary.name
  node_count = 1

  autoscaling {
    min_node_count = 1
    max_node_count = 2
  }

  management {
    auto_repair  = true
    auto_upgrade = false
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
    ]

    labels = {
      es_type = "coordinator_nodes"
    }

    # 6 CPUs, 22GB of RAM
    preemptible  = false
    image_type   = "ubuntu_containerd"
    machine_type = "custom-6-22528"
    local_ssd_count = 0
    disk_size_gb    = 50
    disk_type       = "pd-standard"
    tags         = ["gke-node", "${var.cluster_name}-coord"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

# Separately Managed Kibana Pool
resource "google_container_node_pool" "kibana-pool" {
  name       = "kibana-pool"
  location   = var.zone
  cluster    = google_container_cluster.primary.name
  node_count = 1

  autoscaling {
    min_node_count = 1
    max_node_count = 2
  }

  management {
    auto_repair  = true
    auto_upgrade = false
  }

  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
    ]

    labels = {
      es_type = "kibana_nodes"
    }

    # 4 CPUs, 13GB of RAM
    preemptible  = false
    image_type   = "ubuntu_containerd"
    machine_type = "custom-4-13312"
    local_ssd_count = 0
    disk_size_gb    = 50
    disk_type       = "pd-standard"
    tags         = ["gke-node", "${var.cluster_name}-kibana"]
    metadata = {
      disable-legacy-endpoints = "true"
    }
  }
}

output "kubernetes_cluster_name" {
  value       = google_container_cluster.primary.name
  description = "GKE Cluster Name"
}

And the content of the file vpc.tf

variable "project_id" {
  description = "project id"
}

variable "region" {
  description = "region"
}

provider "google" {
  project = var.project_id
  region  = var.region
}

# VPC
resource "google_compute_network" "vpc-gke" {
  name                    = "${var.cluster_name}-vpc"
  auto_create_subnetworks = "false"
}

# Subnet
resource "google_compute_subnetwork" "subnet" {
  name          = "${var.cluster_name}-subnet"
  region        = var.region
  network       = google_compute_network.vpc-gke.name
  ip_cidr_range = "170.35.0.0/24"

}

#Peering between OLD VMs vpc and GKE K8s vpc
resource "google_compute_network_peering" "to-vms-vpc" {
  name         = "to-vms-vpc-vpc-network"
  network      = google_compute_network.vpc-gke.id
  peer_network = "projects/sigma-scheduler-297405/global/networks/vms-vpc-network"
}

resource "google_compute_network_peering" "to-gke-cluster" {
  name         = "to-gke-cluster-vpc-network"
  network      = "projects/sigma-scheduler-297405/global/networks/vms-vpc-network"
  peer_network = google_compute_network.vpc-gke.id
}

output "region" {
  value       = var.region
  description = "region"
}

#Enable communication from GKE pods to external instances, networks and services outside the Cluster.
resource "google_compute_firewall" "gke-cluster-to-all-vms-on-network" {
  name    = "gke-cluster-k8s-to-all-vms-on-network"
  network = google_compute_network.vpc-portal.id

  allow {
    protocol = "tcp"
  }

  allow {
    protocol = "udp"
  }

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "esp"
  }

  allow {
    protocol = "ah"
  }

  allow {
    protocol = "sctp"
  }

  source_ranges = ["10.96.0.0/14"]
}

Let's deploy this GKE Cluster with Terraform!

Deploy a whole cluster is quite easy:

$ git clone https://github.com/calvarado2004/terraform-gke.git

$ git checkout resize-to-small
Switched to branch 'resize-to-small'
Your branch is up to date with 'origin/resize-to-small'.

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google...
- Installing hashicorp/google v3.49.0...
- Installed hashicorp/google v3.49.0 (signed by HashiCorp)

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.

* hashicorp/google: version = "~> 3.49.0"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform plan -out=gke-cluster.plan

$ terraform apply "gke-cluster.plan"

Posted on

CI/CD example with Python, Django, Kubernetes and Okteto

As a sysadmin with experience providing tech support for enterprise applications for around a decade, all this DevOps stuff happened suddenly and little bit silently to be honest, mostly because when your main concerns are to keep the daily operations working properly and the IT infrastructure doing well, there is no much time to look for new and amazing technologies. I arrived little bit late to this wave but in Mexico nowadays (late 2020), many companies are not even understanding what's going on with all this stuff.

But fortunately for me, in 2017, my colleague César Olea told me on a nice conversation that all this DevOps, CI/CD and remarkably, Containers and Kubernetes were being a great success on the IT Industry; so, after that conversation, I started to look into all this new world on my own, first with Docker and after that with Rancher 1.6 directly, which was simply great for me because that version of Rancher was a glorified Docker-Compose application, that I felt like having my own data center working inside my laptop, with each container performing pretty much like a server. And that was pretty important for me, to be able to understand and adopt all this new technology and this new approach.

Sooner than later, I realized that all this containers are not just another fancy way to deal with applications, nope, is much more than that, is a complete set of practices to enhance and improve the entire IT department, even sometimes called as Digital Transformation, term that if your company is really being involved into that, could be fairly appropriate to use.

Everything working together makes a ton of sense

Now in 2020, everything is still moving forward quite fast, even this basic example will become irrelevant in few months, but anyway, this blog is mine and is pretty much an attempt to demonstrate for myself and for others that actually I have all these skills.

The list of technologies that I'm using is just the basic for a minimal CI/CD architecture:

  • Programming IDE, Visual Studio Code with all the relevant plugins installed.
  • Python with Django as programming language, I'm learning right now Python and Django/Flask
  • Github account and a repository to push all my code.
  • A Kubernetes cluster to deploy all my stuff and deploy Jenkins, I choose Okteto because is cheap and is nice.
  • Okteto internal registry for the container images.
  • Okteto ingress controllers to publish the application.
  • Jenkins as Continuous Integration/Continuous Deployment engine, this approach actually could be different on many companies, because many organizations just set a trigger on every new image pushed to the container registry.

Backend

  • PostgreSQL database managed by Django using Django Models.
  • Python using Django.
  • I'm generating an API Rest to be consumed by the Frontend, however, there are some parts that actually comes directly from Django because this framework can act as frontend as well as backend.
  • Nginx Load Balancer, this would not be needed on this very basic application, but I decided to include it just for the challenge and because on more complex applications Nginx is widely used.
  • Repository URL: https://github.com/calvarado2004/django-api-rest
  • Backend URL: https://nginx-api-rest.calvarado04.com/api/element

Frontend

Infrastructure

Kubernetes is the professional way to deploy and use containers on enterprise graded environments, change my mind hahaha.

So, that is the natural step that you must take if you are being involved with containers.

My K8s namespace looks like:

CI/CD Pipelines!

As you should know, Jenkins works with Groovy to build its pipelines, Groovy it is not my favorite language but is still usable and Jenkins have some useful help on the application itself as well as on its documentation.

Database deployment:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.


node {
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    
    stage('Deploy the PostgreSQL Database'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/db-k8s
            ${OKTETO_DIR}/okteto namespace
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest-db-deployment
            '''
            println output
        }
    }
}

Backend deployment pipeline, Django:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/backend-django'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    stage ('Build and Push Image with Okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            ${OKTETO_DIR}/okteto build -t ${CONTAINER_IMAGE}:${BUILD_TAG} .
            '''
            println output
        }
    }
    
    stage('Deploy the new image to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/backend-k8s
            ${OKTETO_DIR}/okteto namespace
            cat kubernetes.j2 | sed "s#{{ CONTAINER_IMAGE }}:{{ TAG_USED }}#${CONTAINER_IMAGE}:${BUILD_TAG}#g" > kubernetes.yaml
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest
            '''
            println output
        }
    }
}

Nginx pipeline

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/backend-django'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/django-api-rest.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            git url: "${GIT_PROJECT}"
    }
    
    
    stage('Deploy Nginx to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/nginx-k8s
            ${OKTETO_DIR}/okteto namespace
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/nginx-api-rest
            '''
            println output
        }
    }
}

Vue pipeline:

#!/usr/bin/env groovy

//Author: Carlos Alvarado
//Jenkins Pipeline to handle the Continuous Integration and Continuous Deployment on Okteto.
//Prerequisites: you should install the Custom tools plugin on Jenkins, ... 
//...get the okteto CLI and Kubectl. You also need to get your Okteto Token and save it on a Jenkins Credential


node {
    
    env.OKTETO_DIR = tool name: 'okteto', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.HOME = "${WORKSPACE}"
    env.CONTAINER_IMAGE = 'registry.cloud.okteto.net/calvarado2004/frontend-vue'
    env.KUBECTL_DIR = tool name: 'kubectl', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
    env.GIT_PROJECT = 'https://github.com/calvarado2004/vuedjango.git'
    
    stage ('Prepare Environment with Okteto ') {
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            cleanWs deleteDirs: true
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            '''
            println output
        }
    }
    
    stage ('Download the source code from GitHub'){
            def output = sh returnStdout: true, script: '''git clone "${GIT_PROJECT}"'''
            println output
    }
    
    stage ('Build and Push Image with Okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/vuedjango
            ${OKTETO_DIR}/okteto build -t ${CONTAINER_IMAGE}:${BUILD_TAG} .
            '''
            println output
        }
    }
    
    stage('Deploy the new image to okteto'){
        withCredentials([string(credentialsId: 'okteto-token', variable: 'SECRET')]) {
            def output = sh returnStdout: true, script: '''
            ${OKTETO_DIR}/okteto login --token ${SECRET}
            cd ${HOME}/vuedjango/frontend-k8s
            ${OKTETO_DIR}/okteto namespace
            cat kubernetes.j2 | sed "s#{{ CONTAINER_IMAGE }}:{{ TAG_USED }}#${CONTAINER_IMAGE}:${BUILD_TAG}#g" > kubernetes.yaml
            ${KUBECTL_DIR}/kubectl apply -f kubernetes.yaml
            ${KUBECTL_DIR}/kubectl rollout status deployment.apps/django-api-rest
            '''
            println output
        }
    }
}

Dockerfiles

Docker is just a company that works with containers, but its Dockerfiles became the standard way to define almost all of them.

Here is the Dockerfile for the backend:

FROM python:3.8.5

COPY djangovue /djangovue
COPY requirements.txt /djangovue/requirements.txt
WORKDIR /djangovue
RUN pip install -r requirements.txt && chmod 755 /djangovue/manage.py
CMD python manage.py runserver 0.0.0.0:8000

And the frontend one:

# build environment
FROM node:12.2.0-alpine as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json /app/package.json
RUN npm install --silent
RUN npm install @vue/cli@3.7.0 -g
COPY . /app
RUN npm run build

# production environment
FROM nginx:1.16.0-alpine
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Working with more than one environment

If you check this frontend, you will see two environment files, concretely a file called .env.development with the following content:

VUE_APP_DJANGO_HOST=localhost
VUE_APP_DJANGO_PORT=8000
VUE_APP_DJANGO_PROTOCOL=http

and a file called .env.production with something more interesting:

VUE_APP_DJANGO_HOST=nginx-api-rest.calvarado04.com
VUE_APP_DJANGO_PORT=443
VUE_APP_DJANGO_PROTOCOL=https

So, yes, this is the proper way to deal with more than one environment, define inside your code variables that you can check and modify later. Nevermore the developers mantra: but it works on my machine!...

Kubernetes definitions

Database:

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: django-api-rest-pvc
  namespace: calvarado2004
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: django-api-rest-credentials
  namespace: calvarado2004
type: Opaque
data:
  user: UG9zdGdyZXM=
  password: UG9zdGdyZXNrOHMk 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-api-rest-db-deployment
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: django-api-rest-db-container
  template:
    metadata:
      labels:
        app: django-api-rest-db-container
        tier: backend
    spec:
      containers:
        - name: django-api-rest-db-container
          image: postgres:12.4
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user

            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password

            - name: POSTGRES_DB
              value: djangovuedb

            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata

          ports:
            - containerPort: 5432
          volumeMounts:
            - name: django-api-rest-volume-mount
              mountPath: "/var/lib/postgresql/data"

      volumes:
        - name: django-api-rest-volume-mount
          persistentVolumeClaim:
            claimName: django-api-rest-pvc
---
kind: Service
apiVersion: v1
metadata:
  name: django-api-rest-db-service
  namespace: calvarado2004
spec:
  selector:
    app: django-api-rest-db-container
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432

Backend:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: django-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: django-api-rest-container
  template:
    metadata:
      labels:
        app: django-api-rest-container
    spec:
      containers:
        - name: django-api-rest-container
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          ports:
            - containerPort: 8000
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password
            - name: POSTGRES_HOST
              value: django-api-rest-db-service
      initContainers:
        - name: django-api-rest-init
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          command: ['python', 'manage.py', 'migrate']
          env:
            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: user
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: django-api-rest-credentials
                  key: password
            - name: POSTGRES_HOST
              value: django-api-rest-db-service
---
kind: Service
apiVersion: v1
metadata:
  name: django-api-rest
  namespace: calvarado2004
spec:
  selector:
    app: django-api-rest-container
  type: ClusterIP
  ports:
  - name: django-http
    protocol: TCP
    port: 8000
    targetPort: 8000

Nginx:

Note that I'm consuming the Django application making reference to the internal DNS that is the standard way to do it on Kubernetes {{application}}.{{namespace}}.svc.cluster.local when you want to consume a service with another service internally. This approach will not work for Vue because that application is effectively consuming the API on client side (literally is doing its duty on your browser) and because of that, it needs to be referenced to the API published to Internet (or Intranet if is an internal app).

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config-map
data:
  nginx.conf: |-
    events {
      
    }
    http {
      include /etc/nginx/conf.d/*.conf;
      upstream backend_server {
          server django-api-rest.calvarado2004.svc.cluster.local:8000;
      }
      server {
          listen 80 default_server;
          server_name nginx-api-rest.calvarado04.com;
          location / {
              proxy_pass http://backend_server;
              proxy_set_header Host $http_host;
              proxy_redirect off;
              proxy_set_header X-Real-IP $remote_addr;
              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header X-Forwarded-Proto $https;
              proxy_connect_timeout 360s;
              proxy_read_timeout 360s;
              proxy_hide_header Access-Control-Allow-Origin;
              proxy_hide_header Access-Control-Allow-Credentials;
              set $CORS_CREDS true;
              set $CORS_ORIGIN $http_origin;
              set $CORS_METHODS 'GET, POST, PUT, DELETE, OPTIONS';
              set $CORS_HEADERS 'Authentication-Token, Cache-Control, Cookie, If-Modified-Since, Range, User-Agent, X-Requested-With';
              set $CORS_EXPOSE_HEADERS 'Content-Disposition, Content-Length, Content-Range, Set-Cookie';
              set $CORS_PREFLIGHT_CACHE_AGE 600;
              set $X_FRAME_OPTIONS '';
              if ($request_method = 'OPTIONS') {
                add_header Access-Control-Allow-Origin $CORS_ORIGIN;
                add_header Access-Control-Allow-Methods $CORS_METHODS;
                add_header Access-Control-Allow-Headers $CORS_HEADERS;
                add_header Access-Control-Allow-Credentials $CORS_CREDS;
                add_header Access-Control-Max-Age $CORS_PREFLIGHT_CACHE_AGE;
                add_header Content-Type 'text/plain; charset=utf-8';
                add_header Content-Length 0;
                return 204;
              }
              if ($request_method != 'OPTIONS') {
                add_header Access-Control-Allow-Origin $CORS_ORIGIN;
                add_header Access-Control-Allow-Methods $CORS_METHODS;
                add_header Access-Control-Allow-Headers $CORS_HEADERS;
                add_header Access-Control-Allow-Credentials $CORS_CREDS;
                add_header Access-Control-Expose-Headers $CORS_EXPOSE_HEADERS;
                add_header X-Frame-Options $X_FRAME_OPTIONS;
              }
          }
      }
 
    }  
---      
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-api-rest-container
  template:
    metadata:
      labels:
        app: nginx-api-rest-container
    spec:
      containers:
        - name: nginx-api-rest-container
          image: nginx:latest
          volumeMounts: 
            - name: nginx-config
              mountPath: /etc/nginx/nginx.conf
              subPath: nginx.conf
          ports:
            - containerPort: 443
          command: ["/bin/sh"]
          args: ["-c", "while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\""]
      volumes:
        - name: nginx-config
          configMap:
            name: nginx-config-map
---
kind: Service
apiVersion: v1
metadata:
  name: nginx-api-rest
  namespace: calvarado2004
  annotations:
    dev.okteto.com/auto-ingress: "true"
spec:
  selector:
    app: nginx-api-rest-container
  type: ClusterIP
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

Frontend:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vue-api-rest
  namespace: calvarado2004
spec:
  replicas: 1
  selector:
    matchLabels:
      app: vue-api-rest-container
  template:
    metadata:
      labels:
        app: vue-api-rest-container
    spec:
      containers:
        - name: vue-api-rest-container
          image: {{ CONTAINER_IMAGE }}:{{ TAG_USED }}
          ports:
            - containerPort: 80

---
kind: Service
apiVersion: v1
metadata:
  name: vue-api-rest
  namespace: calvarado2004
  annotations:
    dev.okteto.com/auto-ingress: "true"
spec:
  selector:
    app: vue-api-rest-container
  type: ClusterIP
  ports:
    - name: "vue-api-rest"
      protocol: TCP
      port: 80
      targetPort: 80

Last thoughts

As you can realize, DevOps adoption is not easy at all because implies to understand and know how to make it work together a huge range of technologies, that used to be very specialized and kind of isolated ones from each others. Developers needs to know more in deep about infrastructure, and Sysadmins, DBA's, Testers and Security teams needs to understand and make some effort to achieve a confortable way to deploy easily to production but warranting the best levels of quality at the same time.

This is the deal, but at the end of the day, it's not rocket science... 😉