Industrial Use Case of KUBERNETES

WHAT IS KUBERNETES ?


Kubernetes is a cluster management system for managing containerized applications across multiple hosts, providing mechanisms for deployment, maintenance, and scaling of applications. It groups containers that make up an application into logical units for easy management and discovery.

Kubernetes builds upon years of experience at Google running production workloads at scale using a system called Borg, combined with best-of-breed ideas and practices from the community. Google released Kubernetes in 2014, donated it as a seed technology for the founding of the Cloud Native Computing Foundation in 2015, and continues actively using and developing the project.


Kubernetes Master

The Kubernetes master is the primary control unit for the cluster. The master is responsible for managing and scheduling the workloads in addition to the networking and communications across the entire cluster.

These are the components that run on the master:

  • Etcd Storage – Is an open-source key-value data store that can be accessed by all nodes in the cluster. It stores configuration data of the cluster’s state.
  • Kube-API-Server – The API server manages requests from the worker nodes, and it receives REST requests for modifications, and serves as a front-end to control cluster.
  • Kube-scheduler – Schedules the pods on nodes based on resource utilization and also decides where services are deployed.
  • Kube-controller-manager – It runs a number of distinct controller processes in the background to regulate the shared state of the cluster and perform routine tasks. When there is a change to a service, the controller recognizes the change and initiates an update to bring the cluster up to the desired state.

Worker Nodes

These nodes run the workloads according the schedule provided by the master. The interaction between the master and worker nodes are what’s known as the control plane.

  • Kubelet – Kubelet ensures that all containers in the node are running and are in a healthy state.  If a node fails, a replication controller observes this change and launches pods on another healthy pod. Integrated into the kubelet binary is ‘cAdvisor` that auto-discovers all containers and collects CPU, memory, file system, and network usage statistics and also provides machine usage stats by analyzing the ‘root’ container.
  • Kube Proxy – It acts as a network proxy and a load balancer. Additionally, it forwards the request to the correct pods across isolated networks in a cluster.
  • Pods - A pod is the basic building block on Kubernetes. It represents the workloads that get deployed. Pods are generally collections of related containers, but a pod may also only have one container. A pod shares network/storageand also a specification for how to run the containers.
  • Containers – Containers are the lowest level of microservice. These are placed inside of the pods and need external IP addresses to view any outside processes.

Benefits of Kubernetes

As the first Cloud Native Cloud Foundation (CNCF) project, Kubernetes is the fastest growing project in the history of Open Source software. It became popular for the following key reasons:

Portability

Kubernetes offers portability, and faster, simpler deployment times. This means that companies can take advantage of multiple cloud providers if needed and can grow rapidly without having to re-architect their infrastructure.

Scalability

With Kubernetes ability to run containers on one or more public cloud environments, in virtual machines, or on bare metal means that it can be deployed almost anywhere. And because Kubernetes has fundamentally changed the way development and deployments are made, teams can also scale much faster than they could in the past.

High Availability

Kubernetes addresses high availability at both the application and the infrastructure level. Adding a reliable storage layer to Kubernetes ensures that stateful workloads are highly available. In addition to this, the master components of a cluster can be configured for multi-node replication (multi-master) and this also ensures a higher availability.

Open Source

Since Kubernetes is open source, you can take advantage of the vast ecosystem of other open source tools designed specifically to work with Kubernetes without the lock-in of a closed/proprietary system.

Proven, and Battle Tested

A huge ecosystem of developers and tools with 5,608 GitHub repositories and counting means that you won’t be forging ahead into new territory without help.

Market Leader

It was developed by and used and maintained by Google which not only gives it instant credibility, but can be trusted to fix bugs and release new features on a regular basis.

Pinterest’s Kubernetes story



With over 250 million monthly active users and serving over 10 billion recommendations every single day, that is huge. As they knew these numbers are going to grow day by day, they began to realize the pain of scalability and performance issues.

Their initial strategy was to move their workload from EC2 instances to Docker containers; hence they first moved their services to Docker to free up engineering time spent on Puppet and to have an immutable infrastructure.

And then the next strategy was to move to Kubernetes:) Now they can take ideas from ideation to production in a matter of minutes whereas earlier they used to take hours or even days. They have cut down so much of overhead cost by utilizing Kubernetes and have removed a lot of manual work without making engineers worry about the underlying infrastructure.


Why Pinterest switched to kubernetes?


In 2016, Pinterest decided — after a fair amount of evaluation — that it needed to move its production infrastructure to containers, Benedict said. One of the biggest developments in enterprise computing this decade, containers allow software developers to package their apps in small lightweight packages that can be run across different environments and squeeze even more performance out of hardware than virtual machines.


It’s hard to migrate those workloads off these systems because we lack the context of how they work,” Benedict said. Containers give Pinterest a lot of the same benefits as those custom tools in a standard format, and while he said the company has no plans to move any workloads off AWS, containerizing one’s applications at least allows the company to have the discussion.


Then they shifted to container, and then Pinterest put its Kubernetes strategy into play. Pinterest is using Kubernetes to manage clusters of containers in production, starting with some non-critical workloads that tend to be predictable.

The project was simplify life for Pinterest’s software development teams and give the company better tools for controlling and managing its infrastructure, Benedict said.

Challenge

After eight years in existence, Pinterest had grown into 1,000 microservices and multiple layers of infrastructure and diverse set-up tools and platforms. In 2016 the company launched a roadmap towards a new compute platform, led by the vision of creating the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

Solution

The first phase involved moving services to Docker containers. Once these services went into production in early 2017, the team began looking at orchestration to help create efficiencies and manage them in a decentralized way. After an evaluation of various solutions, Pinterest went with Kubernetes.

Impact

"By moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins," says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest. "We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster."


Hope you liked my blog :)

Comments

Post a Comment

Popular posts from this blog

Automation Using Python

Image
  Welcome to my menu program This program contains fully automated technologies that can be run directly in any flavor of linux operating system This menu program contains some great technologies like : 1. Amazon Web Services (AWS) 2. Big Data Hadoop 3. LVM partition technique 4. Web Server 5. Yum  6. Docker This menu-driven program is built in python that can be useful in automation of above mentioned technologies. 1. Amazon Web Services (AWS) Using this menu program, you can do following things : - AWS key pairs      - Describe key pair     - create key pair     - Delete key pair - EC2 instance     - Create Instance     - Describe Instance     - Start Instance     - Stop Instance     - Reboot Instance     - Terminate Instance - Create AWS security group - EBS volume     - Create EBS volume     - Attach EBS volume     - Detach EBS volume     ...
Read more

NETWORK TOPOLOGY

Image
In this blog I'm going to create a  network Topology Setup in such a way that System A can  ping to two Systems System B and System C but both these systems should not be pinging each other without using any security rule e.g firewall etc . Every body thinks that in a network topology, if System A can ping to B and System B can ping to C, then it's obvious that system C can ping system A. But NO, Not Always... So in this blog, I'm going to show you how to create such topology. And believe me, doing this is so simple. It's just a game of netmask and routing table. Let's jump to the practical : System A : 1. Change the ip and netmask of system A to 192.168.1.20/16 2. Then add network range in the routing table .      Here I've selected the range such that it can ping to both of my other system. 3. By using route -n command, you can check the routing table  of System. 4. Using ifconfig enp0s3 command, you can check the ip of the system . System B : 1. C...
Read more

Chat Server using UDP

Image
  Create Chat Server  Specifications : Use Socket Programming Use UDP Data Transfer protocol Create server and receiver both Use multi-threading to get and receive data parallely Now creating Chat Server with above given specifications. This is the code for Chat Server using UDP protocol and using Socket programming. Here same program can send receive messege both at the same time. I've used multi-threading concept to send and receive data parallely from multiple Server sides. These are the two servers that I used for my chat server - chatApp : And using multi-threading , i sent data parallely from two servers. one from Linux and another one from windows : Hope you liked it :)
Read more