Kubernetes in ShareThis production environment in practice

ShareThis is a start-up enterprise that provides social sharing technology, founded in 2007. It has developed a number of social sharing of small parts and tools, publishers use these tools can be very social sharing. The company claims to have more than 2 million affiliate publishers, with 120 social channels and 95% of US Internet users. The goal is to allow readers to share content as easily as possible while allowing publishers to expand their reach.

In order to achieve the customer's business objectives, ShareThis share technology also with the analysis and advertising tools. ShareThis engine handles large amounts of data to help marketers reach the right audience and provide information on how to do marketing. Its social quality index (SQI) can be a comparative analysis of the effect of social activities, so that publishers to more in-depth understanding of their social sharing strategy implementation status and their own in the social impact of the location.

ShareThis customers include Delish, the Food Network, AllThingsD and Cosmopolitan.

Kubernetes has become an important tool and component of our barriers to infrastructure reduction based on infrastructure:

  • Push Docker implementation
  • Simplify container management
  • Guide developers to understand the infrastructure
  • Achieve continuous integration and delivery

In order to achieve these goals, we fundamentally adopt Kubernetes and transform our DevOps team into a cloud platform team, with containers and micro services as a technical form. Including the construction of a series of tools to solve their own long-term face of the original technical problems.

problem lies in

The biggest problem is that cloud computing is still a new thing, and we are still very young. We are still in the beginning with the use of traditional data center mentality. We manage all of our services: MySQL, Cassandra, Aerospike, Memcache and everything else. We set up virtual machines in much the same way that we used traditional servers, where we installed our applications and managed them with Nagios or Ganglia.

Unfortunately, this way of thinking and the cloud as the core concept can be completely opposed. Compared to the cloud program to serve as a form of thinking, we have been based on the server thinking. In addition, we use scripting settings, server deployments, and the avoidance of vendor lockdowns, as compared to modern cloud scenarios such as automatic scale scaling, micro services, or managed virtual machines.

These methods itself is not bad, but often less efficient. This approach does not give full play to the advantages of change, but the cloud is able to quickly change and put it into the business process. It also means that these changes can be described as natural enemies of the data center when changes have to be made, but in the cloud they are only in small-scale, quick-start adjustment initiatives.


Use Kubernetes as a tool to facilitate Docker implementation

As Docker became an important force in the technology industry, ShareThe company's engineers began to test and achieve good results. Facts quickly prove that we need to deploy a matching container for each application in the enterprise, thereby significantly simplifying testing in the development environment.

Some applications can be more quickly integrated into the Docker environment because they are relatively simple and less relevant to other system components. For those applications where these associations are not strong, we are able to manage it with Fig (Fig as the initial name of Docker Compose). Of course, most of our data channels or associative applications are still too rough, and can not directly implement Dockerization. In this case, the power of Docker alone can not solve the problem.

At the end of 2015, our dissatisfaction with the original infrastructure has finally accumulated to the critical point. In order to get the response, we began to evaluate the Docker tools, ECS, Kubernetes and Mesosphere. We quickly concluded that Kubernetes had a more stable performance and was able to provide our customers with a better level of user friendliness than their competitors. As a business, we were able to easily achieve the goal of migrating all our infrastructure to Kubernetes, and to further consolidate the infrastructure and Docker.

The engineers were initially skeptical. However, after witnessing the scale of the application to easily achieve the significant effect of each application of hundreds of examples, they began to cheer. Now, it is not only the inherent pain that we are moving forward to Docker, and the great potential of the technology is really motivated by our enthusiasm. This allows us to quickly achieve the original extremely difficult load migration. At present, we have to Kubernetes run in a number of onions 65 sets of large virtual machine system which, and plans in the next few months this figure will be further upgraded to 100 sets. Our Kubernetes cluster processes 80 million requests in parallel every day, and in the next few months we plan to raise this processing capacity directly to 2 billion requests per day.

Use Kubernetes as a tool to manage containers

We originally planned to use Docker to handle development tasks rather than scale it into the production environment. One of the biggest contradictions is that we do not have the ability to manage Docker components in a large scale – specifically, knowing which containers are running, when a deployment version is running, how and how the current state of an application is managed Subnet and VPC and so on. Faced with these problems, we simply can not use it as a production platform. Obviously, we need a huge supply of tools to solve these problems.

While looking at Kubernetes, we found that there are several key features that can immediately grab the user's eye:

  • Can be easily installed on AWS (and all of our apps are running here)
  • Dockerfile provides a directory path for the replication controller through a yaml / json file
  • Each Pod can easily achieve the scale of the scale of scaling
  • We can easily scale the number of virtual machines above the AWS in the Kubernetes cluster
  • The rolling deployment and rollback mechanism is built into the tooling system
  • Each Pod can be monitored by running a status check
  • The service endpoint is managed by the tool
  • Have a positive and dynamic technology community

Unfortunately, the biggest problem with tool supply is that it can not adapt to our original infrastructure – it can only provide a set of potential infrastructure solutions as a migration target. In addition, a series of network limitations make it impossible for us to transfer your application directly to the new VPC. Again, the reorganization of so many applications requires developers to have to deal with a number of issues that were originally handled by system administrators and operations teams.

Use Kubernetes as a tool to guide developers to familiarize themselves with the infrastructure

When we decided to turn from the original Chef platform to Kubernetes, I felt that we did not understand the potential problems we could face at that time. We have actually used a variety of different ways to use a variety of network configuration for server operation, and its overall design ideas and Kubernetes VPC streamlined style completely inconsistent.

In the production environment, we run AWS VPC and AWS classic services across multiple regions. This means that we need to manage a large number of subnets with different access control mechanisms for a variety of different applications. Our newly launched applications are very secure and do not have public endpoints. This means that we have a complex set of VPC interconnections, network address translation (NAT), and proxy mechanisms running in different configurations.

In the world of Kubernetes, VPC is the only component. All Pods can communicate with each other at the theoretical level, and service endpoints are well defined. Developers can easily solve some of the details of the problem, while eliminating most of the operation and maintenance needs.

We decided to turn all the infrastructure / DevOps developers into application developers (not joking). At the beginning of our employment, we were asked to develop a technical basis rather than a technology foundation, so the decision was not as impractical as it sounds.

After that, we decided to transform the overall engineering and technical department into the operation and maintenance department. Developers naturally have excellent flexibility, they love to challenge and willing to learn. this point is very important. Just a month later, our business has been transformed from the state of the original DevOps personnel, each engineer has the ability to modify the business structure.

And for the network, product, problem solving, root cause analysis and other objectives of the personnel training work is also a smooth Kubernetes way to scale on the line. In the first month, I have been laughing at my own decision will not bring big trouble. But after two months, the feasibility has begun to show up. Three months later, we can achieve ten times a week to deploy. Four months, 40 applications per week successfully on the line. At that time we have 30% of the application has been successfully completed the migration, this result is more than excellent, it is incredible. Kubernetes has allowed us to transition from an "infrastructure-driven" business to a "infrastructure-driven" business.

To Kubernetes as a means to achieve continuous integration and delivery

How do we achieve more than 40 deployment tasks per week? In simple terms, it is the ability to implement technology integration and deployment (referred to as CI / CD) while migrating. The first application we deployed in Kubernetes was Jenkins, and after that, each of the applications involved was added to Jenkins. As the work progressed, we were able to add and remove all the work of the Pod in Kubernetes by Jenkins in an automated way, and even faster than our ability to track. Interestingly, our scale stretching problem has now become another kind of distress – we want to make too many changes in one time, people need to wait in line to change their order on the line. Our current goal is to achieve 100 deployments per week through the new infrastructure. This goal is fully achievable if we can continue to move forward and stay based on Kubernetes and Jenkins to maintain this continuous integration / delivery process.

Future outlook

Next we need to complete the migration task. This problem has been largely resolved, and the most troublesome part of the work is currently the hands of the more tedious. Moving the load from the legacy infrastructure means that we need to change the network configuration to implement Kubernetes VPC and access across the region. This is still a serious challenge, and we are also actively addressed.

In addition, part of the service in Kubernetes performance is not satisfactory – for example, stateful distributed database. Fortunately, we can usually migrate these loads to third-party platforms and take care of them. At the end of this round of migration, we will only need to run the Pod on Kubernetes, which means that our infrastructure will be further simplified.

Of course, there is no free lunch in the world; moving all the infrastructure to Kubernetes means that we need to hire a highly skilled Kubernetes technical expert. Our team has been reorganized in the form of infrastructure, and they are also busy with application development (which should also be their own work) to achieve business value. However, we do not yet have a team of engineers who can keep up with Kubernetes and the latest features of cloud computing.

With this in mind, we have appointed an engineer to build a new "cloud platform team" and fill in several new members. They will be responsible for developing the appropriate tools to help us to better interact with Kubernetes, while managing all the cloud resources. In addition, they will adjust the Kubernetes source code and some Kubernetes SIG, and if possible, provide code for the open source project.

to sum up

In summary, Kubernetes' first impression seems to be a bit of a stranger, but its actual complexity and subversive level are not as exaggerated as we thought. On the other hand, Kubernetes has also brought us a huge return – let us become a company that can respond as quickly as possible to the customer's wishes.

Original link: ShareThis: Kubernetes In Production

Heads up! This alert needs your attention, but it's not super important.