Mesos vs OpenStack? Who is the future of a private cloud?

[Editor's note] This article is a question on Quora , the questioner for the future development trend of private cloud is puzzled, Mesos and OpenStack relationship is what, whether they can replace each other? Experts from Mesosphere and Rackspace have interpreted this issue from the functions and product positioning of OpenStack and Mesos. The relationship between them is not a substitute for each other, but with a focus on each other, depending on the user's actual needs. I hope this article will help readers to the private cloud field of open source products and related concepts have a better understanding.

Quora on the issue <br /> private cloud future development trend is what is Mesos? Or OpenStack?
In the private cloud, Mesos and Docker seems to be a perfect combination, Google has been committed to this research, I am curious about the future of private cloud will be what, how do you think?

Respondents: Lukas Löschea (working at Mesosphere, before using OpenStack at work)

Mesosphere and Openstack solve a completely different problem. In fact, they can work together. Users can run Mesos on top of OpenStack's cluster, which can bring a lot of additional benefits, such as improving system utilization and fault tolerance.

I found that OpenStack's initial configuration work was much more difficult than Mesos, but the overall understanding of the OpenStack system was much easier because the OpenStack cluster was basically running in a manner similar to how we ran the system in the past few years. OpenStack's basic philosophy is to provide a virtual server, in order to achieve this purpose, OpenStack also provides a good hardware layer abstraction. But the virtual machine constitutes a cluster may make you feel difficult to maintain, for each virtual machine maintenance is not easy, the user encountered the problem and basically in the previous maintenance of the physical machine cluster is basically similar. You still need an operating system, some configuration and management tools, like Puppet, Chef, Salt, Ansible, etc., and need to use a separate host to run these tools. The idea of ​​containerization is the technology that is available after that. Although there is already a Docker Nova driver on OpenStack, it seems to me that most people use OpenStack to create a virtual machine and not a container. They use the way OpenStack is more similar to the use of AWS.

The reality is that users do not need so many servers, users just want to be able to run their services , so that it can play a role. From this point of view, OpenStack can do many things. Providing a server is the ultimate solution, and in the end, all the services are running on the server. But if I had other options, I would prefer to run my service in such a way that services could be infinitely extended (I did not care how they were running and where they were running) and could be automatically detected when the service failed , I do not need to wake up at 3 am and maintain the service. This is what Mesos can do.

Mesos users can adjust their work patterns in a new way, which can greatly simplify the workflow. For example, if a user uses Marathon to run the service for a long time, he or she can set up the workflow: the developer can submit the code to the staging branch, and the CI (continuous integration) server (running in the Mesos cluster) builds a docker image And upgrade the Marathon REST API to deploy through the docker image. If the service is running normally during the staging phase, the developer will merge his newly submitted code into the master / production branch to complete the product system upgrade and deployment.

The final effect should be this: for the user, the so-called deployment process only means that the developer will submit his code to the corresponding branch. Similar to rolling deployments, dependencies between services, health checks, and automatic restart of these functions can be provided by Mesos / Marathon.

This is a completely different way of working, because in this mode, the operation and maintenance personnel no longer need to package application deployment code, or start a specific monitoring services.

From a complex point of view, I think that if users want to run a private cloud platform based on OpenStack, at least need to understand Heat, Nova, Glance, Keystone, Neutron and Cinder these components. I have done the OpenStack deployment work, I found that because of OpenStack's own design reasons, there will be some very complex problems in the maintenance process. Because Openstack itself is an enterprise-class product, not a community product, you will find that the installation and operation of the OpenStack platform requires at least several people from different departments to be responsible. Therefore, if you work in a large company or organization, the maintenance work of the OpenStack platform may happen to be organized according to the organization's own organization (system support, server hardware, infrastructure and network, security, etc.)

On the Internet can find the relevant tutorial, basically can help you set up in about half an hour can be used for production environment Mesos cluster.

From a personal point of view, Mesos's work may be the way we run our data centers in the future. The so-called data center is that all my hardware resources can work together like a whole computer, rather than a single host to work, and users do not need to manually determine where the service to run.

Respondents: Florian Leibert

For the construction of the private cloud platform, Mesos + Marathon + Docker is a good solution. Mesos is a proven tool that can provide scalability services (Mesos is adopted by Twitter, Airbnb, Netflix, ebay, PayPal, and so on). Marathon is a cluster-wide initialization and control system that helps users run Linux services in cgroup and Docker containers. Many companies are optimistic about this technology. All of the key components of the technology are open source and also a core component of our company's DCOS (Mesosphere Datacenter Operating System) commercial product.

Mesos and Mesophere series of technologies can provide a similar cloud environment that can run existing Linux tasks or provide a local environment for building new distributed systems.

Mesos is a distributed system kernel that provides the API required for programming directly to the data center. It will be the underlying hardware (bare metal or virtual machine) to abstract, these hardware in the form of consistent resources to the user. It contains basic functions for building distributed systems (for example, MesS supports Spark App, and Chronos, etc.) such as messaging, task execution, and so on. In this way, users can use Mesos to build a complete service. Apache Spark is one of the examples, Spark framework built on the basis of Mesos, which greatly enhance the efficiency of the development work. For example, Spark developers do not have to worry about network-related issues, do not consider how the task will be distributed in different nodes to perform, these are Mesos have the basic core functions.

You can refer to Mesos' page on Apache.org, where Mesosphere and its partners are certifying these services and are packaging these services into the Mesosphere DCOS Datacenter Service so that they can be installed directly through an instruction.

Mesos Another very good feature is that it can run on the existing OpenStack cluster, you can also run directly on the bare metal, you only need to run on each node a small Linux process can be.

I think Mesos is the future of cloud computing because it provides a good service scalability, is conducive to data center automation operation and maintenance, can be self-diagnosis and provide a rich local service ecosystem (rich native application ecosystem), at the same time, You can also do not have to modify the code to allow existing services to run properly.

Respondent: Nati Shalom (built the first Java PaaS solution, working with Amazon, RackSpace, Azure, VMware, Xen, JClouds experts to design a product that provides automated services and scheduling services in the cloud.)

In the IT industry, from time to time, there will be new technology, instead of the original "new technology", Mesos's case seems to be in line with this situation.

In fact, although Mesos was successful, it did not make it a common solution to replace existing infrastructure. In the previous article: if I use Docker I still need OpenStack? , I pointed out that in a particular area, OpenStack and Docker as popular, so Docker temporarily can not replace OpenStack.

My point is that if we want to answer this question, we need to look back at what we wanted to get from a cloud infrastructure like OpenStack at first, and whether Mes Mes could provide an alternative to all of these features.

The same cloud environment as OpenStack provides multi-tenant-based resource management for core functions such as computing, storage, and networking, in addition to being able to take resources actively and allocate workloads.

As many of the major cloud service providers continue to invest in the development of public cloud infrastructure, OpenStack has gradually developed into an ecosystem.

Mesos is doing well for assigning workloads across hosts. But I do not think it should be seen as an alternative to the OpenStack cloud platform. If you want to replace OpenStack, you need to provide the same level of security, storage, network and multi-tenant services with OpenStack. I do not think Mesos needs to do it better than OpenStack in all respects. In my opinion, Mesos can run as much as OpenStack, and not replace OpenStack.

Respondent: Dale Bracey (Rackspace Private Cloud OpenStack Product Engineer)

Although I am not familiar with Mesos, but I generally understand that this is another can share the host kernel container services.

I try to add some more to Lukas Lösche 's point of view. You can combine the content mentioned in the previous answer to serve you. OpenStack can be used to provide scalability, and you can more easily manage your development tools, websites, services and servers. Even if the direct management of the bare metal, OpenStack can also be completed ( Ironic-OpenStack )

OpenStack overall deployment and maintenance is more difficult, although the document content is very detailed, but still lacks detailed guidance, such as telling the user from the beginning to the last steps should be how to complete. After all, OpenStack is a big project, this complexity is normal.

OpenStack is trying to become a distribution layer that can manage multiple hypervisors, virtual machines, and storage services, but there is still a lack of capacity. Previously mentioned the OpenStack-Docker Nova driver, which installs the Docker on the compute node and submits the container to Nova / Glance in the form of a virtual machine, but in fact the container is not a virtual machine so that the user can not only use the container technology itself Advantage, you can also use the OpenStack provided control management functions. In addition there is a CoreOS-OpenStack project, this project is also very popular, these two projects have similar characteristics.

Of course, users still need to use some configuration system to help you manage the system, unless you want things to become more complicated. We used Chef and then used Ansible. It was sad that I had just learned that Ansible's CTO Michael Dehaan had left Ansible today. I do not know how the company will develop in the future, of course, this is a digression.

If you only want your system to run, not to care about virtual machines and servers, OpenStack out of the box does not provide you with a fully automated environment, but you can use some of the tools to achieve this purpose. These tools are also completely open, you can customize them and follow the way you want to work. You can combine Heat-OpenStack, Ceilometer-OpenStack, and various DevOps tools to provide you with automation services.

Finally, for the new product to do an ad: Rackspace also recognized OpenStack deployment work is indeed very difficult, we have been committed to helping customers simplify the deployment of the difficulty, we provide github for the deployment of private cloud OpenStack node, allows users to use (There is a community project in stackgorge stackforge / os-ansible-deployment ) We also provide other private cloud solutions.

You can try to use it to read the deployment document, submit a bug to submit new feature requests, and contribute to the community, let us work together to simplify the OpenStack deployment process. Users only need to focus on the development process of the service, without regard to the underlying infrastructure. We will always be able to help you, including the updated github above the document. Here are some relevant information:

Docs: Private Cloud Support and Management by Rackspace

Install: rackspace.comPrivate Cloud Computing, Storage & Hosting by Rackspace & Openstack

Learn more: Rackspace Community

Original is the future of the private cloud world, Mesos or OpenStack? (Translation: Wang Zhe)

===========================
Translator introduction <br /> Wang Zhe, Zhejiang University SEL laboratory graduate, currently in the cloud platform team engaged in scientific research and development work. Zhejiang University team PaaS, Docker, large data and mainstream open source cloud computing technology has in-depth research and secondary development experience, the team is now part of the community will contribute to the technical articles, hoping to help readers.

Heads up! This alert needs your attention, but it's not super important.