Private cloud myth: what is the future?

[Editor's note] A very good article, the author from the current plight of OpenStack talk about, talk to the background of the private cloud, and then introduced the history of cloud computing. From the original intention of cloud computing and the popular distributed applications now extend their core view: the server and the virtual machine will not disappear, but we will be less and less direct interaction between them.

Private cloud market is declining, but do not blame OpenStack. Because from the beginning, the idea of ​​a private cloud may be wrong.

As AWS is considered the cloud service standard, VMware is considered the local server virtualization standards, and OpenStack was once known as VMware's open source implementation, which spawned a number of private cloud startups and private cloud products. But now talk about OpenStack, everyone is mixed.

According to an article in BusinessInsider in April , the OpenStack Foundation works with a number of technology vendors, but each partner is in order to engrave its own brand. The author writes that the project provides "lifelong support" and "provides venture capital support for OpenStack's start-up companies, but it seems that resources are drying up." "

Some supporters argue that the OpenStack project (claiming to be undeniable) needs to become mature and win the user, but others point out that the user grows slower than expected, and that some high-profile start-up business changes or closures are very bad omen.

Some skeptics are fair, but the heart of the problem is that OpenStack – neither neither nor before – it is playing a game that can not win in the cloud software layer . Because Opentack is working with other private cloud technologies in the same field to try to build a localized AWS.

History of the cloud

At the beginning of this century, there was a lot of discussion about what is the essential technology of private cloud systems. They also define related terms such as grid computing , utility computing and flexible application platforms. The technical term is different, but the general idea is the same: create a resource pool that provides the application with the required resources and provides features such as schedulability, high availability, and failover.

We often use the grid to analog cloud computing, this metaphor allows us to more intuitive understanding of these technologies. In theory, we do not need to care about a single server. Just as we do not need to care about a generator in the grid, we only need to connect the "application" to the grid and then get the power needed The

At the end of 2006, the Amazon AWS released a cloud of elasticity . You do not need to have a local server, all operations are running in the Amazon data center! You can pay for these resources by hour! (Sun Microsystems made the original idea , but AWS first realized and promoted this model and made it mainstream.)
With the constant innovation of AWS, it gradually overturned the use of the original server, but also implies the importance of the server. Users can deploy computing power quickly and easily compared to ever before, while users still rent virtual machines, including CPU, memory, and local storage, as well as their own unique IP addresses.

Infrastructure as a service, that is, IaaS, which means that R & D personnel should be as little as possible to consider the server, but they still have to consider.

This is also the direction of cloud computing vendors have been working hard. When everyone realizes that cloud computing is the future, Microsoft Azure, Google App Engine and Heroku are trying to abstract the application platform from the server management level, which is the later platform as a service, that is, PaaS.

But that's already the case, whether it's good or bad (and probably better in many ways), the AWS model has won. Traditional application providers face all the risk of being closed or forgotten, and replaced by the Amazon model of private cloud start-ups and large-scale application providers' blue sea – which spawned OpenStack and its entrepreneurial companies around its ecosystem Class technology of large-scale application providers. In addition, even if the original PaaS products still exist, Microsoft and Google eventually forced to launch similar to the AWS IaaS cloud services.

Private cloud dilemma

Despite this, technically, rebuilding an AWS-like platform, many private clouds lack funds, knowledge reserves and vision, let alone innovation. To be aware that when OpenStack was launched in 2010 , AWS had spent nearly four years adding new features and services.

To a certain extent, to build all sectors of the universal products, not so much this is AWS, as it is a resource-intensive platform. Especially in the first few years, AWS is intended to deploy a one-time platform, and then tell the user to either accept, or leave. If the essence of the problem is to configure the virtual server, shared storage and network, there may be some certification requirements, then AWS and VMware already exist.
It's easy to understand why people will be excited about the private cloud and then talk about OpenStack-based private clouds. But when they wait for these technologies to become mature and commoditized, it seems that many people will only say, " Okay. We will use AWS and VMware .

Perhaps some big IT vendors are driving OpenStack to become a basic private cloud, its purpose is more to provide a vision for enterprise users. If any customer has enough money, wisdom and experience, they can do so (translator note: use OpenStack to build a private cloud). Or, no matter how they and the world will change, embrace cloud computing is the trend.

The rise of distributed applications

A few years ago, two important changes took place almost simultaneously: the rise of developers and large-scale Web applications. Facts have proved that the two are intertwined so that they are almost inseparable. They help to encourage innovation, people need to rethink the nature of the private cloud is what.

For application developers, AWS means freedom . By easily accessing computing resources and services, they develop more applications and try to make their lives easier. This model produces a lot of new tools and new technologies – a brand new business market, a developer-centric, and a significant increase in developer productivity.

Some of them are very successful, from the media reports, the number of users and investment situation, Docker , MongoDB and Elastic , each in the past few years raised more than 100 million US dollars of venture capital. New Relic is now a listed company . Several other well-known vendors include GitHub and Stripe.

But they are just the tip of the iceberg. Wherever you go, there are always other start-up companies or open source projects that try to improve the user experience of the database, either through the API to provide a business process analysis, or otherwise accelerate from concept to code, from testing to production The
While AWS liberates developers and lets them see the world in new horizons, some large Internet companies, such as Google, Facebook, LinkedIn, Yahoo and Twitter, already have a lot of size. In response to the crazy challenge of serving millions of dollars, or even billions of users, they began to hire the smartest computer scientists to build new databases and infrastructure in the world to manage and analyze data.

Then they start talking about these systems – in many cases open source them. This is largely due to these companies, so that we can use, for example, Hadoop (and Hive ), Cassandra , Mesos and Kafka , as well as dozens of important open source technologies. Even if they are not always easy to use, but they are really strong.

When cloud-based applications (mostly based on AWS) increase in user volume, developers need to build a scalable application. Some cloud applications will grow steadily every month, and some applications may spread out of the virus overnight.

Suddenly, Google's scalability challenge became a challenge for Pinterest (or Airbnb, or Instagram, or Spotify) for scalability . Google's analysis of user data opportunities, but also look into the Pinterest, Instagram and Spotify the opportunity.

A benign cycle has already begun. Now, start-up companies from the beginning to use these ultra-distributed technology, and ultimately make their projects benefit, and often open source their own tools to make other people benefit. This cycle has affected every industry in the start-up companies.

The true value of cloud computing

With the development and integration of developers and distributed applications, the importance of it is beyond doubt. This requires us to reflect on the nature of cloud computing. At least that is far from simply buying a virtual server from Amazon, Microsoft or Google and taking some of the technology they created.

Nowadays, businesses of all sizes are aware that, regardless of whether their infrastructure is running in the cloud or in their own data centers, they need to be as flexible as a system of large-scale network companies – flexible, efficient and automated. This requires changes from the hardware level ( open computing ) to the network level ( software definition network ), to the scheduling level ( Mesos ), application level ( container ).

The latter two levels are particularly interesting because they need to rethink how the application is deployed in the cloud (shared or private). Finally, after the cloud has become the dominant standard for ten years, we should stop talking about the server, just like the utility of those folks who describe us.

Google and Microsoft are no longer worried about this because they can manage their own data centers and applications. The most famous example is Google's Borg system (and the subsequent Omega system), they run in the Linux control group, take intelligent scheduling work to ensure that the application to obtain the necessary resources. Developers only need to specify the link resources of the service, as well as their application needs the resources, the rest are done by the intelligent scheduling system.

But so far, Borg and Omega are still used internally by Google. In the rest of the world, only through other similar technologies to achieve, such as Mesos, Docker and Kubernetes (Google contribution). Docker enables developers to discover a much easier and portable configuration approach, and Mesos and Kubernetes allow developers to dynamically extend their container-based applications dynamically.
This combination, along with some other important new technologies, allows cutting-edge companies to manage all servers in a pooled way (either physically or in the cloud). Large data and database systems and Web services run on the same cluster, intelligently share resources, rather than competing for resources. Dynamic scalability makes a number of server downtime is not so important, micro-service architecture is replacing the monopoly application architecture.

This architecture is very powerful for managing data centers and building up their own applications, and AWS, Google and Microsoft have proven that they have been earning billions of dollars a year from virtual server business and are now fast Turned to a container-based management platform, and then quickly appeared in the interface on the container management options.

Servers and virtual machines will not go away, but the direct interaction between us and them will be less and less.

The scalable, distributed, service-based architecture that large Internet companies have developed over the past decade will have a profound impact on the future. But yesterday's architecture is not necessarily not suitable for tomorrow's application, the premise is whether it can improve the developer's work model, improve the operation and maintenance team's work model, making the company more competitive. If it can not be done, then it will be replaced.

Ask: what is the private cloud?

The rise, fall and resurrection of the private cloud (translation: Yang Feng proofreading: Li Yingjie)

Translator introduction Yang Feng, currently working at EMC Corporation, responsible for ISILON cluster storage related technology work.

    Heads up! This alert needs your attention, but it's not super important.