In one of the previous post, I gave an example of how an analytics team can benefit from fast prototype environment deployment. Rapid experimentation and instant infrastructure are both crucial parts of the tech team's success. It is vital to respond to the clients' requests and adapt to emerging needs. Many companies have already noticed that innovation requires transformation. Otherwise, there is a considerable risk of losing customers. Unlike old and very conservative organizations, relatively new technological giants like Google or Facebook know how to adjust their business models with new rules of the game. The bespoke companies are highly agile players, and they adopt hybrid cloud very quickly.
Let's start with the definition and simple use case
Cloud computing has been within the regular operations of enterprises already for a while. It continues to be one of the fastest-growing segments of IT spend.
"Hybrid cloud is a computing environment that connects a company's on-premises private cloud services and third-party, public cloud into a single, flexible infrastructure for running the organization's applications and workloads." - IBM Cloud Learn Hub
When we speak about the hybrid cloud, we primarily refer to a combination of public and private clouds. For example, let's take the most straightforward use case when connecting on-premise cloud with third-party providers can be called 'a hybrid cloud.' Imagine a highly agile team of data scientists who frequently need to access significant computational resources for quick experimentation on big data sets of non-sensitive information. It would be reasonable to spin up, just for the sake of the example, an AWS SageMaker, if it would need to happen a couple of times per month. It would make absolute sense from a costing point of view. We always consider the importance of spending money, right? However, let's imagine a very realistic scenario when the team needs to access this capacity daily. Then, most probably, it pays off more to buy the server and put it into the server room in the company's headquarter. A couple of powerful GPUs, CPUs, and a lot of RAM would do the trick. Let's don't forget that we need to store our data somewhere. While accessing the computing resources from AWS might not be the best idea, storing data in AWS S3 sounds like a great plan. Therefore our simple, yet capable hybrid cloud infrastructure boils down to having a 'beast' server in the basement and cheap data warehouse in the cheap cloud file storage. It seems like a straightforward plan, but it is easier to say than to actually do it.
Popularity for hybrid cloud is growing
It is hard to strike the right balance between the public and private clouds. Some companies prefer to rely on the cutting-edge solutions offered by third-party providers, whereas other companies are more comfortable with on-premise servers. The latter can be caused by various reasons starting from conservative views and skills to legal issues. There are multiple vital reasons as to why the notion of a hybrid cloud is gaining demand.
One of them is flexibility, and it is visible now like never before. Remote access to our network drives, remote work, or home office was an essential part of our lives before the recent pandemic, but the post-COVID world will bring much more interest in remote collaboration. The high level of flexibility wouldn't be achievable solely by relying on private clouds, because scaling and access are better managed when a public cloud is utilized.
The other concern is security, which has recently become a hot topic for all of us. There are still many companies that manage sensitive data and do not risk exposing them to cloud providers. It is dependent on the level of security required for a given application or service, but not all clouds ensure the expected protection. This puts the decision about architecture more towards the private cloud. Though the flexibility concern doesn't allow us to implement only a private cloud. Hence the hybrid infrastructure should be set up.
The third point that naturally requires attention is cost. Hybrid cloud ecosystems allow enterprises to leverage the resources they already have, without the necessity to embrace new tools or splash out on new hardware. Since pay-as-you-go charges are not always the most comfortable for firms or specific projects, it is crucial to have an alternative.
High-level architecture
I am glad you got familiar with the concept of containers, and I believe we are on the same page now. Just in case you missed my post about the containers, I suggest you quickly go through it - it will be much easier to understand the following diagram. Also, it might be interesting to see how Susan's team used a similar architecture in a real-world project implementing an Apache Spark environment - here is the post.
There is no surprise for you on the diagram, most probably. These are parts that connect private, public, and local machines (e.g., our laptops). We leverage the computational capacity of our powerful on-prem server and stock massive data sets in the cloud. We can spin up multiple containers inside the server and then source the data directly from the cloud file storage, depending on the purpose.
The above picture shows a drill-down of one of the possible containers and gives a bit more details on the infrastructure described in this post.
Summing up
There lots of pros for the hybrid cloud and the decision of the implementation depends on individual organization needs. However, it is clear that the overall trend is increasing, and the majority of companies that used to utilize any form of cloud service are changing the approach and seriously consider the hybrid cloud.
The described use case and high-level architecture are both widespread and highly applicable to most organizations that are presently changing their digital strategy and plan to adopt more medium-term innovation.