With the rise of cloud computing and the on-demand available capacity, I sometimes hear people say that software automatically scales, all you have to do it place it in a cloud environment such as Amazon. The reason why people tend to believe this, comes from all the propaganda made by such companies. They tell you tales of how your system adapts the the load automatically, and use cases of companies which had a drastic increase in load overnight, and how well it worked with the cloud environment.
While it is true that in a cloud environment it is pretty easy to start multiple ‘instances’ that run the software, this does by no means has anything to do with true scalability. Just considered we have a common desktop application, or even something that normally is installed on a server on-premises. We now decide to abandon the costly infrastructure required to run this software, and move it to a cloud environment as is.
How do you think it will perform? Much like before, nothing gained, nothing lost. What about if load increases? Again, it will react the same way as if it is running on a desktop or server. Then what about all those promises of automatic scaling?
Well, the truth is, nothing comes free. If you expect a new instance to be started when load becomes high, you need to add a trigger for it. But before you do, did you even consider how your users will be connecting to the new instance? I hope you didn’t forget to put a load balancer in front of your instance and use that as a gateway to your software. If so, starting multiple instances will lead to nothing but a higher bill, as users will not be able to connect to your new instance.
But even if you did use a load balancer and users are being distributed over the different instances there are some other things you have to take into account. Does the software rely on a locally installed database? I can imagine a normal desktop or on-premises application to do so, but in the cloud this becomes impossible. Not only because users will be tied to a certain instance as the database with their data is located there and such a binding is not desired, after all, that’s why we have the load balancer for. Binding users to a certain instance means we can not terminate an instance that is still being used by a user. Even more, what do you do if indeed an instance is not being used? Do you terminate it? But this means losing all the data in the database.
Separating the database is the first change you need to make to the software, such that it can handle connecting to a remote database. But it is not the only one because the same problem arises with the file system. Most applications just store data on their hard drive, but for the same reasons as with the database this can not be done in a scaling system.
But now at least we have a system that can automatically scale and benefit from it to the fullest, right? Well, did you consider that maybe the load on your software is now moved to the database or the file system? Software that heavily depends on the file system or a database may be scaled up, but that only leads to a database and file system that are overloaded. Scaling these is a lot harder. Not only does it mean data needs to be copied, it also means that data will not be directly available on the different servers. If we would want this, we would have to write all the changes to the different database and file systems, leading again to them becoming the bottleneck.
While a cloud environment allows us to utilize a scaling system to the fullest, just as with software that would run on a server it has to be designed for it. Software that was not designed to be scaled will lead to unfeasible situations, even worse than a server that is overloaded. Note that these problems are not related to the cloud environment, they are present on an on-premises system as well, but the cloud environment does not solve them either, we as developers have to do that.
As a final remark I would like to point out that this is very similar to a multi-core architecture, where changes have to be made to the software to utilize multiple cores. If not all you can do is run multiple instances of it next to each other in complete isolation. While this does allow us to process twice the amount of data, this is not the form of scaling we really want.
Working As A Software Engineer 