Can Your eCommerce Platform Handle One Hundred eCommerce Orders Per Second?
What is there in common between a rush hour in aviation when dozens of aircraft are approaching the airport at the same time, and an online sales peak with hundreds of buyers simultaneously pressing the ‘Checkout’ button on an ecommerce website? The correct answer is that, no matter what, all requests must be served anyway. The pilots have to land the aircrafts safely, and online shoppers must be rewarded with a ‘Thank You’ screen for successful order handling.
You might be surprised to learn that you can use the same order-handling algorithm in ecommerce as in aviation to achieve a 100% customer satisfaction rating. Let’s talk through how to do this.
Don’t Miss Even a Single Order in B2C/B2B eCommerce
Why is handling a huge number of concurrent orders so critical for ecommerce? The answer is obvious: to not miss a single order, while giving your customers a great user experience and preventing the website from failing under the high load.
Black Friday is a perfect example of the peak load on ecommerce sites around the world. And while it is a good example, it is mitigated by the fact that your IT team knows the date of this big sale in advance and can prepare for the increased sales volume by launching more powerful servers in the cloud.
More important for assessing platform scalability is how it will handle a completely unexpected peak load when the date and time is not anticipated. Sometimes a marketing team can prompt such a peak load by sending mass emails with promotional products without warning their IT people in advance. In that instance, it is necessary to work on enhancing business processes rather than improving ecommerce software.
Recent ecommerce trends show that more and more brands in both B2C and B2B are doing limited runs and using scarcity to drive larger profit margins. This also happens due to supply chain shortages such as the lack of electronic chips or other components and materials during the pandemic. However, such small batches of products can even be called “Limited Edition” to force customers to buy them for a short period of time.
An unexpected peak in demand can also be triggered by third-party-driven events such as a press release about a shortage of some products. The bursts of demand are typical, not only in B2C, but also in B2B ecommerce too. For example, when Bloomberg published an article about the coffee crop failure in 2021, suppliers saw B2B buyers immediately rush to place bulk orders for coffee for restaurants and hotels before the price rose.
Also, surges in demand can provoke the actions of regulators to ban some goods due to new environmental standards. The peak sales could even be driven by messages from the vendors themselves about the termination of some popular models of goods and the impending scarcity of spare parts for them.
So you can see there are many independent sources of unexpected peak demand in ecommerce, both B2B and B2C. Suppliers, regardless of the size of their enterprise, need to be ready by having a best-in-breed order-handling system in terms of its scalability.
Order-Handling Speed Is Crucial for eCommerce Platforms
Without fail, every ecommerce platform vendor declares their system to be a highly scalable software that can handle a barrage of customers. In fact, this might not be entirely true and the devil is in the details. Namely, the key is in the elasticity of scaling and the time the system takes to respond to an unexpected influx of customers.
We previously drew an analogy between aeronautics and ecommerce in processing requests, so let’s use aeronautical examples to illustrate ecommerce scaling. Usually, we talk about two scenarios for scaling the resources of an ecommerce platform:
- The simplest and most basic scaling ecommerce order handling is relying on the capabilities of the purchased IT infrastructure subscription plan. For example, you know you usually have 20 orders per minute on average and, therefore, you purchased a subscription with some reserve of resources up to handling 40 orders per minute. If the number of buyers becomes greater, some of the customers will be denied service. In aeronautics, this scaling might be illustrated by scaling within the capabilities of the airfield infrastructure. For example, on average, around 20 aircraft per hour are landing, with the max capability of 40 aircraft per hour for the airfield.
- The so-called ecommerce resource auto-scaling. In this case, the ecommerce platform has a special load analyzer that calculates the load for repeating periods of time, such as every 30 seconds. If the load reaches the upper limit, a new copy of the IT infrastructure is initialized. In this way, the ecommerce order handling can indeed be scaled many times within the resources of a data center or even a cluster of data centers. The disadvantage of such an auto-scaling algorithm is an inertia that usually lasts for a few minutes. This time is needed to start an additional server instance with another copy of the ecommerce order-handling module in the cloud or on the premises.
The aeronautical analogy here continues when considering how you can scale the airfield’s capability according to growing air traffic. For example, you can initiate construction for an additional runway. Of course, it will take years, not seconds and minutes like in the case of scaling IT capabilities, but, in essence, the way of scaling is identical, i.e., creating new copies of the same kind of resources.
The Virto Commerce platform also has order-handling scaling based on the principle of launching additional instances in case of a significant increase in the number of orders. We consider this our basic regular order-handling module. Our tests show that the launch time for an additional instance takes up to 1 minute, which is quite acceptable for small and medium-sized ecommerce businesses.
But how to handle orders with absolutely no inertia in scaling? Let me continue to apply how we used an idea from aviation to handle peak orders without launching additional platform instances.
Take an Aviation Holding Pattern and Wait to Be Served
So, now we have to get peak order handling to be instantaneous and without auto-scaling inertia. This is where a technique from aeronautics known as the holding pattern comes to the rescue.
It is also important to understand that the entire application of Virto Commerce architecture does not need to be redesigned. We intend for the new FastOrder module to be used on demand, and besides, it is fairly inexpensive to host the FastOrder module on a server in the cloud.
The holding procedure in aeronautics is an algorithm for servicing arriving aircraft during airport runway overload, with a redirection to the holding area. This is a circular-shaped route that pilots fly near the airport while waiting for their turn to land. Aircraft are positioned in this loop of the holding area at various altitude levels (1,000 feet between, according to FAA safety regulations). The lowest aircraft is cleared to land first, and then all other aircraft move one level down and the lowest one is again the next in line to land.
As you can see, the use of a buffer in the form of a holding area is an example of an inexpensive but very effective solution for providing service under peak load conditions.
When the Virto DevLabs team was tasked with developing a high performance order-handling module, the concept of a scaling algorithm using queue buffering was implemented here as well. This high performance order module is now included in the Virto Enterprise Pack and can be easily installed instead of the basic order-handling module. The module recomposing is simple to do thanks to the Virto Atomic Architecture.
The availability of the Azure Queue buffer in the high performance order module allows subsequent order execution software modules to work in a standard mode without overloading. An important feature of the high performance order-handling module in the Virto Commerce platform is its low cost of deployment in the cloud.
If we look at the diagram, we see that a new instance of the platform appears with the FastOrder module, which is only responsible for accepting a request to create an order, validate it, create an order number, and put it in the queue. This operation is very fast; we were guided by a time of less than 300 ms, and not resource-intensive — just a 1-Core CPU with 1 GB RAM is enough.
The main platform still works as usual, only orders do not come from the storefront, but from the FastOrder module. This high performance module extracts orders from the queue according to available resources. If the queue grows, then auto-scaling can increase the number of instances.
The advantages of the high performance order-handling module are as follows:
- Enrichment of Virto Commerce Solution with enterprise functionality by demand;
- Guaranteed order creation UX for end customer; and
- Cost-optimized solution for integration and IT expense.
Testing Virto High Performance Order Module on Minimal Azure Server
Our customers interested in Virto’s high performance order module want to make sure that an online store based on the Virto Commerce platform can easily handle an unexpected high load. With this in mind, we decided to test and confirm that Virto’s platform can handle peak traffic of 100 orders per second and it will only cost you $3 extra per day. Yes, there is no typo here, only $3!
Of course, when customers place 100 orders per second in your store, at first, you will be happy that all the orders were accepted. A little later, you will think that, in reality, delivering so many purchases, especially in B2B, will be a hard logistical task. But the good news is that, if you sell virtual products such as game subscriptions or access to online movies, the high ordering speed is your competitive advantage while content downloads can be arranged through partners.
We used the Apache JMeter load testing service to generate a given number of orders from a given number of users and measure throughput on different server hardware configurations in the Azure Cloud. Apache JMeter is an open-source Java application designed to load test functional behavior and measure performance. You can see the results here:
As we tried a small, cheap 1 Core + 1 GB RAM server hardware configuration in Azure Cloud, each order was placed in 300-millisecond intervals with an average concurrent throughput of 97 orders per second. If necessary, it is possible to scale up both the FastOrder module and the Virto Commerce Platform to support the acceptance of even more orders, up to several times more.
Another interesting feature was a smoother load on the CPU and DB of the main platform. Sometimes it rose to 80–85% of the available resources, but peaks and reaching 100% were not observed in the tests.
Summary
Order-handling speed in an ecommerce platform is a key differentiator in competitive markets. If you Google the article “Speed is now a landing page factor for Google Search and Ads”, you’ll see that site speed is a critical top-ranking factor for both desktop and mobile devices. If you spend a lot of money on the fast website front end, but forget about the speed of the middleware and back end, then you have not reached your business goal.
Any low performance from the software responsible for handling orders has a detrimental effect on customer loyalty. Dissatisfied customers, unable to check out due to site overload, may make purchases from your competitors in the future.
By losing an order, you not only lost a sale, but you also wasted money spent on marketing your online store and on your IT team, who ultimately failed to deliver the service. On the contrary, customers who succeed in placing an order, even when there is a large peak of buyers to the site, such as a preannounced sale, will readily turn to your brand in the future.
If you need more information about performance, integration methods, and how to run the Virto high performance order software module, please contact the Virto Commerce team.
***
About Virto Commerce DevLabs
Virto Dev Labs is a group of highly professional developers and technology experts and business visionaries within the Virto Commerce team. DevLabs drives ideas for Virto Commerce’s road ahead and runs experiments to test cutting-edge technologies for implementation in future releases of our B2B/B2C ecommerce platform. New developments in this category are for broad usage. We encourage you to try them and provide feedback in GitHub or on our contact page.
