Use of Predictive Analytics in Application Performance Management

According to Forrester “You can’t see into the future (yet), but with predictive analytics you can make an educated guess.” 

Predictive models utilize the past data to determine the likelihood of certain future outcomes. With predictive analytics businesses can take informed decisions to plan in an optimized manner. 

Predictive analytics can benefit your IT teams in many ways. It gives the ability to monitor the health or status of an application so that you can predict and respond to application outages. The ability to prevent failures before they happen is a big win. It saves time and money and creates a more resilient IT infrastructure. Predictive analytics tools can also be used to predict the user experience at a projected user load and to evaluate if the current infrastructure can support the projected user growth. 

Application Performance Monitoring (APM) helps monitor the user experience and leverages predictive analytics to improve the application performance whenever there is a sudden spike in number of user’s accessing the application/whenever there is a sudden degradation in application performance. 

Predictive Analytics Usage 

Organizations can leverage predictive analytics to improve their application performance in the following areas: 

• Identify root causes for application performance issues 
• By identifying root causes for application performance using machine learning techniques, organizations can focus on the right set of areas in which to take action. Predictive analytics can then study the characteristics of the various attributes within each cluster that can provide deep insights into what changes need to be made to achieve ideal performance and avoid specific bottlenecks. 

• Monitor application health in real–time 
• Performing real-time monitoring of application health via multi-variate machine-learning (ML) techniques allows organizations to catch and respond to the degradation of application health in a timely manner. Most applications rely on multiple services to capture the true health of the application. The data might consist of configuration data, application logs, network logs, error logs, performance logs and more. Predictive analytics models can analyze the past data during a time in which the application was in a good state and subsequently identify whether the incoming data exhibits normal behavior or not. 

• Predict user load 
• Predictive analytics can help in predicting the user load by analyzing the past data. Organizations can use this data to better prepare in order to handle the predicted user load and provide experience assurance which would help in reducing customer churn. This data can also help the organizations better plan their future IT infrastructure requirements and capacity utilization. 

• Predict application outages before they happen 
• Predicting application downtime or outages before they happen helps perform the needed maintenance on the application without any downtime. This can save an organization time, money, and much more. Before an application outage, the IT infrastructure leaves lots of indirect clues hours, or even days, before it dies. The predictive analytics model can learn those patterns and continue to monitor for similar occurrences, predicting future failures before they happen. With this type of predictive model in place, preventive action can be taken at the right time. 

Applying predictive analytics to forecast application performance 

Predictive analytics in the area of application performance improvement focuses on three main areas – Forecasted User Load, Response Time prediction, and Infrastructure Assessment. 

User Load Prediction 

Traditionally organizations have relied on peak user traffic in the past in order to come up with the number of users that might access the application in the future. This model has its own limitations as it does not consider factors such as emergence of new technologies, change in user behavior and other disruptive factors. By using AI/ML predictive analytics, businesses can avoid these pitfalls as forecasting models can be built by analyzing the user behavior in real time.  

Using the data provided by the APM we have built a model to predict the number of users who will access the application six months into the future. We used the hourly monitoring data from production monitoring for the past 2 years as input to the model. We found that the accuracy provided by the model was low when we used classification and regression models due to missing data and data imbalance.  

The data was normalized to handle missing data and data wrangling performed to mitigate the data imbalance and fed to a neural network. This increased the accuracy of the model to approx. 75–80%. The model was then converted to a self-learning algorithm by retraining the model on live data which further increased the model accuracy by 8-10%. 

Such accurate forecast helps decrease the customer churn rate. 

Response Time Prediction 

APM tools capture a lot of application data and hardware utilization details. This data can be leveraged to predict the application response time. 

In order to predict the application response time for a given user load, using Exploratory Data Analysis using the user load as the input provides low accuracy and correlation. Even adding the application server hardware utilization details to the inputs and using multi linear regression models, does not help achieve the required accuracy.  

However, when we add the database server hardware utilization details, network telemetry and middle ware utilization details to the inputs and use the ARIMA models, we find that the model accuracy may increase to more than 75%.  

This model helped our customers to tweak the application architecture and server hardware to improve the application performance for the anticipated user load. This also helped in improving the user experience and better customer retention during the peak season. 

Infrastructure Capacity Prediction 

Apart from building the models to predict the user load and response times, we have also come up with a model that would estimate whether a given hardware will be able to handle a given user load. By tweaking the input parameters to the model, we can also predict the infrastructure required to handle a particular user load. In order to achieve this, a model was built by leveraging the APM data from production for a period of two years. 

With a 75% accurate model, we then modified the hardware configuration to predict the application health. 

This model helped our customers to plan for cloud migration by evaluating the infrastructure requirement and the costs associated with infrastructure upgrade requirement versus cloud migration. 

Data is the new oil – are you analyzing it? 

According to Forrester “They say data is the new oil. They say data is the new currency. They say data is the key competitive differentiator. All true. But reality is sobering: Only 7% of firms report advanced, insights-driven practices.” 

Businesses need to spend some time to realize the benefits. The predictive analytics implementation needs time as data has to be prepared, cleansed, correlated, the right algorithm has to be chosen to process the data, the algorithmic output to be understood, and the algorithm has to be retrained with new sets of data and optimized till the required success factors are met. We can accelerate the deployment time by leveraging our experience with similar type of application or domain. 

Since predictive analytics depend on the data to predict the future outcomes, any problems with the data quantity or quality will adversely impact the results. We can mitigate this problem by integrating our models with the APM tool in production. 

In order to simplify the implementation of predictive analytics, we at Cigniti are developing a process that will monitor key performance indicators and develop its own unique system for setting dynamic thresholds. Essentially, the machine-based learning process will figure out high and low thresholds for when it’s appropriate to alert the operations team. The new model would shift these thresholds over time as it learns from the aggregation of historical data. It will allow for thresholds that are based on seasonality and history rather than a single static threshold set by a human.  

With these algorithm-set thresholds, an APM typically will send fewer alerts, but these alerts will be far more intelligent, actionable, and valuable. 

Predictive analytics can predict the future behavior and it can provide certain recommendations, but it cannot provide a complete course of action to handle the various scenarios. Organizations will have to plan how to handle the various outcomes coming out of the Predictive analytics models. 

Connect with our experts today to resolve your Performance Engineering and Management needs.