Executive Summary:
A property solutions provider faced challenges with their on-premises server and manual search processes. These challenges limited their scalability and flexibility while facilitating public searches. To address these limitations and improve efficiency, the property solutions provider partnered with the Jahnel Group team to implement AWS OpenSearch.
By migrating to AWS our client streamlined their search processes, eliminated costly hardware investments, and overcame scalability limitations. AWS OpenSearch provided enhanced search capabilities, including efficient indexing, powerful querying, and resilient security measures.
The implementation of AWS OpenSearch improved operational efficiency and enhanced the search experience. It provided a reliable and scalable infrastructure, eliminating constraints. This transition not only resolved their challenges but positioned our client for future growth.
About our Customer:
Driven by a commitment to the efficient and effective administration of claims, our client employs various strategies. They provide modern resources for claims, innovative management systems, compliance services, and program consulting. Consumer protection is paramount to them, and they leverage their industry expertise to help organizations navigate the complexities of claims management.
Navigating the Challenge:
Supporting public searches in their app posed the primary challenge for our client. This was driven by a substantial volume of search requests, averaging around 250,000 per day across all pods using OpenSearch. The traffic load varied significantly among different pods, occasionally exceeding 1 million searches daily. To effectively manage these fluctuations, our client ensured an adequate number of nodes and adapted their sharding strategy, which partially resolved the issue of slow searches.
On-premises server and manual search processes posed substantial costs and operational challenges for our client. They had to invest in expensive hardware and allocate resources to manage and scale their infrastructure. The reliance on Lucene indexing has further added to their expenses as it necessitates additional hardware for each new contractor. Our client was seeking a more cost-effective and scalable solution to streamline their search operations.
If these challenges continued, our client would face several potential consequences. There would be performance issues with slow search queries, leading to an unsatisfactory user experience. The surge in search traffic during peak periods could overwhelm their infrastructure, resulting in service disruptions or downtime. Costly and inefficient search processes, along with the burden of maintaining and scaling their on-premises server, would strain their financial resources. The limited scalability would impede their ability to meet growing demands effectively, resulting in delays and missed growth opportunities.
The JG Solution:
One of the key components of the solution was the implementation of AWS OpenSearch, a fully managed search and analytics service that served as an open-source search and analytics engine. By migrating to OpenSearch, our client was able to eliminate the need for physical hardware and reduce associated costs. OpenSearch also provided scalability and flexibility, allowing our client to easily add new contractors by adjusting the configuration settings. This scalability eliminated the necessity of investing in new servers or hardware, resulting in a more cost-effective solution.
To simplify the codebase and seamlessly incorporate their existing code with OpenSearch's powerful search capabilities, the Jahnel Group team integrated AWS OpenSearch. This integration enhanced the solution's search operations, optimizing the client's search functionality while providing scalability and operational efficiency.
The Jahnel Group team incorporated AWS EMR (Elastic MapReduce) and AWS Lambda to further enhance the solution. AWS EMR facilitated efficient processing and analysis of large datasets, optimizing the data management process. AWS Lambda, on the other hand, enabled the automation of functions in response to events, streamlining the data pipeline and reducing manual effort.
We also used a Kubernetes CRON Job template to process delta reloads for indices. We accessed OpenSearch and Kibana securely through a Bastion host, using SSH keys and port forwarding. System monitoring and metrics were gathered from OpenSearch using Datadog, with customized dashboards and alerts set up.
Results and Benefits:
The implementation of AWS OpenSearch brought significant cost savings for our client. They no longer needed to invest tens of thousands of dollars in new hardware per new contract. With OpenSearch, they could modify the cluster configuration, resulting in a cost reduction of a few hundred dollars per contract. The transition to OpenSearch also eliminated the requirement for a dedicated team to manage on-premises infrastructure, saving valuable time and resources.
The improved scalability of OpenSearch proved beneficial for our client. It enabled them to handle increasing data volumes and new contracts without significant hardware upgrades. The solution delivered faster and more efficient search performance, ensuring that the client's search operations could keep up with their growing needs.
To further enhance the system, our client implemented several improvements. By increasing the size of instances and optimizing the sharding strategy, they experienced improved search performance. The scalability of the client's infrastructure was also improved, allowing it to handle fluctuating search traffic effectively. The appropriate number of nodes and optimized configurations ensured smooth operation even during periods of high traffic.
Efficient index management was achieved by integrating Kubernetes CRON Jobs and delta reload processes. This streamlined the management of indices, enabling easy data manipulation in OpenSearch.
Why the Client Chose Jahnel Group:
Jahnel Group's expertise in implementing AWS OpenSearch made them the preferred choice for our client. We offered a scalable and cost-effective solution that eliminated the need for physical hardware. The Jahnel Group team showcased our technical proficiency by seamlessly integrating the client's existing code with OpenSearch's search capabilities, simplifying the transition process.
Jahnel Group excels in streamlining the transformation of customer concepts into fully launched digital products. With Jahnel Group’s established process and dedicated development team, we can supplement existing projects and offer valuable assistance throughout the entire development process.
About Jahnel Group
Jahnel Group is a consulting firm that specializes in helping companies leverage technology to improve their business operations. We provide end-to-end strategic consulting and deployment services, helping companies optimize their operations and reduce costs through the use of technology. We are an Advanced Tier AWS Services Partner, with expertise in AWS OpenSearch, AWS Lambdas, Amazon API Gateway, Amazon DynamoDB, and other AWS services.
