Lädt...


🔧 Optimizing Search Performance with Elastic Search's Profile API


Nachrichtenbereich: 🔧 Programmierung
🔗 Quelle: dev.to

Recently, a production application faced significant delays in query times, with some searches taking up to 60 seconds. The application utilizes Elastic Search for customer searches, but as the database has grown over the years, the performance has suffered. This article explores how the Profile API in Elastic Search can be leveraged to diagnose and improve query performance.

Key Takeaways

  • Elastic Search's Profile API provides detailed insights into query performance.
  • Limiting the number of fields in a query can significantly reduce search times.
  • Avoiding wildcard searches can enhance query efficiency.

Understanding the Problem

The initial query used for searching customer data was written a decade ago and was designed to search across all fields in the customer index. As the number of documents increased, the time taken to execute these queries also grew, leading to frustrating delays for users.

The original query structure was as follows:

{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "Ryan*"
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must": [
              {
                "terms": {
                  "merchant_id": [2]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Utilizing the Profile API

To diagnose the slow query, the Profile API was employed by adding profile: true to the query. This modification allowed for a detailed breakdown of the query's execution, revealing how long each part took to process. The output indicated that the query was searching through an excessive number of fields, which contributed to the slowdown.

Optimizing the Query

To enhance performance, the number of searchable fields was limited. The revised query focused only on the fields relevant to the user interface, such as first name, last name, and email. The updated query structure looked like this:

{
  "profile": true,
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "Ryan*",
            "fields": [
              "first_name",
              "last_name",
              "email",
              "reference",
              "card_token",
              "card_number",
              "public_id"
            ]
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must": [
              {
                "terms": {
                  "merchant_id": [2]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

This change resulted in a dramatic reduction in query time, from several seconds to just a few milliseconds.

Further Improvements

The next step was to evaluate the necessity of using wildcard searches. By removing the wildcard from the query, the performance improved even further. The new query structure was:

{
  "profile": true,
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "Ryan",
            "fields": [
              "first_name",
              "last_name",
              "email",
              "reference",
              "card_token",
              "card_number",
              "public_id"
            ]
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must": [
              {
                "terms": {
                  "merchant_id": [2]
                }
              }
            ]
          }
        }
      ]
    }
  }
}

This adjustment led to query execution times in the milliseconds range, showcasing the efficiency of Elastic Search when configured correctly.

Conclusion

The experience highlighted the importance of using the Profile API for diagnosing slow queries in Elastic Search. By limiting the fields searched and avoiding wildcard queries, significant improvements in performance were achieved. This case serves as a valuable lesson for developers looking to optimize their search functionalities in large databases.

...

🔧 Optimizing Search Performance with Elastic Search's Profile API


📈 51.04 Punkte
🔧 Programmierung

🔧 Optimizing Search Performance with Elastic Search's Profile API


📈 51.04 Punkte
🔧 Programmierung

📰 Elastic Stack 7.3 bietet mit Elastic Maps ein Tool zur Geodatenanalyse


📈 24.2 Punkte
📰 IT Nachrichten

🔧 Managing Elastic IPs and Elastic Block Storage (EBS) in AWS - Week Eleven


📈 24.2 Punkte
🔧 Programmierung

📰 Elastic präsentiert Elastic Security for Cloud - Dev-Insider


📈 24.2 Punkte
📰 IT Security Nachrichten

📰 Suche: Der Elastic Agent bringt Daten auf neue Art in den Elastic Stack 7.9


📈 24.2 Punkte
📰 IT Nachrichten

🔧 Understanding and Optimizing AWS RDS Performance Using Performance Insights


📈 22.26 Punkte
🔧 Programmierung

🔧 Optimizing Cloud Performance: An In-Depth Guide to Cloud Performance Testing and its Benefits


📈 22.26 Punkte
🔧 Programmierung

🕵️ Cisco Elastic Services Controller REST API API Request weak authentication


📈 21.89 Punkte
🕵️ Sicherheitslücken

🔧 Mastering GraphQL Performance: 8 Expert Strategies for Optimizing Your API


📈 21.31 Punkte
🔧 Programmierung

🔧 Optimizing Performance: Efficient Use of Google's Geolocation API for High-Traffic Applications


📈 21.31 Punkte
🔧 Programmierung

🕵️ CVE-2023-49923 | Elastic Enterprise Search up to 7.17.15/8.11.1 Documents API log file


📈 21.3 Punkte
🕵️ Sicherheitslücken

🔧 Optimizing Vector Search Performance With Elasticsearch


📈 20.72 Punkte
🔧 Programmierung

🔧 Optimizing website performance with a Search Console bubble chart


📈 20.72 Punkte
🔧 Programmierung

📰 Elastic Enterprise Search: Giving users the tools to bring search experiences to market quickly


📈 20.72 Punkte
📰 IT Security Nachrichten

🕵️ GitLab: Group search with Elastic search enable leaks unrelated data


📈 20.72 Punkte
🕵️ Sicherheitslücken

🔧 Full-text search using MongoDB and Elastic Search


📈 20.72 Punkte
🔧 Programmierung

🔧 Elastic Search and Open Search - a brief history of the license war


📈 20.72 Punkte
🔧 Programmierung

🔧 MuleSoft API Gateway: Securing and Optimizing API Management


📈 20.36 Punkte
🔧 Programmierung

🔧 Seamless API Management: Optimizing Spring Cloud Gateway with AWS API Gateway


📈 20.36 Punkte
🔧 Programmierung

🔧 Optimizing Google Search via API: A Guide to Using Serper


📈 19.77 Punkte
🔧 Programmierung

🔧 Optimizing Machine Learning Models: Comparing Grid Search, Randomized Search, and Optuna


📈 19.18 Punkte
🔧 Programmierung

🔧 Unlocking the Power of Bing Search Engine API: Your Guide to a Free Search API


📈 18.41 Punkte
🔧 Programmierung

🕵️ GetSimple CMS 3.x Profile admin/profile.php name Cross Site Scripting


📈 18.02 Punkte
🕵️ Sicherheitslücken

🕵️ ImageMagick Profile magick/profile.c Denial of Service


📈 18.02 Punkte
🕵️ Sicherheitslücken

matomo