Sep 9, 2018

Elasticsearch Query DSL : Query context and Filter context. (Elasticsearch and Kibana Devtool)

                6 comments   
Elasticsearch provides a full Query DSL (Domain Specific Language) based on JSON to define queries. Query clauses behave differently depending on whether they are used in query context or filter context.

Query context

A query clause used in query context works on the principle of relevancy score of documents and it answers the question “How well does this document match this query clause?”. It can be viewed as lists all relevant document order by relevancy score. Relevancy score is formalised in terms of _score computed by query clause in query context. "_score" represents how well the document matches, relative to other documents.
Query context is in effect whenever a query clause is passed to a query parameter, such as the query parameter in the search API. Blow query with query context returns all document where course description matches word science.
GET /courses/_search
{
  "query": {
    "match": { 
      "course_description": "science" 
    }
  }
}

Filter context

Filter clause can be viewed as binary tool with outcome 0/1. A query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No .
Filter context is mostly used for filtering structured data, e.g. Range query(date in given range), status check, etc. Frequently used filters will be cached automatically by Elasticsearch, to speed up performance.
Filter context is in effect whenever a query clause is passed to a filter parameter, such as the filter or must_not parameters in the bool query, the filter parameter in the constant_score query, or the filter aggregation. Below query with filter context returns all course document where student enrolled  >=33.
GET /courses/_search
{
  "query": {
    "bool": {
      "filter": {
          "range":  { "students_enrolled": { "gte": 33 }}
        }
      
    }
  }
}
Note: The fundamental difference between Query and Filter context is that - query context is associated with _score(relevancy score) however filter context is associated with binary outcome(either True or False)

Download this document and Index in elastic search for illustrating concept of Query and Filter context. Once all documents are indexed, verify with GET command and it should list 10 documents.

Query context and filter context query samples:

1. Only query context : It gives two document with _score value based on doc relevancy. 
GET /courses/_search
{
  "query": {
    
    "match": { 
      "course_description": "science" 
    }
  }
}
Show response

2. Only query context with place holder for filter: Multiple match clause with bool, here filter parameters is empty. The filter parameter indicates filter context. It returns 4 documents as output with score relevance. Here bool query combines one or moreboolean clauses each clause with a typed occurrence. 
GET /courses/_search
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "professor.facutly_type": "full-time" }},
        { "match": { "professor.department": "finance" }}
      ],
      "filter": [ 
         
      ]
    }
  }
}
Show response

3. Query context with filter: Add filter criteria in above query. Range filters remove one document and returns 3 document in response.
GET /courses/_search
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "professor.facutly_type": "full-time" }},
        { "match": { "professor.department": "finance" }}
      ],
      "filter": [ 
         { "range":  { "students_enrolled": { "gte": 16 }}}
      ]
    }
  }
}
Show response

4. Using must_not clause : must_not clause reduces response document count to 1.
GET /courses/_search
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "professor.facutly_type": "full-time" }},
        { "match": { "professor.department": "finance" }}
      ],
      "must_not": [
        { "match": { "course_description": "business" }}
      ], 
      "filter": [ 
         { "range":  { "students_enrolled": { "gte": 16 }}}
      ]
    }
  }
}
Show response
5. Using multi_match :
GET /courses/_search
{
  "query": {
    "multi_match": {
      "query": "computer",
      "fields": ["name","professor.department"]
    }
  }
}
Show response
6. Using multi_phrase: It requires complete and valid phase for search. Partial or broken phrase will result empty documents.
GET /courses/_search
{
  "query": {
    "match_phrase": {
      "course_description": "computer science introduction teaching"
    }
  }
}
Show response
7. Using match_phase_prefix: Partial or broken phase will also work out and returns documents in response.
GET /courses/_search
{
  "query": {
    "match_phrase_prefix": {
      "course_description": "computer science"
    }
  }
}
Show response
8. Using range clause: gte stands for greater than equal and lte stands for less than equal. Other options are gt(greater than), lt (less than equal).
GET /courses/_search
{
  "query": {
    "range": {
      "students_enrolled": {
        "gte": 20,
        "lte": 30
      }
    }
  }
}
Show response
9. Using Should clause : Should clause is good to have when we need most relevant document. If we remove "minimum_should_match" clause then we get multiple documents along with score. But when want to retrieve most relevant doc use "minimum_should_match".
GET /courses/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "name": "accounting"
          }
        }
      ],
      "must_not": [
        {
          "match": {
            "room": "e7"
          }
        }
      ],
      "should": [
        {
          "range": {
            "students_enrolled": {
              "gte": 10,
              "lte": 20
            }
          }
        }
      ],
      "minimum_should_match": 1
    }
  }
}
Show response
Location: Bengaluru, Karnataka, India

6 comments:

  1. RakeshAug 31, 2019, 1:52:00 AM

    Fabulous post admin, it was too good and helpful. Waiting for more updates.
    Tally course in Chennai
    Tally Training in Chennai
    Tally institute in Chennai
    AngularJS course in Chennai
    ccna Training in Chennai
    PHP course in Chennai
    Salesforce course in Chennai
    Tally Course in Velachery
    Tally Course in Anna Nagar
    Tally Course in T Nagar

    ReplyDelete
  2. SameersrolOct 17, 2019, 1:30:00 AM

    Can you share the JSON documents or commands so that I can try these queries. Regards

    ReplyDelete
  3. UnknownFeb 27, 2020, 3:46:00 AM

    Thank for this blog are more informative step by step and useful contents. I here by attached my site would you see this blog

    7 tips to start a career in digital marketing

    “Digital marketing is the marketing of product or service using digital technologies, mainly on the Internet, but also including mobile phones, display advertising, and any other digital medium”. This is the definition that you would get when you search for the term “Digital marketing” in google. Let’s give out a simpler explanation by saying, “the form of marketing, using the internet and technologies like phones, computer etc”.

    we have offered to the advanced syllabus course digital marketing for available join now.

    more details click the link now

    https://www.webdschool.com/digital-marketing-course-in-chennai.html

    ReplyDelete
  4. onlineidealabMay 29, 2020, 3:38:00 AM

    Learn about Digital Marketing Courses. What Is Digital Marketing and How You Can Use It For Your Business. Please visit our website to know more information about our courses. Please visit our website to know more information.
    https://onlineidealab.com/digital-marketing-courses-in-bangalore/

    ReplyDelete
  5. vé máy bay đi CanadaMay 31, 2021, 8:32:00 PM

    Aivivu chuyên vé máy bay, tham khảo

    mua ve may bay di my

    giá vé máy bay từ mỹ về vn

    giá vé máy bay từ canada về Việt Nam

    taxi từ sân bay nội bài về hà nội

    ReplyDelete
  6. InnomaticsSep 2, 2021, 5:50:00 AM

    Data science course with placements in hyderabad
    we provide classroom training on IBM certified Data Science at hyderabad for the individuals who believe hand-held training. We teach as per the Indian Standard Time (IST) with In-depth practical Knowledge on each topic in classroom training, 80 – 90 Hrs of Real-time practical training classes. There are different slots available on weekends or weekdays according to your choices.

    ReplyDelete