Elasticsearch provides a full Query DSL (Domain Specific Language) based on JSON to define queries. Query clauses behave differently depending on whether they are used in query context or filter context.
Query context
A query clause used in query context works on the principle of relevancy score of documents and it answers the question “How well does this document match this query clause?”. It can be viewed as lists all relevant document order by relevancy score. Relevancy score is formalised in terms of
_score computed by query clause in query context. "_score" represents how well the document matches, relative to other documents.
Query context is in effect whenever a query clause is passed to a query parameter, such as the query parameter in the search API. Blow query with query context returns all document where course description matches word science.
GET /courses/_search
{
"query": {
"match": {
"course_description": "science"
}
}
}
Filter context
Filter clause can be viewed as binary tool with outcome 0/1. A query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No .
Filter context is mostly used for filtering structured data, e.g. Range query(date in given range), status check, etc. Frequently used filters will be cached automatically by Elasticsearch, to speed up performance.
Filter context is in effect whenever a query clause is passed to a filter parameter, such as the filter or must_not parameters in the bool query, the filter parameter in the constant_score query, or the filter aggregation. Below query with filter context returns all course document where student enrolled >=33.
GET /courses/_search
{
"query": {
"bool": {
"filter": {
"range": { "students_enrolled": { "gte": 33 }}
}
}
}
}
Note: The fundamental difference between Query and Filter context is that - query context is associated with _score(relevancy score) however filter context is associated with binary outcome(either True or False)
Download
this document and Index in elastic search for illustrating concept of Query and Filter context. Once all documents are indexed, verify with GET command and it should list 10 documents.
Query context and filter context query samples:
1. Only query context : It gives two document with _score value based on doc relevancy.
GET /courses/_search
{
"query": {
"match": {
"course_description": "science"
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.1380306,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "4",
"_score": 1.1380306,
"_source": {
"name": "Computer Science 101",
"room": "C12",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "full-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2013-08-27",
"course_description": "CS 101 is a first year computer science introduction teaching fundamental data structures and alogirthms using python. "
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "7",
"_score": 0.6727604,
"_source": {
"name": "Computer Internals 250",
"room": "C8",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "part-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2012-08-20",
"course_description": "cpt Int 250 gives students an integrated and rigorous picture of applied computer science, as it comes to play in the construction of a simple yet powerful computer system. "
}
}
]
}
}
2. Only query context with place holder for filter: Multiple match clause with bool, here filter parameters is empty. The filter parameter indicates filter context. It returns
4 documents as output with score relevance. Here
bool query combines one or moreboolean clauses each clause with a typed occurrence.
GET /courses/_search
{
"query": {
"bool": {
"must": [
{ "match": { "professor.facutly_type": "full-time" }},
{ "match": { "professor.department": "finance" }}
],
"filter": [
]
}
}
}
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 1.1143606,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "2",
"_score": 1.1143606,
"_source": {
"name": "Marketing 101",
"room": "E4",
"professor": {
"name": "William Smith",
"department": "finance",
"facutly_type": "part-time",
"email": "wills@onuni.com"
},
"students_enrolled": 18,
"course_publish_date": "2015-06-21",
"course_description": "Mkt 101 is a course from the business school on the introduction to marketing that teaches students the fundamentals of market analysis, customer retention and online advertisements"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "1",
"_score": 0.87546873,
"_source": {
"name": "Accounting 101",
"room": "E3",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 27,
"course_publish_date": "2015-01-19",
"course_description": "Act 101 is a course from the business school on the introduction to accounting that teaches students how to read and compose basic financial statements"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "9",
"_score": 0.7985077,
"_source": {
"name": "Tax Accounting 200",
"room": "E7",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 17,
"course_publish_date": "2016-06-15",
"course_description": "Tax Act 200 is an intermediate course covering various aspects of tax law"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "10",
"_score": 0.7985077,
"_source": {
"name": "Capital Markets 350",
"room": "E3",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 13,
"course_publish_date": "2016-01-11",
"course_description": "This is an advanced course teaching crucial topics related to raising capital and bonds, shares and other long-term equity and debt financial instrucments"
}
}
]
}
}
3. Query context with filter: Add filter criteria in above query. Range filters remove one document and returns 3 document in response.
GET /courses/_search
{
"query": {
"bool": {
"must": [
{ "match": { "professor.facutly_type": "full-time" }},
{ "match": { "professor.department": "finance" }}
],
"filter": [
{ "range": { "students_enrolled": { "gte": 16 }}}
]
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 3,
"max_score": 1.1143606,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "2",
"_score": 1.1143606,
"_source": {
"name": "Marketing 101",
"room": "E4",
"professor": {
"name": "William Smith",
"department": "finance",
"facutly_type": "part-time",
"email": "wills@onuni.com"
},
"students_enrolled": 18,
"course_publish_date": "2015-06-21",
"course_description": "Mkt 101 is a course from the business school on the introduction to marketing that teaches students the fundamentals of market analysis, customer retention and online advertisements"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "1",
"_score": 0.87546873,
"_source": {
"name": "Accounting 101",
"room": "E3",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 27,
"course_publish_date": "2015-01-19",
"course_description": "Act 101 is a course from the business school on the introduction to accounting that teaches students how to read and compose basic financial statements"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "9",
"_score": 0.7985077,
"_source": {
"name": "Tax Accounting 200",
"room": "E7",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 17,
"course_publish_date": "2016-06-15",
"course_description": "Tax Act 200 is an intermediate course covering various aspects of tax law"
}
}
]
}
}
4. Using must_not clause : must_not clause reduces response document count to 1.
GET /courses/_search
{
"query": {
"bool": {
"must": [
{ "match": { "professor.facutly_type": "full-time" }},
{ "match": { "professor.department": "finance" }}
],
"must_not": [
{ "match": { "course_description": "business" }}
],
"filter": [
{ "range": { "students_enrolled": { "gte": 16 }}}
]
}
}
}
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.7985077,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "9",
"_score": 0.7985077,
"_source": {
"name": "Tax Accounting 200",
"room": "E7",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 17,
"course_publish_date": "2016-06-15",
"course_description": "Tax Act 200 is an intermediate course covering various aspects of tax law"
}
}
]
}
}
5. Using multi_match :
GET /courses/_search
{
"query": {
"multi_match": {
"query": "computer",
"fields": ["name","professor.department"]
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 0.9331132,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "4",
"_score": 0.9331132,
"_source": {
"name": "Computer Science 101",
"room": "C12",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "full-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2013-08-27",
"course_description": "CS 101 is a first year computer science introduction teaching fundamental data structures and alogirthms using python. "
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "7",
"_score": 0.64072424,
"_source": {
"name": "Computer Internals 250",
"room": "C8",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "part-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2012-08-20",
"course_description": "cpt Int 250 gives students an integrated and rigorous picture of applied computer science, as it comes to play in the construction of a simple yet powerful computer system. "
}
}
]
}
}
6. Using multi_phrase: It requires complete and valid phase for search. Partial or broken phrase will result empty documents.
GET /courses/_search
{
"query": {
"match_phrase": {
"course_description": "computer science introduction teaching"
}
}
}
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 3.959425,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "4",
"_score": 3.959425,
"_source": {
"name": "Computer Science 101",
"room": "C12",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "full-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2013-08-27",
"course_description": "CS 101 is a first year computer science introduction teaching fundamental data structures and alogirthms using python. "
}
}
]
}
}
7. Using match_phase_prefix: Partial or broken phase will also work out and returns documents in response.
GET /courses/_search
{
"query": {
"match_phrase_prefix": {
"course_description": "computer science"
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 4.5521226,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "4",
"_score": 4.5521226,
"_source": {
"name": "Computer Science 101",
"room": "C12",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "full-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2013-08-27",
"course_description": "CS 101 is a first year computer science introduction teaching fundamental data structures and alogirthms using python. "
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "7",
"_score": 2.6910417,
"_source": {
"name": "Computer Internals 250",
"room": "C8",
"professor": {
"name": "Gregg Payne",
"department": "engineering",
"facutly_type": "part-time",
"email": "payneg@onuni.com"
},
"students_enrolled": 33,
"course_publish_date": "2012-08-20",
"course_description": "cpt Int 250 gives students an integrated and rigorous picture of applied computer science, as it comes to play in the construction of a simple yet powerful computer system. "
}
}
]
}
}
8. Using range clause: gte stands for greater than equal and lte stands for less than equal. Other options are gt(greater than), lt (less than equal).
GET /courses/_search
{
"query": {
"range": {
"students_enrolled": {
"gte": 20,
"lte": 30
}
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "1",
"_score": 1,
"_source": {
"name": "Accounting 101",
"room": "E3",
"professor": {
"name": "Thomas Baszo",
"department": "finance",
"facutly_type": "part-time",
"email": "baszot@onuni.com"
},
"students_enrolled": 27,
"course_publish_date": "2015-01-19",
"course_description": "Act 101 is a course from the business school on the introduction to accounting that teaches students how to read and compose basic financial statements"
}
},
{
"_index": "courses",
"_type": "classroom",
"_id": "3",
"_score": 1,
"_source": {
"name": "Anthropology 230",
"room": "G11",
"professor": {
"name": "Devin Cranford",
"department": "history",
"facutly_type": "full-time",
"email": "devinc@onuni.com"
},
"students_enrolled": 22,
"course_publish_date": "2013-08-27",
"course_description": "Ant 230 is an intermediate course on human societies and cultures and their development. A focus on the Mayans civilization is rooted in this course"
}
}
]
}
}
9. Using Should clause : Should clause is good to have when we need most relevant document. If we remove "minimum_should_match" clause then we get multiple documents along with score. But when want to retrieve most relevant doc use "minimum_should_match".
GET /courses/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "accounting"
}
}
],
"must_not": [
{
"match": {
"room": "e7"
}
}
],
"should": [
{
"range": {
"students_enrolled": {
"gte": 10,
"lte": 20
}
}
}
],
"minimum_should_match": 1
}
}
}
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.6099695,
"hits": [
{
"_index": "courses",
"_type": "classroom",
"_id": "8",
"_score": 1.6099695,
"_source": {
"name": "Accounting Info Systems 350",
"room": "E3",
"professor": {
"name": "Bill Cage",
"department": "accounting",
"facutly_type": "full-time",
"email": "cageb@onuni.com"
},
"students_enrolled": 19,
"course_publish_date": "2014-05-15",
"course_description": "Act Sys 350 is an advanced course providing students a practical understanding of an accounting system in database technology. Students will use MS Access to build a transaction ledger system"
}
}
]
}
}
Fabulous post admin, it was too good and helpful. Waiting for more updates.
ReplyDeleteTally course in Chennai
Tally Training in Chennai
Tally institute in Chennai
AngularJS course in Chennai
ccna Training in Chennai
PHP course in Chennai
Salesforce course in Chennai
Tally Course in Velachery
Tally Course in Anna Nagar
Tally Course in T Nagar
Can you share the JSON documents or commands so that I can try these queries. Regards
ReplyDeleteThank for this blog are more informative step by step and useful contents. I here by attached my site would you see this blog
ReplyDelete7 tips to start a career in digital marketing
“Digital marketing is the marketing of product or service using digital technologies, mainly on the Internet, but also including mobile phones, display advertising, and any other digital medium”. This is the definition that you would get when you search for the term “Digital marketing” in google. Let’s give out a simpler explanation by saying, “the form of marketing, using the internet and technologies like phones, computer etc”.
we have offered to the advanced syllabus course digital marketing for available join now.
more details click the link now
https://www.webdschool.com/digital-marketing-course-in-chennai.html
Learn about Digital Marketing Courses. What Is Digital Marketing and How You Can Use It For Your Business. Please visit our website to know more information about our courses. Please visit our website to know more information.
ReplyDeletehttps://onlineidealab.com/digital-marketing-courses-in-bangalore/