generated from adobe/aem-boilerplate
-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat/search improvements #304
Closed
Closed
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
d0818e9
add promotion banner block
kronnox f76e703
improve image fetching
kronnox 6c37c33
Merge branch 'issue/183'
kailasnadh790 e9c32bc
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 a484646
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 9115e7e
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 169baff
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 3451040
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 ce3d2d6
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 3ae7a14
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 99fa335
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 03e1b88
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 d48632f
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 3c77d98
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 0665a23
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 121728a
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 22c7e67
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 1c367d3
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 a4e0c78
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 c735268
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 f140823
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 693767c
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 69ee8f1
Update search-results.js
kailasnadh790 a781465
Update search-results.js
kailasnadh790 3b88716
search improvement
kailasnadh790 e2114b0
adding blurb
kailasnadh790 6889c96
Improve search prioritization
bstopp aa06ead
Update search-results.js
kailasnadh790 c156e5e
Update search-results.js
kailasnadh790 c92f27b
Update search-results.js
kailasnadh790 4772689
Update search-results.js
kailasnadh790 c2477b0
Update search-results.js
kailasnadh790 9bd6f8f
MOre improvements.
bstopp 8e1df01
Update search-results.js
kailasnadh790 ac795ee
Update search-results.js
kailasnadh790 be39a49
Update search-results.js
kailasnadh790 b84a347
Update search-results.js
kailasnadh790 3b73f8b
loader
kailasnadh790 ac16696
Update search-results.js
kailasnadh790 4be8af4
Update search-results.js
kailasnadh790 e32524a
Update search-results.js
kailasnadh790 2ec9db8
workers
kailasnadh790 bb98a10
formatting
kailasnadh790 669ce04
Update helix-query.yaml
kailasnadh790 dca321d
Update hero.js
kailasnadh790 f05d34a
flicker fix
kailasnadh790 2200b12
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 acffe40
Merge branch 'main' of https://github.com/hlxsites/famous-smoke-cigar…
kailasnadh790 5e311d2
Update hero.js
kailasnadh790 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,156 @@ | ||
/* eslint-disable max-len */ | ||
/* eslint-disable no-restricted-globals */ | ||
const ARTICLE_INDEX_PATH = '/cigaradvisor/index/article-index.json'; | ||
const SEARCH_INDEX_PATH = '/cigaradvisor/index/search-index.json'; | ||
const articleIndexData = []; | ||
|
||
const searchIndexData = []; | ||
/** | ||
* Retrieves search index data from the server. | ||
* @returns {Promise<Object>} The search index data. | ||
*/ | ||
async function getSearchIndexData(path = SEARCH_INDEX_PATH, flag = false) { | ||
if (searchIndexData.length === 0 || flag) { | ||
const resp = await fetch(path); | ||
let jsonData = ''; | ||
if (resp.ok) { | ||
jsonData = await resp.json(); | ||
} | ||
jsonData.data.forEach((a) => { | ||
searchIndexData.push({ ...a }); | ||
}); | ||
// If there are more items to load, load them | ||
if ((jsonData.total - jsonData.offset) > jsonData.limit) { | ||
const offset = jsonData.offset + jsonData.limit; | ||
const indexPath = `${SEARCH_INDEX_PATH}?offset=${offset}&limit=${jsonData.total - offset}`; | ||
await getSearchIndexData(indexPath, true); | ||
} | ||
} | ||
// Protected against callers modifying the objects | ||
const ret = []; | ||
if (searchIndexData) { | ||
searchIndexData.forEach((a) => { | ||
ret.push({ ...a }); | ||
}); | ||
} | ||
return ret; | ||
} | ||
|
||
/** | ||
* Loads posts from the specified path asynchronously. | ||
* @param {string} path - The path to fetch the posts from. | ||
* @param {boolean} recurse - Indicates whether to recursively load more articles. | ||
* @returns {Promise<Array<Object>>} - A promise that resolves to an array of post objects. | ||
*/ | ||
async function loadPosts(path = ARTICLE_INDEX_PATH, recurse = false) { | ||
if (articleIndexData.length === 0 || recurse) { | ||
const resp = await fetch(path); | ||
let jsonData = ''; | ||
if (resp.ok) { | ||
jsonData = await resp.json(); | ||
} | ||
jsonData.data.forEach((a) => { | ||
articleIndexData.push({ ...a }); | ||
}); | ||
// If there are more articles to load, load them | ||
if ((jsonData.total - jsonData.offset) > jsonData.limit) { | ||
const offset = jsonData.offset + jsonData.limit; | ||
const indexPath = `${ARTICLE_INDEX_PATH}?offset=${offset}&limit=${jsonData.total - offset}`; | ||
await loadPosts(indexPath, true); | ||
} | ||
} | ||
// Protected against callers modifying the objects | ||
const ret = []; | ||
if (articleIndexData) { | ||
articleIndexData.forEach((a) => { | ||
ret.push({ ...a }); | ||
}); | ||
} | ||
return ret; | ||
} | ||
|
||
const IGNORED_TERMS = []; | ||
|
||
const doMatch = (property, term) => { | ||
const regex = new RegExp(term, 'gi'); | ||
if (property) { | ||
return property.match(regex); | ||
} | ||
return false; | ||
}; | ||
|
||
function filterData(fullTerm, data) { | ||
const searchTokens = []; | ||
searchTokens.push(fullTerm); | ||
|
||
searchTokens.push(...fullTerm.toLowerCase().split(/\s+/).filter((term) => term && term.length > 2 && term !== fullTerm.toLowerCase())); | ||
|
||
// Object | ||
// { | ||
// priority: Number | ||
// article: article | ||
// count: Number | ||
// } | ||
const results = []; | ||
data.forEach((result) => { | ||
const found = { | ||
article: result, | ||
count: 0, | ||
}; | ||
|
||
searchTokens.forEach((token) => { | ||
if (IGNORED_TERMS.includes(token.toLowerCase().trim())) return; | ||
// eslint-disable-next-line no-param-reassign | ||
if (token.endsWith('s')) token = token.substring(0, token.length - 1); // Handle potential pluralization of token. | ||
|
||
if (doMatch(result.title, token)) { | ||
found.rank ||= 1; | ||
found.count += 1; | ||
} | ||
if (doMatch(result.heading, token)) { | ||
found.rank ||= 2; | ||
found.count += 1; | ||
} | ||
if (doMatch(result.description, token)) { | ||
found.rank ||= 3; | ||
found.count += 1; | ||
} | ||
if (doMatch(result.blurb, token)) { | ||
found.rank ||= 4; | ||
found.count += 1; | ||
} | ||
if (doMatch(result.text, token)) { | ||
found.rank ||= 5; | ||
found.count += 1; | ||
} | ||
}); | ||
|
||
if (found.count > 0) { | ||
results.push(found); | ||
} | ||
}); | ||
|
||
return results.sort((l, r) => { | ||
if (l.rank < r.rank) { | ||
return -1; | ||
} | ||
if (l.rank === r.rank) { | ||
if (l.count > r.count) { | ||
return -1; | ||
} | ||
if (l.count < r.count) { | ||
return 1; | ||
} | ||
return 0; | ||
} | ||
return 1; // Left rank is greater than right rank - move it down the list. | ||
}).map((r) => r.article); | ||
} | ||
|
||
self.onmessage = async function handleSearch(event) { | ||
const data = await getSearchIndexData(); | ||
const allArticles = await loadPosts(); | ||
const { searchValue } = event.data; | ||
const searchResults = filterData(searchValue, data); | ||
self.postMessage({ results: JSON.stringify(searchResults), articles: JSON.stringify(allArticles) }); | ||
}; |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this will ever be populated? Can you run some tests and see if this keeps data between calls?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bstopp you are correct...this is not getting cached between calls as we are fetching in worker... basically negating little perf improvement we could get by window.history.pushState and not reloading the page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the worker did the fetch and returned all the results. after that we don't have to actually run a search again per-page, we just change the page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bstopp But I feel that would reset all the performance improvements we achieved...mapping entire list of search results with article-index was taking time. So, it would slow the search on first load.
Can we use Service Workers and Caching API ? Reading about them now...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, current performance is not bad as well even without this caching...these requests are cached on CDN rit?