-
Notifications
You must be signed in to change notification settings - Fork 68
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## High Level Overview of Change In order to improve the usability of the search bar inside of Explorer, matching tokens will now auto-populate as a user is typing. The results are clickable and redirect to the respective token page. The data is pulled from XRPLMeta and cached inside of the Explorer backend to display an open sourced list of tokens and relevant statistics. Backend changes implement serverside caching of tokens data from XRPL Meta, and exposes an API from the explorer backend for search result querying. This is less expensive than the previous approach of websockets (which remain open for the duration of a user session) and are susceptible to rate limits on XRPLMeta's public node infrastructure. This also offers resilience in the case that the public XRPLMeta node has downtime. Desktop: <img width="906" alt="Screenshot 2024-11-05 at 4 36 33 PM" src="https://github.com/user-attachments/assets/8f19503d-74fc-474f-b946-534c028ce4a9"> Mobile: <img width="453" alt="Screenshot 2024-11-05 at 4 37 26 PM" src="https://github.com/user-attachments/assets/3ba5ba0d-1fe0-4bfd-97fe-a32f12420490"> <!-- Please include a summary/list of the changes. If too broad, please consider splitting into multiple PRs. --> ### Context of Change <!-- Please include the context of a change. If a bug fix, when was the bug introduced? What was the behavior? If a new feature, why was this architecture chosen? What were the alternatives? If a refactor, how is this better than the previous implementation? If there is a design document for this feature, please link it here. --> ### Type of Change <!-- Please check relevant options, delete irrelevant ones. --> - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Refactor (non-breaking change that only restructures code) - [ ] Tests (You added tests for code that already exists, or your new feature included in this PR) - [ ] Documentation Updates - [ ] Translation Updates - [ ] Release ### TypeScript/Hooks Update <!-- In an effort to modernize the codebase, you should convert the files that you work with to React Hooks and TypeScript. If this is not possible (e.g. it's too many changes, touching too many files, etc.) please explain why here. --> - [ ] Updated files to React Hooks - [ ] Updated files to TypeScript ## Before / After <!-- If just refactoring / back-end changes, this can be just an in-English description of the change at a technical level. If a UI change, screenshots should be included. --> ## Test Plan <!-- Please describe the tests that you ran to verify your changes and provide instructions so that others can reproduce. --> <!-- ## Future Tasks For future tasks related to PR. --> --------- Co-authored-by: Phu Pham <[email protected]> Co-authored-by: pdp2121 <[email protected]> Co-authored-by: Caleb Kniffen <[email protected]>
- Loading branch information
4 people
authored
Nov 6, 2024
1 parent
4cb0975
commit 5a93a6d
Showing
22 changed files
with
822 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| const axios = require('axios') | ||
| const log = require('../../lib/logger')({ name: 'tokens search' }) | ||
|
|
||
| const REFETCH_INTERVAL = 60 * 60 * 1000 // 1 hour | ||
| const XRPLMETA_QUERY_LIMIT = 1000 | ||
| const cachedTokenSearchList = { tokens: [], last_updated: null } | ||
|
|
||
| const parseCurrency = (currency) => { | ||
| const NON_STANDARD_CODE_LENGTH = 40 | ||
| const LP_TOKEN_IDENTIFIER = '03' | ||
|
|
||
| const hexToString = (hex) => { | ||
| let string = '' | ||
| for (let i = 0; i < hex.length; i += 2) { | ||
| const part = hex.substring(i, i + 2) | ||
| const code = parseInt(part, 16) | ||
| if (!isNaN(code) && code !== 0) { | ||
| string += String.fromCharCode(code) | ||
| } | ||
| } | ||
| return string | ||
| } | ||
|
|
||
| return currency.length === NON_STANDARD_CODE_LENGTH && | ||
| currency?.substring(0, 2) !== LP_TOKEN_IDENTIFIER | ||
| ? hexToString(currency) | ||
| : currency | ||
| } | ||
|
|
||
| async function fetchXRPLMetaTokens(offset) { | ||
| log.info(`caching tokens from ${process.env.XRPL_META_URL}`) | ||
| return axios | ||
| .get( | ||
| `https://${process.env.XRPL_META_URL}/tokens?trust_level=1&trust_level=2&trust_level=3`, | ||
| { | ||
| params: { | ||
| sort_by: 'holders', | ||
| offset, | ||
| limit: XRPLMETA_QUERY_LIMIT, | ||
| }, | ||
| }, | ||
| ) | ||
| .then((resp) => resp.data) | ||
| .catch((e) => log.error(e)) | ||
| } | ||
|
|
||
| async function cacheXRPLMetaTokens() { | ||
| let offset = 0 | ||
| let tokensDataBatch = [] | ||
| const allTokensFetched = [] | ||
|
|
||
| tokensDataBatch = await fetchXRPLMetaTokens(0) | ||
| const { count } = tokensDataBatch | ||
| while (offset < count) { | ||
| allTokensFetched.push(...tokensDataBatch.tokens) | ||
| offset += XRPLMETA_QUERY_LIMIT | ||
| // eslint-disable-next-line no-await-in-loop | ||
| tokensDataBatch = await fetchXRPLMetaTokens(offset) | ||
| } | ||
|
|
||
| cachedTokenSearchList.tokens = allTokensFetched.filter( | ||
| (result) => | ||
| result.metrics.trustlines > 50 && | ||
| result.metrics.holders > 50 && | ||
| result.metrics.marketcap > 0 && | ||
| result.metrics.volume_7d > 0, | ||
| ) | ||
| cachedTokenSearchList.last_updated = Date.now() | ||
|
|
||
| // nonstandard from XRPLMeta, check for hex codes in currencies and store parsed | ||
| cachedTokenSearchList.tokens.map((token) => ({ | ||
| ...token, | ||
| currency: parseCurrency(token.currency), | ||
| })) | ||
| } | ||
|
|
||
| function startCaching() { | ||
| if (process.env.VITE_ENVIRONMENT !== 'mainnet') { | ||
| return | ||
| } | ||
| cacheXRPLMetaTokens() | ||
| setInterval(() => cacheXRPLMetaTokens(), REFETCH_INTERVAL) | ||
| } | ||
|
|
||
| startCaching() | ||
|
|
||
| function queryTokens(tokenList, query) { | ||
| const sanitizedQuery = query.toLowerCase() | ||
|
|
||
| return tokenList.filter( | ||
| (token) => | ||
| token.currency?.toLowerCase().includes(sanitizedQuery) || | ||
| token.meta?.token?.name?.toLowerCase().includes(sanitizedQuery) || | ||
| token.meta?.issuer?.name?.toLowerCase().includes(sanitizedQuery) || | ||
| token.issuer?.toLowerCase().startsWith(sanitizedQuery), | ||
| ) | ||
| } | ||
|
|
||
| function sleep(ms) { | ||
| return new Promise((resolve) => setTimeout(resolve, ms)) | ||
| } | ||
|
|
||
| module.exports = async (req, res) => { | ||
| try { | ||
| log.info('getting tokens list for search') | ||
| const { query } = req.params | ||
| while (cachedTokenSearchList.tokens.length === 0) { | ||
| // eslint-disable-next-line no-await-in-loop -- necessary here to wait for cache to be filled | ||
| await sleep(1000) | ||
| } | ||
| const queriedTokens = await queryTokens(cachedTokenSearchList.tokens, query) | ||
| return res.status(200).json({ | ||
| result: 'success', | ||
| updated: cachedTokenSearchList.last_updated, | ||
| tokens: queriedTokens, | ||
| }) | ||
| } catch (error) { | ||
| log.error(error) | ||
| return res.status(error.code || 500).json({ message: error.message }) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.