FlexSearch v0.8 #415
Replies: 16 comments 15 replies
-
#99 This FR exists for years and has many upvotes. Would really like to see this feature. |
Beta Was this translation helpful? Give feedback.
-
@xoxys I've added a similar feature by some customiziation in a project. This is something which could be done by post-processing. Looks like we have a further candidate for a "resolver" 👍 |
Beta Was this translation helpful? Give feedback.
-
Is there a specific reason to use Web SQL over IndexedDB? Web SQL is deprecated in Blink, removed from WebKit and it was never implemented in Gecko. |
Beta Was this translation helpful? Give feedback.
-
@cutiful IndexedDB of course :) |
Beta Was this translation helpful? Give feedback.
-
When will this be released? :D |
Beta Was this translation helpful? Give feedback.
-
When you search documents for several fields you get a result like this [{
field: "title",
result: [0, 1, 2]
},{
field: "content",
result: [3, 4, 5]
}] Would be useful to get the score from each item. When you don't to separate the search results into two different lists in your UI then the results lists needs to be combined. Combining the list while keeping the scoring/ranking intact is also useful. |
Beta Was this translation helpful? Give feedback.
-
Regarding export/import improvements, I for one forked this project solely to remove the |
Beta Was this translation helpful? Give feedback.
-
Some feedback:
|
Beta Was this translation helpful? Give feedback.
-
Actually I dived into the core functionality of the index and it looks like there is some unlocked capability available by switching over to |
Beta Was this translation helpful? Give feedback.
-
Would be neat if there was EdgeDB support. https://www.edgedb.com/ |
Beta Was this translation helpful? Give feedback.
-
Will the new release include sorting? It was mentioned here but I haven't seen it in the recent documentation. |
Beta Was this translation helpful? Give feedback.
-
To have a Playground would be amazing! About the highlight. |
Beta Was this translation helpful? Give feedback.
-
would be great if it could support sync export/import, to be used on server side index preparation for static websites (I've added a flexsearch plugin for vuepress but failed to use the 0.7 version due to inability to export synchronously) |
Beta Was this translation helpful? Give feedback.
-
Any plans to support fuzzy search as discussed here #118 ? |
Beta Was this translation helpful? Give feedback.
-
I just discovered FlexSearch and was initially very impressed, but my next thought was about the lack of automatic persistence of the index to IndexedDB. I see that there is export/import, but it the current per-key implementation is surely far from ideal. So, I'm very glad to see that this will be the focus of the next version. I wonder if any of the following ideas might be possible?
I stumbled upon this convo from the devs of RxDB and SurrealDB, where the latter seems to say that they store the indexes as ArrayBuffer https://news.ycombinator.com/item?id=32554752 Similar ideas are here as well: |
Beta Was this translation helpful? Give feedback.
-
I just implemented the fulltext search plugin for RxDB, based on flexsearch. It persists the index and incremental updates to it. On page/app reloads it loads the stored index instead of fully reindexing everything. Here are my learning about what could be improved on the flexsearch side:
|
Beta Was this translation helpful? Give feedback.
-
Let's discuss the new version. Also I have some important new features I would like to show you.
FlexSearch Playground
Use a playground to interactively test different settings and compare them.
Feature: Storage Interface including Postgres, SQLite, MongoDB, Clickhouse + IndexedDB
The whole export/import will be improved. A new storage interface lets you use different storage technologies. Those will be provided as builtin:
The storage interface also will provide
enrich
the results from database objects.It's probably better to exclude those libraries from package.json. Alternatively each of the storage could be installed by the developer by just
npm install ...
. Let me know what you think.How we can add several storage adapter on server-side which are easy to install and use?
I would like to exclude all those DBMS dependencies. We could make a dedicated npm package for each of the storages. But it needs some additional configuration anyway (on server-side).
Simple (Non-Flexible)
Instead we can let the developer pass an authorized db client instance to the storage class provided by this library, then theoretically it just needs to specify the table name additionally for the destination. This sounds like a simple solution. But it might not work with all kinds of interface variants of the same storage engine. This solution needs to make a support for each of them.
Flexible (Non-Simple)
A different approach would be let the developer implement some kind of abstract class, where some very basic methods needs to be implement by giving a full working template for each storage DBMS to start with. Here it needs more configuration but also support all interface variants of the same storage engines.
We could provide both. What did you think?
Feature: Resolver
This feature will solve a couple of limitations of the current version. It is providing:
boost
of indexes/fields on searchThe resolve feature will add as opt-in, so older version does not need a migration. For this reason the automatic
resolve
needs either to be disabled or a full resolver should be passed here:The new version also supports document indexes by just using the class
Index
and passing adocument
:When you make a search you will get a non-resolved intermeditate result you can use to pass into post-processing:
You can also switch to raw results when not configured before:
FlexSearch will providing some methods you can use to resolve those results by applying custom logic.
The most simple one is the default resolve, which always runs under the hood when not specified:
Boost
Chain arguments or use an array.
Logical Expression
You can nest those expressions:
and
,or
,xor
andnot
Custom Functions
Custom functions could be used to determine score through a custom function:
This could be used in combination with logical expressions:
This could be also done directly by
.search(query, options)
:Feature: Virtual Fields/Tags
Actually you can just derive fields from data. To add dynamic contents it needs to apply this to the data. Virtual fields are indexes which rely on custom functions (which have access to the data). Also virtual fields could be used as tags.
Assume we have this data:
A document could be initialized as follows by applying a virtual field "location" and a virtual tag "status":
Virtual fields can override existing data fields.
Also the virtual field "location" was used here as a tag, just to demonstrate. But you can't access virtual fields on the
data
inside a virtual definition. Virtual fields will also not export to the data, so you might need to refresh them.Partial Updates
The new version is also providing making partial updates to specific indexes instead of re-indexing the whole data payload at all.
So
.update(<id>, data, fields)
is getting a new parameter to limit partial updates on specific fields without changing other fields:Asume you have indexed an event and just the location and date has changed. Then you can use partial data to update indexes without loosing contents from other fields when not included.
Result Highlighting
This will output something like:
You can also specify a regular expression which should be used to use custom wrappers for matches:
Or:
Beta Was this translation helpful? Give feedback.
All reactions