Use Case Compatibility

[sreekanthedayar]

First of all, iam so happy to see a search engine that is well suited for mysql and like a database itself. Kudos to the team who built this, though it's not surfaced anywhere when I search on internet.

We are a small SaaS startup that focus on collaboration segment with a little difference. Our primary USP is collaboration on Emails, Messages, Documents and ability to publish web content via API (Headless). Yes its a mini Sharepoint that we almost built and launching in another month. we had left some features that can be added later, one such thing is Search. We are now evaluating Meilisearch, elastic and Lucene.
We had previous experience working with Lucene, but it doesnt scale well with our backend (especially we have to heavy lift ourself if we require high availability or distributed search). Meilisearch's RAM based storage will not scale well without future requirement and will become a bottlneck as DMS is not similar to websearch or ecom search. Our final candidate was elastic search. But thinking about the heavylifting it need to configure, it's a big turn off for us, we operates with a lean team of 5 with 1 in devops. Then suddenly i came to know about manticore from a blog https://prabhatsharma.in/blog/in-search-of-a-search-engine-beyond-elasticsearch-introducing-zinc/ After going though the docs, it seems a perfect fit for our use case. But before doing anything, i just want to make sure following things
we use Dotnet 6/8 and MySQL as DB with a couple of microservices.

1. Will there be a paid support if we require anything critical in future. This is very important as we are a SaaS, and if we come to a deadlock with anything, will there be people to support (ofcourse on paid consulting). I had saw in website about the support plans, but i just want to make sure when we go in production about this.
2. How well this scale to fit our document search requirement, We store all our documents in Cloudflare R2 (s3 compatible storage engine) now, but we plan to built a small webapi for storing all files in a mysql Db, so that it can be ingested into search. I also saw about document storage feature here, in this case, can we avoid using mysql to store documents just for the purpose of search alone and use this fetaure to store document itself and search?
3. We use mysql connector for working with DB from our C#, how well matured is C# library? Will this fully support data ingestion and search? If not, can we use mysql connector for all operations?
4. Does this support HA? I didnt find any docs relate to this other than mentioned somewhere that you support HA with loadbalancing inbuilt. I saw distributed table and is that how it's implemented?
5. Does manticore support sharding, if we require to store some enterprise tenants on a separate shards, does manticore support sharding and routing inside or do we need to built our own logic for routing?
6. Finally, We plan to use the row wise storage, in this model, do we come to a deadlock with Ram restriction of modern system? do we need to use columnar approach so that there is no restriction on the storage? We don't need real time results as its like a documents, a little bit more latency is not an issue for us.

Again a big kudos for the team behind this.

[sanikolaev]

Hi @sreekanthedayar

Will there be a paid support if we require anything critical in future. This is very important as we are a SaaS, and if we come to a deadlock with anything, will there be people to support (ofcourse on paid consulting). I had saw in website about the support plans, but i just want to make sure when we go in production about this.

Yes - https://manticoresearch.com/services

How well this scale to fit our document search requirement, We store all our documents in Cloudflare R2 (s3 compatible storage engine) now, but we plan to built a small webapi for storing all files in a mysql Db, so that it can be ingested into search. I also saw about document storage feature here, in this case, can we avoid using mysql to store documents just for the purpose of search alone and use this fetaure to store document itself and search?

If you're only using MySQL as intermediate storage, you don’t need it. You can write directly to Manticore and store the data there.

We use mysql connector for working with DB from our C#, how well matured is C# library? Will this fully support data ingestion and search? If not, can we use mysql connector for all operations?

Yes, you can use the MySQL connector. The C# library was added about a year ago, so it's one of the newest compared to libraries for other languages. It's also worth noting that, like the other libraries, it's auto-generated using the OpenAPI generator.

Does this support HA? I didnt find any docs relate to this other than mentioned somewhere that you support HA with loadbalancing inbuilt. I saw distributed table and is that how it's implemented?

Yes - https://manual.manticoresearch.com/Creating_a_cluster/Remote_nodes/Mirroring#Mirroring

Does manticore support sharding, if we require to store some enterprise tenants on a separate shards, does manticore support sharding and routing inside or do we need to built our own logic for routing?

Yes - https://manual.manticoresearch.com/Creating_a_table/Creating_a_distributed_table/Creating_a_distributed_table#Creating-a-distributed-table

Fully-automatic sharding is in progress - https://roadmap.mnt.cr/

Finally, We plan to use the row wise storage, in this model, do we come to a deadlock with Ram restriction of modern system? do we need to use columnar approach so that there is no restriction on the storage? We don't need real time results as its like a documents, a little bit more latency is not an issue for us.

It depends on your schema, but generally, if a bit of extra latency is acceptable and you won't be using in-place updates, you can use columnar storage for your attributes.

[sreekanthedayar]

Thank you for your detailed response. I'm going through the documentation to get some basic concepts and how to do things. Will get back once we start implementation.