Is There An Actively Developed Open Source Crawler/Indexer for OpenSearch for Samba Shares? ( like FSCrawler and fs2es-Indexer for ElasticSearch)

[vepicurean]

Please describe the end goal of this project

There no clear method to use an external crawler application to create an index for OpenSearch on Samba shares ( so users on MacOS can search Samba shares via an index)? I've seen people asking about this for years. I tried to use fs2es-indexer with OpenSearch, but can't get it to work, and that fs2es-indexer project looks dormant. I had thought Open Search was a fork of Elasticsearch and would work with fs2es-indexer, but I can't get Opensearch to work with fs2es-indexer ( the project seems dormant for a while now).

https://wiki.archlinux.org/title/samba - see section 1.3.11 ( Enable Spotlight Searching )

Supporting References

This question about an indexer for Opensearch has been asked before... There's no clear solution ( actively maintained application) that I can see for Open Search for indexing files for search of Samba shares via Opensearch using MacOS Finder (Spotlight) on Samba shares.

https://github.com/Ellerhold/fs2es-indexer

Issues

See here... Ellerhold/fs2es-indexer#39

Related component

Indexing

[peternied]

[Triage - attendees 1 2 3 4 5]
@vepicurean Thanks for asking about this use case, but this functionality is outside of the scope of the OpenSearch's charter. We'd recommend that you look at the forums for tools/tech built on top of OpenSearch [1]

• [1] https://forum.opensearch.org/

[vepicurean]

[Triage - attendees 1 2 3 4 5] @vepicurean Thanks for asking about this use case, but this functionality is outside of the scope of the OpenSearch's charter. We'd recommend that you look at the forums for tools/tech built on top of OpenSearch [1]

• [1] https://forum.opensearch.org/

I already did all that - that is why I'm asking. I don't understand what use OpenSearch is if someone else has to build a tool on top of it to create an index so Samba shares can be searched for content using OpenSearch.

I'm astonished this is not core functionality in Opensearch....

[dblock]

I'm astonished this is not core functionality in Opensearch....

It just means nobody else needed it or took the time to develop it. Maybe you can help?

In #12517 you've mentioned that this is possible with Elasticsearch. Using what tool/functionality? How are you doing this with Elasticsearch?

[vepicurean]

I'm astonished this is not core functionality in Opensearch....

It just means nobody else needed it or took the time to develop it. Maybe you can help?

In #12517 you've mentioned that this is possible with Elasticsearch. Using what tool/functionality? How are you doing this with Elasticsearch?

I am always willing to help....

https://wiki.samba.org/index.php/Spotlight_with_Elasticsearch_Backend

https://www.davidschlachter.com/misc/freebsd-samba-macos-spotlight-elasticsearch

Because OpenSearch promoted the project as a fork of Elasticsearch without the licensing issues, I decided to go with Opensearch first ( think MariaDB vs MySQL). Maybe I made a mistake going with Opensearch.

It's best I go with Elasticsearch and FSCrawler for now. Let me know how I can help with Opensearch to index Samba shares and integrate with Spotlight (MasOS workstations).

[dblock]

I found dadoonet/fscrawler#1274 which talks about supporting OpenSearch, IMO it might actually just work. Have you tried, especially with OpenSearch 1.x?

I opened dadoonet/fscrawler#1824 to add/document/fix support for OpenSearch.

[vepicurean]

Yes, I saw that issue at FSCrawler that contains the post dadoonet/fscrawler#1274 (comment) and read through it in detail and decided it was too much of a hack. I'm not sure why they closed the issue out.

Maybe I'm old school, my view is this should be core functionality of Opensearch ( should work for all versions, including the current version) since it's supposed to be a drop in replacement for Elasticsearch.

If I have to use FSCrawler for the indexing, I'm OK with that, I just want to make sure that at least Opensearch supports working with FSCrawler as core functionality.

Hope this makes sense. Thanks for open that issue dadoonet/fscrawler#1824 at FSCrawler, but I also see that Opensearch needs some development to make the entire process work without specific versions of X and Y program - something along the way will always break - I've been through that before.

Thanks again.

[dblock]

Maybe I'm old school, my view is this should be core functionality of Opensearch ( should work for all versions, including the current version) since it's supposed to be a drop in replacement for Elasticsearch.

OpenSearch is first and foremost an open source project, a fork of an existing one. Please help us implement the missing features.

[vepicurean]

Maybe I'm old school, my view is this should be core functionality of Opensearch ( should work for all versions, including the current version) since it's supposed to be a drop in replacement for Elasticsearch.

OpenSearch is first and foremost an open source project, a fork of an existing one. Please help us implement the missing features.

Understand, I am happy to help with the UX and development testing.

[johndpope]

was looking at this ticket - and after troubleshooting fscrawler - found this repo Fess which uses opensearch and was a better fit for my needs.

https://github.com/codelibs/fess?tab=readme-ov-file
https://github.com/codelibs/docker-fess/tree/master

[dblock]

@johndpope Nice, cc: @krisfreedain

[dblock]

Would love a blog post on https://opensearch.org/ of how to use fess for OP's purposes.