Intermittent Lucene.Net.Spatial Issue

[deiniolcarter]

Hello,

We are currently experiencing intermittent issues when using Lucene for searches in combination with the Lucene.Net.Spatial library. The problem seems to be exacerbated by the use of load-balanced Azure app services, however it has been known to occur on a local development machine.

We are using Umbraco as our CMS with Examine used as an extension to Lucene.Net. However, all of the custom code that I have written interacts directly with the Lucene.Net library. We have extended the built-in Umbraco ExternalIndex and non-geo searches seem to cause no issues. We have also followed all of the recommended steps in the Umbraco Load Balancing Guide for Azure.

Any help/insight would be greatly appreciated!

NuGet Packages: Umbraco 8.15.0, Examine 1.2.1, Lucene.Net 3.0.3, Lucene.Contrib 3.0.3, Lucene.Contrib.Spatial 3.0.3, Spatial4n.Core 0.3.0, System.Spatial 5.8.5

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.

Parameter name: index
           at System.Collections.BitArray.Get(Int32 index)
           at Lucene.Net.Spatial.Util.FixedBitSet.Get(Int32 index) in d:\Lucene.Net\FullRepo\trunk\src\contrib\Spatial\Util\FixedBitSet.cs:line 100
           at Lucene.Net.Spatial.Vector.DistanceValueSource.DistanceDocValues.DoubleVal(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\contrib\Spatial\Vector\DistanceValueSource.cs:line 76
           at Lucene.Net.Spatial.Vector.DistanceValueSource.DistanceDocValues.FloatVal(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\contrib\Spatial\Vector\DistanceValueSource.cs:line 70
           at Lucene.Net.Spatial.Util.ReciprocalFloatFunction.FloatDocValues.FloatVal(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\contrib\Spatial\Util\ReciprocalFloatFunction.cs:line 59
           at Lucene.Net.Search.Function.DocValues.DoubleVal(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\Function\DocValues.cs:line 82
           at Lucene.Net.Spatial.Util.ValueSourceFilter.ValueSourceFilteredDocIdSet.Match(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\contrib\Spatial\Util\ValueSourceFilter.cs:line 67
           at Lucene.Net.Search.FilteredDocIdSet.AnonymousClassFilteredDocIdSetIterator.Match(Int32 docid) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\FilteredDocIdSet.cs:line 70
           at Lucene.Net.Search.FilteredDocIdSetIterator.Advance(Int32 target) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\FilteredDocIdSetIterator.cs:line 77
           at Lucene.Net.Search.FilteredQuery.AnonymousClassWeight.AnonymousClassScorer.Advance(Int32 target) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\FilteredQuery.cs:line 103
           at Lucene.Net.Search.ConjunctionScorer.Advance(Int32 target) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\ConjunctionScorer.cs:line 113
           at Lucene.Net.Search.BooleanScorer2.Advance(Int32 target) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\BooleanScorer2.cs:line 409
           at Lucene.Net.Search.BooleanScorer2.SingleMatchScorer.Advance(Int32 target) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\BooleanScorer2.cs:line 265
           at Lucene.Net.Search.ReqOptSumScorer.Score() in d:\Lucene.Net\FullRepo\trunk\src\core\Search\ReqOptSumScorer.cs:line 78
           at Lucene.Net.Search.BooleanScorer2.Score() in d:\Lucene.Net\FullRepo\trunk\src\core\Search\BooleanScorer2.cs:line 403
           at Lucene.Net.Search.Function.CustomScoreQuery.CustomScorer.Score() in d:\Lucene.Net\FullRepo\trunk\src\core\Search\Function\CustomScoreQuery.cs:line 528
           at Lucene.Net.Search.TopScoreDocCollector.InOrderTopScoreDocCollector.Collect(Int32 doc) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\TopScoreDocCollector.cs:line 49
           at Lucene.Net.Search.Scorer.Score(Collector collector) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\Scorer.cs:line 68
           at Lucene.Net.Search.IndexSearcher.Search(Weight weight, Filter filter, Collector collector) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\IndexSearcher.cs:line 228
           at Lucene.Net.Search.IndexSearcher.Search(Weight weight, Filter filter, Int32 nDocs) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\IndexSearcher.cs:line 188
           at Lucene.Net.Search.Searcher.Search(Query query, Filter filter, Int32 n) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\Searcher.cs:line 108
    at Lucene.Net.Search.Searcher.Search(Query query, Int32 n) in d:\Lucene.Net\FullRepo\trunk\src\core\Search\Searcher.cs:line 118

[Shazwazza]

Examine itself isn't responsible for Lucene specific logic so if there are actually underlying issues with Lucene, there's not a whole lot that Examine can do. That said, it could be how you are implementing spatial search and how you are accessing the underlying lucene resources. There's a spatial search example in the codebase here https://github.com/Shazwazza/Examine/blob/7042580d1aa9936cd8d9cb0a152a0d3cab18e063/src/Examine.Test/Examine.Lucene/Extensions/SpatialSearch.cs

[deiniolcarter]

@Shazwazza Is it still possible to host Examine index files on a blob storage account?

[Shazwazza]

@deiniolcarter answered you on twitter.

Hi, this was only a proof of concept and never really released. It's plagued with issues so wasn't pursued. It could be more easily possible in latest versions but would still be a huge effort.