Defining custom analyzers

[tomsap5]

I couldn't find a simple way to define a custom analyzer.

I'm working with Grails 2.5.3 and elasticsearch 0.0.4.6.
Under config.groovy i use the following to define "my_analyzer":

elasticSearch {
    datastoreImpl = 'hibernateDatastore'
    client.mode = 'transport'
    index {
        analysis {
            filter {
                'my_synonym_filter' {
                    type = 'synonym'
                    synonyms = ['doc,doctor']
                }
            }
        }
        analysis {
            analyzer {
                'my_analyzer' {
                    tokenizer = 'standard'
                    filter = ['lowercase', 'my_synonym_filter']
                }
            }
        }
    }
}

and my Domain searchable configuration looks like this :

static searchable = {
    only = ['title', 'content', 'relatedToMilestone', 'domains', 'source']
    relatedToMilestone component: true
    domains component: true
    source component: true
    all = [analyzer: 'my_analyzer']
}

I get a MapperParsingException: Analyzer [my_analyzer] not found for field [_all] during the bulk indexing stage.

Is it even possible to work with the plugin and use a custom configurable analyzer?
If not,is there some kind of hook to integrate with the plugin indexing stage and use the client index java API to define my analyzer?

Thanks!
Tom

[vonovak]

I was once dealing with a similar situation when I needed to define custom analyzers. I ended up with using the multi_field: true because it fit my situation well and the following code didn't work as expected (although the mapping looked good in the head plugin). Anyway, here's something you can start with (add to your bootstrap.groovy).

def setupES(indexName) {
        elasticSearchHelper.withElasticSearch { client ->
            client.admin().indices().close(new CloseIndexRequest(indexName));

            client.admin().indices().prepareUpdateSettings(indexName).setSettings("{\n" +
                    "\"analysis\":{\n" +
                    "           \"analyzer\":{\n" +
                    "              \"analyzer_keyword\":{\n" +
                    "                 \"tokenizer\":\"keyword\",\n" +
                    "                 \"filter\":\"lowercase\"\n" +
                    "              }\n" +
                    "           }\n" +
                    "          }\n" +
                    "        }").execute().actionGet()

            client.admin().indices().open(new OpenIndexRequest(indexName));

            DeleteMappingRequest deleteMappingRequest = new DeleteMappingRequest(indexName).types("yourtype");
            client.admin().indices().deleteMapping(deleteMappingRequest).actionGet();

            client.admin().indices().preparePutMapping(indexName).setType("yourtype").setSource("{\n" +
                    "        \"properties\":{\n" +
                    "           \"yourproperty\":{\n" +
                    "              \"analyzer\":\"analyzer_keyword\",\n" +
                    "              \"type\":\"string\"\n" +
                    "           }\n" +
                    "        }\n" +
                    "     }").execute().actionGet()
        }
    }

[bruno-lopes]

Hi @tomsap85, I'm using Grails 2.5.1 and elasticsearch 0.0.4.6
Here I defined my custom analyzer this way:

elasticSearch {
    index {
        analysis {
            analyzer {
                "my_sort" {
                    tokenizer = "keyword"
                    filter = ["icu_collation"]
                }
                "default" {
                    tokenizer = "standard"
                    filter = ["standard", "lowercase", "asciifolding"]
                }
            }

        }
    }
}

And in my domain class:

static searchable = {
        all = [analyzer: 'my_sort']
}

After running the application, without exceptions, I did a curl to confirm:

curl -XGET http://localhost:9200/index_v0/domain/_mapping?pretty
{
  "index_v0" : {
    "mappings" : {
      "domain" : {
        "_all" : {
          "analyzer" : "my_sort"
        },
        "properties" : {...}
      }
}

I hope this helps you.