[BUG] exists queries on nested flat_object fields throw a string_index_out_of_bounds_exception

[chamidiz]

Describe the bug

When creating an index mapping with an object field that has a subfield of flat_object exists queries or query strings with <field>:* throw a string_index_out_of_bounds_exception.

Related component

Search

To Reproduce

Create an index with a mapping with a flat_object subfield to an object:

PUT flat-object-test
{
    "mappings": {
        "properties": {
            "foo": {
                "properties": {
                    "bar": {
                        "type": "keyword"
                    },
                    "baz": {
                        "type": "flat_object"
                    }
                }
            }
        }
    }
}

Create a document to search:

POST flat-object-test/_doc
{
    "foo": {
        "bar": "abcdef",
        "baz": {
            "a": "b"
        }
    }
}

Search with an exists query:

GET flat-object-test/_search
{
    "query": {
        "exists": {
            "field": "foo"
        }
    }
}

Result is an exception:

{
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "failed to create query: Index 8 out of bounds for length 8",
        "index": "flat-object-test",
        "index_uuid": "vyS6P9-jTdKcETemZjlr7Q"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "flat-object-test",
        "node": "ehTrfc9eT26MNpzRtJiPYQ",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to create query: Index 8 out of bounds for length 8",
          "index": "flat-object-test",
          "index_uuid": "vyS6P9-jTdKcETemZjlr7Q",
          "caused_by": {
            "type": "string_index_out_of_bounds_exception",
            "reason": "Index 8 out of bounds for length 8"
          }
        }
      }
    ]
  },
  "status": 400
}

NOTE: exists query on foo.bar or foo.baz operate as expected. Only affected for the top-level field.

Expected behavior

Searching for _exists_: foo should not throw an exception.

Additional Details

Tested on Opensearch versions 2.15.0 and 2.18.0

[kkewwei]

It's imported by DynamicKeyFieldTypeLookup, when iterating through all fields, the DynamicKeyFieldTypeLookup imports a dynamic field named "foo. baz.", which casues the exception.

OpenSearch/server/src/main/java/org/opensearch/index/mapper/DynamicKeyFieldTypeLookup.java

Line 100 in 75a2fc3

Iterator<MappedFieldType> fieldTypes() {

@chamidiz, do you mind if I pick this up and fix it?

[chamidiz]

It's imported by DynamicKeyFieldTypeLookup, when iterating through all fields, the DynamicKeyFieldTypeLookup imports a dynamic field named "foo. baz.", which casues the exception.

OpenSearch/server/src/main/java/org/opensearch/index/mapper/DynamicKeyFieldTypeLookup.java

Line 100 in 75a2fc3

Iterator<MappedFieldType> fieldTypes() {

@chamidiz, do you mind if I pick this up and fix it?

Absolutely, thank you for looking!