Performance issues using a vertex property of cardinality SET

[mrckzgl]

Hi,

we have vertex properties with quite large number of values (~4000). When using cardinality SET we see very poor performance when inserting these property values:

commit properties traversal
property iterate: 0.018344
property commit: 0.439884
commit properties traversal
property iterate: 0.028657000000000002
property commit: 0.23183800000000002
commit properties traversal
property iterate: 0.018746
property commit: 0.252911
commit properties traversal
property iterate: 1.463567
property commit: 0.33779600000000004
commit properties traversal
property iterate: 62.840981
property commit: 0.5420280000000001
commit properties traversal
property iterate: 135.000684
property commit: 0.67888
commit properties traversal

These are performance logs (units in seconds) of doing something like this (in pseudo code):

while(true) {
  traversal = db.traversal().v(some_node_id);
  for each item in 500 entry list {
     traversal.property(Cardinality.SET, key, value in list)
  }
  traversal.iterate();
  db.tx().commit();
}

So after around 2000 property values inserts get really slow, we stopped when it reached 2 minutes for another 500 inserts.
We do not get this performance drop when using Cardinality.LIST. So I wonder, if the duplicate check on the SET is naively implemented? Something like every insert takes O(n). Might this be the case? And if so, is this duplicate check part of JanusGraph's codebase, or the backend's codebase, or tinkerpop's codebase?

best