mongoDB addToSet performance

We used mongoDB’s sets to save alist of references to other entities for lookup by using addToSet (classical m:n SQL lookup table use case).

We discovered that in the beginning updates to set set are very fast but get slower and slower up to 300 ms for a single update at more than 10.000 items in the set.

The reason is that a set is unique and on an update the set is searched if the element is already in there.

We decided to not use sets anymore and save one document per item which results in in stable update time of 4 ms.

Another solution of course could be to use and index on the set. We did not try that. Perhaps somebody has and could tell if it works fast.

GD Star Rating

Ein Gedanke zu „mongoDB addToSet performance“

  1. It may also be that Mongo needs to move the data around when the document object gets too large. When the object is first saved, Mongo reserved a bit of space for it to grow over time, but once this buffer is full it will need to save the data somewhere else on the disk and clear this space. This operation is slow and could be a big part of the effect you see.

Kommentar verfassen

This site uses Akismet to reduce spam. Learn how your comment data is processed.