Only requeue compaction when there was activity #759
Conversation
| @@ -41,12 +41,18 @@ public void run() { | |||
| return; | |||
| } | |||
|
|
|||
| tablet.majorCompact(reason, queued); | |||
| CompactionStats stats = tablet.majorCompact(reason, queued); | |||
There was a problem hiding this comment.
I would have thought Spotbugs would have dinged this method returning stats that aren't being used...
| // make blocking calls to gather information. Without the following check these strategies would | ||
| // endlessly requeue. So only check if a subsequent compaction is needed if the previous | ||
| // compaction actually did something. | ||
| if (stats != null && stats.getEntriesRead() > 0) { |
There was a problem hiding this comment.
Won't this prevent compactions from happening when stats.getEntriesRead() == 0?
There was a problem hiding this comment.
No, tablets are checked periodically to see if they need a compaction by the tablet server. This code is just an optimization to re-queue the tablet for compaction if there is still work to do (without waiting for the periodic check). If we run a compaction and it does nothing, its likely there is no follow on compaction work to be done.
For background, tablets are limited in the number of files they will compact at once. So if a tablet had 100 files to compact and the limit is 20, then when it compacts 20 files it would be nice if immediately re-queued instead of waiting for the tablet server to check all tablets again. With these changes, it will still re-queued immediately in that case.
There was a problem hiding this comment.
OK thanks for making that clear.
While working on an experimental compaction strategy for Fluo for apache/fluo#1054 I ran into a situation where Accumulo was calling my compaction strategy hundreds of thousands of times per second. I tracked down the problem and fixed it in this PR.