When reading the article another idea came into my mind: Skip messages and requeue them immediately if the retry count plus one is not a power of 2.
So that would mean:
Retry count : 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 (...)
Process/skip: 1 1 0 1 0 0 0 1 0 0 0 0 0 0 0 1 (...)
where 1 is process, 0 is skip (requeue immediately)

So if we have a TTL of 1 second we would process the messages at 0, 1, 3, 7, 15, (...) seconds.

That would add some complexity in code but the logic could be extracted into a class like this:

public class ExponentialBackoffMustRequeueEstimator {

public boolean mustSkipAndRequeue(Message<?> message) {
long numberOfRetries = getNumberOfRetriesFromHeader(message);
if (numberOfRetries == 0) {
return false;
}
return !isPowerOfTwo(numberOfRetries + 1);
}

public boolean maxRetriesExceeded(Message<?> message, long maxRetries) {
long retries = getNumberOfRetriesFromHeader(message);
return (retries + 1) > (Math.pow(2, maxRetries));
}
}

You see any disadvantage in this approach?

I just can't believe it's that hard to achieve it with Rabbit. Like, isn't it EVERYONE's use case!?

I have a problem with this solution. What happens if a second message arrives on queue.50000 when it has already passed 40000 and the message TTL is set to 30000? The queue will be deleted, and this message will be lost before it has the chance to reach the necessary 30000 milliseconds required for it to be resent to the main queue..

Are maybe priority queues usable here? https://www.rabbitmq.com/pr...

Nice article!

Question. Does backoff ttl jittering has sense in this context to try distribute spikes more evenly like you would do with some basic remote calls or it has no real sense here?

Possibilities:
a) you create q.jitter-val queues with per q ttl - you would end with enormous amount of queues connected to randomness of jitter (eventually you could limit it with narrowing randomness by rounding up)
b) you stick to no-jittered backoff queues (with no-jittered vlaues) but you set jitterd ttl for direct messages which would make rabbit to pick lower. Still I think we would hit the FIFO problem, so if higher TTL msg is at the head our lower TTL messages would need to wait anyway (you initial problem).

So I think only some version of a) might sense but I am not sure if I don't miss anything and jitter has sense at all for queue based solutions. Would do you think?

Excellent explanation. . Thanks . . Crystal clear