Processing Groups

Processing Groups is a way to set specific configurations, such as queues, concurrency, long polling for a set of queues or a single one.

Given this configuration:

concurrency: 25
queues:
  - queue1
  - queue2
  - queue3

Shoryuken will fetch messages from queue1, then queue2 ,then queue3, then repeat.

If you don't want to wait on fetching for queue1 to fetch for queue2, you can move them into separate groups:

groups:
  group1:
    concurrency: 10
    queues:
      - queue2
  group2:
    concurrency: 10
    queues:
      - queue1

Is that all? No, it is not! If you want to process only one message at time for queue1, but keeping concurrency: 25 for queue2 and queue3:

concurrency: 25
queues:
  - queue2
  - queue3

groups:
  group1:
    concurrency: 1
    queues:
      - queue1

Note: If you want make sure to process only one message at time for a given queue, while running multiple Shoryuken processes, have a look at FIFO Queues.

Polling strategy per group

For defining a Polling strategy per group, you can set it a follows:

# config/initializers/shoryuken.rb
Shoryuken.configure_server do |config|
  config.options[:group1][:polling_strategy] = Polling::StrictPriority
end

Long polling per group

# config/initializers/shoryuken.rb
Shoryuken.sqs_client_receive_message_opts[:group1] = { wait_time_seconds: 20 }