How do I set up the RabbitMQ Shovel plugin on my Amazon MQ broker?

5 minute read
0

I want to use the RabbitMQ Shovel plugin to move messages from an on-premises broker to an Amazon MQ for RabbitMQ broker.

Short description

To configure the RabbitMQ Shovel plugin on an Amazon MQ for RabbitMQ broker, create a dynamic shovel (on the RabbitMQ website). To create a dynamic shovel for an Amazon MQ for RabbitMQ broker, you can use either of the following options:

The following resolution requires two brokers: an upstream broker and a downstream broker. You can use an on-premises or cloud-based upstream broker. In this case, the downstream broker is the Amazon MQ for RabbitMQ broker. You can configure a dynamic shovel on either a downstream or upstream broker. The following example procedure shows how to create a dynamic shovel on a downstream broker.

Note: Amazon MQ doesn't support private Amazon MQ brokers as shovel destinations or static shovels. For more information, see Shovel plugin.

Resolution

Create a dynamic shovel on a downstream broker with the RabbitMQ web console

Important: Make sure that you specify the Uniform Resource Identifier (URI) for queues that aren't present in the broker where the shovel is created.

1.    Open the Amazon MQ console.

2.    In the navigation pane, choose Brokers.

3.    In the Name column, choose the name of your downstream broker.

4.    In the Connections pane, select the URL that's listed below RabbitMQ web console. The RabbitMQ web console login page for your downstream broker opens.

5.    Enter your Username and Password for your broker. Then, choose Login to log in to the RabbitMQ web console.

6.    In the RabbitMQ web console, choose the Admin tab. Then, in the right navigation pane, choose Shovel Management.

7.    Choose Add a new shovel.

8.    For Name, enter a name for your shovel.

9.    For Source, select AMQP 0.9.1 from the dropdown list. Then, complete the following steps:
In the dropdown list to the right of URI, select Queue.
In the text box below Queue, enter the name of the upstream broker's source queue.
For URI, delete the default amqp:// text and enter the upstream broker's URI in the following format:

amqps://$UPSTREAM_BROKER_USER:$UPSTREAM_BROKER_PASSWORD@$UPSTREAM_BROKER_ENDPOINT:5671

10.    For Destination, select AMQP 0.9.1 from the dropdown list. Then, complete the following steps:
In the dropdown list to the right of URI, select Queue.
In the text box below Queue, enter the name of the downstream broker's destination queue.
For URI, keep the default amqp:// text and enter the downstream broker's URI in the following format:

amqp://$DOWNSTREAM_BROKER_USER:$DOWNSTREAM_BROKER_PASSWORD@$DOWNSTREAM_BROKER_ENDPOINT:5671

11.    Choose Add shovel.
Note: If the source and destination queues don't already exist, then the RabbitMQ automatically creates them.

Create a dynamic shovel on a downstream broker with the RabbitMQ Management Command Line Tool

Important: Make sure that you specify the Uniform Resource Identifier (URI) for queues not present in the broker where the shovel is created.

Run the following curl command:

curl -u $SHOVEL_BROKER_USER:$SHOVEL_BROKER_PASSWORD \
-X PUT https://$SHOVEL_BROKER_ENDPOINT/api/parameters/shovel/%2f/rmq-lab-shovel \
-H "content-type: application/json" \
-d '{"value": {"src-protocol": "amqp091","src-uri": "amqps://$UPSTREAM_BROKER_USER:$UPSTREAM_BROKER_PASSWORD@$UPSTREAM_BROKER_ENDPOINT:5671","src-queue": "shovel-demo-src-queue","dest-protocol": "amqp091","dest-uri": "amqps://$DOWNSTREAM_BROKER_USER:$DOWNSTREAM_BROKER_PASSWORD@$DOWNSTREAM_BROKER_ENDPOINT:5671","dest-queue": "shovel-demo-dest-queue"}}'

Important:

  • Replace $SHOVEL_BROKER_USER and $SHOVEL_BROKER_PASSWORD with the downstream broker's credentials.
  • Replace $SHOVEL_BROKER_ENDPOINT with the downstream broker's endpoint.
  • Replace $UPSTREAM_BROKER_USER and $UPSTREAM_BROKER_PASSWORD with the upstream broker's credentials.
  • Replace $UPSTREAM_BROKER_ENDPOINT with the upstream broker's endpoint.
  • Replace $DOWNTREAM_BROKER_USER and $DOWNTREAM_BROKER_PASSWORD with the downstream broker's credentials.
  • Replace $DOWNTREAM_BROKER_ENDPOINT with the downstream broker's endpoint.
  • Replace $SOURCE_QUEUE_NAME with the source queue's name in upstream broker.
  • Replace $DESTINATION_QUEUE_NAME with the destination queue's name in the downstream broker.

Test your setup

Send a test message to the upstream broker's source queue

1.    Open the RabbitMQ web console of the upstream broker.
Note: For more information about how to access the RabbitMQ web console, see Management plugin in the RabbitMQ documentation.

2.    Choose the Queues tab. Then, select the source queue's name.

3.    In the navigation pane, choose Publish message. The Publish message section opens.

4.    In the Payload text box, enter a test message.
Note: You can leave the Headers and Properties text boxes blank.

5.    Choose the Publish message button. This sends the test message to the source queue. If you correctly configured your dynamic shovel, then the shovel consumes the message and sends it to the downstream broker's destination queue.

Verify that the test message is received by the downstream broker's destination queue

1.    Open the RabbitMQ web console of the downstream broker. For instructions, see steps 1-5 of the To create a dynamic shovel on a downstream broker using the RabbitMQ web console section.

2.    Choose the Queues tab. Then, choose the destination queue's name.

3.    In the navigation pane, choose Get messages. The Get messages section opens.

4.    Choose the Get messages button. If the dynamic shovel is configured correctly, then the test message that you sent to the source queue appears as the Payload value.

Related information

Migrating message driven applications to Amazon MQ for RabbitMQ

How do I set up the RabbitMQ Federation plugin on my Amazon MQ broker?

AWS OFFICIAL
AWS OFFICIALUpdated 10 months ago