CODEWITHSUNDEEP
rabbitmqkubernetes-helm
peasant
peasant

Reputation: 41

How do I configure my helm chart to pickup a rabbitmq community plugin?

GOAL: To use a rabbitmq community plugin from https://www.rabbitmq.com/community-plugins.html in my rabbitmq-ha helm chart.

Any plugins that do not ship with the server will need to be installed. Plugins are distributed as .ez archives (which are zip files with metadata). The files must be copied to one of the plugins directories specified by $RABBITMQ_PLUGINS_DIR.

PROBLEM: How does the rabbitmq community plugin's .ev file get injected into the chart for use?

BACKGROUND:

For my example I am trying to install the latest version of the community plugin rabbitmq_delayed_message_exchange into the latest stable version of rabbitmq-ha.

The name of the plugin is identified by adding this to my values.yaml:

amqp-rabbitmq:
  extraPlugins: |
    rabbitmq_delayed_message_exchange,

And when I 

helm install example .  --debug

it generates

# Source: example/charts/amqp-rabbitmq/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-amqp-rabbitmq
  namespace: default
  labels:
    app: amqp-rabbitmq
    chart: amqp-rabbitmq-1.38.2
    release: example
    heritage: Helm
data:
  enabled_plugins: |
    [
      rabbitmq_delayed_message_exchange,
      rabbitmq_consistent_hash_exchange,
      rabbitmq_management,
      rabbitmq_peer_discovery_k8s
    ].

  rabbitmq.conf: |
    ## RabbitMQ configuration
    ## Ref: https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.conf.example

    ## Authentification

    ## Clustering
    cluster_formation.peer_discovery_backend  = rabbit_peer_discovery_k8s
    cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
    cluster_formation.k8s.address_type = hostname
    cluster_formation.node_cleanup.interval = 10
    # Set to false if automatic cleanup of absent nodes is desired.
    # This can be dangerous, see http://www.rabbitmq.com/cluster-formation.html#node-health-checks-and-cleanup.
    cluster_formation.node_cleanup.only_log_warning = true
    cluster_partition_handling = autoheal
    ## The default "guest" user is only permitted to access the server
    ## via a loopback interface (e.g. localhost)
    loopback_users.guest = false

    management.load_definitions = /etc/definitions/definitions.json

    ## Memory-based Flow Control threshold
    vm_memory_high_watermark.absolute = 256MB

    ## Auth HTTP Backend Plugin

    ## LDAP Plugin

    ## MQTT Plugin

    ## Web MQTT Plugin

    ## STOMP Plugin

    ## Web STOMP Plugin

    ## Prometheus Plugin

    ## AMQPS support

and creates 3 k8s_amqp-rabbitmq_example-amqp-rabbitmq-n_default_... containers.

The log in each container shows

2020-01-11 01:21:16.826 [info] <0.8.0> Server startup complete; 6 plugins started.

* rabbitmq_management

* rabbitmq_web_dispatch

* rabbitmq_consistent_hash_exchange

* rabbitmq_peer_discovery_k8s

* rabbitmq_management_agent

* rabbitmq_peer_discovery_common

2020-01-11 01:23:06.954 [info] <0.529.0> node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up

2020-01-11 01:23:12.776 [info] <0.529.0> rabbit on node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up

I can successfully log into rabbitmq, create exchanges, queues, etc.

However, the desired rabbitmq_delayed_message_exchange plugin is no where to be found.

When my application tries to use the helm installed rabbitmq-ha to create the delayed exchange it gets:

'Error: Connection closed: 503 (COMMAND-INVALID) with message "COMMAND_INVALID - unknown exchange type 'x-delayed-message'"

It works with the non kubernetes instances of rabbitmq where I could easily install the plugin.

I have spent several days searching the web for a clear example, hints, and clues as to how to avail the rabbitmq-ha chart of a community plugin and not progressed.

Any ideas out there? What am I missing?

Upvotes: 4

Views: 7698

Answers (4)

Fergus_Hinn
Fergus_Hinn

Reputation: 11

add this to your value.yaml

## @param plugins List of default plugins to enable (should only be altered to remove defaults; for additional plugins use `extraPlugins`)
##
plugins: "rabbitmq_management rabbitmq_peer_discovery_k8s"

## @param queue_master_locator Changes the queue_master_locator setting in the rabbitmq config file
##
queue_master_locator: min-masters

## @param communityPlugins List of Community plugins (URLs) to be downloaded during container initialization
## Combine it with extraPlugins to also enable them.
##
communityPlugins: "https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/v4.0.2/rabbitmq_delayed_message_exchange-4.0.2.ez"
## @param extraPlugins Extra plugins to enable (single string containing a space-separated list)
## Use this instead of `plugins` to add new plugins
##
extraPlugins: "rabbitmq_auth_backend_ldap rabbitmq_delayed_message_exchange"

Upvotes: 0

Dag Vilmar Tveit
Dag Vilmar Tveit

Reputation: 71

Building a custom chart is no longer nececarry i did it with this values.yaml


## Plugins to enable
##
plugins: 'rabbitmq_management rabbitmq_peer_discovery_k8s'

## Community plugins to download during container initialization.
## Combine it with extraPlugins to also enable them.
##
communityPlugins: |
  https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/3.8.9/rabbitmq_delayed_message_exchange-3.8.9-0199d11c.ez

## Extra plugins to enable
## Use this instead of `plugins` to add new pluginsmv ra  
##
extraPlugins: |
  rabbitmq_auth_backend_ldap 
  rabbitmq_delayed_message_exchange

Upvotes: 6

Yuci
Yuci

Reputation: 30109

Here is an example to install and enable two RabbitMQ community plugins:

  • rabbitmq_delayed_message_exchange
  • rabbitmq_message_timestamp

Step 1: Get the Helm chart:

$ helm search repo rabbitmq
NAME               CHART VERSION    APP VERSION DESCRIPTION                                       
bitnami/rabbitmq   6.25.4           3.8.3       Open source message broker software that implem...

$ helm pull --untar bitnami/rabbitmq

Step 2: Update file statefulset.yaml

...
# Install community plugins
pushd /opt/bitnami/rabbitmq/plugins/
curl -LO https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/v3.8.0/rabbitmq_delayed_message_exchange-3.8.0.ez
curl -LO https://github.com/rabbitmq/rabbitmq-message-timestamp/releases/download/v3.8.0/rabbitmq_message_timestamp-3.8.0.ez
popd
# End of installing community plugins
exec rabbitmq-server
...

enter image description here

Step 3: Install RabbitMQ with the two plugins enabled

$ cd rabbitmq
$ helm install rmq . --set rabbitmq.extraPlugins="rabbitmq_delayed_message_exchange rabbitmq_message_timestamp"

Now you can verify that the two plugins have been installed

$ kubectl get pod
NAME             READY   STATUS    RESTARTS   AGE
rmq-rabbitmq-0   1/1     Running   0          3m32s

$ kubectl exec -it rmq-rabbitmq-0 bash
I have no name!@rmq-rabbitmq-0:/$ rabbitmq-plugins list
Listing plugins with pattern ".*" ...
 Configured: E = explicitly enabled; e = implicitly enabled
 | Status: * = running on rabbit@rmq-rabbitmq-0.rmq-rabbitmq-headless.default.svc.cluster.local
 |/
[  ] rabbitmq_amqp1_0                  3.8.3
[  ] rabbitmq_auth_backend_cache       3.8.3
[  ] rabbitmq_auth_backend_http        3.8.3
[  ] rabbitmq_auth_backend_ldap        3.8.3
[  ] rabbitmq_auth_backend_oauth2      3.8.3
[  ] rabbitmq_auth_mechanism_ssl       3.8.3
[  ] rabbitmq_consistent_hash_exchange 3.8.3
[E*] rabbitmq_delayed_message_exchange 3.8.0
[  ] rabbitmq_event_exchange           3.8.3
[  ] rabbitmq_federation               3.8.3
[  ] rabbitmq_federation_management    3.8.3
[  ] rabbitmq_jms_topic_exchange       3.8.3
[E*] rabbitmq_management               3.8.3
[e*] rabbitmq_management_agent         3.8.3
[E*] rabbitmq_message_timestamp        3.8.0
[  ] rabbitmq_mqtt                     3.8.3
...

Note: My Helm version is v3.1.1

$ helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}

Upvotes: 2

menya
menya

Reputation: 1525

I think rabbitmq-ha default helm chart use the official rabbitMQ image, so it only contains core plugins.

I check the chart config, it just set the /etc/rabbitmq/enabled_plugins conf file, plugin file will not be downloaded automatically.

There are two solutions:

  • Build your custom rabbitmq image, ship the rabbitmq_delayed_message_exchange file with it.
  • Download the plugin file in initContainer, add it to your chart values extraInitContainers

Upvotes: 0

Related Questions