Sticky session not working with multiple apache vhosts and multiple JBoss 7.2 server-groups

Question

Having a specific cluster architecture with the following details:

• Load balancer (10.10.0.1) using Apache's mod_cluster 1.2.0 without advertising using multicast
• 2 JBoss AS 7.2 servers (jboss-instance-1, jboss-instance-1) in domain mode (host and slave), i'm using proxy-list instead of multicast

In domain.xml i have the following settings (only important snippets):

    ...
        <subsystem xmlns="urn:jboss:domain:modcluster:1.1">
            <mod-cluster-config advertise-socket="modcluster" connector="ajp" balancer="${mycluster.modcluster.balancer:DefaultBalancer}" proxy-list="10.10.0.1:6677">
                <dynamic-load-provider>
                    <load-metric type="busyness"/>
                </dynamic-load-provider>
            </mod-cluster-config>
        </subsystem>
    ...
    ...
    <server-group name="SG1" profile="ha">
        <system-properties>
            <property name="mycluster.modcluster.balancer" value="SG1Balancer"/>
        </system-properties>
        <jvm name="default"/>
        <socket-binding-group ref="ha-sockets"/>
    </server-group>
    <server-group name="SG2" profile="ha">
        <system-properties>
            <property name="mycluster.modcluster.balancer" value="SG2Balancer"/>
        </system-properties>
        <jvm name="default"/>
        <socket-binding-group ref="ha-sockets"/>
    </server-group>
    ....

And in host.xml (same as in slave and host):

    ....
    <servers>
        <server name="server-1" group="SG1" auto-start="true">
            <socket-bindings port-offset="1"/>
        </server>
        <server name="server-2" group="SG2" auto-start="true">
            <socket-bindings port-offset="2"/>
        </server>
    ....

I need to point each virtual hosts to a specific server-group, so this is my solution for it:

• www.vhost1.com -> SG1Balancer, which is attached to group=SG1, so it will load balance between:
  • jboss-instance-1's server-1
  • jboss-instance-2's server-1
• www.vhost2.com -> SG2Balancer, which is attached to group=SG2, so it will load balance between:
  • jboss-instance-1's server-2
  • jboss-instance-2's server-2

Here's my httpd configuration sets:

loadbalancer.conf:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

LoadModule slotmem_module modules/mod_slotmem.so
LoadModule manager_module modules/mod_manager.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule advertise_module modules/mod_advertise.so

<IfModule manager_module>
  Listen 0.0.0.0:6677
  ManagerBalancerName ddrct-cluster

  <VirtualHost *:6677>
    ServerName RecetteDtvLb1
    <Location />
      Order deny,allow
      Allow from 0.0.0.0
    </Location>

    # No server advertising
    # we're using proxy-list
    ServerAdvertise Off

    KeepAliveTimeout 300
    MaxKeepAliveRequests 0
    EnableMCPMReceive

    <Location /mcm>
      SetHandler mod_cluster-manager
      Order deny,allow
      Allow from 0.0.0.0
    </Location>
  </VirtualHost>
</IfModule>

vhosts.conf:

ProxyRequests Off
NameVirtualHost *:80

<VirtualHost *:80>
  ServerName www.vhost1.com

  ProxyPass / balancer://SG1Balancer stickysession=JSESSIONID
  ProxyPassReverse / balancer://SG1Balancer
  ProxyPreserveHost On

  ErrorLog "logs/vhost1_error.log"
  CustomLog "logs/vhost1_access.log" common
</VirtualHost>

<VirtualHost *:80>
  ServerName www.vhost2.com

  ProxyPass / balancer://SG2Balancer stickysession=JSESSIONID
  ProxyPassReverse / balancer://SG2Balancer
  ProxyPreserveHost On

  ErrorLog "logs/vhost2_error.log"
  CustomLog "logs/vhost2_access.log" common
</VirtualHost>

Everything is running fine, deployed apps in G1 can be used through www.vhost1.com and vice versa for G2, the problem is all about session stickyness, here are the symptoms:

• My browser is accepting JSESSIONID cookie, but having a content like this: KhH7gInyAFPsILN6mYDQ84Kf.jboss-instance-1:server-1 will not get my next request sticked to jboss-instance-1's server-1, the load balancer will switch my requests between jboss-instance-1's server-1 and jboss-instance-2's server-1 and each time it changes the JSESSIONID cookie content.
• When i manually edit the content of JSESSIONID cookie (using firecookie plugin) to remove the server name like this: KhH7gInyAFPsILN6mYDQ84Kf.jboss-instance-1, my requests will be sticking on jboss-instance-1, leading to a correct behaviour

Did not get too much documentation about multi-vhost and multi-server-group clustering and load balancing configurations, it was my own guess to implement such configuration, i may get it wrong ... any help ?

Answer 1

Regarding node names
The relevant issue MODCLUSTER-435 was resolved in Nov 2014. It is safe to use upper case characters and dashes in any decent mod_cluster version, i.e. 1.3.x.

Regarding ProxyPass
One doesn't need to set anything ProxyPass related unless one wants to do something special. The aforementioned setting is redundant and weird. Mod_cluster uses mod_proxy as its backend, so it actually dynamically configures proxy pass balancer members for you. It makes sense to have your own ProxyPass additional setting if you wish to alter the balancer's behaviour in a custom way, e.g.

• One needs to have static content served from the Apache HTTP Server while the dynamic one being proxied to Wildfly workers: ProxyPassMatch ^/app/static/ ! ProxyPass /app balancer://qacluster stickysession=JSESSIONID|jsessionid nofailover=on ProxyPass / ! ProxyPassReverse /app balancer://qacluster ProxyPassReverseCookieDomain / /app/ ProxyPassReverseCookiePath / /app/ ProxyPreserveHost on

Answer 2

Issue resolved, from KB182813:

• Do not put the "-" character in your balancer name as it has been known to create sticky session issues
• Do not use uppercase characters in your balancer name