Multiple region caches with Doctrine 2 second level cache and Symfony 3.3

Question

Have a distributed SF3.3 application running on multiple AWS EC2 instances with a central ElastiCache (redis) cluster.

Each EC2 instance also runs a local Redis instance which is used for Doctrine meta and query caching.

This application utilises Doctrines Second Level Cache, which works very well from a functional point of view. But performance is poor (900-1200ms page loads) on AWS due to the 400+ cache calls it makes to load in our Country and VatRate entities required on many of our pages.

As these Country and VatRate entities change rarely I'd like to utilise both the local Redis instance and ElastiCache for result caching by using different regions defined in the second level cache. This should reduce the latency problem with the 400+ cache calls as when running on a single box page loads are sub 100ms. Reading the documentation this all seems to be possible, just not entirely sure how to configure it with Symfony and PHP-Cache.

An example of the current configuration:

app/config/config.yml

doctrine:
    dbal:
        # .. params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    region_cache_driver:
                        type: service
                        id: doctrine.orm.default_result_cache

cache_adapter:
    providers:
        meta: # Used for version specific
            factory: 'cache.factory.redis'
            options:
                host: 'localhost'
                port: '%redis_local.port%'
                pool_namespace: "meta_%hash%"
        result: # Used for result data
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: result

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        result:
            service_id:         'cache.provider.result'
            entity_managers:    [ default ]

src/AppBundle/Entity/Country.php

/**
 * @ORM\Table(name = "countries")
 * @ORM\Cache(usage = "READ_ONLY")
 */
class Country
{
    // ...

    /**
     * @var VatRate
     *
     * @ORM\OneToMany(targetEntity = "VatRate", mappedBy = "country")
     * @ORM\Cache("NONSTRICT_READ_WRITE")
     */
    private $vatRates;

    // ...
}

src/AppBundle/Entity/VatRate.php

/**
 * @ORM\Table(name = "vatRates")
 * @ORM\Cache(usage = "READ_ONLY")
 */
class VatRate
{
    // ...

    /**
     * @var Country
     *
     * @ORM\ManyToOne(targetEntity = "Country", inversedBy = "vatRates")
     * @ORM\JoinColumn(name = "countryId", referencedColumnName = "countryId")
     */
    private $country;

    // ...
}

src/AppBundle/Entity/Order.php

/**
 * @ORM\Table(name = "orders")
 * @ORM\Cache(usage = "NONSTRICT_READ_WRITE")
 */
class Order
{
    // ...

    /**
     * @var Country
     *
     * @ORM\ManyToOne(targetEntity = "Country")
     * @ORM\JoinColumn(name = "countryId", referencedColumnName = "countryId")
     */
    private $country;
    // ...
}

Attempted Configuration

app/config/config.yml

doctrine:
    dbal:
        # .. params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    region_cache_driver: array
                    regions:
                        local:
                            type: service
                            service: "doctrine.orm.default_result_cache" # TODO: needs to be local redis 
                        remote:
                            type: service
                            service: "doctrine.orm.default_result_cache" # TODO: needs to be remote redis

cache_adapter:
    providers:
        meta: # Used for version specific
            factory: 'cache.factory.redis'
            options:
                host: 'localhost'
                port: '%redis_local.port%'
                pool_namespace: "meta_%hash%"
        result: # Used for result data
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: result

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        result:
            service_id:         'cache.provider.result'
            entity_managers:    [ default ]

src/AppBundle/Entity/Country.php

/**
 * @ORM\Table(name = "countries")
 * @ORM\Cache(usage = "READ_ONLY", region = "local")
 */
class Country
{
    // as above
}

src/AppBundle/Entity/VatRate.php

/**
 * @ORM\Table(name = "vatRates")
 * @ORM\Cache(usage = "READ_ONLY", region = "local")
 */
class VatRate
{
    // as above
}

src/AppBundle/Entity/Order.php

/**
 * @ORM\Table(name = "orders")
 * @ORM\Cache(usage = "NONSTRICT_READ_WRITE", region = "remote")
 */
class Order
{
    // as above
}

Which results in

Type error: Argument 1 passed to Doctrine\ORM\Cache\DefaultCacheFactory::setRegion() must be an instance of Doctrine\ORM\Cache\Region, instance of Cache\Bridge\Doctrine\DoctrineCacheBridge given,

Not too sure where to go from here, been working from the tests here: https://github.com/doctrine/DoctrineBundle/blob/74b408d0b6b06b9758a4d29116d42f5bfd83daf0/Tests/DependencyInjection/Fixtures/config/yml/orm_second_level_cache.yml but the lack of documentation for configuring this makes it a little more challenging!

Answer 1

After much playing around with the PHP-Cache library, it's clear from looking in the CacheBundle compiler that it will only ever support one DoctrineBridge instance from the configuration. https://github.com/php-cache/cache-bundle/blob/master/src/DependencyInjection/Compiler/DoctrineCompilerPass.php

Solution was to create my own compiler, not pretty but it seems to work.

src/AppBundle/DependencyInjection/Compiler/DoctrineCompilerPass.php

namespace AppBundle\DependencyInjection\Compiler;

use Cache\Bridge\Doctrine\DoctrineCacheBridge;
use Cache\CacheBundle\Factory\DoctrineBridgeFactory;
use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Reference;

class DoctrineCompilerPass implements CompilerPassInterface
{
    /** @var ContainerBuilder */
    private $container;

    public function process(ContainerBuilder $container)
    {
        $this->container = $container;

        $this->enableDoctrineCache('local');
        $this->enableDoctrineCache('remote');
    }

    private function enableDoctrineCache(string $configName)
    {
        $typeConfig = [
            'entity_managers' => [
                'default'
            ],
            'use_tagging' => true,
            'service_id' => 'cache.provider.' . $configName
        ];

        $bridgeServiceId = sprintf('cache.service.doctrine.%s.entity_managers.bridge', $configName);

        $this->container->register($bridgeServiceId, DoctrineCacheBridge::class)
            ->setFactory([DoctrineBridgeFactory::class, 'get'])
            ->addArgument(new Reference($typeConfig['service_id']))
            ->addArgument($typeConfig)
            ->addArgument(['doctrine', $configName]);
    }
}

src/AppBundle/AppBundle.php

use AppBundle\DependencyInjection\Compiler\DoctrineCompilerPass;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\HttpKernel\Bundle\Bundle;

class AppBundle extends Bundle
{
    public function build(ContainerBuilder $container)
    {
        parent::build($container);

        $container->addCompilerPass(new DoctrineCompilerPass());
    }
}

app/config/config.yml

doctrine:
    dbal:
    # ... params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    regions:
                        remote:
                            cache_driver:
                                type: service
                                id: cache.service.doctrine.remote.entity_managers.bridge
                        local:
                            cache_driver:
                                type: service
                                id: cache.service.doctrine.local.entity_managers.bridge

cache_adapter:
    providers:
        local:
            factory: 'cache.factory.redis'
            options:
                host: '%redis_local.host%'
                port: '%redis_local.port%'
                pool_namespace: "local_%hash%"
        remote:
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: 'result'

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.local'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.local'
            entity_managers:    [ default ]

While this seems to work to some extent, there's some inconsistencies local cache calls resulting in 500 errors when theres probably something missing in the cache. Overall think I'm trying to bend the second level cache more than it was designed to.

Answer 2

The error message you are getting entirely reflects the root of your issue. You are passing DoctrineCacheBridge instances (the underlying class of doctrine.orm.default_result_cache) when instances of the Doctrine\ORM\Cache\Region interface expected:

            second_level_cache:
                #...
                regions:
                    local:
                        type: service
                        service: "region_service_not_cache_service" # Here is a Region instance expected 
                    remote:
                        type: service
                        service: "region_service_not_cache_service" #Here is a Region instance expected

In your former configuration the doctrine.orm.default_result_cache cache service is set as the default cache through the region_cache_driver setting. \Doctrine\ORM\Cache\DefaultCacheFactory generates instances of DefaultRegion on flight (as none was preconfigured) and feeds the default cache to them.

The latter configuration is expected to have pre-configured regions and could be fixed several ways. I suggest the next:

dbal:
    # .. params

orm:
    #...
            second_level_cache:
                #...
                regions:
                    local:
                        type: default
                        cache_driver: 
                            type: service
                            id: "doctrine.orm.default_query_cache" # NOTE that this is the service id of your local cache generated by PHP-Cache Bundle
                    remote:
                        type: default
                        cache_driver: 
                            type: service
                            id: "doctrine.orm.default_result_cache" # NOTE that this is the service id of your remote cache generated by PHP-Cache Bundle

Here you tell Doctrine to create 2 DefaultRegion regions under the local and remote keys and pass local_cache and remote_cache to them correspondingly.

And it's better to return region_cache_driver to the former value otherwise DefaultRegions generated on flight will use array cache:

            second_level_cache:
                enabled: true
                region_cache_driver: 
                    type: service
                    id: doctrine.orm.default_result_cache