doctrine2 extra lazy fetching of association

Question

I have a Thread entity which has a OneToMany association with a Message entity. I am fetching a thread with a DQL query, and I want to limit its amount of messages to 10. Therefore I am setting the fetch mode to EXTRA_LAZY as below.

class Thread
{
    // ...

    /**
     * @var ArrayCollection
     * @ORM\OneToMany(targetEntity="Profile\Entity\Message", mappedBy="thread", fetch="EXTRA_LAZY")
     * @ORM\OrderBy({"timeSent" = "ASC"})
     */
    protected $messages;
}

This allows me to use the slice method to issue a LIMIT SQL query to the database. All good so far. Because my messages are encrypted, I need to decrypt them in my service layer before handling the thread object off to the controller (and ultimately view). To accomplish this, I am doing the following in my service:

foreach ($thread->getMessages()->slice(0, 10) as $message) {
    // Decrypt message
}

The call to slice triggers an SQL query that fetches 10 messages. In my view, I am doing the following to render the thread's messages:

$this->partialLoop()->setObjectKey('message');
echo $this->partialLoop('partial/thread/message.phtml', $thread->getMessages());

The problem is that this fetches the entire collection of messages from the database. If I call slice as in my service, the same SQL query with LIMIT 10 is issued to the database, which is not desirable.

How can I process a limited collection of messages in my service layer without issuing another SQL query in my view? That is, to have doctrine create a single SQL query, not two. I could simply decrypt my messages in my view, but that kind of defeats the purpose of having a service layer in this case. I could surely fetch the messages "manually" and add them to the thread object, but if I could do it automatically through the association, then that would be much preferred.

Thanks in advance!

Answer 1

How about a slightly different approach than most have suggested:

Slice

In the Thread entity, have a dedicated method for returning slices of messages:

class Thread
{
    // ...

    /**
     * @param  int      $offset
     * @param  int|null $length
     * @return array
     */
    public function getSliceOfMessages($offset, $length = null)
    {
        return $this->messages->slice($offset, $length);
    }
}

This would take care of easily retrieving a slice in the view, without the risk of fetching the entire collection.

Decrypting message content

Next you need the decrypted content of the messages.

I suggest you create a service that can handle encryption/decryption, and have the Message entity depend on it.

class Message
{
    // ...

    /**
     * @var CryptService
     */
    protected $crypt;

    /**
     * @param CryptService $crypt
     */
    public function __construct(CryptService $crypt)
    {
        $this->crypt = $crypt;
    }
}

Now you have to create Message entities by passing a CryptService to it. You can manage that in the service that creates Message entities.

But this will only take care of Message entities that you instantiate, not the ones Doctrine instantiates. For this, you can use the PostLoad event.

Create an event-listener:

class SetCryptServiceOnMessageListener
{
    /**
     * @var CryptService
     */
    protected $crypt;

    /**
     * @param CryptService $crypt
     */
    public function __construct(CryptService $crypt)
    {
        $this->crypt = $crypt;
    }

    /**
     * @param LifecycleEventArgs $event
     */
    public function postLoad(LifecycleEventArgs $event)
    {
        $entity = $args->getObject();

        if ($entity instanceof Message) {
            $message->setCryptService($this->crypt);
        }
    }
}

This event-listener will inject a CryptService into the Message entity whenever Doctrine loads one.

Register the event-listener in the bootstrap/configuration phase of your application:

$eventListener = new SetCryptServiceOnMessageListener($crypt);
$eventManager  = $entityManager->getEventManager();
$eventManager->addEventListener(array(Events::postLoad), $eventListener);

Add the setter to the Message entity:

class Message
{
    // ...

    /**
     * @param CryptService $crypt
     */
    public function setCryptService(CryptService $crypt)
    {
        if ($this->crypt !== null) {
            throw new \RuntimeException('We already have a crypt service, you cannot swap it.');
        }

        $this->crypt = $crypt;
    }
}

As you can see, the setter safeguards against swapping out the CryptService (you only need to set it when none is present).

Now a Message entity will always have a CryptService as dependency, whether you or Doctrine instantiated it!

Finally we can use the CryptService to encrypt and decrypt the content:

class Message
{
    // ...

    /**
     * @param string $content
     */
    public function setContent($content)
    {
        $this->content = $this->crypt->encrypt($content);
    }     

    /**
     * @return string
     */
    public function getContent()
    {
        return $this->crypt->decrypt($this->content);
    }
}

Usage

In the view you can do something like this:

foreach ($thread->getSliceOfMessages(0, 10) as $message) {
    echo $message->getContent();
}

As you can see this is dead simple!

Another pro is that the content can only exist in encrypted form in a Message entity. You can never accidentally store unencrypted content in the database.

Answer 2

I believe you should have a different class for decrypted message and push them separately into your view. Because Message is actually your entity-model and should be used for mapping data into your db but your purpose is quite different.

Because as I understood from your code you make smth. like:

$message->setText(decrypt($message->getText));

and after that you making even worse

$thread->setMessages(new ArrayCollection($thread->getMessages()->slice(0, 10)));

Imagine then, that after you made this changes, somewhere in the code $em->flush() happens. You will loose every message except those 10, and they will be stored decrypted.

So you can push them as a separate array of decrypted messages (if you have only one thread on this view) or multi-dimensional array of messages with ThreadID as first level.

Or you can implement the answer Stock Overflaw gave you.

It's up to you, but you certainly should change your approach.

And by my opinion the best solution would be an implementation of decryption as ViewHelper.

Answer 3

It seems as if I figured it out. When calling slice, I get an array of entities. If I convert this array to an instance of \Doctrine\Common\Collections\ArrayCollection and call the setMessages method, it seems to work.

$messages = $thread->getMessages()->slice(0, 10);

foreach ($messages as $message) {
    // Decrypt message
}

$thread->setMessages(new ArrayCollection($messages));

Then, in my view, I can just call $thread->getMessages() as I normally would. This has the advantage that my view does not need to know that any decryption is happening or know about any other properties. The result is that only one SQL query is executed, which is the one "generated by" slice - exactly what I wanted.

Answer 4

The slice method's comment says:

Calling this method will only return the selected slice and NOT change the elements contained in the collection slice is called on.

So calling slice has no effect on the global PersistentCollection returned by your getMessages method: I don't think what you try to achieve here is doable.

As a workaround, you could declare a $availableMessages attribute in your Thread class, said attribute not being mapped to Doctrine. It would look like:

class Thread {
    /**
     * @var ArrayCollection
    */
    protected $availableMessages;
    ...
    public function __construct() {
        ...
        $this->availableMessages = new ArrayCollection();
    }
    ...
    public function getAvailableMessages() {
        return $this->availableMessages;
    }
    public function addAvailableMessage($m) {
        $this->availableMessages->add($m);
    }
    ...
}

When you process your messages in your service, you could:

$messages = $thread->getMessages()->slice(0, 10);
foreach ($messages as $message) {
    //do your process...
    ...
    //add the unpacked message to the proper "placeholder"
    $thread->addAvailableMessage($message);
}

Then in your view:

echo $this->partialLoop(
    'partial/thread/message.phtml',
    $thread->getAvailableMessages()
);

There might be some differences in your implementation, like you might prefer having an ID-indexed array instead of an ArrayCollection for $availableMessages, and/or use a set instead of an add method, but you get the idea here...

Anyway, this logic allows you to control the amount of output messages from the service layer, without any implication of later-called layers, which is what you want from what I understood :)

Hope this helps!