Symfony 4 - performance issue on form with a selectbox with many options

Question

I built a form with a selectbox (EntityType) with a big amount of choices (about 50 000) :

->add(
    'destination', EntityType::class, array(
        'label' => 'à', 
        'multiple' => false,
        'required' => false,
        'class' => Stop::class,
        'attr' => array(
            'class' => 'form-control',
        )
    )
);

I am facing a big performance issue : dozens of seconds before the list is displayed when I click on it.

I guess the solution would be to initially only load a few elements (e.g. a hundred), and then use Ajax to request the DB when the user starts typing (I am using a select2 box with search field). Problem is that I cannot figure out the most efficient way to do it through Symfony.

I have seen that the choice_loader functionality could do it, but there is no detailed documentation available : https://symfony.com/blog/new-in-symfony-3-2-lazy-loading-of-form-choices

Would be great if somebody can help on this,

Thanks for your support,

Answer 1

When I face this kind of trouble, I use another approach. If the select option will have more than 20 entries, so I change it to a Input Text with autocomplete.

The Steps are:

1. Install a good autocomplete Javascript lib like jquery-typeahead

   I like to use Wepack Encore in Symfony. With Webpack, Npm and Yarn the installation is easy like

   yarn add jquery-typeahead --dev
   

   You would need to run yarn run encore dev after installation.

2. Create a new FieldType for your form to replace the EntityType

   Lets suppose that we need to create a register form with city field. The default behaviour will use EntityType and show a Select Option with all cities. To change it to autocomplete lets create another FieldType.

   <?php    
   // src/Form/Type/AutocompleteCityType.php
   namespace App\Form\Type; 
   use Doctrine\ORM\EntityManagerInterface; 
   use Symfony\Component\Form\AbstractType; 
   use Symfony\Component\Form\Extension\Core\Type\SearchType; 
   use Symfony\Component\OptionsResolver\OptionsResolver; 
   class AutocompleteCityType extends AbstractType
   {
   
       public function configureOptions(OptionsResolver $resolver)
       {
           $resolver->setDefaults(array(
               'attr' => ['autocomplete' => 'off']
           ));
       }
   
       public function getParent()
       {
           return SearchType::class;
       }
   }
   

NOTE: On the code above, I am extending SearchType::class that is a Input Type Search (HTML 5).

Our new field type can be used on our forms but it is just another string field. Won't work correctly to replace EntityType. We need to convert this string to an Entity. So we need a DataTransformer

3. Create a City to String DataTransformer

   <?php
   // src/Form/DataTransformer/CityToStringTransformer.php    
   namespace App\Form\DataTransformer;                
   use App\Entity\City; // Pay attention to use your Entities correctly
   use Symfony\Component\Form\DataTransformerInterface;
   use Symfony\Component\Form\Exception\TransformationFailedException;
   
   class CityToStringTransformer implements DataTransformerInterface
   {
       private $entityManager;
   
       public function __construct(EntityManagerInterface $entityManager)
       {
           $this->entityManager = $entityManager;
       }
   
       /**
        * Transforms an object (City) to a string.
        *
        * @param  City|null $city
        * @return string
        */
       public function transform($city)
       {
           if (null === $city) {
               return '';
           }
   
           return $city->getSomethingUnique();
       }
   
       /**
        * Transforms a string to an object (city).
        *
        * @param  string $somethingUnique
        * @return City|null
        * @throws TransformationFailedException if object (city) is not found.
        */
       public function reverseTransform($somethingUnique)
       {
           // empty City? It's optional, so that's ok
           if (!$somethingUnique) {
               return;
           }        
   
           $city = $this->entityManager
                   ->getRepository(City::class)
                   ->findByThatSomethingUnique($somethingUnique);                
   
           if (null === $city) {
               // causes a validation error
               // this message is not shown to the user
               // see the invalid_message option
               throw new TransformationFailedException(sprintf(
                   'The city "%s" cannot be found!',
                   $somethingUnique
               ));
           }
   
           return $city;
       }
   }
   

Note: The string must be some kind of unique key to work correctly and need to be good to show on Autocomplete and fill the field (Like [CityCode] CityName). The DataTransformation cannot return more than one result on findByThatSomethingUnique() method.

Note 2 : For exemple, $city->getSomethingUnique() cant be $city->getId()."-".$city->getName() and ->findByThatSomethingUnique($somethingUnique) can be ->findOneById(explode("-", $somethingUnique)[0])

Almost done. We can use both classes on our FormType to replace EntityType.

4. Using on the FormType

   // src/Form/MyFormType.php
   
   // (...) Other declarations(...)
   use App\Form\DataTransformer\ContactToStringTransformer;
   use App\Form\Type\AutocompleteContactType;
   
   class MyFormType extends AbstractType
   {
       private $cityTransformer;
   
       public function __construct(CityToStringTransformer $cityTransformer)
       {
           $this->cityTransformer = $cityTransformer;
       }
   
       public function buildForm(FormBuilderInterface $builder, array $options)
       {
           /* @var $myEntity MyEntity */
           $myEntity = $builder->getData();
           $builder
               ->add('city', AutocompleteCityType::class, [
                   'label' => 'Custom City Label',
               ]) 
           // (...) Other fields (...)
           ;
   
           $builder->get('city')
               ->addModelTransformer($this->cityTransformer);
   
       }
   

With the code until here, the form will be shown correctly, but the typeahead must be configured properly.

You can create a new twig block type for this new field type. The code below must reside in the your custom form_theme

5. The Twig block

   {% block autocomplete_city_widget %}
       {% spaceless %}
           <div class="typeahead__container">
               <div class="typeahead__field">
                   <div class="typeahead__query">
                       {{ form_widget(form) }}
                   </div>
               </div>
           </div>
       {% endspaceless %}
   {% endblock %}
   

NOTE: The Twig code above is related to jquery-typeahead and only works with a field called AutocompleteCityType. If you install another lib or change the name of FieldType class, change it properly. Also pay attention to form names that change the block name to be rendered.

The last thing is to write the javascript for typeahead get the entries.

6. The Typeahead Javascript

   jQuery.typeahead({
       input: "#myForm_city", // Related to FormName and Autocomplete Field Name
       minLength: 1,
       maxItem: 20,
       order: "asc",
       dynamic: true,
       delay: 500,
       backdrop: { "background-color": "#eeeeee" },
       template: "<small style='color:#999;'>{{ '[{{citycode}}] {{cityname}}' }}</small>", // remember that this is a Twig template...
       emptyTemplate: "No results for typed string",
       source: {
           city: {
               display: ["citycode", "cityname"],
               ajax: function (query) {
                   return {
                       type: "POST",
                       url: '{{ path('controller_with_city_list_json_response') }}',
                       path: "city",
                       data: {
                           "q": "{{ '{{query}}' }}",
                           "length" : "40",
                       },
                       callback: {
                           done: function (res) {
                               var d = {};
                               d.city = [];
                               jQuery(res.data).each(function(index, value) {
                                   d.city.push(value.city);
                               });
                               return d;
                           }
                       }
                   }
               }
           }
       },
       callback: {
           onClickAfter: function (node, a, item, event) {
               event.preventDefault();
               jQuery(node).val("[" + item.citycode + "] " + item.cityname);
           }
       },
       debug: false
   });
   

NOTE: The Typeahead code above want a json response in format

{"data":
    [
        {"city":
            {"cityname":"City Name X", "citycode": "NXNX"}
        },
        {"city":
            {"cityname":"City Name Y", "citycode": "NYNY"}
        }
    ]
 }