In CakePHP 3, how can I create a new Entity with BelongsToMany records including join data?

Question

In the parlance of the classic BelongsToMany example, I'm trying to create a new Student with associated Courses records, and final_grade data for each of the Student's Courses. (The Courses records themselves already exist.) When I attempt to create the Student, I'm sending courses[0][_joinData][course_id] and courses[0][_joinData][final_grade], but when it tries to save the new CoursesStudents record, the validation complains that it doesn't have a student_id field. I'm not able to pass it in, because it doesn't exist yet!

I can associate the new Student records with existing Courses by sending data as courses[_ids], and that correctly picks up the newly provisioned Course's id for the course_id for the join table's new record. But I haven't been able to figure out a way to include non-default join data.

(My real models are AccessGroups and Markets, but the relationships are the same. I want to be able to associate a new AccessGroup with an existing Market at a specific access_level at the time of creation, without having to generate the AccessGroupsMarkets record after the fact.)

Edit

Code:

I'm adding my code and the responses I'm getting back from CakePHP, as suggested by @burzum:

// in a controller
$accessGroupsTable = TableRegistry::get('AccessGroups');

$data = [
    'name' => 'Test for Access Group with Market Join Data',
    'access_group_type_id' => '1',
    'project_id' => '49',
    'markets' => [
        0 => [
            'id' => '46',
            '_joinData' => [
                'market_id' => '46',
                'access_level' => '2',
            ]
        ],
        1 => [
            'id' => '47',
            '_joinData' => [
                'market_id' => '47',
                'access_level' => '2',
            ]
        ]
    ],
];

$newAccessGroup = $accessGroupsTable->newEntity($data);
debug(["newAccessGroup", $newAccessGroup], false); // debug!
$saved = $accessGroupsTable->save($newAccessGroup);
debug(["saved", $saved], false); // debug!

Response:

/src/Controller/AccessGroupsController.php (line 161)
########## DEBUG ##########
[
    (int) 0 => 'newAccessGroup',
    (int) 1 => object(App\Model\Entity\AccessGroup) {

        'name' => 'Test for Access Group with Market Join Data',
        'access_group_type_id' => (int) 1,
        'project_id' => (int) 49,
        'markets' => [
            (int) 0 => object(App\Model\Entity\Market) {

                'id' => (int) 46,
                'project_id' => (int) 49,
                'name' => 'United States - Flammmo',
                'created' => object(Cake\I18n\Time) {

                    'time' => '2016-05-13T18:42:11+00:00',
                    'timezone' => 'UTC',
                    'fixedNowTime' => false

                },
                'modified' => object(Cake\I18n\Time) {

                    'time' => '2016-05-13T18:42:11+00:00',
                    'timezone' => 'UTC',
                    'fixedNowTime' => false

                },
                'deleted' => null,
                '_joinData' => object(App\Model\Entity\AccessGroupsMarket) {

                    'market_id' => (int) 46,
                    'access_level' => (int) 2,
                    '[new]' => true,
                    '[accessible]' => [
                        'market_id' => true,
                        'access_group_id' => true,
                        'access_level' => true
                    ],
                    '[dirty]' => [
                        'market_id' => true,
                        'access_level' => true
                    ],
                    '[original]' => [],
                    '[virtual]' => [],
                    '[errors]' => [
                        'access_group_id' => [
                            '_required' => 'This field is required'
                        ]
                    ],
                    '[invalid]' => [],
                    '[repository]' => 'AccessGroupsMarkets'

                },
                '[new]' => false,
                '[accessible]' => [
                    '*' => true
                ],
                '[dirty]' => [
                    '_joinData' => true
                ],
                '[original]' => [
                    '_joinData' => [
                        'market_id' => '46',
                        'access_level' => '2'
                    ]
                ],
                '[virtual]' => [
                    (int) 0 => 'flag_url'
                ],
                '[errors]' => [],
                '[invalid]' => [],
                '[repository]' => 'Markets'

            },
            (int) 1 => object(App\Model\Entity\Market) {

                'id' => (int) 47,
                'project_id' => (int) 49,
                'name' => 'Indonesia - Flammmo',
                'created' => object(Cake\I18n\Time) {

                    'time' => '2016-06-24T16:24:57+00:00',
                    'timezone' => 'UTC',
                    'fixedNowTime' => false

                },
                'modified' => object(Cake\I18n\Time) {

                    'time' => '2016-06-24T16:24:57+00:00',
                    'timezone' => 'UTC',
                    'fixedNowTime' => false

                },
                'deleted' => null,
                '_joinData' => object(App\Model\Entity\AccessGroupsMarket) {

                    'market_id' => (int) 47,
                    'access_level' => (int) 2,
                    '[new]' => true,
                    '[accessible]' => [
                        'market_id' => true,
                        'access_group_id' => true,
                        'access_level' => true
                    ],
                    '[dirty]' => [
                        'market_id' => true,
                        'access_level' => true
                    ],
                    '[original]' => [],
                    '[virtual]' => [],
                    '[errors]' => [
                        'access_group_id' => [
                            '_required' => 'This field is required'
                        ]
                    ],
                    '[invalid]' => [],
                    '[repository]' => 'AccessGroupsMarkets'

                },
                '[new]' => false,
                '[accessible]' => [
                    '*' => true
                ],
                '[dirty]' => [
                    '_joinData' => true
                ],
                '[original]' => [
                    '_joinData' => [
                        'market_id' => '47',
                        'access_level' => '2'
                    ]
                ],
                '[virtual]' => [
                    (int) 0 => 'flag_url'
                ],
                '[errors]' => [],
                '[invalid]' => [],
                '[repository]' => 'Markets'

            }
        ],
        '[new]' => true,
        '[accessible]' => [
            'project_id' => true,
            'access_group_type_id' => true,
            'name' => true,
            'slug' => true,
            'features' => true,
            'assets' => true,
            'markets' => true,
            'child_groups' => true,
            'users' => true
        ],
        '[dirty]' => [
            'name' => true,
            'access_group_type_id' => true,
            'project_id' => true,
            'markets' => true
        ],
        '[original]' => [],
        '[virtual]' => [],
        '[errors]' => [],
        '[invalid]' => [],
        '[repository]' => 'AccessGroups'

    }
]
###########################
/src/Controller/AccessGroupsController.php (line 177)
########## DEBUG ##########
[
    (int) 0 => 'saved',
    (int) 1 => false
]
###########################

Note the validation errors on both of the markets' _joinData access_group_id

Working code that's not quite what I want

$accessGroupsTable = TableRegistry::get('AccessGroups');

$data2 = [
    'name' => 'Test for Access Group with Market IDs',
    'access_group_type_id' => '1',
    'project_id' => '49',
    'markets' => [
        '_ids' => [ 46, 47 ],
    ],
];

$newAccessGroup = $accessGroupsTable->newEntity($data2);
$saved = $accessGroupsTable->save($newAccessGroup);

This code successfully creates the new AccessGroup, along with the associated new AccessGroupsMarkets records, but with the default values for access_level, not the value that I'd like to be able to specify.

@systematical asked about the table setup. They were generated from a migration and then cake/bake'd (see the Table associations)

Answer 1

The problem (pointed out by @npm on IRC #cakephp) was that I was doing validation on the foreign keys in my join table's model:

// in AccessGroupsMarketsTable::validationDefault
$validator
    ->requirePresence('access_group_id', 'create')
    ->notEmpty('access_group_id')
    ->add('access_group_id', 'valid', ['rule' => 'numeric']);

$validator
    ->requirePresence('market_id', 'create')
    ->notEmpty('market_id')
    ->add('market_id', 'valid', ['rule' => 'numeric']);

This was in addition to the $rules in buildRules:

$rules->add($rules->existsIn(['access_group_id'], 'AccessGroups'));
$rules->add($rules->existsIn(['market_id'], 'Markets'));

@npm suggested that I remove the validation for those foreign keys (keeping the $rules in place). And his suggestion works. I can now save the new AccessGroups entity with its related AccessGroupsMarkets records and their associated access_level information.