Unity nav mesh agent twitches at a certain direction

Question

I have a scene with an agent and a player. The agent follows the player, while the main camera also follows the player and the player moves by joystick on the screen. When the agent follows the player and its direction is right or down, everything works smoothly and clearly, but when the agent's direction is left or up, it starts to lag. Who knows what could be the problem?

player code:

void FixedUpdate()
{
    Vector2 movement = new (moveJoystick.Horizontal * speed, moveJoystick.Vertical * speed);
    rb.MovePosition(rb.position + movement * Time.fixedDeltaTime);
}

agent code:

void FixedUpdate()
{
    if (player != null)
    {
        agent.SetDestination(player.transform.position);
    }
}

camera code:

void LateUpdate()
{
    if (player != null)
    {
        Vector3 temp = transform.position;
        temp.x = player.position.x;
        temp.y = player.position.y;

        transform.position = temp;
    }
}

I tried everything I could, changed the camera movement code, the player. Nothing worked.