Why can't I decide which node to navigate to at runtime based on the value of state?

Question

Traceback (most recent call last):
  File "/Users/zuoyuxuan/Documents/Code/QINIU/agent_flow/swarm_workflow.py", line 46, in <module>
    graph.init_graph()
  File "/Users/zuoyuxuan/Documents/Code/QINIU/agent_flow/graph/CreatAgentGraph.py", line 65, in init_graph
    self.add_conditional_edges()
  File "/Users/zuoyuxuan/Documents/Code/QINIU/agent_flow/graph/CreatAgentGraph.py", line 56, in add_conditional_edges
    self.graph.add_conditional_edges(
  File "/Users/zuoyuxuan/anaconda3/envs/agent_flow/lib/python3.10/site-packages/langgraph/graph/graph.py", line 293, in add_conditional_edges
    raise ValueError(
ValueError: Branch with name `route_to_next_agent` already exists for node `router_agent`

#This is my route funciton:
def route_to_agent(self,state):
        if not state["next_agent"]: 
            return self.entry_agent_node
        last_message = state["messages"][-1]
        if isinstance(last_message,AIMessage):
            return END
        return state["next_agent"]

#This how I define conditional edges  
self.graph.add_conditional_edges(
            START,
            self.route_to_agent,
            {agent.name: agent.name for agent in self.agent_factory.get_all_agents().values()}
        )
for agent in self.agent_factory.get_all_agents().values():
    self.graph.add_conditional_edges(
        agent.name,
        self.route_to_agent,
        {next_agent: next_agent for next_agent in agent.next_agents}
    )

I want to define nodes based on the order of agents, and then navigate to the next node through the next node next_agent determined by the agent at runtime, but this fails in langgraph