Q-Learning Issue: Flow and Pressure Relationship in Glue Dispensing

Question

I am working on a Q-learning agent to optimize glue dispensing with pressure adjustments. However, I'm facing an issue where the flow rate seems inconsistent with the pressure changes. Specifically, the flow rate increases when the pressure decreases, which seems counterintuitive as I expect the flow rate to decrease with lower pressure.

Here is a brief description of my environment:

Environment Details: The pressure ranges between 25 to 40 units. The target flow is 2997.The flow calculation includes noise and degradation factors.

Problem: The Q-table doesn't reflect the expected behavior between states and actions. When pressure decreases, the flow rate sometimes unexpectedly increases.

Reward Function: The reward is calculated as the negative absolute difference between the target flow rate and the actual flow, aiming to minimize the error.

Attempts to Fix: I have tried adjusting the reward function and tweaking state-action definitions, but the inconsistency remains.

Questions: How can I make the flow rate consistently decrease when the pressure is reduced? Are there any recommended approaches to adjust the reward function or state scaling to achieve more realistic behavior?

Is there a way to better align the Q-table actions with the expected state transitions? Any insights or suggestions would be greatly appreciated!