TensorFlow RNN limitations in regards to grouped data

Question

This is a bit of an abstract problem and I hope someone has a good answer if it's feasable to start with, I'm planning to implement it as a RNN using TensorFlowSharp wrapper.

However with implementing this I ran into a few questions that are pretty much the basis of the platform; and I would like to avoid some pitfalls this early ;)

I have a lot of data that can be grouped up, however how I can enforce a weight/bias that data from another group does not get included, i.e. always stay within the group .. or .. should I use a session for each group and only feed/learn it data that pertains to that group (55+ groups atm.).

What would be the best course of action, let it learn everything and hope it learns that it should stick with its group, or enforce it in some way or split it up.

And if anyone knows, how would the performance be affected between the three.