How do I select from a set of uncertain guesses?

Question

I'm trying to determine if this is a solved problem or not and what algorithms I should be searching.

The issues is this: I send a resource to a remove service and it responds with a number of guesses at what it is, each with its own percentage level of certainty. Some of these guesses will be aliases for others.

Thus, if the service guesses that the resource is an A with one guess at 80% certainty, but a B with 3 guesses at 40%, 50% and 60% (for instance). There may also be other single guesses at lower percentages.

In such an instance, what algorithm would apply that would allow be to choose between A and B? Alternatively, what field of probability would apply?

At first, I thought about The Two Generals problem, but that's more to do with uncertainty in delivery as opposed to uncertainty of content.

If this is a solved problem, what algorithm should I be investigating?

Edit

Per comments and answers below, here's some more information.

• The guesses are independant of one another
• The individual guesses do not add up to 100%. Each guess is an attempt to match the incoming resource to one that exists within a collection. The certainty is simply on how accurate the match is.
• Each of these resources (which are unique) has some metadata that identifies it... a text label, if you will. These may not be unique. Thus, multiple unique resources may have the same label, thence the aliasing.

Answer 1

The "best" answer depends on how you interpret multiple guesses (guesses with different aliases) for the same resource. If you think that each guessed alias is a fresh "independent" guess, not just a partial duplicate guess influenced by another guess, then the appropriate way to interpret your reported "probabilities/confidences" of guesses is to add up all the percentages for all aliases guessed for a resource, and whatever resource has the highest total probability score, choose that one.

It's going to be hard to give a better answer unless you can give some explanation about how the reported "probabilities/confidences" for guesses are computed, because for example ideally all the percentages for all guesses should add up to 100% (or less) if they really are probability scores for each guess and there is only one correct resource that matches your query. If your "probability/confidence" scores add up to more than 100%, then there may be a better way to interpret them, if you can tell us how they are computed/what they really "mean."