Various errors using VisionServiceClient in XamarinForms

Question

I am trying to create a simple Xamarin forms app which allows the user to browse for or take a photo and have azure cognitive services tag the photo using a custom vision model.

I am unable to get the client to successfully authenticate or find a resource per the error message in the exception produced by the VisionServiceClient. Am I missing something? What would be the correct values to use for the arguments to VisionServiceClient?

All keys have been removed from the below images, they are populated.

Exception thrown in VS2017:

'Microsoft.ProjectOxford.Vision.ClientException' in System.Private.CoreLib.dll

Call to VisionServiceClient:

private const string endpoint = @"https://eastus2.api.cognitive.microsoft.com/vision/prediction/v1.0";
private const string key = "";

VisionServiceClient visionClient = new VisionServiceClient(key, endpoint);
VisualFeature[] features = { VisualFeature.Tags, VisualFeature.Categories, VisualFeature.Description };
try
{
     AnalysisResult temp = await visionClient.AnalyzeImageAsync(imageStream,                    
     features.ToList(), null);

     return temp;
}
catch(Exception ex)
{
     return null;
}

VS Exception Error:

Azure Portal for cognitive services:

Custom Vision Portal:

Answer 1

Thank you to cthrash for the extended help and talking with me in chat. Using his post along with a little troubleshooting I have figured out what works for me. The code is super clunky but it was just to test and make sure I'm able to do this. To answer the question:

Nuget packages and classes

Using cthrash's post I was able to get both the training and prediction nuget packages installed, which are the correct packages for this particular application. I needed the following classes:

Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models

Endpoint Root

Following some of the steps Here I determined that the endpoint URL's only need to be the root, not the full URL provided in the Custom Vision Portal. For instance,

https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/

Was changed to

https://southcentralus.api.cognitive.microsoft.com

I used both the key and endpoint from the Custom Vision Portal and making that change I was able to use both a training and prediction client to pull the projects and iterations.

Getting Project Id

In order to use CustomVisionPredictionClient.PredictImageAsync you need a Guid for the project id and an iteration id if a default iteration is not set in the portal.

I tested two ways to get the project id,

   Using project id string from portal

1. Grab the project id string from the portal under the project settings.
2. For the first argument to PredictImageAsync pass

   Guid.Parse(projectId)

   Using the training client

1. Create a new CustomVisionTrainingClient

2. To get a list of <Project> use

   TrainingClient.GetProjects().ToList()

3. In my case I only had a single project so I would just need the first element.

   Guid projectId = projects[0].Id

Getting Iteration Id

To get the iteration id of a project you need the CustomVisionTrainingClient.

1. Create the client
2. To get a list of <Iteration> use

   client.GetIterations(projectId).ToList()

3. In my case I had only a single iteration so I just need the first element.

   Guid iterationId = iterations[0].Id

I am now able to use my model to classify images. In the code below, fileStream is the image stream passed to the model.

public async Task<string> Predict(Stream fileStream)
{
    string projectId = "";
    //string trainingEndpoint = "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.2/Training/";
    string trainingEndpoint = "https://southcentralus.api.cognitive.microsoft.com/";
    string trainingKey = "";
    //string predictionEndpoint = "https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/";
    string predictionEndpoint = "https://southcentralus.api.cognitive.microsoft.com";
    string predictionKey = "";

    CustomVisionTrainingClient trainingClient = new CustomVisionTrainingClient
    {
        ApiKey = trainingKey,
        Endpoint = trainingEndpoint
    };

    List<Project> projects = new List<Project>();

    try
    {
        projects = trainingClient.GetProjects().ToList();
    }
    catch(Exception ex)
    {
        Debug.WriteLine("Unable to get projects:\n\n" + ex.Message);
        return "Unable to obtain projects.";
    }

    Guid ProjectId = Guid.Empty;

    if(projects.Count > 0)
    {
        ProjectId = projects[0].Id;
    }

    if (ProjectId == Guid.Empty)
    {
        Debug.WriteLine("Unable to obtain project ID");
        return "Unable to obtain project id.";
    }

    List<Iteration> iterations = new List<Iteration>();

    try
    {
        iterations = trainingClient.GetIterations(ProjectId).ToList();
    }
    catch(Exception ex)
    {
        Debug.WriteLine("Unable to obtain iterations.");
        return "Unable to obtain iterations.";
    }

    foreach(Iteration itr in iterations)
    {
        Debug.WriteLine(itr.Name + "\t" + itr.Id + "\n");
    }

    Guid iteration = Guid.Empty;

    if(iterations.Count > 0)
    {
        iteration = iterations[0].Id;
    }

    if(iteration == Guid.Empty)
    {
        Debug.WriteLine("Unable to obtain project iteration.");
        return "Unable to obtain project iteration";
    }

    CustomVisionPredictionClient predictionClient = new CustomVisionPredictionClient
    {
        ApiKey = predictionKey,
        Endpoint = predictionEndpoint
    };

    var result = await predictionClient.PredictImageAsync(Guid.Parse(projectId), fileStream, iteration);

    string resultStr = string.Empty;

    foreach(PredictionModel pred in result.Predictions)
    {
        if(pred.Probability >= 0.85)
            resultStr += pred.TagName + " ";
    }

    return resultStr;
}

Answer 2

It looks like you're confusing the Computer Vision and the Custom Vision APIs. You are attempting to use the client SDK for the former using the API key of the latter.

For .NET languages, you'll want the Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction NuGet package.

Your code will end up looking something like this:

ICustomVisionPredictionClient client = new CustomVisionPredictionClient()
{
    ApiKey = PredictionKey,
    Endpoint = "https://southcentralus.api.cognitive.microsoft.com"
};
ImagePrediction prediction = await client.PredictImageAsync(ProjectId, stream, IterationId);