How to send a local image instead of URL to Microsoft Computer Vision API using JAVA

Question

The question is how to load image file and pass it as object to Microsoft Computer Vision API, all the sample code in Microsoft website is reading image from url.

// This sample uses the Apache HTTP client library(org.apache.httpcomponents:httpclient:4.2.4)
// and the org.json library (org.json:json:20170516).
package com.mparnisari.test;

import java.awt.image.BufferedImage;
import java.io.File;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;

import javax.imageio.ImageIO;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.FileEntity;
import org.apache.http.entity.StringEntity;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.json.JSONObject;


public class Main
{    
    public static final String subscriptionKey = "MY-KEY-HERE";

    public static final String uriBase = 
        "https://westcentralus.api.cognitive.microsoft.com/vision/v1.0/analyze";

    public static void main(String[] args)
    {
        HttpClient httpclient = new DefaultHttpClient();

        try
        {
            URIBuilder builder = new URIBuilder(uriBase);

            builder.setParameter("visualFeatures", "Categories,Description,Color");
            builder.setParameter("language", "en");

            URI uri = builder.build();
            HttpPost request = new HttpPost(uri);

            request.setHeader("Content-Type", "application/json");
            request.setHeader("Ocp-Apim-Subscription-Key", subscriptionKey);

            // Request body.
            BufferedImage image = null;
            File f = null;
            f = new File("C:\\Coffee.jpg"); //image file path
            image = ImageIO.read(f);

            File file = new File("C:\\Coffee.jpg");

            FileEntity reqEntityF = 
                new FileEntity(file, ContentType.APPLICATION_OCTET_STREAM);

            request.setEntity(reqEntityF);

            HttpResponse response = httpclient.execute(request);
            HttpEntity entity = response.getEntity();

            if (entity != null)
            {
                // Format and display the JSON response.
                String jsonString = EntityUtils.toString(entity);
                JSONObject json = new JSONObject(jsonString);
                System.out.println("REST Response:\n");
                System.out.println(json.toString(2));
            }
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }
}

The output is:

REST Response:

{
  "code": "BadArgument",
  "requestId": "7ecf2198-1b7f-44d0-9cc2-e05e28791281",
  "message": "JSON format error."
}

As in other post in stackoverflow guide to use FileEntity to upload the image. But it is not working.

i think this part should somehow refactor to read image instead of a URL.

// Execute the REST API call and get the response entity.
HttpResponse response = httpclient.execute(request);
HttpEntity entity = response.getEntity();

Let me know what is best solution to solve this problem, because if its possible to pass the image from local to the API, it would be great to have a for loop to analyze an image set.

Answer 1

my version for Android (Java)

private static void doRequest(final String image){
    new AsyncTask<Void, Void, String>() {
        DataOutputStream request;
        HttpURLConnection connection;
        @Override
        protected String doInBackground(Void... voids) {
            try {
                URL url = new URL(requestUrl);
                boolean isUrl = image.contains("http");
                openConnection(url);
                setPostData(isUrl);
                setBody(image, isUrl);
                return getResponse();
            }catch (IOException e) {
                e.printStackTrace();
            }
            return null;
        }
            private void openConnection(URL url) throws IOException {
                connection = (HttpURLConnection) url.openConnection();
            }
            private void setPostData(boolean isUrl) throws ProtocolException {
                connection.setRequestMethod("POST");
                connection.setRequestProperty("details", "{string}");
                connection.setRequestProperty("Host", "westeurope.api.cognitive.microsoft.com");
                if(isUrl) {
                    connection.setRequestProperty("Content-Type", "application/json");
                }
                else {
                    connection.setRequestProperty("Content-Type", "application/octet-stream");
                }
                connection.setRequestProperty("Ocp-Apim-Subscription-Key", "yourKey");
            }
            private void setBody(String image, boolean isUrl) throws IOException {
                request = new DataOutputStream(connection.getOutputStream());
                if(isUrl){
                    request.writeBytes(urlBody(image));
                }
                else {
                    request.write(FileManager.getBytesFrom(image));
                }

            }
                private String urlBody(String image){
                    return  "{\"url\":\"" + image + "\"}";
                }
            private String getResponse() throws IOException{
                InputStream inputStream = connection.getInputStream();
                String response = inputStream2String(inputStream);
                return response;
            }
                private String inputStream2String(InputStream inputStream) throws IOException{
                    BufferedReader bufferedReader = bufferResult(inputStream);
                    return resultString(bufferedReader).toString();
                }
                    private StringBuilder resultString(BufferedReader bufferedReader) throws IOException {
                        StringBuilder stringBuilder = new StringBuilder();
                        String line;
                        while ((line = bufferedReader.readLine()) != null) {
                            stringBuilder.append(line);
                        }
                        return stringBuilder;
                    }
                    private BufferedReader bufferResult(InputStream inputStream){
                        InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
                        return new BufferedReader(inputStreamReader);
                    }

        @Override
        protected void onPostExecute(String result) {
            //...
        }

    }.execute();
}

Answer 2

Change

request.setHeader("Content-Type", "application/json");

To:

request.setHeader("Content-Type", "application/octet-stream");

and instead of using

StringEntity reqEntity = new StringEntity(url)

Use:

FileEntity reqEntity = new FileEntity(image, ContentType.APPLICATION_OCTET_STREAM);

after this changes just give the image path as a file

File image = new File("D:\\coffee3.jpg"); //image file path

and Microsoft computer vision api will send back the result as Json.

Answer 3

As @Jon said, you need to change the content-type header.

// Request headers.
request.setHeader("Content-Type", "application/octet-stream");

// Request body.
File file = new File(imagePath);
FileEntity reqEntity = new FileEntity(file);
request.setEntity(reqEntity);