How to deserialise or format WebAPI result into specific json structure

Question

I was working with a .net core 3.1 Web API. Which is getting data from an external API. Following is my code Controller part

[HttpGet("transinfo/{id}")]        
public Object GettransactionData(int id)
{
  var result=_transaction.GettransactionDetails(id).Result;
  List<PipeLineResponse> P = JsonConvert.DeserializeObject<List<PipeLineResponse>>(result.ToString());
  PipeLineResponseObject P1 = new PipeLineResponseObject();
  P1.data = P;
  return P1;
}

And my service code as follows

public async Task<Object> GettransactionDetails(int id)
{
  string request=//fetched from db 
  var stringContent = new StringContent(request);            
  Client = utilities.GetHttpClient();
  string apiEndpoint=//External API URL
  HttpResponseMessage httpResponseMessage = await Client.PostAsync(apiEndpoint, stringContent);
  if (httpResponseMessage.IsSuccessStatusCode)
  {
      return await httpResponseMessage.Content.ReadAsAsync<Object>();     
  }
}

But i am getting the result in following format (response from postman)

{
  "data": [
    {
      "Tranid": "34540d40-7db8-44c1-9a2a-5072c2d01756",
      "fields": {
        "Fields.10": "1001",
        "Fields.11": "Test1",
        "Fields.12": "Fixed1"
      }
    },
    {
      "Tranid": "145800f9-c4a5-4625-84d7-29af5e674a14",
      "fields": {
        "Fields.10": "1002",
        "Fields.11": "Test2",
        "Fields.12": "Fixed2"
      }
    }
  ]
}

But i need the data in following format

{
  "data": [
    {
      "TransactionID": "34540d40-7db8-44c1-9a2a-5072c2d01756",
      "fieldsList": [
        {
          "fieldId": "10",
          "fieldValue": "1001"
        },
        {
          "fieldId": "11",
          "fieldValue": "Test1"
        },
        {
          "fieldId": "12",
          "fieldValue": "Fixed1"
        }
      ]
    },
    {
      "TransactionID": "145800f9-c4a5-4625-84d7-29af5e674a14",
      "fieldsList": [
        {
          "fieldId": "10",
          "fieldValue": "1002"
        },
        {
          "fieldId": "11",
          "fieldValue": "Test2"
        },
        {
          "fieldId": "12",
          "fieldValue": "Fixed2"
        }
      ]
    }
  ]
}

How can i achieve this ? is possible to deserialise using JObject or JArray? Please help. i have tried to create following model class and tried to deserialise but not getting result as expected.

public class PipeLineResponse
{
   public string TransactionID { get; set; }
   public List<Dictionary<string, string>> fields { get; set; }
}    

public class PipeLineResponseObject
{
 public List<PipeLineResponse> data { get; set; }
}

How to create that json in that format any DTO or Automapper will work ? Please help me with samples.

Answer 1

The solution that I am laying down here takes the DTO approach. The response from the service is being deserialized to the DTO, which further is being manually mapped to the final ViewModel that we are sending to the client. By no means, this implementation is production-ready and there is scope for improvement, for which I am adding in comments. But this gives a detailed understanding of how we can handle these kind of scenarios. We are making use of Newtonsoft.Json, which can be pulled into your project via the NuGet package manager.

Structure of the DTO

// RootDTO.cs
// This structure is directly based on the response obtained from remote service.

public class Fields
{
    [JsonProperty(PropertyName ="Fields.10")]
    public string Fields10 { get; set; }
    [JsonProperty(PropertyName = "Fields.11")]
    public string Fields11 { get; set; }
    [JsonProperty(PropertyName = "Fields.12")]
    public string Fields12 { get; set; } 
}

public class Datum
{
    public string Tranid { get; set; }
    public Fields fields { get; set; }
}

public class RootDTO
{
    [JsonProperty(PropertyName ="data")]
    public List<Datum> data { get; set; }
}

Structure of ViewModel

// PipelineResponse.cs
public class FieldsList
{
    public string fieldId { get; set; }
    public string fieldValue { get; set; }
}

public class ResponseDatum
{
    [JsonProperty(PropertyName = "TransactionID")]
    public string TransactionID { get; set; }
    public List<FieldsList> fieldsList { get; set; }
}
   
public class PipelineResponse
{        
    public List<ResponseDatum> data { get; set; }
}

Deserializing the response to the DTO

// ...other code
var responseString = await httpResponseMessage.Content.ReadAsAsync<Object>();
// This is where the DTO object is created. This should be mapped to view model type.
var responseDTO = JsonConvert.DeserializeObject<RootDTO>(responseString);

Mapping the DTO to ViewModel

The mapping from DTO type to ViewModel type needs to be done before sending the response to the client. It is the view model type that is sent to the client. This logic can be placed within a separate helper (ideally, to separate concerns) or any other location as per the practices you are following.

public PipelineResponse ConvertResponseDTOToResponse(RootDTO responseDTO)
{
    // FieldId is being hardcoded here. Instead, you can use Reflection to 
    // fetch the property name, split on '.' and take the item at index 1.
    // Notice that DTO properties have "JsonProperty" attributes for this.
    try
    {        
        List<ResponseDatum> responseList = new List<ResponseDatum>();
        if (responseDTO != null)
        {
            // Reflection can be used to avoid hardcoding on 'fieldId'
            foreach (var item in responseDTO.data)
            {
                var responseDataObj = new ResponseDatum
                {
                    TransactionID = item.Tranid,
                    fieldsList = new List<FieldsList>
                    {
                        new FieldsList
                        {
                            fieldValue = item.fields.Fields10,
                            fieldId = "10"
                        },
                        new FieldsList
                        {
                            fieldValue = item.fields.Fields11,
                            fieldId = "11"
                        },
                        new FieldsList
                        {
                            fieldValue = item.fields.Fields12,
                            fieldId = "12"
                        }
                    }
                };
                responseList.Add(responseDataObj);
            }
        }

        // This object is what you return from your controller endpoint finally.
        // The serialized response of this object is of the json structure you need 
        return new PipelineResponse { data = responseList };
    }
    catch (Exception ex)
    {
        throw ex;
    }
}