Need to make multiple HTTP calls and merge results

Question

I have an issue which given my inexperience with Angular (v6.0), RxJs (v6.0) and observables in general is causing some considerable headache over the past week.

A service used by a component needs to output a data model collection but to do so will require four separate api calls, and I am having issues with related data that is optional stopping the return of the data model.

Basic flow

• The first api call gets the main entity and creates the initial data model collection.
• The second api call will go and get related data and add it to the data model.
• The third api call will go and get related data if it exists and add it to the data model.
• The fourth api call will go and get data related to the most recent related entry from the third api call. Data model collection is then returned.

Design

• For the second, third and fourth api calls a batch query method is used based on the an ID collection to reduce the number of calls, this is okay as we only request 10, 20 records at a time.

Expectations

• If there are no main entities returned from the first api call all following calls will not occur.
• There will always be data returned for the second api call.
• If no related data was returned from the third api call this is not a problem but the data model collection should be returned.
• If no related data was returned from the third api call the fourth api call will not happen but the data model collection should be returned.

Issues

• Ensuring the data model collection is returned when no results are found for the third and forth api calls occur.
• I can't change the api.

Known

• I do not have much in the way of error handling.
• There is a likely a lot of bugs, sorry.
• It is one long chain it would be better to be broken into smaller chunks.
• My experience with JavaScript is rusty.

Tried

• I have looked extensively at the '*map' functions and 'forkJoin' but have not been able to get anything working correctly, and with RxJs most examples are for versions less than 6.0 making it hard to translate.

Code
I would love to make this shorter but the way we call the api is important here.

return this.agentService.GetByFilter(query)
  .pipe(
    map(agents => {
      let deployments : AgentDeployment[] = [];

      for(let i = 0; i < agents.length; ++i) {          
        deployments.push(new AgentDeployment(agents[i].id, agents[i].name, agents[i].tags));
      };

      return deployments;
    }),
    map(deployments => {
      // Getting related tags assoiciated with the agent

      let tagIds : Array<number> = [];

      // Get tag id's
      for(let i = 0; i < deployments.length; ++i) {          
        if (deployments[i].tagIds) {
          for(let y = 0; y < deployments[i].tagIds.length; ++y) {
            if (!tagIds.includes(deployments[i].tagIds[y])) {
              tagIds.push(deployments[i].tagIds[y]);
            }              
          };
        }
      }

      // Get the tags associated with the agents
      let filter = new Filter();
      filter.Field = 'ID';
      filter.Value = tagIds.join(',');

      // Get tags using list of tag id's
      return this.tagService.GetByFilter(filter).pipe(
        map(tags => {        
          if (tags.length) {
            for(let i = 0; i < tags.length; ++i) {     
              // Find the agent
              deployments.forEach(deployment => {
                if (deployment.tagIds && deployment.tagIds.includes(tags[i].id)) {
                  deployment.tags.push(tags[i]);
                }
              });
            }              
          }
          return deployments;
        }));
    }),
    mergeMap(deployments => deployments),
    map(deployments => {
      // Getting deployments related to the agent, an agent may not have any deployment packages associated

      let agentIds : Array<number> = [];

      // Get tags
      for(let i = 0; i < deployments.length; ++i) {          
        if (deployments[i].id) {
            agentIds.push(deployments[i].id);
        }
      }

      // Get the deployment packages for the agents
      let filter = new Filter();
      filter.Field = 'AgentId';
      filter.Value = agentIds.join(',');

      // Get agent deployment packages using list of agent id's
      return this.agentDeploymentPkgService.GetByFilter(filter)
        .mergeMap(packages =>  packages)
        .groupBy(pkg => pkg.id)            
        .map(pkg$ => {
          if (pkg$) {
            // Add packages to agent deployment
            pkg$.map(pkg => {
            // Find the agent
            let i : number = deployments.findIndex(agent => agent.id == pkg$.key);

              if (pkg.deploymentPackageIds) {
                // Keep last deployment ID handy
                deployments[i].lastDeploymentId = Math.max.apply(null, pkg.deploymentPackageIds);;
                deployments[i].deployments.push(new Deployment(pkg.id, pkg.name))
              }
            }); 
          }
          return deployments;
        });
    }),
    mergeMap(deployments => deployments),
    map(deployments => {
      // Get the logs for the most recent deployment package for the agent, may not exist if no recent deployment package

      let deploymentPackageIds : Array<number> = [];

      // Get tags
      for(let i = 0; i < deployments.length; ++i) {          
        if (deployments[i].id) {
          deploymentPackageIds.push(deployments[i].lastDeploymentId);
        }
      }

      // Get the deployment packages for the agents
      let filter = new Filter();
      filter.Field = 'DeploymentPackageId';
      filter.Value = deploymentPackageIds.join(',');

      // Get agent deployment log based on the last 
      this.logService.GetByFilter(filter)
        .map(logs => {
          logs.forEach(log => {
            // Find the agent
            let i : number = deployments.findIndex(agent => agent.lastDeploymentId == log.deploymentPackageId);

            deployments[i].logId = log.id;
            deployments[i].status = log.lastReportedStatusMessage;
          });

          return deployments;
        });


      return deployments;
    })
  );

Answer 1

The quick answer here is

forkJoin

Check the documentation

Rxjs - ForkJoin