CODEWITHSUNDEEP
c#wcfsoapchunks
tesicg
tesicg

Reputation: 4053

How to read big amount of data in chunks?

I have a project where I should use external WCF service that has method that looks as following:

Items catalogItems = externalClient.getCatalogItems(auth, idCatalog, 1, 100);

After I call getCatalogItems service method, I should transform the returned array of items to raw SOAP message in this manner:

Message response = Message.CreateMessage(MessageVersion.Default, ReplyAction_GetCatalogItems, catalogItems);

The last 2 parameters in getCatalogItems service method designates the size of chunk of data that should be obtained in each call. For example, if we have 1050 records all of them should be obtained 10 times in chunks of 100 and 1 time in chunk of 50.

I understand I should read the data until they are available. I have 2 questions:

  1. How do I know where I should continue to read? For example, if I've read first portion of 100 records, how do I know where's the current position of reader?

  2. How do I know when I reach the end?

Upvotes: 2

Views: 325

Answers (1)

coder_bro
coder_bro

Reputation: 10773

One approach would be to make it the responsibility of the client to remember the state (ie the page number where the client currently is).

So you can change your method call to include a page number and items per page parameters:

Items catalogItems = externalClient.getCatalogItems(auth, idCatalog, pageNumber, itemsPerPage);

The service can then essentially select a set of items based on the pageNumber and itemsPerPage values and it need not hold the state. (Note: this can be easily translated to a select query if you are using a database as repository for the items)

You can possibly alter the return value to include the total number of items as well: Example:

CatalogResponse respone = externalClient.getCatalogItems(auth, idCatalog, pageNumber, itemsPerPage);

public class CatalogResponse 
{
   private _totalItems;
   private _items;
}

This also provides the flexibility for the client to determine the chunk of items to receive in each call and to the end user to select a page size.

Upvotes: 1

Related Questions