CODEWITHSUNDEEP
azure-cognitive-search
Aravind Bala
Aravind Bala

Reputation: 11

Size of Azure Search Index

I have a search service with 12 S1 with a total capacity of 300 GB. My index used to be around 240 GB, and I added about 12 GB of data (I added new fields to the existing index). The new size is 303 GB instead of the 252 GB of so that I would expect).

I understand that Azure search uses a merge process where the index size increases temporarily but then the merge catches up to reduce the index.

My question is how do I know whether it is a temporary increase in size or am I really running out of space? Also, is there a way to force the index merge process to reduce the index size?

Upvotes: 0

Views: 1738

Answers (1)

Ishan Srivastava
Ishan Srivastava

Reputation: 620

I work on the Azure Search team.

The first thing I want to call out is that adding 12 GB of raw content doesn't necessarily mean that the index size will increase only by 12 GB. The indexed data could be significantly larger depending on the schema and field attributes especially if fields are searchable.

Although, the service runs a merge periodically to store docs more efficiently and free up space, the index size from azure portal or the Service Statistics API is a good approximation of the actual index size as it already excludes the size that might be freed up. This process is non-deterministic and depends on multiple factors like indexing rate and size of the docs. The expected amount of reduction is also influenced by the type of operation (add, update, delete).

In short, there isn't any need to differentiate since storage size already reflects the actual size of the index. Also, there is no way for forcing a merge directly from the outside. This is intentional to ensure stability of the service.

Regardless, I will advise to do capacity planning according to the storage size statistic surfaced from Azure Search and keep some breathing room to ensure future indexing is not affected. If you are over the max storage limit or want more breathing room, consider upgrading to a higher SKU - Choose a pricing tier or SKU for Azure Search. Note that this will require a re-index of the data though. Hope this helps.

Upvotes: 1

Related Questions