CODEWITHSUNDEEP
json
user6604655
user6604655

Reputation: 1043

is there an easy way to estimate size of a json object?

I'm working on a security service that will return a list of permissions and I'm trying to estimate the size of the json response object. Here's a piece of sample data:

ID=123 VariableName=CanAccessSomeContent

I'm looking for an easy way to estimate what size of the json response object will be with 1500 rows. Is there an online estimation tool or some other technique I can use to easily get a rough size estimate?

Upvotes: 9

Views: 22798

Answers (4)

Adi Stav
Adi Stav

Reputation: 41

I solved it, when I needed to, by adding a File-like object that just counted the characters and json.dump()ing into it:

# File-like object, throws away everything you write to it but keeps track of the size.
class MeterFile:
    def __init__(self, size=0):
        self.size = size

    def write(self, string):
        self.size += len(string)

# Calculates the JSON-encoded size of an object without storing it.
def json_size(obj, *args, **kwargs):
    mf = MeterFile()
    json.dump(obj, mf, *args, **kwargs)
    return mf.size

The advantage is that encoding is not stored in memory, which could be large especially in cases you care about the size to begin with.

Upvotes: 4

Shankar ARUL
Shankar ARUL

Reputation: 13720

Function to estimate file size (Mash of JSON-Size & UTF-8 Length node repos)

function json_filesize (value) {  
    // returns object size in bytes
    return (~-encodeURI(JSON.stringify(value)).split(/%..|./).length)/1048576
}

json_filesize({foo: 'bar'}) >> 13

Upvotes: 2

Nick M.
Nick M.

Reputation: 142

Using Python you can estimate the size by creating the dictionary or just make one... 

import json
import os
import sys

dict = {}

for a in range(0, 1500):
    dict[a] = {'VariableName': 'CanAccessSomeContent'}

output = json.dumps(dict, indent = 4)   

print ("Estimated size: " + str(sys.getsizeof(output) / 1024) + "KB")

with open( "test.json", 'wb') as outfile:
    outfile.write(output)

print ("Actual size: " + str(os.path.getsize('test.json') / 1024) + "KB")

Output:

Estimated size: 100KB
Actual size: 99KB

Upvotes: 10

Evert
Evert

Reputation: 99687

I'm not sure if this is what you're after, as this seems extremely basic, but here goes:

  1. First start with 0 rows, encode it and measure the size (We'll call this A).
  2. Then, get a decent sample from your database and encode 1 row at a time.
  3. For each of those outputs, calculate the size and then store the average (we'll call this B).

Now for X rows, the estimated json response will be X * (B-A) + A

So if A was 100 bytes, and B was 150 bytes, for 1500 rows we'll get:

1500 * (150-100) + 100 = 75100 bytes = 73 KB

Upvotes: 1

Related Questions