CODEWITHSUNDEEP
javagoogle-app-engineindexinggoogle-cloud-datastore
Craig B
Craig B

Reputation: 1

Excessive Datastore Writes when creating

Hi asked a question on this yesterday and have now simplified the problem to see if it makes it easier to solve / diagnose.

I am trying to bulk create 500 JDO entities, each entity has a "Key" as primary key, two indexed Strings and one unindexed String.

When I create the 500 entities it uses 6% of the datastore write quoto (from a quoto of 500000) which is 30000 write operations which equates to 60 writes per entity. On the development server the "write ops" is 6 which agrees with the various comments made about datastore writes. There is no searching and no composite indexes, I'm just creating 500 entities.

So why do my entities take 60? The JDO Entity is as below:

import javax.jdo.annotations.Extension;
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;

import com.google.appengine.api.datastore.Key;

@PersistenceCapable
public class Product {

    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;
    @Persistent
    private String productId;
    @Persistent
    private String productName;

    @Persistent
    @Extension(vendorName="datanucleus", key="gae.unindexed", value="true")
    private String merchantProductId;   

      .
      .// getters and setters here but not included in post
      .
}

The 500 entities are created in the loop:

private void createBulkProducts(PrintWriter out){
    int cnt = 0;
    Product tmpProd = null;
    PersistenceManager pm = PMF.get().getPersistenceManager();
    ArrayList<Product> products = new ArrayList<Product>();
    for(cnt = 0; cnt < 500; cnt++){
        tmpProd = createBulkProduct(new Integer(cnt).toString());
        products.add(tmpProd);
    }
    if(products.isEmpty() == false){
        pm.makePersistentAll(products);
        out.println(products.size() + " bulk products committed to database");
    }
    pm.close();
    out.println(cnt + " bulk products inserted");           
}

private Product createBulkProduct(String cnt){
    Product product = new Product();
    product.setMerchantProductId("mid" + cnt);
    product.setProductId("pid" + cnt);
    product.setProductName("the quick brown fox jumps over the lazy dog " + cnt);
    return product;
}

I am running on GAE 1.6.1

All help / suggestions greatly appreciated Craig

Upvotes: 0

Views: 110

Answers (1)

Peter Knego
Peter Knego

Reputation: 80350

Daily quota for free apps is 50k write operations not 500k. See your Dashboard where it should say n.nn of 0.05 Million Ops.

Taken this into account all calculations are OK.

Upvotes: 1

Related Questions