CODEWITHSUNDEEP
javadatabaseserializationthroughputprevayler
user2699942
user2699942

Reputation: 143

Prevayler Serialization Optimization

I'm working with Prevayler, doing some very simple stress testing.

On my program, I can get 10,000 reads in 500ms, but writing or changing one object takes 23ms. (I don't have detailed specs on this computer. Nothing impressive. Dual core 3.0ghz, 7200rpm HD, and likely 1333mhz ddr3 memory.)

Serialization seems to be the bottleneck.

What can be done to increase write/change performance?

JProfiler I ran the program in JProfiler (10,000 User Objects being written to disk.), it ran for 250 seconds, of that time 99.6% was used in running org.implementation.PrevaylerImpl.execute

UPDATE I switched over to XStreamSerializer, which increased write speed to 21ms per write, and decreased storage from

3555KB journal/10,000 Objects -> 2421KB

350KB snapshot/10,000 Objects -> 313KB

I would like to see 1000+ writes/second.

---- Main Method ----

Transactor trans = new Transactor();

    for(int i =0;i<10000;i++)
    {
       trans.createUser("Username", "PasswordHash");
    }  


---- Transactor Class ----

public void createUser(String Username, String PasswordHash) {    

    String id = Helper.randomID();

    //Avoids id duplicates, id is 7 digit base58. (Like Bitcoin, but shorter.)
    //ID generation takes .0078ms.
    while(getUser(id)!=null)
    {
        id=Helper.randomID();
    }

    prevayler.execute(new CreateUser(Username, PasswordHash,id));
}


---- CreateUser Class ----

public class CreateUser implements Transaction, Serializable{

    String Username;
    String PasswordHash;
    String id;

    public  CreateUser(String Username, String PasswordHash, String id)
    {
        this.Username = Username;
        this.PasswordHash = PasswordHash;
        this.id = id;
    }

    public void executeOn(Object core,Date date) {
          ((Core)core).store(new User(Username, PasswordHash), id);
    }
}


---- Core Class (Store method) ----

private final Map<String,Object> dataMap = new HashMap<String,Object>();

public void store(Object object, String id) {

    dataMap.put(id, object);

    //Adds User id to a seperate ArrayList<String>
    if(object instanceof User )
    {
      userList.add(id);
    }
}

Upvotes: 2

Views: 712

Answers (2)

Stephan Eggermont
Stephan Eggermont

Reputation: 15917

23ms is about right for one spinning disk. With Disc Sync, I guess a file is opened, the transaction logged, flushed to disk and file closed. If the disk head has to move for that (updating the directory information for instance), you look at those kinds of times. Random access to a sector of the disk is about 20ms, perhaps half of that for highest end drives.

It is often better to just flush to disk 10 times a second, then you can use that disk sectors are large enough to hold multiple transactions and that disks are very good at writing consecutive data, i.e. logs.

Upvotes: 3

user2699942
user2699942

Reputation: 143

I turned off Disk Sync (Which is on by default.), now I'm getting 2500 writes/sec on this computer. With an SSD I was getting 10,000 writes/sec. (Reads are much faster too.) That works for me, but I'm not sure why Disk Sync slows it down that much, maybe someone can speak to that?

public Transactor() {

    if (prevayler == null) {
        try {

            PrevaylerFactory<Core> factory = new PrevaylerFactory<Core>();
            factory.configurePrevalentSystem(new Core());

            //This
            factory.configureJournalDiskSync(false);

            factory.configureJournalSerializer("Journal",new XStreamSerializer());
            prevayler = factory.create();
...

Upvotes: 1

Related Questions