CODEWITHSUNDEEP
phpsqldatabasepostgresqlconcurrency
Liqun
Liqun

Reputation: 4171

Obtain an unique sequence order number concurrently from PostgreSQL

We are designing an order management system, the order id is designed as a bigint with Postgresql, and the place structure is implemented as follows:

Take 2015072201000010001 as an order id example, the first eight places are considered as the date which is 20150722 here, the next seven places are considered as the region code which is 0100001 here, and the last four places are for the sequence number under the aforementioned region and date.

So every time a new order is created, the php logic application layer will query PostgreSQL with the following like sql statement:

select id from orders where id between 2015072201000010000 and 2015072201000019999 order by id desc limit 1 offset 0

then increase the id for the new order, after this insert the order to PostgreSQL database.

This is ok if there is only one order generation process at one time. But with hundreds of concurrent order generation request, there are such a lot of chances that the order ids will collide since the database read/write lock mechanism of PostgreSQL.

Let's say there are two order requests A and B. A tries to read the the latest order id from the database, then B reads the latest order id too, then A writes to the database, finally B writes to the db will failed since the order id primary key collides.

Any thoughts on how to make this order generation action concurrently feasible?

Upvotes: 1

Views: 1977

Answers (3)

Patrick
Patrick

Reputation: 32316

In the case of many concurrent operations your only option is to work with sequences. In this scenario you would need to create a sequence for every date and region. That sounds like a lot of work, but most of it can be automated.

Creating the sequences

You can name your sequences after the date and the region. So do something like:

CREATE SEQUENCE seq_201507220100001;

You should create a sequence for every combination of day and region. Do this in a function to avoid repetition. Run this function once for every day. You can do this ahead of time or - even better - do this in a scheduled job on a daily basis to create tomorrow's sequences. Assuming you do not need to back-date orders to previous days, you can drop yesterday's sequences in the same function.

CREATE FUNCTION make_and_drop_sequences() RETURNS void AS $$
DECLARE
  region    text;
  tomorrow  text;
  yesterday text;
BEGIN
  tomorrow  := to_char((CURRENT_DATE + 1)::date, 'YYYYMMDD');
  yesterday := to_char((CURRENT_DATE - 1)::date, 'YYYYMMDD');
  FOREACH region IN 
    SELECT DISTINCT region FROM table_with_regions
  LOOP
    EXECUTE format('CREATE SEQUENCE %I', 'seq_' || tomorrow || region);
    EXECUTE format('DROP SEQUENCE %I', 'seq_' || yesterday|| region);
  END LOOP;
  RETURN;
END;
$$ LANGUAGE plpgsql;

Using the sequences

In your PHP code you obviously know the date and the region you need to enter a new order id for. Make another function that generates a new value from the right sequence on the basis of the date and the region:

CREATE FUNCTION new_date_region_id (region text) RETURN bigint AS $$
DECLARE
  dt_reg  text;
  new_id  bigint;
BEGIN
  dt_reg := tochar(CURRENT_DATE, 'YYYYMMDD') || region;
  SELECT dt_reg::bigint * 10000 + nextval(quote_literal(dt_reg)) INTO new_id;
  RETURN new_id;
END;
$$ LANGUAGE plpgsql STRICT;

In PHP you then call:

SELECT new_date_region_id('0100001');

which will give the next available id for the specified region for today.

Upvotes: 3

Gary - Stand with Ukraine
Gary - Stand with Ukraine

Reputation: 7561

The usual way to avoid locking ids in Postgres is through the sequences.

You could use Postgresql sequences for each region. Something like

create sequence seq_0100001;

then you can get a number from that using:

select nextval('seq_'||regioncode) % 10000 as order_seq

That does mean the order numbers will not reset to 0001 each day, but you do have the same 0000 -> 9999 range for order numbers. It will wrap around.

So you may end up with:

2015072201000010001 -> 2015072201000017500 
2015072301000017501 -> 2015072301000019983
2015072401000019984 -> 2015072401000010293

Alternatively you could just generate a sequence for each day/region combination, but you'd need to be on top of dropping the previous days sequences at the start of next day.

Upvotes: 2

Trung Huynh
Trung Huynh

Reputation: 79

Try to use UUIDv1 type which is a combination of timestamp and MAC adress. You can have it auto-generated on server side if the order of inserts is important for you. Otherwise, the IDs can be generated from any of your clients before inserting (you might need their clock synchronized). Just be aware that with UUIDv1 is you can disclose the MAC address of the host where the UUID was generated. In this case, you may want to spoof the MAC address.

For your case, you can do something like

CREATE TABLE orders (
    id uuid PRIMARY KEY DEFAULT uuid_generate_v1(),
    created_at timestamp NOT NULL DEFAULT now(),
    region_code text NOT NULL REFERENCES...
    ...
);

Read more at http://www.postgresql.org/docs/9.4/static/uuid-ossp.html

Upvotes: -1

Related Questions