CODEWITHSUNDEEP
diskspaceapache-pulsarpulsar
punekr12
punekr12

Reputation: 743

Apache Pulsar disk usage in a container

I am running Apache pulsar in a Docker container in standalone mode.

I am publishing log file lines into the pulsar topic. The total logs are about 60G in size.

The pulsar container keeps running out of disk space on the / disk and has to terminate.

It is a default configuration. The docker-compose file looks as follows 

  pulsar:
    image: apachepulsar/pulsar
    container_name: pulsar
    command: bin/pulsar standalone
    ports:
        - 6650:6650
        - 8080:8080

Is there a way to limit pulsar disk usage so that it does not run out of all the space ?

Upvotes: 2

Views: 556

Answers (1)

Sergii Zhevzhyk
Sergii Zhevzhyk

Reputation: 4202

Apache Pulsar allows to specify retention policies and expiry for a namespace.

To specify a retention policy, you can use the Pulsar admin CLI:

bin/pulsar-admin namespaces set-retention my-tenant/my-namespace --size 10G --time 1d

Another way is to execute a POST request to the Pulsar REST API:

localhost:8080/admin/v2/namespaces/my-tenant/my-namespace/retention

where the body has the following JSON structure:

{
    "retentionTimeInMinutes": 1440,
    "retentionSizeInMB": 10240
}

Update the commands above with your values for my-tenant, my-namespace, localhost

Upvotes: 3

Related Questions