nixmind
nixmind

Reputation: 2266

Postgresql Automatic archive purge not occuring

I read several document on postgresql archive recovery and cleanup but my postgresql server still don't purge archive, or I didn't really understand how it works.

Simply, I wrote a shell script for WAL archive cleanup. When I run the script from the command line, it works and purges my archive (I leave only archive newer than three days). My script is named pg_archive_cleanup, is executable and put here : /usr/sbin/

And I configuire my /etc/postgresql/9.3/main/recovery.conf like this :

# -------------------------------
# PostgreSQL recovery config file
# -------------------------------
#
# Edit this file to provide the parameters that PostgreSQL needs to
# perform an archive recovery of a database, or to act as a replication
# standby.
#
# If "recovery.conf" is present in the PostgreSQL data directory, it is
# read on postmaster startup.  After successful recovery, it is renamed
# to "recovery.done" to ensure that we do not accidentally re-enter
# archive recovery or standby mode.
#
# This file consists of lines of the form:
#
#   name = value
#
# Comments are introduced with '#'.
#
# The complete list of option names and allowed values can be found
# in the PostgreSQL documentation.
#
#---------------------------------------------------------------------------
# ARCHIVE RECOVERY PARAMETERS
#---------------------------------------------------------------------------
# archive_cleanup_command
#
# specifies an optional shell command to execute at every restartpoint.
# This can be useful for cleaning up the archive of a standby server.
#
archive_cleanup_command = '/usr/sbin/pg_archive_cleanup'
#
#---------------------------------------------------------------------------
# RECOVERY TARGET PARAMETERS
#---------------------------------------------------------------------------
#
# By default, recovery will rollforward to the end of the WAL log.
# If you want to stop rollforward at a specific point, you
# must set a recovery target.
#
# You may set a recovery target either by transactionId, by name,
# or by timestamp. Recovery may either include or exclude the
# transaction(s) with the recovery target value (ie, stop either
# just after or just before the given target, respectively).
#
#
#recovery_target_name = ''      # e.g. 'daily backup 2011-01-26'
#
#recovery_target_time = ''      # e.g. '2004-07-14 22:39:00 EST'
#
#recovery_target_xid = ''
#
#recovery_target_inclusive = true
#
#
# If you want to recover into a timeline other than the "main line" shown in
# pg_control, specify the timeline number here, or write 'latest' to get
# the latest branch for which there's a history file.
#
#recovery_target_timeline = 'latest'
#
#
# If pause_at_recovery_target is enabled, recovery will pause when
# the recovery target is reached. The pause state will continue until
# pg_xlog_replay_resume() is called. This setting has no effect if
# hot standby is not enabled, or if no recovery target is set.
#
#pause_at_recovery_target = true
#
#---------------------------------------------------------------------------
# STANDBY SERVER PARAMETERS
#---------------------------------------------------------------------------
#
# standby_mode
#
# When standby_mode is enabled, the PostgreSQL server will work as a
# standby. It will continuously wait for the additional XLOG records, using
# restore_command and/or primary_conninfo.
#
standby_mode = on
restore_command = 'cp /var/lib/postgresql/database/archive/%f "%p"'
#
# primary_conninfo
#
# If set, the PostgreSQL server will try to connect to the primary using this
# connection string and receive XLOG records continuously.
#
primary_conninfo = 'host=db-master port=5432 user=repli password=Esibfegiav4'           # e.g. 'host=localhost port=5432'
#
#
# By default, a standby server keeps restoring XLOG records from the
# primary indefinitely. If you want to stop the standby mode, finish recovery
# and open the system in read/write mode, specify path to a trigger file.
# The server will poll the trigger file path periodically and start as a
# primary server when it's found.
#
trigger_file = '/var/lib/postgresql/database/failover_trigger'
#
#---------------------------------------------------------------------------
# HOT STANDBY PARAMETERS
#---------------------------------------------------------------------------
#
# Hot Standby related parameters are listed in postgresql.conf
#
#---------------------------------------------------------------------------

You can see the line

archive_cleanup_command = '/usr/sbin/pg_archive_cleanup'

My cleanup script :

#!/bin/bash

ARCHIVEDIR='/var/lib/postgresql/database/archive'
CHECKPOINT=$(find $ARCHIVEDIR -type f -mtime +3 -type f -printf '%f\n' | sort -r | head -1)
cd $ARCHIVEDIR
/usr/bin/pg_archivecleanup $ARCHIVEDIR $CHECKPOINT

find $ARCHIVEDIR -type f -mtime +3 -a -type f -a ! -newer $CHECKPOINT -delete

But after 4 days the disk space has grown. I saw in documentation that the recovery.conf file is read at at check_point and restart_point... So I would like to know why the archive are not purged automatically? Where do I set the occurency? When is postgresl supposed to proceed a purge? I want this to occure dayly, I'm obliged to put my cleanup script in a crontab instead? Or somewhere else? And I also have no trace in my postgresql log files. Where are cleanup log writen?

Thanks for your replies.

Upvotes: 2

Views: 6734

Answers (2)

nixmind
nixmind

Reputation: 2266

I solved the issue.

My script and my configuration were good. I just didn't restart postgresql service to have a new restart point taking into account my modification.

I did restart the servicec and it's now working fine as expected.

Thanks @Craig

Upvotes: 0

Craig Ringer
Craig Ringer

Reputation: 324305

pg_archivecleanup takes arguments:

$ /usr/pgsql-9.4/bin/pg_archivecleanup 
pg_archivecleanup: must specify archive location
Try "pg_archivecleanup --help" for more information.

If you look in the logs on the replica you'll probably see a repeated help message from pg_archivecleanup.

the manual shows that archive_cleanup_command has the %r substitution for the last valid restart point, and shows an example configuration:

archive_cleanup_command = 'pg_archivecleanup /mnt/server/archivedir %r'

Upvotes: 1

Related Questions