CouchDB Replication verify

Question

I am searching easy way to verify replicated documents, that all of them were transferred. Also, when some documents have error during replication process, how to check, which one and/or why ?

Now, I have an idea, that I could check current checkpointed sequence on the source. Then list all ids from source/_changes and then HEADing (or GETing) documents from target database and verify their presence (or values).

Answer 1

Having slept on it i think the below would cover the comparing of all docs and _revs

    #!/usr/local/bin/bash

    db1=`curl -s http://192.168.1.2:5984/mydb/_all_docs`
    db2=`curl -s http://192.168.1.3:5984/mydb/_all_docs`

    dif=`diff -y --suppress-common-lines -b -s <(echo "$db1")  <(echo "$db2")`

    echo $dif

Answer 2

Simple nagios script to check two servers dbs and doc counts match - could be improved to check that _rev for each between dbs match.

    #!/usr/local/bin/bash
    # nagios check script to make sure 2 couch servers are the same
    #
    # ie: have same dbs
    #     have same number of documents in each db
    #
    # ./check_couch_dbs 192.168.1.2 192.168.1.3
    #
    # Needs fdescfs for bash redirects
    # 
    # Needs npm - as root: 'pkg install npm'
    # Needs json - as root: 'npm install -g json'
    # https://github.com/zpoley/json-command
    #
    #
    # check_couch_dbs command definition example
    #
    #    define command{
    #       command_name    check_couch_dbs
    #       command_line    $USER1$/check_couch_dbs $ARG1$ $ARG2$
    #       } 
    #

    host1=$1
    host2=$2

    difference_threashold=10

    dbs1=`curl -s http://$host1:5984/_all_dbs | json -ga  | grep -v -E '_replicator|_users' | sort`
    dbs2=`curl -s http://$host2:5984/_all_dbs | json -ga  | grep -v -E '_replicator|_users' | sort`

    dif=`diff -y --suppress-common-lines -b -s <(echo "$dbs1")  <(echo "$dbs2")`

    err=""
    msg=""
    exitcode=0
    if [[ "$dif" == *identical* ]]; then
            msg+="Couchdbs lists match"
    fi
    if [[ "$dif" == *\<* ]]; then
            err+="ERROR - db missing from $host2 \n"
            exitcode=2
    fi
    if [[ "$dif" == *\>* ]]; then
            err+="ERROR - db missing from $host1 \n"
            exitcode=2
    fi

    if [[ $exitcode -gt 0 ]]; then
            echo -e -n $err
            echo "$dif"
            err=""
    #       exit $exitcode
    fi

    dbs=`echo -e "$dbs1\n$dbs2" | sort | uniq`
    #echo "$dbs"

    for db in $dbs; do
            count1=`curl -s http://$host1:5984/$db | json doc_count`
            if [ -z "$count1" ]; then continue; fi  #no db
            count2=`curl -s http://$host2:5984/$db | json doc_count`
            if [ -z "$count2" ]; then continue; fi  #no db
            if [ "$count1" -ne "$count2" ]; then

                    if [ "$count1" -gt "$count2" ]; then
                            difference=$(($count1 - $count2))
                    else
                            difference=$(($count2 - $count1))
                    fi
                    if [[ $difference -gt $difference_threashold ]]; then
                    err+="ERROR - $db count difference $host1: $count1  != $host2: $count2 - difference: $difference\n"
                            exitcode=2
                    else
                            err+="WARNING - $db count difference $host1: $count1 != $host2: $count2 - difference: $difference\n"
                            if [[ $exitcode -lt 1 ]]; then
                                    exitcode=1
                            fi
                    fi
            fi
    done

    if [[ $exitcode -gt 0 ]]; then
            echo -e -n $err $msg
            exit $exitcode
    else
            echo -e "OK - $msg - doc_count match"
            exit $exitcode
    fi