streaming replication dropped between master and slave and how to bring them back on replication ?
Switch Secondary/slave database to become read/write
On the secondary server create the trigger file (as specified in recovery.conf)
touch opt/PostgresPlus/9.2AS/data2/recovery.trigger
Now connect again and check if your current live sessions/new sessions to the secondary database are able to do write operations to the database. If yes, then great! You just completed a successful failover.
Rebuilding the Master Database
Now, we need to re-build the master (assuming that the master database server is up).
First of all clean up the database cluster on old Primary server:
rm -Rf /opt/PostgresPlus/9.2AS/data
Now, take a backup from new primary (dbserver2) to rebuild the master:
pg_basebackup -D /opt/PostgresPlus/9.2AS/data -h 192.168.160.150 -W -p 5222 -Fp --xlog-method=stream
Once the backup is complete, copy the postgresql.conf.dbserver1 as postgresql.conf and then remove recovery.done and copy the recover.conf.dbserver2_is_master as recovery.conf.
Start the Primary DB as new slave
Now start the db cluster on master node. Once the start-up is successful, make sure everything is alright by connecting to the database and firing the below command:
SELECT pg_is_in_recovery(); #expected output is “t”
Then fire the below command on dbserver1 and dbserver2 and both should be same:
SELECT txid_current_snapshot();
Carefully inspect the log files on secondary node (dbserver1) to confirm the recovery is in progress and there is no issues in the replication.