2019独角兽企业重金招聘Python工程师标准>>> 
一、系统
[root@barman ~]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m[root@barman ~]# uname -a
Linux barman 2.6.32-431.11.2.el6.x86_64 #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux# cat /etc/hosts
192.168.100.146 barman
192.168.100.147 pg二、安装
2.1 简介
Barman (backup and recovery manager) is an administration tool for disaster recovery of PostgreSQL servers written in Python. Barman can perform remote backups of multiple servers in business critical environments, and helps DBAs during the recovery phase.
Barman’s most wanted features include: backup catalogues, retention policies, remote recovery, archiving and compression of WAL files and of backups. Barman is written and maintained by PostgreSQL professionals 2ndQuadrant.
Barman的版本更新情况：
2.2 安装依赖包
已安装软件版本检查：
[root@barman ~]# python --version
Python 2.6.6
[root@barman ~]# rpm -qa | grep rsync
rsync-3.0.6-9.el6_4.1.x86_64下载地址：http://dl.fedoraproject.org/pub/epel/6/x86_64/
[root@barman ~]# rpm -ivh python-argparse-1.2.1-2.el6.noarch.rpm
warning: python-argparse-1.2.1-2.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]1:python-argparse ########################################### [100%]
[root@barman ~]# rpm -ivh python-argcomplete-0.6.7-2.el6.noarch.rpm
warning: python-argcomplete-0.6.7-2.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]1:python-argcomplete ########################################### [100%]
[root@barman ~]# rpm -ivh python-argh-0.23.2-1.el6.noarch.rpm
warning: python-argh-0.23.2-1.el6.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
Preparing... ########################################### [100%]1:python-argh ########################################### [100%][root@barman ~]# yum install python-dateutil python-psycopg22.3 安装barman
下载地址：http://sourceforge.net/projects/pgbarman/files/1.3.0/
[root@barman ~]# rpm -ivh barman-1.3.0-1.rhel6.noarch.rpm
Preparing... ########################################### [100%]1:barman ########################################### [100%]三、安装配置数据库
3.1 安装
在barman、pg上分别安装pg数据库，安装过程略
在barman上安装PostgreSQL仅为用于客户端，确保barman能够连上需要备份的pg
3.2 创建用户
在barman上创建barman用户并设置密码：
[root@barman ~]# useradd barman
[root@barman ~]# passwd barman在pg上创建postgres用户并设置密码：
[root@pg ~]# useradd postgres
[root@pg ~]# passwd postgres配置barman/postgres用户环境变量：
export PATH=/opt/pgsql/bin:$PATH:$HOME/bin
export PGDATA=/opt/pgsql/data
export PGUSER=postgres
export PGPORT=5432
export LD_LIBRARY_PATH=/opt/pgsql/lib:$LD_LIBRARY_PATH3.3 配置
[postgres@pg data]$ vim postgresql.conf
listen_addresses = '*'
port = 5432
wal_level = archive 若为HOT STANDBY环境则此处设置为hot_standby
archive_mode = on
archive_command = 'rsync -a %p barman@barman:/backup/pg1/incoming/%f'[postgres@pg data]$ vim pg_hba.conf
添加如下一条
host all all 192.168.100.146/32 trust四、配置barman
4.1 创建归档目录
[root@barman ~]# mkdir -p /backup/pg1/incoming
[root@barman ~]# chown barman:barman /backup -R4.2 配置ssh互信
在barman上设置：
[barman@barman ~]$ ssh-keygen -t rsa
[barman@barman ~]$ ssh-copy-id -i .ssh/id_rsa.pub postgres@pg
[barman@barman ~]$ ssh postgres@pg date
Thu Apr 10 02:11:43 CST 2014在pg上设置：
[postgres@pg ~]$ ssh-keygen -t rsa
[postgres@pg ~]$ ssh-copy-id -i .ssh/id_rsa.pub barman@barman
[postgres@pg ~]$ ssh barman@barman date
Thu Apr 10 02:14:01 CST 2014配置完成后启动数据库：
[postgres@pg ~]$ pg_ctl start
server starting测试：
[barman@barman ~]$ psql -c 'SELECT version()' -h pg -U postgresversion
--------------------------------------------------------------------------------------------------------------PostgreSQL 9.3.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4), 64-bit
(1 row)4.3 barman.conf设置
（设置/etc/barman.conf或～/.barman.conf）
[root@barman ~]# vi /etc/barman.conf
[barman]
barman_home = /var/lib/barman 备份文件路径
barman_user = barman
log_file = /var/log/barman/barman.log
compression = gzip
[main]
description = "Main PostgreSQL Database"
incoming_wals_directory = /backup/pg1/incoming
ssh_command = ssh postgres@pg
conninfo = host=pg user=postgres4.4 主要参数说明
immediate_checkpoint
1.3.0加入了该参数，当设置为true时，barman在执行备份时不需要再像之前一样等待PostgreSQL服务器发生一个checkpoint，而是主动触发一次checkpoint。默认设置为false。 该参数也可即设即用，可在执行barman backup时加上--immediate-checkpoint / --no-immediate-checkpoint。
bandwidth_limit / tablespace_bandwidth_limit
用于限制带宽使用，避免备份过多占用带宽资源。默认设置为0,即不限制。bandwidth_limit用作全局带宽限制，也可通过tablespace_bandwidth_limit对每个表空间所用带宽做特定限制。
network_compression = true|false
设定是否启用网络数据压缩。默认为false。
minimum_redundancy
最小冗余设置。当备份集个数减少到该值时，剩下的备份集将禁止删除，避免误删了所有的备份。
例如：当设置为1时，
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 82.8 KiB
[barman@barman ~]$ barman delete main 20140411T015540
Skipping delete of backup 20140411T015540 for server main due to minimum redundancy requirements (1)
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 82.8 KiB保留策略
retention_policy：指定备份集的保留策略
wal_retention_policy：指定归档的保留策略
retention_policy_mode：策略的执行模式，目前只有auto
保留策略的执行为自动或手动，目前只实现了自动，手动将在之后版本中提供。自动根据保留策略对无用的备份或归档进行清理是通过barman cron。
五、barman管理
[barman@barman ~]$ barman --help
usage: barman [-h] [-v] [-c CONFIG] [-q] [-f {console}]{status,list-server,list-files,list-backup,show-backup,cron,rebuild-xlogdb,recover,backup,check,show-server,delete}...
positional arguments:{status,list-server,list-files,list-backup,show-backup,cron,rebuild-xlogdb,recover,backup,check,show-server,delete}cron Run maintenance taskslist-server List available servers, with useful informationshow-server Show all configuration parameters for the specifiedserversstatus Shows live information and status of the PostgreSQLservercheck Check if the server configuration is working. Thiscommand returns success if every checks pass, orfailure if any of these failsbackup Perform a full backup for the given serverlist-backup List available backups for the given server (supports'all')show-backup This method shows a single backup informationlist-files List all the files for a single backuprecover Recover a server at a given time or xiddelete Delete a backuprebuild-xlogdb Rebuild the WAL file database guessing it from thedisk content.
optional arguments:-h, --help show this help message and exit-v, --version show program's version number and exit-c CONFIG, --config CONFIGuses a configuration file (defaults: ~/.barman.conf,/etc/barman.conf, /etc/barman/barman.conf)-q, --quiet be quiet (default: False)-f {console}, --format {console}output format (default: console)5.1 检查备份信息
5.1.1 配置检查
[barman@barman ~]$ barman show-server main
Server main:
conninfo: host=pg user=postgres
ssh_command: ssh postgres@pg
retention_policy: None
archive_mode: None
minimum_redundancy: 0
custom_decompression_filter: None
compression: gzip
retention_policy_mode: auto
archive_command: None
basebackups_directory: /var/lib/barman/main/base
data_directory: None
tablespace_bandwidth_limit: None
lock_file: /var/lib/barman/main/main.lock
current_xlog: None
description: Main PostgreSQL Database
bandwidth_limit: None
custom_compression_filter: None
incoming_wals_directory: /backup/pg1/incoming
immediate_checkpoint: False
backup_directory: /var/lib/barman/main
active: True
network_compression: False
post_backup_script: None
wals_directory: /var/lib/barman/main/wals
last_shipped_wal: None
server_txt_version: None
wal_retention_policy: main
pre_backup_script: None5.1.2 状态检查
[barman@barman ~]$ barman check main
Server main:
ssh: OK
PostgreSQL: OK
archive_mode: OK
archive_command: OK
directories: OK
retention policy settings: OK
compression settings: OK
minimum redundancy requirements: OK (have 0 backups, expected at least 0)[barman@barman ~]$ barman status main
Server main:
Description: Main PostgreSQL Database
PostgreSQL version: 9.3.4
PostgreSQL Data directory: /opt/pgsql/data
PostgreSQL 'archive_command' setting: rsync -a %p barman@barman:/backup/pg1/incoming/%f
Archive status: last shipped WAL segment 000000010000000000000008
Current WAL segment: 000000010000000000000009
Retention policies: not enforced
No. of available backups: 3
First available backup: 20140410T031606
Last available backup: 20140410T032409
Minimum redundancy requirements: satisfied (3/0)5.2 查看可监听的数据库服务器
[barman@barman ~]$ barman list-server
main - Main PostgreSQL Database5.3 查看指定备份集的详细信息
[barman@barman ~]$ barman show-backup main 20140410T031606
Backup 20140410T031606:Server Name : mainStatus : DONEPostgreSQL Version: 90304PGDATA directory : /opt/pgsql/dataBase backup information:Disk usage : 19.5 MiBTimeline : 1Begin WAL : 000000010000000000000005End WAL : 000000010000000000000005WAL number : 1Begin time : 2014-04-10 03:16:06.343719End time : 2014-04-10 03:16:11.085018Begin Offset : 40End Offset : 184Begin XLOG : 0/5000028End XLOG : 0/50000B8WAL information:No of files : 1Disk usage : 16.0 KiBLast available : 000000010000000000000006Catalog information:Retention Policy: not enforcedPrevious Backup : - (this is the oldest base backup)Next Backup : 20140410T0317385.4 执行全备
[barman@barman ~]$ barman backup main --若想对多个库同时进行备份，可执行barman backup all
Starting backup for server main in /var/lib/barman/main/base/20140410T031606
Backup start at xlog location: 0/5000028 (000000010000000000000005, 00000028)
Copying files.
Copy done.
Asking PostgreSQL server to finalize the backup.
Backup end at xlog location: 0/50000B8 (000000010000000000000005, 000000B8)
Backup completed基础备份：
[barman@barman ~]$ ls /var/lib/barman/main/base/
20140410T031606wal备份：
[barman@barman ~]$ ls /var/lib/barman/main/wals/
0000000100000000 xlog.db xlog.db.lock5.5 查看备份集
[barman@barman ~]$ barman list-backup main
main 20140410T032409 - Thu Apr 10 03:24:13 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T031738 - Thu Apr 10 03:17:41 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T031606 - Thu Apr 10 03:16:11 2014 - Size: 19.5 MiB - WAL Size: 16.0 KiB5.6 恢复
确保用户在指定的恢复路径中有权限创建PGDATA目录
barman recover --help
usage: barman recover [-h] [--target-tli TARGET_TLI][--target-time TARGET_TIME] [--target-xid TARGET_XID][--target-name TARGET_NAME] [--exclusive][--tablespace NAME:LOCATION][--remote-ssh-command SSH_COMMAND]server_name backup_id destination_directory
Recover a server at a given time or xid
positional arguments:server_name specifies the server name for the commandbackup_id specifies the backup ID to recoverdestination_directorythe directory where the new server is created
optional arguments:-h, --help show this help message and exit--target-tli TARGET_TLItarget timeline (default: None)--target-time TARGET_TIMEtarget time. You can use any valid unambiguousrepresentation. e.g: "YYYY-MM-DD HH:MM:SS.mmm"(default: None)--target-xid TARGET_XIDtarget transaction ID (default: None)--target-name TARGET_NAMEtarget name created previously withpg_create_restore_point() function call (default:None)--exclusive set target xid to be non inclusive (default: False)--tablespace NAME:LOCATIONtablespace relocation rule (default: None)--remote-ssh-command SSH_COMMANDThis options activates remote recovery, by specifyingthe secure shell command to be launched on a remotehost. It is the equivalent of the "ssh_command"serveroption in the configuration file for remoterecovery. Example: "ssh postgres@db2" (default: None)5.6.1 本地恢复
[barman@barman ~]$ barman recover main 20140410T032409 /tmp/data
Starting local restore for server main using backup 20140410T032409
Destination directory: /tmp/data
Copying the base backup.
Copying required wal segments.
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.5.6.2 远程恢复
[barman@barman ~]$ barman recover --remote-ssh-command "ssh postgres@pg" main 20140410T032409 /tmp/data
Starting remote restore for server main using backup 20140410T032409
Destination directory: /tmp/data
Copying the base backup.
Copying required wal segments.
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.5.6.3 迁移表空间
测试未成功
5.6.4 PITR
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140410T071548 - Thu Apr 10 07:15:53 2014 - Size: 19.5 MiB - WAL Size: 0 B
postgres=# create table t1(id int);
CREATE TABLE
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140410T072226 - Thu Apr 10 07:22:32 2014 - Size: 19.5 MiB - WAL Size: 0 B
main 20140410T071548 - Thu Apr 10 07:15:53 2014 - Size: 19.5 MiB - WAL Size: 0 B5.6.4.1 target-time
[barman@barman ~]$ barman recover --target-time "2014-04-10 07:16:00" main 20140410T071548 /opt/pgsql/data
Starting local restore for server main using backup 20140410T071548
Destination directory: /opt/pgsql/data
Doing PITR. Recovery target time: '2014-04-10 07:16:00'
Copying the base backup.
Copying required wal segments.
Generating recovery.conf
The archive_command was set to 'false' to prevent data losses.
Your PostgreSQL server has been successfully prepared for recovery!
Please review network and archive related settings in the PostgreSQL
configuration file before starting the just recovered instance.[barman@barman ~]$ pg_ctl start
server starting
postgres=# \d
No relations found.5.6.4.2 target-xid
[barman@barman ~]$ barman backup main
[barman@barman ~]$ barman list-backup main
main 20140411T015540 - Fri Apr 11 01:55:45 2014 - Size: 19.5 MiB - WAL Size: 0 B
postgres=# create table t1(id int);
CREATE TABLE
postgres=# select txid_current();txid_current
--------------1824
(1 row)
postgres=# create table t2(id int);
CREATE TABLE
postgres=# select txid_current();txid_current
--------------1826
(1 row)
postgres=# create table t3(id int);
CREATE TABLE
postgres=# select txid_current();txid_current
--------------1828
(1 row)
postgres=# checkpoint ;
CHECKPOINT
postgres=# select pg_switch_xlog();pg_switch_xlog
----------------0/18016390
(1 row)包含所指事务
恢复：
[barman@barman ~]$ barman recover --target-xid "1825" main 20140411T015540 /opt/pgsql/data验证：
postgres=# \dList of relationsSchema | Name | Type | Owner
--------+------+-------+----------public | t1 | table | postgrespublic | t2 | table | postgres
(2 rows)
{说明包括了1825事务}不包含所指事务
恢复：
[barman@barman ~]$ barman recover --target-xid "1825" --exclusive main 20140411T015540 /opt/pgsql/data验证：
postgres=# \dList of relationsSchema | Name | Type | Owner
--------+------+-------+----------public | t1 | table | postgres
(1 row)5.6.4.3 target-name
接着上一部分执行：
postgres=# \dList of relationsSchema | Name | Type | Owner
--------+------+-------+----------public | t1 | table | postgrespublic | t2 | table | postgrespublic | t3 | table | postgres
(3 rows)
postgres=# create table t4(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target1');pg_create_restore_point
-------------------------0/19015098
(1 row)
postgres=# create table t5(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target2');pg_create_restore_point
-------------------------0/19016170
(1 row)
postgres=# create table t6(id int);
CREATE TABLE
postgres=# select pg_create_restore_point('target3');pg_create_restore_point
-------------------------0/19019160
(1 row)
postgres=# select pg_switch_xlog();pg_switch_xlog
----------------0/19019180
(1 row)恢复：
[barman@barman ~]$ barman recover --target-name "target1" main 20140411T015540 /opt/pgsql/data
[barman@barman pgsql]$ cat data/recovery.conf
restore_command = 'cp barman_xlog/%f %p'
recovery_end_command = 'rm -fr barman_xlog'
recovery_target_name = 'target1'
[barman@barman pgsql]$ pg_ctl start
server starting验证：
postgres=# \dList of relationsSchema | Name | Type | Owner
--------+------+-------+----------public | t1 | table | postgrespublic | t2 | table | postgrespublic | t3 | table | postgrespublic | t4 | table | postgres
(4 rows)5.6.4.4 target-tli
用于恢复时间线的设置
5.7 删除备份集
[barman@barman ~]$ barman delete main 20140410T032409
Deleting backup 20140410T032409 for server main
Done5.8 快捷的备份集ID代替（latest/oldest）
适用：
show-backup | delete | list-files | recover
例如：
barman show-backup main latest5.9 重新生成归档
[barman@barman ~]$ barman rebuild-xlogdb main
Rebuilding xlogdb for server main
Done rebuilding xlogdb for server main (history: 0, backup_labels: 1, wal_file: 4)5.10 列举备份文件
[barman@barman ~]$ barman list-files main 20140411T015540
/var/lib/barman/main/base/20140411T015540/backup.info
/var/lib/barman/main/base/20140411T015540/pgdata/pg_hba.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postmaster.opts
/var/lib/barman/main/base/20140411T015540/pgdata/pg_ident.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postgresql.conf
/var/lib/barman/main/base/20140411T015540/pgdata/postgresql.conf.origin
/var/lib/barman/main/base/20140411T015540/pgdata/backup_label.old
/var/lib/barman/main/base/20140411T015540/pgdata/backup_label
/var/lib/barman/main/base/20140411T015540/pgdata/PG_VERSION
/var/lib/barman/main/base/20140411T015540/pgdata/pg_clog/0000
/var/lib/barman/main/base/20140411T015540/pgdata/pg_notify/0000
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12733
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12707_fsm
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12718_fsm
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12702
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12848
/var/lib/barman/main/base/20140411T015540/pgdata/base/12891/12707
……同时可指定需要列出的文件类型，如：
[barman@barman ~]$ barman list-files --target data main 20140411T015540--target 值为data/standalone/wal/full，含义如下：
data: lists just the data files;
standalone: lists the base backup files, including required WAL files;
wal: lists all WAL files from the beginning of the base backup to the start of the following one (or until the end of the log);
full: same as data + wal.
5.11 附加执行脚本
barman支持在备份之前或之后/归档之前或之后附带执行定制脚本，关于该脚本在barman.conf有相关设置：
; Pre/post backup hook scripts
;pre_backup_script = env | grep ^BARMAN
;post_backup_script = env | grep ^BARMAN
; Pre/post archive hook scripts
;pre_archive_script = env | grep ^BARMAN
;post_archive_script = env | grep ^BARMAN
六、总结
barman功能特点总结如下：
使用简单，通过简单的命令就可实现所有功能;
支持本地/远程备份恢复;
能够同时管理多个备份端，并能同时对多个备份端进行操控;
支持在线全备、wal增量备份;
支持多种压缩功能，如：gzip、bzip2及用户自定义压缩工具;
提供灵活多变的备份、归档保留策略;
支持流量压缩，可定义到表空间;
支持迁移表空间（未测）;
支持自动维护;
支持PITR。
七、参考文献
官方网站：
http://www.pgbarman.org/
http://2ndquadrant.com/
八、license
Barman is the exclusive property of 2ndQuadrant Italia and its code is distributed under GNU General Public License 3.
Contributions to Barman are welcome, and will be listed in the file AUTHORS.
2ndQuadrant Italia requires that any contributions provide a copyright assignment and a disclaimer of any work-for-hire ownership claims from the employer of the developer.
This lets us make sure that all of the Barman distribution remains free code.
