{"id":248,"date":"2018-06-04T16:44:04","date_gmt":"2018-06-04T06:44:04","guid":{"rendered":"http:\/\/www.mrmarkyoung.com\/oracle\/?p=248"},"modified":"2018-06-05T17:15:49","modified_gmt":"2018-06-05T07:15:49","slug":"backup-and-purge-rds-snapshots-in-aws","status":"publish","type":"post","link":"http:\/\/www.mrmarkyoung.com\/oracle\/2018\/06\/04\/backup-and-purge-rds-snapshots-in-aws\/","title":{"rendered":"Backup and Purge RDS Snapshots in AWS"},"content":{"rendered":"<p>Here is a script I&#8217;ve put together to backup both cluster and non clustered databases in AWS. (Still in progress).<\/p>\n<pre class=\"lang:default decode:true \" >import boto3\r\nimport datetime\r\nimport sys\r\n#   Author: Mark Young\r\n#   Date:   3rd October 2017\r\n#   Detail: Automatic database snapshots are restricted to 35 days\r\n#           This script enables us to keep snapshots for extended periods\r\n#   Audit:  1.0 - M.Y. Original\r\n#           1.1 - M.Y. Added retention variables and cluster name\r\n\r\nINTERVAL_TYPE   = 'days'\r\nINTERVAL_NUM    = 1\r\n#RDS_NAME    = 'AURORATEST'\r\n\r\n\r\n#for arg in sys.argv:\r\nrds_type    = sys.argv[1].upper()\r\nRDS_NAME    = sys.argv[2].upper()\r\nPurge       = sys.argv[3].upper()\r\n\r\nclient = boto3.client('rds')\r\nSNAPSHOT_NAME=datetime.datetime.now().strftime(\"%y-%m-%d-%H-%S\")\r\n\r\n\r\n\r\n# Backups for cluster databases\r\ndef Cluster_backup(RDS_NAME):\r\n        print(\"Connecting to Cluster Database\")\r\n        print(\"Cluster Database snapshot backups stated... for \" + RDS_NAME + ' - Snapshot ' + SNAPSHOT_NAME)\r\n        snapshot_description = client.describe_db_cluster_snapshots(DBClusterIdentifier= RDS_NAME)\r\n        while snapshot_description['DBClusterSnapshots'][0]['Status'] != 'available' :\r\n            print(\"still waiting\")\r\n            time.sleep(15)\r\n            snapshot_description = client.describe_db_cluster_snapshots(DBClusterIdentifier= RDS_NAME)\r\n            response = client.create_db_cluster_snapshot(\r\n                DBClusterSnapshotIdentifier= RDS_NAME + '-s-' + SNAPSHOT_NAME,\r\n                DBClusterIdentifier= RDS_NAME,\r\n                Tags=[\r\n                    {\r\n                        'Key': 'Backup',\r\n                        'Value': 'MyToll'\r\n                    },\r\n                    ]\r\n                    )\r\n\r\n\r\n#For debugging\r\n#print(response)\r\ndef Cluster_purge(RDS_NAME):\r\n        print('Looking for items to purge - Retention ' + str(INTERVAL_NUM) + ' ' + INTERVAL_TYPE )\r\n        for snapshot in client.describe_db_cluster_snapshots(DBClusterIdentifier= RDS_NAME, MaxRecords=50)['DBClusterSnapshots']:\r\n            create_ts = snapshot['SnapshotCreateTime'].replace(tzinfo=None)\r\n            if create_ts &lt; datetime.datetime.now() - datetime.timedelta(**{INTERVAL_TYPE: INTERVAL_NUM}):\r\n                print(\"Deleting snapshot id:\", snapshot['DBClusterSnapshotIdentifier'])\r\n                client.delete_db_cluster_snapshot(\r\n                    DBClusterSnapshotIdentifier=snapshot['DBClusterSnapshotIdentifier']\r\n                )\r\n        #For debugging\r\n        #print(response)\r\n        my_response =\"Completed snapshot copy and purge for cluster database...\"\r\n        return my_response\r\n\r\n    # Backups for cluster databases\r\ndef RDS_backup(RDS_NAME):\r\n        print(\"Connecting to RDS Database\")\r\n        print(\"Database snapshot backups stated... for \" + RDS_NAME + ' - Snapshot ' + SNAPSHOT_NAME)\r\n        response = client.create_db_snapshot(\r\n            DBSnapshotIdentifier= RDS_NAME + '-s-' + SNAPSHOT_NAME,\r\n            DBIdentifier= RDS_NAME,\r\n            Tags=[\r\n                {\r\n                    'Key': 'Backup',\r\n                    'Value': 'MyToll'\r\n                },\r\n            ]\r\n        )\r\n\r\ndef RDS_purge(RDS_NAME):\r\n        #For debugging\r\n        #print(response)\r\n        print('Looking for items to purge - Retention ' + str(INTERVAL_NUM) + ' ' + INTERVAL_TYPE )\r\n        for snapshot in client.describe_db_snapshots(DBIdentifier= RDS_NAME, MaxRecords=50)['DBSnapshots']:\r\n            create_ts = snapshot['SnapshotCreateTime'].replace(tzinfo=None)\r\n            if create_ts &lt; datetime.datetime.now() - datetime.timedelta(**{INTERVAL_TYPE: INTERVAL_NUM}):\r\n                print(\"Deleting snapshot id:\", snapshot['DBSnapshotIdentifier'])\r\n                client.delete_db_cluster_snapshot(\r\n                    DBSnapshotIdentifier=snapshot['DBSnapshotIdentifier']\r\n                )\r\n        #For debugging\r\n        #print(response)\r\n        my_response =\"Completed snapshot copy and purge for RDS database...\"\r\n        return my_response\r\n\r\nif rds_type == '-H' or rds_type == '':\r\n    print(\"Usage: rds_backup.py RDS NAME: &lt;CLUSTER\/RDS&gt; DB NAME:&lt;RDS\/CLUSTER NAME&gt; PURGE:&lt;PURGE\/NOPURGE&gt;\")\r\n    print:(\"i.e. rds_backup.py CLUSTER AURORATEST TRUE\")\r\n    sys.exit\r\n\r\nif rds_type == 'CLUSTER':\r\n    Cluster_backup(RDS_NAME)\r\n    if Purge == 'PURGE':\r\n        Cluster_purge(RDS_NAME)\r\nif rds_type == 'RDS':\r\n    RDS_backup(RDS_NAME)\r\n    if Purge == 'PURGE':\r\n        Cluster_purge(RDS_NAME)\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Here is a script I&#8217;ve put together to backup both cluster and non clustered databases in AWS. (Still in progress). import boto3 import datetime import sys # Author: Mark Young # Date: 3rd October 2017 # Detail: Automatic database snapshots are restricted to 35 days # This script enables us to keep snapshots for extended &#8230; <a title=\"Backup and Purge RDS Snapshots in AWS\" class=\"read-more\" href=\"http:\/\/www.mrmarkyoung.com\/oracle\/2018\/06\/04\/backup-and-purge-rds-snapshots-in-aws\/\" aria-label=\"Read more about Backup and Purge RDS Snapshots in AWS\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[39],"tags":[12,31,61],"class_list":["post-248","post","type-post","status-publish","format-standard","hentry","category-aws","tag-aws","tag-python","tag-rds"],"_links":{"self":[{"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/posts\/248","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/comments?post=248"}],"version-history":[{"count":1,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/posts\/248\/revisions"}],"predecessor-version":[{"id":251,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/posts\/248\/revisions\/251"}],"wp:attachment":[{"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/media?parent=248"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/categories?post=248"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.mrmarkyoung.com\/oracle\/wp-json\/wp\/v2\/tags?post=248"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}