How to automate MongoDB backup and upload it to S3 storage using mongodump-s3-service docker image

Database backups are essential for protection against data loss that can completely disrupt business operations. Regular backups are crucial when you have transactional or very dynamic data in your database. Unlike SQL, MongoDB does not come with shadow paging, unless you are ready to work with replica sets (To deploy mongo in replica sets, click here).

TLDR; The docker image, developed by our CTO and Co-Founder Rohan Dhamapurkar, can be found here.

How to use the container:

Highlights:

• The package will take a full database instance backup locally as well as on the server.
• You can change the number of days the local backup stays.
• You can take regular backups or instant backups whenever needed with the help of the regular Linux CRONs.
• Automatic backups are possible on your specified time of day.
• The mongodump created will be named as mongodump_date-and-time.zip

Note: A sample .env file is to be made comprising of all the required configurations and keys. You can find a sample .env file in the mongodump-s3-service repository.

Below is the sample .env file:

# mongo connection credentials
MONGO_HOST=localhost
MONGO_PORT=27017
MONGO_USER=user
MONGO_PASSWORD=pass
AUTH_DB=admin
# aws/digital ocean access key and secret key
ACCESS_KEY_ID=asdaidebfaeiu
SECRET_ACCESS_KEY=bJa9PBeeK7pO4CL2xDHStxRZEI6DqslkCHL5GxwCtFY
# skip for provider digital ocean
DEFAULT_REGION=us-west
# skip for provider aws
ENDPOINT=https://nyc3.digitaloceanspaces.com
# s3 bucket details
BUCKET_NAME=bucket_name
BUCKET_PATH=files/test
# additional options for deleting old files
KEEP_OLD_FILES_DAYS=7
# cron string to run the back up on specified day and time
CRON_EXPRESSION=00 09 * * *
# to set timezone
TZ=Asia/Kolkata

1. To run the container, use the following command:

This command when executed, will start the backup process and take regular backups of your MongoDB database at your specified day and time.

docker run --name mongodump-s3-service \
		   -v ~/backup_files:/usr/files \
		   --network "host" \
		   --env-file .env \
		   -d \
		   rohandhamapurkar/mongodump-s3-service:latest

Explanation:

• - -name: Name of the docker image container. You can use it when referencing the container within a Docker network.
• -v: To bind the docker db file path to local file. For reference to volumes in dockers, click here.
• - -network: We have used host as the network. It means this container will use the Docker host network stack.
• - -env-file: The file path to the .env file.
• -d: To run a container in detached mode. For more information, click here.

2. To take instant backups, use the following command:

This command is used to take instant backups.

docker exec -it mongodump-s3-service instant

Note: The docker instance needs to be running and the mongodump-s3-service container should be there on the server for this command to run.

Explanation:

• exec: To run a command in a running container.
• -it: To run an interactive container.

3. For building a local image and running the container, use the following commands:

This command is used to create a local image first and run the container locally. Run these commands only if you want to make changes to the repo code.

docker build --tag mongodump-s3-service:latest .

docker run --name mongodump-s3-service \
		   -v ~/backup_files:/usr/files \
		   --network "host" \
		   --env-file .env \
		   -d \
		   mongodump-s3-service:latest

Explanation:

• build: The docker build command builds Docker images from a DockerFile and a “context”.
• - -tag: To tag the mongo backup image to a local image.

If we missed out on something you were looking for, drop down a message on our social media handles:

Facebook: OrderStack
Instagram: orderstack
Twitter: @orderstack