Recent years have seen an increase in data volume used by the IT industry to perform analytics and processing. Traditional storage methods include a relational database and an off-premises server. This setup is not able to handle all requests due to limited infrastructure capabilities.
Data is dynamic, unstructured and constantly changing. A no-SQL database can be used to store this type of data. Instead of using SQL queries to query your data, we use programmable languages. The No-SQL Database is a popular choice because of its lightness and ability to handle rapidly changing data as well as the large amount of unstructured and complex data.
AWS DynamoDB and Mongo DB are two of the many services that we have today. Both databases are No-SQL and have similar capabilities. AWS DynamoDB, on the other hand, is managed AWS service. MongoDB’s capabilities are limited to the infrastructure on which it is hosted. AWS DynamoDB, on the other hand, is highly scalable and provides high throughput with low latency. It also supports global tables.
Below are some key terms that are similar, but are not named the same.
You might be interested in migrating your MongoDB data to AWS DynamoDB to increase your data handling capabilities. This will allow you to have a highly scalable managed infrastructure.
To migrate the data, we will use an AWS service called AWS Data Migration Service.
The AWS Database Migration Service allows you to migrate from on-premises, EC2 instances, Standalone servers or AWS RDS to an AWS backed Database Service with virtually no downtime.
To deploy my source on-premises MongoDB Database, I used a Ubuntu 18.04 OS server. I assume that you have some data in your MongoDB database node. This data is now ready for Amazon DynamoDB.
Let us get Started
AWS DMS offers two options for migration
Document Mode: This mode transfers all data from the collection as a single column to the DynamoDB table
Table Mode: DMS scans some documents in the table to create an schema. This allows the DMS to place them in DynamoDB.
For the migration, we will use the ‘table mode’
Let’s start by creating an AWS role that will allow DynamoDB to be accessed by the DMS service.
Go to IAM and then create a new role
DMS
Click Next Permissions to add DynamoDB full Access Policy to the role
If necessary, use tags and name the role as DMS–dynamo-role
Click Save
2. Let’s go to DMS (Database Migration Service) in AWS Console
To perform the migration, we will need to create a replica server in a VPC.
Select the name you prefer
Select VPC where you want it to be launched
Select the instance type: Here, we have selected t3.micro
For Demo purposes, you can choose to use single AZ for Dev Workload. If you need production work, you can choose multi-AZ.
3. Jump to your MongoDB Database and give the permissions
These permissions are required by the Replication server in order to access the DB
To create the role below, use Admin
Create a user with this role using your Database
4. Now, we will create Target and Source endpoints
These endpoints are required to perform the DMS task of transferring the data from DynamoDB to source DB.
Create source endpoint
Name the endpoint source
Server Name is public Ip of MongoDB Machine
Name the DB that has the permissions we created in step 1.
The following will be the endpoint
Create Target endpoint
Give Target endpoint name
Choose Target engine as DynamoDB
The following will be the endpoint
5. Check the endpoint connection to the Source DB connection. If configured correctly, it will show up as Successful
6. In the DMS console, create a Database migration task
Create a Task
Give the task a title
Select the replicat