Bulk data load shared instances

Bulk data load tasks in Amoprhic uses Replication Instances of DMS(Database Migration Service) to ingest the data from source to target. This feature allows to setup multiple data load tasks with a single instance rather than one instance for one task. This Shared Instances are independent to each connection, i.e., when an shared instance is created for a connection, only the tasks created in that connection can use this Instance and the tasks from other connections cannot use this Instance.

How to create a Shared Instance

Below are the steps that are required to create a share instance in Amorphic.

• Instance Name : Name of the instance to be used, an identifier of the Instance.

• Description : Purpose of this Instance which will be helpful for all the authorized users to understand the requirement of this Instance

• Instance Availability Zone : Availability zone to be used to launch the replication instance. When an AZ is selected, all the available instance classes in that AZ will be shown and one can be selected from the list.

• Instance Class : Type of DMS instance class to be used. Please choose approx Instance based on the usability in tasks.

• Allocated Storage: Amount of storage space you want for your replication instance. AWS DMS uses this storage for log files and cached transactions while replication tasks are in progress.

• Multi-AZ : Specifies whether the replication instance is a Multi-AZ deployment. When Multi-AZ setting is enabled Instance Availability Zone setting will be nullified in the backend.

• Auto Minor Version Upgrade : A value that indicates whether minor engine upgrades are applied automatically to the replication instance during the maintenance window. This parameter defaults to true if not specified.

• Auto Terminate : This enables Shared Instance termination to save resource costs based on the termination time value provided by the user. Auto termination process is triggered every hour and looks for any Shared Instance that needs to be notified or deleted and sends an email when one of the below criteria is met. The user will receive a notification email in the following scenarios:

  • If the difference between the auto-terminate process trigger run (every whole hour) and the termination time is less than 30 minutes.
  • If the auto-termination process was successfully able to delete the Instance after the termination time
  • If the auto-termination process wasn't able to delete the Instance due to some fatal errors.

• Auto Termination Time : You can set auto termination time for Shared Instance to less than 168 hours (7 days). If current time is greater than termination time, Instance will be deleted next whole hour.

• DmsVersion(API only): Users have the option to specify the DMS engine version to be used on the replication instance. This attribute is optional and specific to the API. If no version is provided, the replication instance will be created with the latest DMS engine version. Users can update it as well.

Here is the API and sample payload for creating shared instances in Amorphic.

/connections/{connection_id}/instances/{instance_id} & POST method

Sample payload for creating Shared-Instance

    {
        {
            "InstanceName": "CDC-Shared-Instance",
            "InstanceClass": "dms.t3.medium",
            "InstanceAZ": "eu-west-1c",
            "AllocatedStorage": 50,
            "Description": "Shared instance for CDC Tasks",
            "InstanceMultiAZ": false,
            "AutoMinorVersionUpgrade": false,
            "IsAutoTerminateEnabled": false,
            "DmsVersion": "3.4.7"
        }
    }

How to edit a Shared Instance

Once the Instance creation is successful and is in active status, Instance configuration can be updated with Edit Instance option. Only certain configurations can be updated and the same will be shown when an update instance is clicked and the configuration that are not editable will be greyed out. Instance Name and Instance Availability Zone are not editable and the rest all can be changeable.

View Instance details and Tasks associated

To view additional details of the Instance and also the list of all tasks associated with the Instance, one can click on View Instance button which will show all the additional details and the tasks. Task Name & Migration Type can be seen in the Instance details page.

How to delete a shared Instance

A shared instnace can be deleted with the Delete Instance option that can be seen in the options of a particular Instance. If there are any active tasks associated with the Ins

Note

In the Instance listing page all the Instances that are being used in the connection will be shown (Shared + Dedicated) but any action can be made only for shared instances. If the Instance Configuration needs to be updated for a dedicated instance then it can be done either in the task listing page in Edit Task option of Edit Task Instance option.