Data ingestion using the connections(JDBC bulk-load type) for datasets with Redshift target are failing with S3 Access Denied error. AWS has changed the behavior of the replication instance provisioning call and not honoring the version passed that is passed in the input request and using a version(3.4.7) that is not compatible with Redshift.
Affected Versions: 2.1 2.0 1.14 1.13
Fix Version: 2.2 2.1 2.0* 1.14* 1.13*
Customers upgrading to 2.0, 1.14, 1.13 after March 18, 2023 does not require explicit patching as the fix will be part of the release artifacts.
Root cause(s)
As per the AWS support team's response, when Auto minor version upgrade configuration is enabled, DMS uses the current default engine version when you create a replication instance. For example, if you set the Engine version to a lower version number than the current default version, DMS uses the default version than the version that is passed in the input request. In short, DMS instances will default to a region's "preferred version" when the option "Automatic version upgrade" is enabled, Instead of happening during the maintenance window, it would be applied during the launch of the instance. Because of the above behavior, an incompatible version(3.4.7) of the Replication instance is being created which has some known issues in the data ingestion process for Redshift datasets and causing the error S3 Access denied. By default Amorphic sets the Auto minor version upgrade to true as per the recommendations to safeguard against any security vulnerabilities.
Impact
Users will not be able to ingest data into Redshift datasets using jdbc bulk data load feature.
Mitigation
Patch is available for Amorphic versions v2.1, v2.0, v1.14 and v1.13.
Timeline
- 2023-04-18: Bug reported/identified (CLOUD-3246)
- 2023-04-18: Bug triaged
- 2023-04-19: Bug fixed
- 2023-04-19: Testing of patch on v2.1, v2.0, v1.14, v1.13 is completed
- 2023-04-19: Patch for v2.1, v2.0, v1.14, v1.13 released(v2.1, v2.0, v1.14, v1.13 artifacts updated)