In a multi-tenant environment(with non ra3 redshift nodes), Users were unable to process files in a reload type of dataset with target location as redshift when the selected domain is under a non-default tenant.
Ex: In a multi-tenant environment, User created a 'tenant01' tenant and created a domain 'domain01' under this tenant. Now user created a Reload type of redshift dataset and uploaded file to it. This will fail with below error.
Data load failed in previous run with error - Failed to rename temporary table after job success with error - DWH-1038 - Failed, ERROR: schema "abcd" does not exist
Affected Versions: 2.1 2.0 1.14 1.13
Fix Version: 2.2
Root cause(s)
In 1.13, Redshift DATA APIs were introduced in Amorphic to improve the Redshift data copy process. Amorphic is creating redshift connection with the default tenant (database) instead of tenant where the dataset was created, which is leading to the 'schema does not exist' error as redshift is searching the table in default tenant instead of dataset specific tenant.
Note: Redshift does not allow cross-tenant queries in non-ra3 type of nodes.
Impact
In a multi-tenant environment, Users will be unable to process files in a reload type of redshift dataset with domain created under non-default tenant.
Mitigation
Use the ra3 family redshift nodes in a multi-tenant environment.
Timeline
- 2023-04-10: Bug reported / identified (CLOUD-3217)
- 2023-04-10: Bug triaged
- 2023-04-10: Bug fixed
- 2023-04-11: Testing completed
- 2023-06-01: Planned for Release v2.2