While performing the following operations, the directory path for S3 connection type datasets is getting deleted from the metadata:
- Updating a dataset schema
- Updating DataConversionParameters for Redshift datasets
- Updating GlobalSecondaryIndexes for DynamoDB datasets
- Conducting batch edit operations on datasets
Affected Versions: 2.6
2.6.1
Fix Version: 2.6*
2.6.1*
2.7
Customers upgrading to these versions after Aug 09, 2024 does not require explicit patching as the fix will be part of the release artifacts.
Root cause(s)
While making few enhancements and refactors in 2.6 related to s3 connection type of datasets resulted in these errors.
Impact
After upgrading to 2.6/2.6.1, When users try to update the dataset schema or above specified operations then DirectoryPath (S3 prefix) for s3 connection type of dataset will be deleted from the dataset metadata.
Mitigation
Fix will be available in Amorphic version 2.7.
Patch is available for Amorphic versions v2.6, 2.6.1
Timeline
- 2024-08-03: Bug reported/identified (CLOUD-4810)
- 2024-08-04: Bug triaged
- 2024-08-05: Bug fixed
- 2024-08-07: Testing of patch on v2.6 and v2.6.1 has been completed
- 2024-08-07: Patch for v2.6 and v2.6.1 has been released