Provide resources access to an ETL Entity

There are 2 ways of providing resources access to an ETL job.

Note

To receive notifications about the status of resource access update associated with an ETL entity, users must subscribe to notifications for that specific ETL entity during the resource update action.

Resource access through API

Use the following API to provide datasets (read or write), parameters and shared libraries access to an ETL Entity in large number.

Property	Description
resource_type	jobs, glueendpoints or notebooks
resource_id	unique id (uuid) associated with the jobs, glueendpoints or notebooks
API Path	/{resource_type}/{resource_id}/resourceaccess
HTTP Method	POST

Input Body:

    {
        "DatasetAccess": {
            "Owner": [
                {
                    "DatasetId": "uuid",
                    "DatasetName": "dataset_name"
                },
                {
                    "DatasetId": "uuid2",
                    "DatasetName": "dataset_name2"
                }, ...
            ],
            "ReadOnly": [
                {
                    "DatasetId": "uuid3",
                    "DatasetName": "dataset_name3"
                },
                {
                    "DatasetId": "uuid4",
                    "DatasetName": "dataset_name4"
                }, ....
            ]
        },
        "ParameterAccess": ["param1", "param2"],
        "SharedLibraries": ["uuid_lib1", "uuid_lib2"]
    }

To retrieve the status of the latest resource update, use the GET HTTP method without input body. Output retrieved would be

    {
        "Status": "in_progress"/"failed"/"updated",
        "Message": "update failed because of some reason", # When it fails this will appear
        "ParameterAccess": ["param1", "param2"],
        "SharedLibraries": ["uuid_lib1", "uuid_lib2"]
        "DatasetAccess": {
            "ReadOnly": [{
                "DatasetId" : "uuid1",
                "DatasetName": "name1"
            }, ...],
            "Owner": [
            {
                "DatasetId" : "uuid2",
                "DatasetName": "name2"
            }
            ]
        }
    }

Below image shows how the extra resources access provided are shown in User Interface

Resource access through UI

Provide datasets (read or write), parameters and shared libraries access to an ETL Entity in large number using the 'Update Extra Resource Access' button on the top right side of the ETL job details page.

Click on the 'Update Extra Resource Access' button and select the required resources from the dropdown and click 'Submit' to update the resource access.

Once the resource access is submitted, the extra resources will be updated and displayed in the ETL job details page eventually.

The following picture depicts how to update extra resources:

If the extra resources are not getting displayed then refresh the job details page to view the extra resources.

User can also update the extra resources or remove them all again through the same process.

Note

All the extra resources added to the job till that time will be displayed in the UI popup every time the popup is opened so that user can either remove or add accordingly.

Resource Limits

To comply with the character limit for IAM managed policies in AWS Roles, we recommend that the combined total of read and write access datasets not exceed 40. However, below are the independent hard limits on the resource allocation for an ETL entity

Type	Limit
Datasets Read	72
Datasets Write	144
Shared Libraries	62
Parameters	35