Connect on-premise Python application with AWS services using Roles

In this tutorial I’ll show how you can safely connect your local or on-premise Python application to AWS services like S3. In many cases the connection is made by just creating an IAM user and attach the correct policy. This will work but is not the recommended approach by AWS. It’s recommended to use IAM roles, even if your application is not running inside AWS.

The Python application will have a web interface in which files can be uploaded to an S3 bucket. The application is running on a server inside the on-premise data center of a company. The focus of this tutorial is not on the application itself but on the connection made to AWS.

Upload a file to S3

AWS recommends to use IAM Roles to make applications connect to AWS services like S3. IAM Roles can be reused by multiple applications and do not have standard long-term credentials (credentials are rotated thanks to the AWS STS service).

A role is intended to be assumable by anyone who needs it. In our case the application. In this tutorial we will create an IAM user which has as only permission (IAM policy) to assume a role. Roles can be attached directly to AWS resources like EC2 but not to on-premise servers. That’s why we need to create an IAM user. The user will assume a role which allows the application to write objects to S3.

For this tutorial you also the following resources:

• AWS account + account ID
• A (private) S3 Bucket
• Sample Python application (available on Github)
• AWS CLI (configured for your AWS user)

The Python application will write to the following private S3 bucket. You can configure the application to write to your personal bucket later this tutorial.

Create IAM resources

Now let’s start the tutorial. Create the application IAM user using the AWS CLI or directly in the console. This user will be used to assume roles in your on-premise environment.

$ aws iam create-user --user-name application-user

Now create a policy which we will attach to this user.
It will allow the IAM user to list and assume roles. Here is the JSON document which describes the IAM policy.

$ aws iam create-policy --policy-name assume-role-policy --policy-document file://policy-document-1.json

And finally attach the policy to the application-user. Update the command below with your own account ID number.

$ aws iam attach-user-policy --user-name application-user --policy-arn "arn:aws:iam::123456789012:policy/assume-role-policy"
$ aws iam list-attached-user-policies --user-name application-user

We have created the IAM user called application-user who is allowed to assume roles. Before we create the role we have to create an IAM policy which will be attached to the role. We will call this policy “allow-put-s3-object”.

This policy allows to put objects in the S3 bucket called demo-lvthillo-bucket.

$ aws iam create-policy --policy-name allow-put-s3-object --policy-document file://policy-document-2.json

Create the IAM role and attach the allow-put-s3-object to it. Update the account ID to your personal account ID.

$ aws iam create-role --role-name application-role --assume-role-policy-document file://application-role-trust-policy.json

Attach the allow-put-s3-object to the application-role. Do not forget to update again to your account ID.

$ aws iam attach-role-policy --role-name application-role --policy-arn "arn:aws:iam::123456789012:policy/allow-put-s3-object"

We’re all set. We have created the following IAM resources:

• IAM user application-user
• IAM policy assume-role-policy (attached to the application-user)
• IAM policy allow-put-s3-object
• IAM role application-role (allow-put-s3-object policy attached to this role)

Configure application-user on on-premise server

Create access keys for the application-user.

$ aws iam create-access-key --user-name application-user

Use the output from the command above to configure the access keys on your on-premise server where the Python application will run.
Connect to the on-premise server.

$ aws configure
AWS Access Key ID []: xxx
AWS Secret Access Key []: xxx
Default region name []:
Default output format [None]:

Print the details about the IAM identity whose credentials are used to call the API. This is the application-user which only permission is to assume roles.

$ aws sts get-caller-identity
{
    "UserId": "AXXXXX",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/application-user"
}

This single user can be used by different applications through your on-premise environment to assume roles.

Configure and run Python application

Clone the Python application from GitHub and update the config file which is used by the Python application.

[AWS]
RoleArn = arn:aws:iam::123456789012:role/application-role
BucketName = demo-lvthillo-bucket

These values will be used in the application. Call the assume_role method of the STSConnection object and pass the role ARN and a role session name.

sts_client = boto3.client('sts')
role_arn = config['AWS']['RoleArn']
assumed_role_object = sts_client.assume_role(
    RoleArn=role_arn,
    RoleSessionName="AssumeRoleSession"
)

From the response that contains the assumed role, get the temporary credentials that can be used to make subsequent API calls.

credentials = assumed_role_object['Credentials']

These credentials can be used to put objects in the demo-lvthillo-bucket because the allow-put-s3-object policy is attached to the role.

Test application

Run the Python application and upload a .txt or .png file. Instructions on how to run the application can be found in the README.md on Github.

Upload a test.txt file

You can download the file from S3 to verify if the content is the same.

Conclusion

We have created an IAM user who has only two policies attached (listRole and AssumeRole). This user can be used to assume a role. We have created the IAM role “application-role” and attached a policy which allows the role to put objects in a certain S3 bucket. At last we have created a Python application which is configured to assume this role using the application-user. Roles can be reused by multiple applications and do not have standard long-term credentials which improves the security of our application.

If it really helped you… :)