Amazon S3 Setup

Amazon S3 Setup #

Overview #

Configuring Amazon S3 work with MongoDB Realm through the Cosync Storage module is relatively easy. The developer must create an Amazon S3 user with some access keys and a storage bucket. In order to set up the MongoDB Realm Application through the Cosync Portal, the developer will need to gather the following information.

  • AWS user access key
  • AWS user secret access key
  • AWS S3 bucket name
  • AWS bucket region code

In the following section we show the developer how to create and Amazon S3 user, along with a set of API keys, as well as create an S3 bucket, which will hold all of the assets uploaded from from a MongoDB Realm application through the Cosync Storage module.

Amazon S3 Users & Groups #

This document does not purport to be a comprehensive tutorial on configuring the Amazon AWS S3 Storage Service, as Cosync makes the assumption that a developer is familiar with it as a prerequisite. For more detailed information from Amazon on the S3 Service, please consult this link. We shall however discuss the necessary tweaks to S3 that are needed to integrate it into a MongoDB Realm application running with the Cosync Storage module.

Programatic Access #

In order for MongoDB Realm functions to talk to the Amazon S3 Storage Service, the developer will need to add a user with Programmatic Access to their AWS account through the IAM service. By Programmatic Access, we mean that the developer will need to create an access Key ID and a secret access key for the MongoDB Realm functions to call into the AWS API. These keys are created as part of the process of adding a new user with programmatic access to AWS. During this process, the developer will need to save the keys to a downloaded CSV file - as these keys cannot be retrieved after the fact. Furthermore, the developer should not share these keys with anyone as they would permit full access to the Amazon S3 account. Although the Cosync Portal uses these keys to configure the MongoDB Realm application, it will not save them to its database. The developer is fully responsible for the safe keeping of these AWS access keys.

The process for creating a group and a user through IAM that is described below is not necessarily the optimal strategy for configuring Amazon S3, but it is simple enough for the average developer to get started.

As a prerequisite for creating a user with programmatic access to AWS, the developer will first create a group with an attached policy of AmazonS3FullAccess that permits full access to AWS.

Group Creation #

  1. Create the group

Group1

  1. Attach the policy

Group2

  1. Create and review

Group3

In Amazon S3, the group object essentially bundles a policy with a group entity that can then be referenced by a newly created user. In Amazon S3, users can belong to groups, through which they inherit policies that provide access control to the S3 resources.

User Creation #

  1. Create the user

User1

  1. Add the user to the new group

User2

  1. Add optional tags to the user (default is none)

User3

  1. Review and create user

User4

  1. Download .csv file with keys

User5

  1. Inspect and safeguard .csv file

User6

Amazon S3 Buckets #

The next step in the process of configuring Amazon S3 to work with a MongoDB Realm application is to add a bucket to contain all of the assets. For S3, a bucket is the topmost container used to store asset data - all files stored in a bucket have a unique path and can be accessed through URLs by the outside world (expiring or not).

Anatomy of a bucket #

The directory structure underneath an Amazon S3 bucket used to with the Cosync Storage module is organized as follows. The topmost directories directly underneath the bucket will be named with the Realm user Id of the user who uploaded the asset to the bucket. There is also a public directory used to store non-expiring public assets. This public directory in turn has subdirectories for each user Id of the user who uploads a public asset.

Underneath every user directory, the developer can specify an arbitrary path like /avatar. For example, if user with user Id equal to 5ff481a161c490458f8e2c3f uploaded an asset called mugshot.jpg to a path called /avatar, the Cosync Storage module would create cuts with a time stamp included as follows - the time stamp is 1609859729341. Similarly if a user with user Id equal to cb6b7d76cb515cc07557b8a6 were to upload a non-expiring public asset to the path /backgrounds, the resulting asset would be placed under the public branch as shown below.


CosyncStorageBucket
├── 5ff481a161c490458f8e2c3f
│   └── avatar
│       ├── mugshot-1609859729341.jpg
│       ├── mugshot-large-1609859729341.jpg
│       ├── mugshot-medium-1609859729341.jpg
│       └── mugshot-small-1609859729341.jpg
└── public
    ├── 5ff481a161c490458f8e2c3f
    └── cb6b7d76cb515cc07557b8a6
        └── backgrounds
            ├── texture-1609859729348.jpg
            ├── texture-large-1609859729348.jpg
            ├── texture-medium-1609859729348.jpg
            └── texture-small-1609859729348.jpg

Add a bucket to Amazon S3 #

Adding a bucket to Amazon S3 is very easy. To do so, the developer must first go to S3 page within the AWS Console and hit Create Bucket. The bucket name must be lowercase.

Bucket1

Before the bucket is created, the developer must uncheck Block all public access to make the assets in the bucket accessible to the MongoDB Realm Application. After the bucket is created, the developer must click on the bucket and go to the Permissions tab and edit the bucket policy. The developer must change with their bucket name as needed.


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<name of bucket>/public/*"
        }
    ]
}

In practice this looks like this:


Bucket2


Once the bucket policy has been set, the developer is ready to integrate the Amazon S3 bucket into the MongoDB Realm Application using the Cosync Storage module.

Region Code #

The region code for the AWS S3 bucket can be retrieved directly from the list of buckets. In the example below, the region code for the bucket cosyncstoragetest is us-east-1.


Bucket3