قالب وردپرس درنا توس
Home / Tips and Tricks / Using AWS Object Lambda to Transform S3 Objects On Demand – CloudSavvy IT

Using AWS Object Lambda to Transform S3 Objects On Demand – CloudSavvy IT



AWS logo

Object Lambda allows you to place a Lambda function for S3 objects so that they can be transformed on demand with your own custom code. Since it runs automatically on Lambda, you don’t have to worry about running your own proxy layer.

What is object Lambda?

In fact, Object Lambda takes the place of an API before S3. Previously, you had to set up a proxy layer on your own infrastructure to handle transforming objects on demand. This adds complexity, so AWS has added a better solution.

RELATED: What are lambda functions and how do you use them?

Instead of directly accessing objects, you do this through an Object Lambda Access Point. When you make a GET request for a file in an S3 bucket, the Lambda function for that access point is automatically called, accesses the original object, and a transformed object is returned to the application.

When you make a GET request for a file in an S3 bucket, the Lambda function for that access point is automatically called, accesses the original object, and a transformed object is returned to the application.

The use for this can be simple, such as editing information or converting JSON to XML, but since it’s your own code, you can do whatever you want. For example, you can perform a database lookup and return a transformed object with new data, or make requests to external APIs.

You can have multiple access points per bucket, each representing multiple “views” of the underlying data. You do not need to update a client code to use different access points. Simply change the bucket name to the ARN of the Object Lambda Access Point.

s3.get_object( 
    Bucket='arn:aws:s3-object-lambda:us-east-1:123412341234:accesspoint/myolap', 
    Key='s3.txt' )

You also do not need to open the original object with the exact name. For example, your request can be picture_1920x1080.jpg, who would find picture.jpg and resize it to the specified dimensions. In this case, the Lambda function needs additional permissions to access the contents of the bucket.

You have to pay for all the time you spend running Lambda functions, of course. If you’re running a lot of functions through a user-centric access point, this can start to add up. If your transformations are static, consider caching the objects in a separate S3 bucket. For example, if you have a feature that applies filters / compression to an image, you may want to cache the results instead of rebuilding each request. However, for things that depend on the external state, this is not possible.

RELATED: How to Back Up an S3 Bucket (And Why You Would Even)

Using the Lambda object

Head over to the S3 management console to get started. Each Object Lambda Access Point needs a regular access point behind it. You need to create this via Access points> Create in the sidebar.

Each Object Lambda Access Point needs a regular access point behind it.  You need to create this via Access points> Create in the sidebar.  “Width =” 700 “height =” 316 “src =” / pagespeed_static / 1.JiBnMqyl6S.gif “onload =” pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon (this);  “onerror =” this.onerror = null;  pagespeed.lazyLoadImages.loadIfVisibleAndMaybeBeacon (this);  “/></p>
<p>Enter a name and select a bucket, making sure to select ‘Internet’ unless this bucket is limited to one VPC.  Once created, copy the ARN for the access point.</p>
<p><img class=

Create an Object Lambda Access Point:

Create an Object Lambda Access Point:

Name it and paste it into the access point’s ARN, and the console should display the name of the child bucket.

At this point you need to select a Lambda function. Once you have prepared one, you can enter the ARN or select it from the list. Otherwise, you have to go to the Lambda Management Console to create one.

Once you have prepared one, you can enter the ARN or select it from the list.  Otherwise, you have to go to the Lambda Management Console to create one.

At this point, the code is up to you, although AWS provides the following example, which converts the original object to uppercase. Whatever language you use, you need to grab the event context, make a request to S3 using the URL, transform the object, then write the response using the new WriteGetObjectResponse API, then return an HTTP status code.

import boto3
import requests

def lambda_handler(event, context):
    print(event)

    object_get_context = event["getObjectContext"]
    request_route = object_get_context["outputRoute"]
    request_token = object_get_context["outputToken"]
    s3_url = object_get_context["inputS3Url"]

    
    response = requests.get(s3_url)
    original_object = response.content.decode('utf-8')

    
    transformed_object = original_object.upper()

    
    s3 = boto3.client('s3')
    s3.write_get_object_response(
        Body=transformed_object,
        RequestRoute=request_route,
        RequestToken=request_token)

    return {'status_code': 200}

The event object that Lambda receives looks something like this:

{
    "xAmzRequestId": "1a5ed718-5f53-471d-b6fe-5cf62d88d02a",
    "getObjectContext": {
        "inputS3Url": "https://myap-123412341234.s3-accesspoint.us-east-1.amazonaws.com/s3.txt?X-Amz-Security-Token=...",
        "outputRoute": "io-iad-cell001",
        "outputToken": "..."
    },
    "configuration": {
        "accessPointArn": "arn:aws:s3-object-lambda:us-east-1:123412341234:accesspoint/myolap",
        "supportingAccessPointArn": "arn:aws:s3:us-east-1:123412341234:accesspoint/myap",
        "payload": "test"
    },
    "userRequest": {
        "url": "/s3.txt",
        "headers": {
            "Host": "myolap-123412341234.s3-object-lambda.us-east-1.amazonaws.com",
            "Accept-Encoding": "identity",
            "X-Amz-Content-SHA256": "e3b0c44297fc1c149afbf4c8995fb92427ae41e4649b934ca495991b7852b855"
        }
    },
    "userIdentity": {
        "type": "IAMUser",
        "principalId": "...",
        "arn": "arn:aws:iam::123412341234:user/myuser",
        "accountId": "123412341234",
        "accessKeyId": "..."
    },
    "protocolVersion": "1.00"
}

There are two important pieces of information here: the userRequest section, which contains information about the first request, such as URL and HTTP headers, and the userIdentity section, which can be used to personalize the answer based on the IAM user.

RELATED: AWS IAM Users vs. IAM roles: which one should you use?


Source link