S3-SSE: An empirical performance analysis

My current contract involves setting up a serverless processing pipeline in AWS, storing all the data in S3. During development of the system, we have been working with a bucket that does not have versioning or encryption enabled, but we have reached a point where I have asked if the bucket ought to be encrypted.

We are dealing with electrical meter data (electricity consumption), so it is not PII but it is still confidential. Server-side encryption (SSE) on S3 only really protects against an attack vector involving access to Amazon’s physical storage, which is vastly more challenging than this data would be worth; however, enabling SSE on a bucket is trivial, so the configuration cost is essentially zero. With this in mind, encrypting the bucket seems like a reasonable choice to make.

The system architect agreed with this assessment, with the caveat that he would like to be sure that having encryption enabled doesn’t degrade performance too far. I had a brief search of the web to find any other evaluations of performance impacts, and found

• Performance impact of S3 encryption (Reddit)
• Performance Impact of Encryption (Cloudera)

Since these didn’t supply any numbers, I figured: why not do some basic measurements myself?

All our data access is from within AWS (Lambda and Glue), so I needed to perform the measurements from there. With that in mind, I created this Lambda code:

import io
import time
import uuid

import boto3

MB = 2**20
BASE_MEMORY_REQUIRED = 80  # MB


def create_file(size_mb: int) -> io.BytesIO:
    """
    Create and return a file-like buffer of 'size' random megabytes.
    """
    buffer = io.BytesIO()
    with open('/dev/urandom', 'rb') as random_source:
        for _ in range(size_mb):
            buffer.write(random_source.read(1 * MB))
    buffer.seek(0)
    return buffer


def lambda_handler(event: dict, context):
    file_size_mb = event['file_size_mb']
    unencrypted_bucket = event['unencrypted_bucket']
    encrypted_bucket = event['encrypted_bucket']
    iterations = event['iterations']

    # Check that the lambda has been set up properly
    if int(context.memory_limit_in_mb) <= file_size_mb + BASE_MEMORY_REQUIRED:
        raise ValueError("Insufficient memory allocated")

    def upload(bucket):
        big_file.seek(0)
        s3.put_object(
            Bucket=bucket,
            Key=file_name,
            Body=big_file,
        )
        
    def download(bucket):
        obj = s3.get_object(
            Bucket=bucket,
            Key=file_name,
        )
        obj['Body'].read()
        
    s3 = boto3.client('s3')
    unencrypted_upload_times = []
    encrypted_upload_times = []
    unencrypted_download_times = []
    encrypted_download_times = []
    print(f"Starting {iterations} uploads and downloads")
    for i in range(iterations):
        big_file = create_file(file_size_mb)
        file_name = uuid.uuid4().hex

        # Upload the data and time the operation (twice),
        # then download the data and time it (twice)
        t0 = time.time()
        upload(encrypted_bucket)
        encrypted_upload_times.append(time.time() - t0)
        print(f"{i}: uploaded to encrypted bucket")

        t0 = time.time()
        upload(unencrypted_bucket)
        unencrypted_upload_times.append(time.time() - t0)
        print(f"{i}: uploaded to unencrypted bucket")
        # Try to avoid doubling up on in-memory storage of the large files
        big_file.truncate(0)

        t0 = time.time()
        download(encrypted_bucket)
        encrypted_download_times.append(time.time() - t0)
        print(f"{i}: downloaded from encrypted bucket")

        t0 = time.time()
        download(unencrypted_bucket)
        unencrypted_download_times.append(time.time() - t0)
        print(f"{i}: downloaded from unencrypted bucket")

    result = {
        "iterations": iterations,
        "size": file_size_mb,
        "encrypted_upload_times": encrypted_upload_times,
        "unencrypted_upload_times": unencrypted_upload_times,
        "encrypted_download_times": encrypted_download_times,
        "unencrypted_download_times": unencrypted_download_times,
    }
    print(result)
    return {
        'statusCode': 200,
        'body': result,
        'headers': {'Content-Type': 'application/json'}
    }

and triggered it directly from the Lambda console “test” utility with the payload

{
  "file_size_mb": 800,
  "unencrypted_bucket": "alex-test-unencrypted",
  "encrypted_bucket": "alex-test-encrypted",
  "iterations": 10
}

I only requested 10 iterations because even that took nearly ten minutes to run with the 800MB file size that I specified. So I called it 5 times to obtain 50 measurements. The results looked like this:

Which suggests that upload times are largely unaffected by the encryption setting, and that download times are interestingly faster for encrypted files (at least for these files).

In conclusion, enabling encryption on the S3 bucket has no cost (direct financial or performance) so there is no reason not to do so. Note, however, that this measurement only used SSE-S3, not SSE-KMS. From some brief searching of the web, SSE-KMS can run into performance problems in terms of rate of files uploaded/downloaded (as opposed to rate of bytes uploaded/downloaded) because it requires S3 to make API calls to KMS to obtain the key for each operation; this can be largely mitigated by using a Bucket Key, but that’s well beyond the scope of this brief investigation.

Appendix - the raw time measurements (in seconds):