Dart File Server - AWS Lambda - Flutter Web Link Previews

06/26/2020

My 💙 for Flutter has gotten me more interested in the Dart programming language. I recently worked on a project where I used Flutter to build an app for Android, iOS and the Web. When it came to hosting the web app I had a dilemma. Here is a little about how I solved this problem.

To enabling sharable links with rich media I needed to implement open graph tags for things like messengers and social media to display the link nicely. Unfortunately, this isn’t possible in single page applications. After all, there is only 1 HTML file. The content changes but still there is only 1 HTML file and the open graph meta data needs to change on each “page”.

Another problem, if a user copys and pastes directly from the address bar the URL looks something like this https://example.com/#/browse/1 where #/browse/1 fragment is not actually sent to servers. This is just how browsers work, the fragment is used by the browser to find the position on the single page. I needed a way to dynamically generate HTTP responses with open graph meta data in the HTML.

Web Hosting with S3?

My first thought for static sites is usually “easy peasy, S3 it is!”. S3 is great for static hosting and is extremely cheap and reliable. After all, every project has a a budget. However, with static hosting you have no control of incoming HTTPS requests. Thats not going to work. With no control I cannot dynamically construct open graph meta data. I’m a big fan of AWS Lambda. It’s cost effective, zero maintenance and simple to use. Sure, I could spin up a conventional server with Nginx or something but I wanted to keep things as simple as possible.

Dart AWS Lambda Runtime

I’m already working with Dart for mobile and the web, why not use it serverside too?! I now have a single codebase that runs Android, iOS, web AND now server side!

New Dart features have come out since I hand rolled my own runtime, Dart VM with AWS Lambda Custom Runtime, and better yet AWS has published their own open source version of the Dart custom runtime! Check out their official blog post.

Lambda as a File Server

With two 3rd party dependecies I implented my own, naive, file server in Dart.

This is all you need to get up and running! Check out the full implementation on Gitlab, aws_lambda_server.dart.

final Handler<AwsApiGatewayEvent> fileServer =
    (Context context, AwsApiGatewayEvent event) => serveRequest(context, event);

Future<void> main() async {
  Runtime()
    ..registerHandler<AwsApiGatewayEvent>('main.staticFileServer', fileServer)
    ..invoke();
}

A simple deployment includes compiling this program with dart2native. Here is my build script after running flutter build web. I created the folder web-lambda at the root of my Flutter project to hold all things lambda.

function build() {
    [[ -z $1 ]] && { echo "file name not specified"; exit 1; }
    mkdir -p web-lambda/public

    echo 'copy web build...'
    cp -R build/web/* web-lambda/public

    echo 'getting packages...'
    cd web-lambda
    pub get

    echo 'compiling...'
    dart2native main.dart -o bootstrap

    echo 'set bootstrap executable...'
    chmod +x bootstrap

    echo 'zipping...'
    zip -r $1 public/ bootstrap
}

The contents of the zip file now has a Dart executable and a public folder with all of your Flutter Web files. I deployed this Lambda with Api Gateway integration to invoke it from the web.

Great, now I have my Flutter web app being served up via Api Gateway and Lambda! But how do I distinguish between a normal request vs a request that I need to generate meta data? User-Agent Headers to the rescue! aws_lambda_server.dart simply serves files. I need to check incoming requests. I created a new main.dart and pass the custom runtime my own handler. This way I can inspect incoming requests and do what I please with them, reverting to aws_lambda_server.dart to handle “normal” request.

 // main.dart
import 'dart:convert';
import 'dart:io';

import 'package:aws_lambda_dart_runtime/aws_lambda_dart_runtime.dart';
import 'package:aws_lambda_dart_runtime/runtime/context.dart';
import './aws_lambda_server.dart';

 final Handler<AwsApiGatewayEvent> apiGateway = (
  Context context,
  AwsApiGatewayEvent event,
) async {
    if (event.path.contains('/browse/')) {
        final id = event.path.split('/').last;
        if (isBot(event)) { // <-- Is this a preview link request?
            final metaData = await getMetaHtml(id); // <-- generates HTML with dynamic meta data
            return InvocationResult(
                context.requestId,
                AwsApiGatewayResponse(
                    body: metaData,
                    headers: {'Content-Type': 'text/html; charset=utf-8'},
                    statusCode: HttpStatus.ok,
                    isBase64Encoded: false,
                ),
            );
        } else {
            // Normal user wants to open page, redirect them
            return InvocationResult(
                context.requestId,
                AwsApiGatewayResponse(
                    headers: {'Location': 'https://example.com/#/browse/$id'},
                    statusCode: HttpStatus.movedPermanently,
                    isBase64Encoded: false,
                ),
            );
        }
     }

    // Default to serving requests as usual
    return serveRequest(context, event);
}

Future<void> main() async {
    Runtime()
      ..registerHandler<AwsApiGatewayEvent>('main.handler', apiGateway)
      ..invoke();
}

When sharing a link from the app I contruct a URL. For example https://example.com/#/browse/1 turns into https://example.com/browse/1 dropping the #. This way, when the entire URL is sent to my Lambda. When a request comes in I can inspect the request path, if (event.path.contains('/browse/')) and check if the User Agent is a bot.

isBot inspects the user agent so I can dynamically generate meta data only HTML for a rich link sharing experience! If it is a bot I generate meta data, otherwise I redirect the user’s request back to my web app. Going the other direction, https://example.com/browse/1 is turned back into https://example.com/#/browse/1. This allows the web app to load and then open the correct view.

bool isBot(AwsApiGatewayEvent event) =>
    event.headers.userAgent.contains('facebookexternalhit') ||
    event.headers.userAgent.contains('Twitterbot') ||
    event.headers.userAgent.contains('Facebot');

Note: API Gateway and Binary Files

One thing that tripped me up was the use of binary files with API Gateway. Simply responding with isBase64Encoded: true in the Lambda is not enough. You also have to configure API Gateway to know which Content-Types to treat as binary. Without it, all of your binary files will not be sent over the wire correctly and your site will look wrong.

I use CloudFormation and set BinaryMediaTypes to */*. This will deploy however I have not had much success with it working correctly in CloudFormation. I ultimately had to manually set these settings in the AWS Console in the settings section for API Gateway. Once set, be sure to deploy the API again. This ensures the BinaryMediaTypes is in use.

*/* might feel strange, I agree, but I did not have success individually listing content types like image/png.

  RestApi:
    Type: AWS::ApiGateway::RestApi
    Properties:
      Description: My Web Server
      BinaryMediaTypes:
        - "*/*" # <-- this is key!
      EndpointConfiguration:
        Types:
          - EDGE
      FailOnWarnings: false
      Name: !Ref ApiName