Dashboard Framework Part 2: Running Shiny in AWS Fargate with CDK

In the previous post we outlined the architecture of a dashboard framework to run dashboards based on multiple technologies including Shiny and Flask in production. We will now show how to run a basic Shiny dashboard in AWS Fargate behind an Application Load Balancer in less than 60 lines of CDK code. To define our stack in a reproducible manner we will make use of the Amazon Cloud Development Kit (CDK) with Typescript. Starting from a basic CDK stack we now specify the most important components of our stack:

1. The Application Load Balancer (ALB) to route traffic to our dashboards.
2. The Fargate cluster to run our dashboard tasks in a scalable manner.

The deployed stack will finally run an example Shiny dashboard behind an Application Load Balancer. Note that the resulting stack will only run one dashboard without encryption. We’ll implement these features as part of the next post. The resulting CDK code can also be downloaded from Github at https://github.com/quantargo/dashboards.

Prerequisites

To run the following code examples make sure to have

1. an AWS Account
2. a locally configured AWS account by running e.g. aws configure with the aws CLI
3. a local Node.js installation (version >= 14.15.0)
4. Typescript: npm -g install typescript
5. CDK (version >= 2.0): npm install -g aws-cdk

Initialize CDK and Deploy first App

To initialize a sample project we first create a project folder and within the folder execute cdk init:

> mkdir dashboards
  cd dashboards
  cdk init app --language typescript

This command creates a new CDK Typescript project and installs all required packages. The following 2 files are relevant for stack development:

• bin/dashboards.ts: Main file which initializes CDK stack class. You can explicitly set the environment env if you use a different account or region for deployment.
• lib/dashboards-stack.ts: CDK Stack class to which all components of our stack will be added.

Specify Application Load Balancer (ALB)

Next, we need to create an Application Load Balancer (ALB) within a new VPC which is responsible for secure connections and routing. We create a new VPC and add an internetFacing load balancer to it. This means that the load balancer will be accessible from the public internet and will therefore be placed into a public subnet. Within the lib/dashboards-stack.tsfile we put the following lines:

> // Put imports on top of the file
  import * as ec2 from 'aws-cdk-lib/aws-ec2';
  import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'

  // Put below lines within the DashboardsStack constructor
  const vpc = new ec2.Vpc(this, 'MyVpc');

  const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', {
    vpc: vpc,
    internetFacing: true,
    loadBalancerName: 'DashboardBalancer'
  });

Specify Dashboard Cluster

Next, we need to add an ECS cluster to our VPC to run our dashboards efficiently:

> // Put imports on top of the file
  import * as ecs from 'aws-cdk-lib/aws-ecs'

  // Put below lines within the DashboardsStack constructor
  const cluster = new ecs.Cluster(this, 'DashboardCluster', {
    vpc: vpc
  });

Add First Fargate Task Definition

We can now add our first Fargate dashboard to the cluster by specifying a task definition. We use the rocker/shiny Docker container as an example running on port 3838. This also requires respective port mappings in the container definition. Additionally, we use half a virtual CPU (512)—1024would equal a full one—and a memory size of 1024 MiB. By specifying the Fargate service we are already finished with the specification to run our first container in the cluster:

> const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDefinition', {
   cpu: 512,
   memoryLimitMiB: 1024,
  });

  const port = 3838

  const container = taskDefinition.addContainer('Container', {
    image: ecs.ContainerImage.fromRegistry('rocker/shiny'),
    portMappings: [{ containerPort: port }],
  })

  const service = new ecs.FargateService(this, 'FargateService', {
    cluster: cluster,
    taskDefinition: taskDefinition,
    desiredCount: 1,
    serviceName: 'FargateService'
  })

Put Service Behind ALB

Next, we put the Fargate service into an ALB target group so that traffic can be routed through the ALB:

> const tg1 = new elbv2.ApplicationTargetGroup(this, 'TargetGroup', {
   vpc: vpc,
   targets: [service],
   protocol: elbv2.ApplicationProtocol.HTTP,
   stickinessCookieDuration: cdk.Duration.days(1),
   port: port,
   healthCheck: {
     path: '/',
     port: `${port}`
   }
 })

Note that we added 2 parameters to the ALB target group definition:

• stickinessCookieDuration: Since Shiny sessions are stateful we need to prevent the ALB to switch instances (in case there are more) during a session. The session duration set to one day should be sufficient.
• healthCheck: The health check needs to specify the port (as string) and set to the container port 3838, as well.

Finally, we add an HTTP listener which directly forwards all incoming traffic to our dashboard:

> const listener = lb.addListener(`HTTPListener`, {
   port: 80,
   defaultAction: elbv2.ListenerAction.forward([tg1]) 
 })

Deploy

Before deployment you should also bootstrap your CDK environment:

> cdk bootstrap

Now the stack should be ready for deployment. As an extra step, you can now check if the stack can be successfully synthesized using

> cdk synth

Any errors popping up during cdk synth need to be fixed immediately. By continously using cdk synth we make sure that the feedback cycles during development are as short as possible. If cdk synth is successful we can now run

> cdk deploy

Finally, you should see the successful output message including the DashboardsStack.LoadBalancerDNSName which you can directly access through the browser:

> Outputs:
  DashboardsStack.LoadBalancerDNSName = DashboardBalancer-<9-DIGIT-NUMBER>.<region>.elb.amazonaws.com
  Stack ARN:
  arn:aws:cloudformation:<region>:<AWS-ACCOUNT-NO>:stack/DashboardsStack/<uuid>

    Total time: 297.67s

Destroy

If you don’t use the stack any more and to reduce cloud costs just run:

> cdk destroy

Visit Quantargo for additional insight on this topic: https://www.quantargo.com/blog/2022-03-11-creating-dashboard-framework-with-aws-part2.