Serverless Event Driven Dashboards

Serverless has recently become the thing that get everyone excited. So when we were looking at rebuilding a Build Dashboard, serverless was the way to go.

The architecture has five basic components.
  • Build/Deploy Pipeline
  • CloudWatch Events Rule
  • Lambda Function
  • DynamoDB Table
  • API Gateway
  • PIPELINE/STAGES
      Based on the granularity of visibility into the pipeline, the events could be per stage or per pipeline execution. The most relevant way to achieve this is to trigger a start event when a stage/pipeline starts. And capture the execution status and publish an event after the stage finishes. This could either be pass or fail.
      aws events put-events --entries file://startevent.json
      Example Event: startevent.json
      {
        "version": "0",
        "detail-type": "Pipeline Stage Execution State Change",
        "source": "Pipeline",
        "account": "123456789012",
        "time": "2017-04-22T03:31:47Z",
        "region": "ap-southeast-1",
        "resources": [
          "TestPipeline"
        ],
        "detail": {
          "pipeline": "TestPipeline",
          "version": "13",
          "build-id": "01234567-0123-0123-0123-012345678901",
          "stage": "Build",
          "state": "STARTED"
        }
      }

    CLOUDWATCH EVENTS RULE LAMBDA FUNCTION DYNAMODB TABLE API GATEWAY
    
    Description: Creates a cloudformation stack to deploy dynamodb table and API Gateway proxy
    AWSTemplateFormatVersion: '2010-09-09'
    
    Resources:
    
      PipelinesDashboardApiGatewayExecutionRole:
        Type: AWS::IAM::Role
        Properties:
          RoleName: 'PipelinesDashboardApiGatewayExecutionRole'
          AssumeRolePolicyDocument:
            Version: '2012-10-17'
            Statement:
            - Action: ['sts:AssumeRole']
              Effect: Allow
              Principal:
                Service:
                  - apigateway.amazonaws.com
          Path: /
          Policies:
            - PolicyName: 'PipelinesDashboardApiGatewayExecutionRolePolicy'
              PolicyDocument:
                Version: '2012-10-17'
                Statement:
                  - Action:
                    - dynamodb:Scan
                    Effect: Allow
                    Resource: '*'
    
      PipelinesDashboardDynamoDbTable:
        Type: 'AWS::DynamoDB::Table'
        Properties:
          TableName: 'pipelines_dashboard_data'
          AttributeDefinitions:
            -
              AttributeName: 'PipelineName'
              AttributeType: 'S'
          KeySchema:
            -
              AttributeName: 'PipelineName'
              KeyType: 'HASH'
          ProvisionedThroughput:
            ReadCapacityUnits: '2'
            WriteCapacityUnits: '2'
    
      PipelinesDashboardStage:
        Type: 'AWS::ApiGateway::Stage'
        Properties:
          DeploymentId: !Ref 'ApiDeployment'
          Description: 'API Gateway for Pipelines Dashboard'
          MethodSettings:
            - ResourcePath: /
              HttpMethod: GET
          RestApiId: !Ref 'DashboardApi'
          StageName: 'data'
    
      ApiDeployment:
        Type: 'AWS::ApiGateway::Deployment'
        DependsOn: GetPipelinesDetailsMethod
        Properties:
          RestApiId: !Ref DashboardApi
    
      GetPipelinesDetailsMethod:
        Type: "AWS::ApiGateway::Method"
        Properties:
          AuthorizationType: 'NONE'
          HttpMethod: 'GET'
          Integration:
            Type: 'AWS'
            IntegrationHttpMethod: 'POST'
            Uri: !Sub 'arn:aws:apigateway:${AWS::Region}:dynamodb:action/Scan'
            PassthroughBehavior: 'WHEN_NO_TEMPLATES'
            Credentials: !Sub 'arn:aws:iam::${AWS::AccountId}:role/PipelinesDashboardApiGatewayExecutionRole'
            IntegrationResponses:
            -
              ResponseTemplates:
                application/json: "#set($inputRoot = $input.path('$'))\n[\n    #foreach($elem\
                  \ in $inputRoot.Items) {\n        \"name\": \"$elem.PipelineName.S\"\
                  ,\n        \"status\": \"$elem.PipelineStatus.S\",\n        \"stage\"\
                  : null\n    }#if($foreach.hasNext),#end\n\
                  \t#end\n\n]"
              StatusCode: '200'
            RequestTemplates:
              application/json: "#set($inputRoot = $input.path('$'))\n{ \"TableName\"\
                : \"pipelines_dashboard_data\" }"
          OperationName: 'GetPipelinesData'
          ResourceId: !GetAtt 'DashboardApi.RootResourceId'
          RestApiId: !Ref 'DashboardApi'
          MethodResponses:
          -
            StatusCode: '200'
    
      DashboardApi:
        Type: 'AWS::ApiGateway::RestApi'
        Properties:
          Name: 'PipelinesDashboardApi'
          Description: 'API used for Pipelines Dashboard requests'