Private S-Less Three Trick Pony

2021-07-04

A python, a lamb and a node go through a private gateway to get to a spa. “That ripped green massage bench looks like someone CURL’ed up and died on it,” python says. Node suggests: “I’ll strap this scented candle on lambs back, snakey old pal, just close your eyes and imagine you are in a room with a VUE”. Ah, du-du-dish.

Ever had the need to build something in AWS, stringing services together in a flurry of “back office activity” but no quick way to make it pretty and hook up some quick auth to go with it?

I’ve had those special ‘needs’ - plenty, so here is a rig I have used a couple of times. It avoids all the usual rigmarole of amplify console, CloudFront, lambda@edge, Cognito and other public-facing orifices to get a user to click on something that resembles a button and see a value.

The basic gist is to:

nest a spa inside a lambda,
make it speak through an Authoriser
which protects a BFFesque lambda that does applicationy stuff.
serve it all up in one stack

In this example, we use serverless back-endy bits to implement an sfn workflow Todo app

Noco Flow

foo

The flow is from the browser is:

hit the ui lambda and get served with static content straight up from the ui lambda’s dist/* folder
once vue/react/svelte compiled bits kicks in, and the meat-bag types up some credentials axios calls back home with some basic auth in tow.
the Authorizer gets a call to validate those uid:pwd bits and
if ldap bind works api lambda kicks in, and does something. In this case: launch a new SFN, save some stuff in DyndeeB or complete a SFN callback.
if the callback was successful, it removes the todo from dyndb and the sfn carries on and completes

The project looks like this:

foo

Le Lambdas

Starting from what the browser sees…

U/I have been served

ui.py - literally nothing much to see here - just a quick little map an extension to mime-type, serve a file rig

import os
import json
import base64

extMap = {
        '.js': {
            'ctype': 'application/javascript',
            'enc': False
            },
        '.css': {
            'ctype': 'text/css',
            'enc': False
            },
        '.html': {
            'ctype': 'text/html',
            'enc': False
            },
        '.ico': {
            'ctype': 'image/vnd.microsoft.icon',
            'enc': True
            },
        '.png': {
            'ctype': 'image/png',
            'enc': True
            },
        'default': {
            'ctype': 'application/octet-stream',
            'enc': True
            },
        }

def handler(event, context):
    print ('Received event:', json.dumps(event));
    path = event["path"] if event["path"] != '/' else '/index.html'
    if os.path.exists(f'dist/{path}'):
        emap = extMap[os.path.splitext(path)[1]] if os.path.splitext(path)[1] in extMap else extMap['default']
        ctype = emap["ctype"]
        enc = emap["enc"]
        ctent = None
        with open(f'dist/{path}','rb') as f:
            ctent = f.read()
            if enc:
                ctent = base64.b64encode(ctent)
            else:
                ctent = ctent.decode("utf-8") 
        return {
                "statusCode": 200,
                "headers": { 
                    "content-type": ctype
                    },
                "body": ctent,
                "isBase64Encoded": enc,
                }
    else:
        return {
                "statusCode": 404,
                "headers": { 
                    "content-type": "text/plain"
                    },
                "body": f'file {path} not found',
                "isBase64Encoded": False,
                }

Bow to Authorotai

auth.py - again nothing spectacular - just a decode, bind check that flips an Effect bit.

import sys
sys.path.insert(0,'lib')
import json
import os
from basicauth import decode
from ldap3 import Server, Connection, ALL, NTLM

server = Server(os.environ['LDAPURI'], get_info=ALL)

def handler(event, context):
    username, password = decode(event['authorizationToken'])
    ret = {
            "principalId": "user",
            "policyDocument": {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Action": "execute-api:Invoke",
                        "Effect": "Deny",
                        "Resource": event['methodArn']
                        }
                    ]
                },
            }
    validUsers = os.environ['USERS'].split(',')

    if username not in validUsers:
        return ret

    try:
        conn = Connection(server, user=username, password=password, authentication=NTLM)
        conn.bind()
        i_am = conn.extend.standard.who_am_i()
        if i_am is not None:
            ret['policyDocument']['Statement'][0]['Effect'] = 'Allow'
    except Exception as e:
        print (e)
    print (ret)
    return ret

and a requirements.txt with

ldap3
basicauth

A Private Eye

api.py - this is the actual API that does stuff…

GET api/tasks : scan dyndb for active todos

GET api/task/<tid> : read a single todo (dyndb get_item)

POST api/task/<tid> : save a value in a todo (dyndb write_item)

POST api/task/<tid>/complete : send sfn on its merry way (sfn success callback and delete_items’ the record out of dyndb)

import sys
sys.path.insert(0,'/opt')
import os
import json, base64
import boto3
import taskManager

def handler(event, context):
    event["headers"]["Authorization"]='zzz'
    print ('Received event:', json.dumps(event))
    res = {}
    path = event['path'].split('/')
    if path[2] =='tasks':
        if event['httpMethod'] == 'GET':
            res = taskManager.getTaskList()
        if event['httpMethod'] == 'POST':
            res = taskManager.newTask(event["body"])
    if path[2] =='task':
        if len(path)==4:
            if event['httpMethod'] == 'GET':
                tid = path[3]
                res = taskManager.getTask(tid)
            if event['httpMethod'] == 'POST':
                tid = path[3]
                res = taskManager.updateTask(tid, event["body"])
        if len(path)==5:
            if event['httpMethod'] == 'POST' and path[4] == 'complete':
                tid = path[3]
                res = taskManager.completeTask(tid)
    ret = {
        "isBase64Encoded": False,
        "statusCode": 200,
        "headers": { 
            "content-type": 'application/json'
            },
        "body": json.dumps(res,default=str),
    }
    print (ret)
    return ret

taskManager.py - boto3 hacking that goes with it

import boto3
import os

dyndb = boto3.client('dynamodb')
sfn = boto3.client('stepfunctions')

letab = os.environ['DYNTAB']
lesfn = os.environ['SFNARN']

def getTaskList():
    return dyndb.scan(
            TableName=letab,
            Limit=100
            )['Items']

def newTask(task):
    return sfn.start_execution(
            stateMachineArn=lesfn,
            input=task,
            )

def getTask(tid):
    return dyndb.get_item(
            TableName=letab,
            Key={
                "id": {
                    "S": tid
                    }
                }
            )

def updateTask(tid, task):
    itemres = dyndb.get_item(
            TableName=letab,
            Key={
                "id": {
                    "S": tid
                    }
                }
            )
    itemres = dyndb.put_item(
            TableName=letab,
            Item={
                "id": {
                    "S": tid
                    },
                "task": {
                    "S": task
                    },
                "token": {
                    "S": itemres['Item']['token']['S'],
                    }
                }
            )
    return itemres

def completeTask(tid):
    itemres = dyndb.get_item(
            TableName=letab,
            Key={
                "id": {
                    "S": tid
                    }
                }
            )
    print(itemres)
    sfn.send_task_success(
            taskToken=itemres['Item']['token']['S'],
            output=itemres['Item']['task']['S']
            )
    return dyndb.delete_item(
            TableName=letab,
            Key={
                "id": {
                    "S": tid
                    }
                }
            )

And the SFN Lambdas

task.py - a lambda to stash the sfn token in dyndb

import os
import boto3
import json
import uuid

ddb = boto3.client('dynamodb',region_name='af-south-1')

def handler(event, context):
    print (event)
    dyntab = os.environ['DYNTAB']
    ddb.put_item ( 
            TableName=dyntab,
            Item={
                'task': { 'S': json.dumps(event['input'],default=str) },
                'token' : { 'S' : event['token'] },
                'id' : { 'S' : str( uuid.uuid4() ) },
                },
            )

something.py - a lambda to say hello

import boto3

def handler(event, context):
    print ('Hello Cruel World')

A Vue

building ui’s that excite and entertain is “obviously” what I am very good at, and here’s my rendition of a nasty todo app.

Vue ss

This is a stock v3 cli project, with a bit of axios tendrels back to where it came from.

Besides adding tailwindcss to the mix and jigging the dev server to answer some back-end GET’s

vue.config.js - remapping prod/api to python’s default port ( 8000 ) and sending specific gets straight to json files

module.exports = {
  publicPath: '/prod',
  devServer: {
    proxy: {
      '^/prod/api': {
        target: 'http://localhost:8000/',
        ws: true,
        changeOrigin: true,
        pathRewrite: {
          '^/prod/api/tasks.*': () => '/api/tasks/foo.json', // rewrite path
          '^/prod/api/task.*': () => '/api/task/tid/foo.json', // rewrite path
        }
      },
    }
  },
}

package.json - standard except for the serveapi, launching a python http.server

{
  "name": "ui",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "serveapi": "cd test; python -m http.server",
    "serve": "vue-cli-service serve",
    "build": "vue-cli-service build --dest ../handlers/ui/dist"
  },
  "dependencies": {
    "@aws-cdk/aws-dynamodb": "^1.109.0",
    "axios": "^0.21.1",
    "vue": "^3.0.10"
  },
  "devDependencies": {
    "@vue/cli-service": "^4.5.12",
    "@vue/compiler-sfc": "^3.0.10",
    "postcss": "^8.3.5"
  },
  "browserslist": [
    "> 1%",
    "last 2 versions",
    "not dead"
  ]
}

Capture.vue - dumb capture a task component

<template>
  <section class="" >
      <p>Task Details</p>
      <input style="active" class="" placeholder="task details" v-model="details" type="text"/>
      <button @click="$emit('newTask', details)">Submit New Task</button>
  </section>
</template>

<script>

export default {
  data() {
    return {
        details: ''
    }
  },
  methods: {
  },
}

</script>

<style scoped>
section {
  display: flex;
  flex-wrap: wrap;
  background-color: lightskyblue;
  box-shadow: 5px 3px 5px gray ;
  padding: 10px;
  border-radius: 10px 10px;
  margin-bottom: 10px;
}
p {
  margin: 10px 10px 10px 10px;
  padding: 10px;
}

input {
  text-align: center;
  vertical-align: middle;
  padding: 10px;
  margin: 10px;
}
</style>

TopBar.vue - to login with and to give a little colored user feedback

<template>
  <section class="" role="navigation">
    <div class="box" :class="status"/>
    <template v-if="!submitted" >
        <input style="active" class="" placeholder="user name without metmom\" v-model="login.uid" type="text"/>
        <input class="" placeholder="password" v-model="login.pwd" type="password"/>
        <a class="" @click="$emit('login',login);submitted=true">
          Log in
        </a>
    </template>
        <a class="" v-if="submitted" @click="$emit('logout');submitted=false">
          Log out
        </a>
  </section>
</template>

<script>

export default {
  props: [ 'status' ],
  data() {
    return {
      login: { 
        uid: '',
        pwd: '',
      },
      submitted: false,
    }
  },
  methods: {
  },
}

</script>

<style scoped>
.box {
  width:30px;
  height:30px;
  user-select: none;
}
section {
  display: flex;
  flex-wrap: wrap;
  width: 100%;
  background-color: cyan;
}
section * {
  margin: 10px 10px 10px 10px;
  border-radius: 5px;
  border: solid 1px gray;
  box-shadow: 5px 3px 5px darkcyan ;
  padding: 10px;
}

a {
  background-color: lightgray;
  width: 100px;
  text-align: center;
  vertical-align: middle;
  padding: 10px;
  user-select: none;
}

a:hover {
  background-color: gray;
}

.error {
  background-color: red;
}
.ok {
  background-color: lightgreen;
}
.busy {
  background-color: yellow;
}
</style>

and finally App.vue - as a all in one little controller

<template>
  <div>
    <div class="top-bar">
      <top-bar :status="status" @login="setup($event)" @logout="unsetup()"/>
    </div>
    <div v-if="loggedIn" style="width:80%;margin-top:10px;margin-left:10%;">
      <capture @newTask="newTask($event)"/>
      <button @click="fetchupdate()">reload</button>
      <div class="table" v-for="t in tasks" :key="t.id.S">
        <p>{{t.id.S.slice(0,10)}}</p>
        <p>{{JSON.parse(t.task.S)}}</p>
        <input v-model="t.taskData.detail"/>
        <button @click="saveTask(t.id.S,t)">save</button>
        <button @click="completeTask(t.id.S)">complete</button>
      </div>
    </div>
  </div>
</template>

<script>
import axios from 'axios'
import TopBar from './components/TopBar.vue'
import Capture from './components/Capture.vue'

export default {
  name: 'App',
  components: { TopBar, Capture },
  data() {
    return {
      tasks: [],
      loggedIn: null,
      tid: null
    }
  },
  methods: {
    setup(event) {
      axios.defaults.headers.common['Authorization'] = `Basic ${btoa('metmom\\'+event.uid+':'+event.pwd)}`
      axios.defaults.headers.post['Content-Type'] = 'application/json';
      this.loggedIn=event.uid; 
      this.tasks=[];
      this.fetchupdate()
    },
    unsetup() {
      this.loggedIn=false; 
      this.tasks=[];
      clearTimeout(this.tid)
    },
    async fetchupdate() {
      this.status='busy'
      try {
        if (!this.currentTask) {
          console.log('getting full list')
          let res = await axios.get('api/tasks')
          this.tasks = res.data
          this.tasks.forEach(x=>x.taskData=JSON.parse(x.task.S))
          console.log(this.tasks)
        }
        this.status='ok'
      } catch (error) {
        this.status='error'
        alert(error)
      }
    },
    async completeTask(tid) {
      this.status='busy'
      try {
        await axios.post(`api/task/${tid}/complete`)
        this.status='ok'
        this.fetchupdate()
      } catch (error) {
        this.status='error'
        alert(error)
      }
    },
    async newTask(details) {
      this.status='busy'
      try {
        await axios.post(`api/tasks`, { 
          detail: details,
          assignedUser: null,
        })
        this.status='ok'
        this.fetchupdate()
      } catch (error) {
        this.status='error'
        alert(error)
      }
    },
    async saveTask(tid,task) {
      this.status='busy'
      try {
        task.taskData.assignedUser=this.loggedIn
        await axios.post(`api/task/${tid}`, task.taskData)
        this.status='ok'
        this.fetchupdate()
      } catch (error) {
        this.status='error'
        alert(error)
      }
    },
  }
}
</script>

<style>
.table {
  display: grid;
  grid-template-columns: repeat(5, 1fr);
  box-shadow: 5px 3px 5px gray ;
  padding: 10px;
  border-radius: 10px 10px;
}
button {
  background-color: lightgray;
  text-align: center;
  vertical-align: middle;
  padding: 10px;
  user-select: none;
  border-radius: 5px 5px;
}
body {
  font-family: Arial;
  padding: 0;
  margin: 0;

}
</style>

Seed E-Kay

the app

const cdk = require('@aws-cdk/core')
const { HumanStack } = require('../lib/humantask-stack')

const app = new cdk.App();
new HumanStack(app, 'HumanStack', {
  vpc: {
      name: '<yovpc>',
      cidr: '<yocidr>'
  },
  ldapuri: 'ldaps://<yo dir>',
  uiUsers: '<yo user filter list>',
  env: {
    account: '<yo aws account id>',
    region: 'af-south-1'
  },
  vpcEndpoint: 'vpce-<yo api gateway endpoint id>',
});

the stack

const cdk = require('@aws-cdk/core')
const lambda = require('@aws-cdk/aws-lambda')
const apigateway = require('@aws-cdk/aws-apigateway')
const iam = require('@aws-cdk/aws-iam')
const ec2 = require('@aws-cdk/aws-ec2')
const sfn = require('@aws-cdk/aws-stepfunctions')
const tasks = require('@aws-cdk/aws-stepfunctions-tasks')
const dynamodb = require('@aws-cdk/aws-dynamodb');

class HumanStack extends cdk.Stack {

  constructor(scope, id, props) {
    super(scope, id, props);

    /*
                        _     __ _
    __      _____  _ __| | __/ _| | _____      __
    \ \ /\ / / _ \| '__| |/ / |_| |/ _ \ \ /\ / /
     \ V  V / (_) | |  |   <|  _| | (_) \ V  V /
      \_/\_/ \___/|_|  |_|\_\_| |_|\___/ \_/\_/
    */

    const taskTable = new dynamodb.Table(this, 'taskTable', {
      partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING }
    });

    const humanFunc = new lambda.Function(this, 'humanFunc', {
      runtime: lambda.Runtime.PYTHON_3_8,
      handler: 'task.handler',
      code: lambda.Code.fromAsset('./handlers/humantask'),
      timeout: cdk.Duration.seconds(10),
      environment: {
          'DYNTAB': taskTable.tableName,
      },
      memorySize: 512,
    })
    taskTable.grantWriteData(humanFunc)

    const humanTask = new tasks.LambdaInvoke(this, 'humanTask', {
      lambdaFunction: humanFunc,
      integrationPattern: sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN,
      payload: sfn.TaskInput.fromObject({
        token: sfn.JsonPath.taskToken,
        input: sfn.JsonPath.stringAt('$'),
      }),
      resultPath: '$.taskResult',
    })

    const somethingFunc = new lambda.Function(this, 'somethingFunc', {
      runtime: lambda.Runtime.PYTHON_3_8,
      handler: 'something.handler',
      code: lambda.Code.fromAsset('./handlers/something'),
      timeout: cdk.Duration.seconds(10),
      environment: {
      },
      memorySize: 512,
    })

    const somethingTask = new tasks.LambdaInvoke(this, 'somethingTask', {
      lambdaFunction: somethingFunc,
      resultPath: '$.result',
    })

    const sfndef = humanTask
      .next(somethingTask)

    const leSfn = new sfn.StateMachine(this, 'HumanTaskStateMachine', {
      definition: sfndef,
      timeout: cdk.Duration.hours(12),
    })

    /*           _
      __ _ _ __ (_)
     / _` | '_ \| |
    | (_| | |_) | |
     \__,_| .__/|_|
          |_|
    */

    const vpc_name = props.vpc.name
    const levpc = ec2.Vpc.fromVpcAttributes(this, vpc_name, {
      vpcId: cdk.Fn.importValue(vpc_name+'VPC'),
      vpcCidrBlock: props.vpc.cidr,
      privateSubnetIds:[
        cdk.Fn.importValue(vpc_name+'PrivateSNaSubnet'),
        cdk.Fn.importValue(vpc_name+'PrivateSNbSubnet'),
        cdk.Fn.importValue(vpc_name+'PrivateSNcSubnet'),
      ],
      availabilityZones:[
        "af-south-1a",
        "af-south-1b",
        "af-south-1c",
      ]}
    )

    const authFunc = new lambda.Function(this, 'authFunction', {
      runtime: lambda.Runtime.PYTHON_3_8,
      handler: 'auth.handler',
      code: lambda.Code.fromAsset('./handlers/auth'),
      timeout: cdk.Duration.seconds(10),
      vpc: levpc,
      environment: {
          'LDAPURI': props.ldapuri,
          'USERS': props.uiUsers
      },
      memorySize: 1024,
    })

    const apiFunc = new lambda.Function(this, 'apiFunction', {
      runtime: lambda.Runtime.PYTHON_3_8,
      handler: 'api.handler',
      code: lambda.Code.fromAsset('./handlers/api'),
      timeout: cdk.Duration.seconds(10),
      vpc: levpc,
      environment: {
          'SFNARN': leSfn.stateMachineArn,
          'DYNTAB': taskTable.tableName
      },
      memorySize: 1024,
    })

    taskTable.grantReadWriteData(apiFunc)
    leSfn.grantTaskResponse(apiFunc)
    leSfn.grantStartExecution(apiFunc)

    const uiFunc = new lambda.Function(this, 'uiFunction', {
      runtime: lambda.Runtime.PYTHON_3_8,
      handler: 'ui.handler',
      code: lambda.Code.fromAsset('./handlers/ui'),
      timeout: cdk.Duration.seconds(10),
      environment: {
      },
      memorySize: 1024,
    })

    const apigwSG = new ec2.SecurityGroup(this, 'apigwSG', {
        vpc: levpc,
        allowAllOutbound: true,
    })
    apigwSG.addIngressRule(ec2.Peer.ipv4('10.0.0.0/8'), ec2.Port.tcp(443), 'https from on-prem');
    const iEndPoint = ec2.InterfaceVpcEndpoint.fromInterfaceVpcEndpointAttributes(this, 'apigwVpcE', {
      port: 443,
      vpcEndpointId: props.vpcEndpoint,
    })
    const polStatement = new iam.PolicyStatement({
        principals: [new iam.AnyPrincipal()],
        actions: [
            "execute-api:Invoke"
        ],
        resources: ['*'],
    })
    const polDoc = new iam.PolicyDocument({
        statements: [ polStatement ]
    })

    const api = new apigateway.RestApi(this, 'humanTaskApi', {
        defaultIntegration: new apigateway.LambdaIntegration(uiFunc, {
            proxy: true,
        }),
        endpointConfiguration: {
            types: [ apigateway.EndpointType.PRIVATE ],
            vpcEndpoints: [iEndPoint]
        },
        policy: polDoc,
        securityGroups: [apigwSG],
        deploy: true,
        binaryMediaTypes: ['image/png','image/vnd.microsoft.icon','application/octet-stream'],
        minimumCompressionSize: 100,
    });
    const leUi = api.root.addResource('{proxy+}')

    leUi.addMethod('any', new apigateway.LambdaIntegration(uiFunc, {
        proxy: true,
    }), {
        authorizationType: apigateway.AuthorizationType.NONE,
    })

    const auth = new apigateway.TokenAuthorizer(this,  'apiAuth', {
        handler: authFunc
    })
    const leApi = api.root.addResource('api')
    const leApiProxy = leApi.addResource('{proxy+}')
    leApiProxy.addMethod('any', new apigateway.LambdaIntegration(apiFunc, {
        proxy: true,
    }), {
        authorizer: auth
    })

  }

}

module.exports = { HumanStack }

and package.json it rode in on

{
  "name": "humantask",
  "version": "0.1.0",
  "bin": {
    "humantask": "bin/humantask.js"
  },
  "scripts": {
    "install": "cd ui; npm i; cd ../handlers/auth; pip install -r requirements.txt -t lib; cd ../housekeeping; pip install -r requirements.txt -t lib",
    "build": "cd ui; npm run build;",
    "test": "jest",
    "cdk": "cdk"
  },
  "devDependencies": {
    "aws-cdk": "1.109.0",
    "@aws-cdk/assert": "1.109.0",
    "jest": "^26.4.2"
  },
  "dependencies": {
    "@aws-cdk/aws-apigateway": "^1.109.0",
    "@aws-cdk/aws-dynamodb": "^1.109.0",
    "@aws-cdk/aws-ec2": "^1.109.0",
    "@aws-cdk/aws-events": "^1.109.0",
    "@aws-cdk/aws-events-targets": "^1.109.0",
    "@aws-cdk/aws-iam": "^1.109.0",
    "@aws-cdk/aws-lambda": "^1.109.0",
    "@aws-cdk/aws-secretsmanager": "^1.109.0",
    "@aws-cdk/aws-stepfunctions": "^1.109.0",
    "@aws-cdk/aws-stepfunctions-tasks": "^1.109.0",
    "@aws-cdk/core": "1.109.0",
    "source-map-support": "^0.5.16"
  }
}

Afterwards

If you need to point your on-prem bits to AWS, add a nice dns name, point it to nginx, send it to the internal VPC endpoints and remap the host header to the internal api gateway name.
The private REST api gateway aint really supposed to be abused into behaving like an apache server, so it doesn’t serve / very well. You have to launch the ui with a prod/index.html to get the party started.
Get some sleep ffs