Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws-lite #553

Merged
merged 25 commits into from
Feb 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
cf5b8b5
Update deps
ryanblock Oct 10, 2023
eca838a
`@aws-sdk/client-ssm` / `aws-sdk/clients/ssm` → `@aws-lite/ssm`
ryanblock Oct 11, 2023
3a72eba
Temporarily revert `tap-arc`
ryanblock Oct 11, 2023
eba7bc3
`@aws-sdk/[client|lib]-dynamodb` / `aws-sdk/clients/dynamodb` → `@aws…
ryanblock Oct 12, 2023
f779d7c
Eh, just use `.then()` + `.catch()` instead of `callbackify`
ryanblock Oct 12, 2023
15115f5
Expose `@aws-lite/dynamodb` client as `arc.tables()._client`
ryanblock Oct 13, 2023
48db6a2
Ensure arc.tables also works in AWS :sweating:
ryanblock Oct 13, 2023
59dbfd9
Rename legacy AWS SDK interface
ryanblock Oct 13, 2023
3ca8918
`@aws-sdk/client-sns` / `aws-sdk/clients/sns` → `@aws-lite/sns`
ryanblock Oct 16, 2023
d1516a4
Tidy up tables factory a wee bit
ryanblock Oct 16, 2023
9a1ad47
Hand off region selection to `getAwsClient()`
ryanblock Oct 18, 2023
daeac37
`@aws-sdk/client-apigatewaymanagementapi` / `aws-sdk/clients/apigatew…
ryanblock Oct 18, 2023
48c681e
Farewell, old friend!
ryanblock Oct 18, 2023
c614a4b
8.0.0-RC.0
ryanblock Oct 18, 2023
6b711e1
Update readme
ryanblock Dec 11, 2023
82b9d0d
Update deps
ryanblock Dec 12, 2023
d06c36e
Update deps
ryanblock Jan 3, 2024
03d9d4c
8.0.0-RC.1
ryanblock Jan 3, 2024
0455baa
Test matrix update: add Node.js 20.x / remove 14.x
ryanblock Jan 8, 2024
4bff97b
Update CI
ryanblock Jan 10, 2024
d4687c4
Move calls to new `aws-lite` `endpoint` param
ryanblock Jan 29, 2024
c4f5595
8.0.0-RC.3
ryanblock Jan 29, 2024
653db73
Update deps
ryanblock Jan 31, 2024
1929c95
Update types imported from aws-sdk (#555)
tbeseda Feb 2, 2024
792c26b
Update deps
ryanblock Feb 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ coverage/
dist.js
scratch/
src/http/get-index
types/
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
node-version: [ 14.x, 16.x, 18.x ]
node-version: [ 16.x, 18.x, 20.x ]
os: [ windows-latest, ubuntu-latest, macOS-latest ]

# Go
Expand Down Expand Up @@ -53,7 +53,7 @@ jobs:
- name: Notify
uses: sarisia/actions-status-discord@v1
# Only fire alert once
if: github.ref == 'refs/heads/main' && failure() && matrix.node-version == '14.x' && matrix.os == 'ubuntu-latest'
if: github.ref == 'refs/heads/main' && failure() && matrix.node-version == '20.x' && matrix.os == 'ubuntu-latest'
with:
webhook: ${{ secrets.DISCORD_WEBHOOK }}
title: "build and test"
Expand Down
26 changes: 26 additions & 0 deletions _changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,32 @@ Also see: [Architect changelog](https://github.com/architect/architect/blob/main

---

## [8.0.0] 2023-10-17

Architect Functions just got a lot faster. Gone are the days of 500-1000ms cold starts due to instantiating the AWS SDK – Functions v8 is now between 2-5x faster, and uses 2-4x less memory, courtesy of [aws-lite](https://aws-lite.org)!


### Added

- `arc.tables()` now includes a new DynamoDB client: `_client`, an instantiation of [`@aws-lite/dynamodb`](https://github.com/architect/aws-lite/tree/main/plugins/dynamodb)
- `_client` is largely functionally similar to the AWS SDK's DocumentClient, but a bit less fiddly (we think)
- `arc.tables()` methods should be functionally the same, including key error properties


### Changed

- Breaking change: AWS SDK v2 + v3 DynamoDB client + DocumentClient instantiation is now opt-in
- Code depending on `data._db` or `data._doc` must now instantiate with the `awsSdkClient` boolean option, like so: `await arc.tables({ awsSdkClient: true })`
- If you only rely on the DocumentClient (`_doc`), you may want to just try using the new [`@aws-lite/dynamodb`](https://aws-lite.org/services/dynamodb)-based `_client`
- Breaking change: while we've taken efforts to ensure the maximum degree of compatibility with AWS SDK v2 and v3 errors, the errors returned in Arc Functions 8.0 (using `aws-lite`) may still vary slightly
- This only really applies if your error handling relies on specific properties or values
- If you just `console.log()` your errors, you will be totally fine, and the quality of the errors you get via `aws-lite` will most likely improve with this change
- Note: if you're an AWS SDK v2 user considering migrating to v3, error incompatibility will apply even more so; v3 errors are incompatible with v2, whereas `aws-lite` errors attempt to be compatible with both SDK v2 + v3
- Added Node.js 20.x to test matrix
- Breaking change: removed support for Node.js 14.x (now EOL, and no longer available to created in AWS Lambda)

---

## [7.0.0] 2023-07-10

### Added
Expand Down
40 changes: 21 additions & 19 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@architect/functions",
"version": "7.0.0",
"version": "8.0.0-RC.3",
"description": "Runtime utility library for Functional Web Apps (FWAs) built with Architect (https://arc.codes)",
"homepage": "https://github.com/architect/functions",
"repository": {
Expand All @@ -15,7 +15,7 @@
"test:unit": "cross-env tape 'test/unit/**/*-test.js' | tap-arc",
"test:integration": "cross-env tape 'test/integration/**/*-test.js' | tap-arc",
"coverage": "nyc --reporter=lcov --reporter=text npm run test:unit",
"test": "npm run lint && npm run test:integration && npm run coverage",
"test": "npm run lint && npm run test:integration && npm run coverage && npm run test:types",
"test:types": "tsd --files types/*.test-d.ts",
"rc": "npm version prerelease --preid RC"
},
Expand All @@ -25,7 +25,13 @@
"author": "Brian LeRoux <[email protected]>",
"license": "Apache-2.0",
"dependencies": {
"cookie": "^0.5.0",
"@aws-lite/apigatewaymanagementapi": "^0.0.7",
"@aws-lite/client": "^0.16.0",
"@aws-lite/dynamodb": "^0.3.3",
"@aws-lite/sns": "^0.0.4",
"@aws-lite/sqs": "^0.2.0",
"@aws-lite/ssm": "^0.2.2",
"cookie": "^0.6.0",
"cookie-signature": "^1.2.1",
"csrf": "^3.1.0",
"node-webtokens": "^1.0.4",
Expand All @@ -34,29 +40,25 @@
"uid-safe": "^2.1.5"
},
"devDependencies": {
"@architect/asap": "^6.0.3",
"@architect/asap": "^7.0.3",
"@architect/eslint-config": "2.1.1",
"@architect/req-res-fixtures": "git+https://github.com/architect/req-res-fixtures.git",
"@architect/sandbox": "^5.7.4",
"@aws-sdk/client-apigatewaymanagementapi": "^3.319.0",
"@aws-sdk/client-dynamodb": "^3.319.0",
"@aws-sdk/client-sns": "^3.319.0",
"@aws-sdk/client-sqs": "^3.319.0",
"@aws-sdk/client-ssm": "^3.319.0",
"@aws-sdk/lib-dynamodb": "^3.319.0",
"@aws-sdk/node-http-handler": "^3.347.0",
"@types/aws-lambda": "^8.10.119",
"@architect/sandbox": "^6.0.0",
"@aws-lite/apigatewaymanagementapi-types": "^0.0.9",
"@aws-lite/dynamodb-types": "^0.3.4",
"@aws-lite/sns-types": "^0.0.5",
"@aws-lite/sqs-types": "^0.2.1",
"@types/aws-lambda": "^8.10.133",
"@types/node": "18",
"aws-sdk": "^2.1364.0",
"cross-env": "~7.0.3",
"eslint": "^8.47.0",
"eslint": "^8.56.0",
"nyc": "~15.1.0",
"proxyquire": "~2.1.3",
"sinon": "^15.2.0",
"tap-arc": "~1.0.0",
"tape": "^5.6.6",
"sinon": "^17.0.1",
"tap-arc": "^1.2.2",
"tape": "^5.7.4",
"tiny-json-http": "^7.5.1",
"tsd": "^0.28.1"
"tsd": "^0.30.4"
},
"files": [
"types/*",
Expand Down
109 changes: 42 additions & 67 deletions src/discovery/index.js
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
let { isNode18, useAWS } = require('../lib')
let ssm, ssmClient
let { getAwsClient, useAWS } = require('../lib')

/**
* @param {string} type - events, queues, or tables
* @returns {object} {name: value}
*/
module.exports = function lookup (callback) {

let { ARC_APP_NAME: app, ARC_ENV: env, ARC_SANDBOX, ARC_STACK_NAME: stack, AWS_REGION } = process.env
let { ARC_APP_NAME: app, ARC_ENV: env, ARC_SANDBOX, ARC_STACK_NAME: stack } = process.env

let local = !useAWS()

Expand All @@ -19,11 +18,8 @@ module.exports = function lookup (callback) {
app = 'arc-app'
}

let Path = `/${stack || toLogicalID(`${app}-${env}`)}`
let Recursive = true
let values = []
let config

let plugins = [ '@aws-lite/ssm' ]
let config = { plugins }
if (local) {
let port = 2222
if (ARC_SANDBOX) {
Expand All @@ -35,70 +31,49 @@ module.exports = function lookup (callback) {
}
config = {
endpoint: `http://localhost:${port}/_arc/ssm`,
region: AWS_REGION || 'us-west-2',
plugins,
}
}

// shim v2 and v3
if (!ssmClient) {
if (isNode18) {
let { SSMClient: SSM, GetParametersByPathCommand: cmd } = require('@aws-sdk/client-ssm')
let GetParametersByPathCommand = cmd
ssm = new SSM(config)
ssmClient = (params, callback) => {
let command = new GetParametersByPathCommand(params)
return ssm.send(command, callback)
}
}
getAwsClient(config, (err, client) => {
if (err) callback(err)
else {
let SSM = require('aws-sdk/clients/ssm')
ssm = new SSM(config)
ssmClient = (params, callback) => {
return ssm.getParametersByPath(params, callback)
}
}
}

function getParams (params) {
ssmClient(params, function done (err, result) {
if (err && local &&
err.message.includes('Inaccessible host') &&
err.message.includes('localhost')) {
let msg = 'Sandbox internal services are unavailable, please ensure Sandbox is running'
callback(ReferenceError(msg))
}
else if (err) {
callback(err)
}
else if (result.NextToken) {
values = values.concat(result.Parameters)
getParams({ Path, Recursive, NextToken: result.NextToken })
}
else {
values = values.concat(result.Parameters)
let services = values.reduce((a, b) => {
let hierarchy = b.Name.split('/')
hierarchy.shift() // leading slash
hierarchy.shift() // stack name
let type = hierarchy.shift() // i.e. tables, events, queues, plugins
if (!a[type]) a[type] = {}
let parent = a[type]
let child, lastChild, lastParent
/* eslint-disable-next-line */
while (child = hierarchy.shift()) {
if (!parent[child]) parent[child] = {}
lastParent = parent
parent = parent[child]
lastChild = child
let Path = `/${stack || toLogicalID(`${app}-${env}`)}`
client.ssm.GetParametersByPath({ Path, Recursive: true, paginate: true })
.then(result => {
let services = result.Parameters.reduce((a, b) => {
let hierarchy = b.Name.split('/')
hierarchy.shift() // leading slash
hierarchy.shift() // stack name
let type = hierarchy.shift() // i.e. tables, events, queues, plugins
if (!a[type]) a[type] = {}
let parent = a[type]
let child, lastChild, lastParent
/* eslint-disable-next-line */
while (child = hierarchy.shift()) {
if (!parent[child]) parent[child] = {}
lastParent = parent
parent = parent[child]
lastChild = child
}
lastParent[lastChild] = b.Value
return a
}, {})
callback(null, services)
})
.catch(err => {
if (err && local &&
err.message.includes('Inaccessible host') &&
err.message.includes('localhost')) {
let msg = 'Sandbox internal services are unavailable, please ensure Sandbox is running'
callback(ReferenceError(msg))
}
lastParent[lastChild] = b.Value
return a
}, {})
callback(null, services)
}
})
}
getParams({ Path, Recursive })
else {
callback(err)
}
})
}
})
}

function toLogicalID (str) {
Expand Down
23 changes: 1 addition & 22 deletions src/events/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,5 @@ let subFactory = require('./subscribe')
module.exports = function eventsAndQueuesFactory (arc, type) {
let publish = pubFactory(arc, type)
let subscribe = subFactory(type)
return {
/**
* `arc.events|queues.publish`
* publish events and queues
*
* @param {Object} params
* @param {String} params.name - the event name (required)
* @param {Object} params.payload - a json event payload (required)
* @param {Function} callback - a node style errback (optional)
* @returns {Promise} - returned if no callback is supplied
*/
publish,

/**
* `arc.events|queues.subscribe`
* listen for events and queues
*
* @param {Function} handler - a single event handler function
* @returns {Lambda} - a Lambda function sig
*/
subscribe
}
return { publish, subscribe }
}
Loading
Loading