Simple library to retry or poll your payload using SNS, SQS and Lambda
Use SNS and SQS services to hook your application for retrying your message with Lambda (recommended) or NodeJS app.
SNS service can send payload to topic (topic is decided based on configuration) which then gets delivered to queue (if subscribed). Configuration is fully customizable. User can specify which topic to send it to based on the status codes: retryStatusCodes, failureStatusCodes, successStatusCodes.
Messages from SQS can be consumed by Lambda (scheduled with interval) using SQS service. Specify no. of messages to read, queue name, and destination topic and leave the rest to service to read, process, delete, and send it to destination topic
If you are also looking for aws promise libraries? this library will nicely integrate with your code...
Example solution diagram of how you can use this library to retry or poll your message
Installing
In Node.js
The preferred way to install the AWS Serverless Retry for Node.js is to use the npm package manager for Node.js. Simply type the following into a terminal window:
npm install aws-serverless-retry
Usage and Getting Started
const ASR = ;const SNSService = ASRSNS; //Create SNSServicelet region = "us-west-2";let snsService = region; //Paramslet statusCode = 200;let payload = "data": "Test";let snsConfig = retryStatusCodes: 500 failureStatusCodes: 400 successStatusCodes: 200 201 maxRetryAttempts: 2 retryTopicName: "retry-topic" successTopicName: "success-topic" failureTopicName: "failure-topic"; //Now call sendToTopicByStatusCodesnsService ;
Logging
Set DEBUG environment variable to true to enable logging
Actions you can perform:
SNS Service:
-
createTopic(topicName)
Creates topic with the provided topic name. If topic exists it simply returns promise response with TopicArn
Response: Promise aws-sdk standard response
let ASR = ;let SNSService = ASRSNS;let snsService = "us-west-2";//Params//topicName: string value.snsService; -
sendToTopicByStatusCode(statusCode, payload, snsConfig, subject)
Publishes payload to appropriate topic (success/failure/retry) based on statusCodes provided in configuration. If passed in StatusCode is not listed in configuration it is published to failure topic by default. If topic does't exists it creates topic and then sends payload to that topic.
Response: Promise aws-sdk standard publishTopic response with additional topicName property
let ASR = ;let SNSService = ASRSNS;let snsService = "us-west-2";//Params//statusCode: integer value//payload: JSON object only//snsConfig: JSON object only//subject: Optional parameter to be used as the "Subject" line when the message is delivered to email endpoints.// Subjects must be ASCII text that begins with a letter, number, or punctuation mark; must not include line breaks// or control characters; and must be less than 100 characters longlet snsConfig =retryStatusCodes: 500 //optionalfailureStatusCodes: 400 //optionalsuccessStatusCodes: 200 201 //optionalmaxRetryAttempts: 2 //optional, defaults to 1 is not providedretryTopicName: "retry-topic" //requiredsuccessTopicName: "success-topic" //requiredfailureTopicName: "failure-topic" //required;snsService; -
sendToTopic(topicName, payload, subject, phoneNumber, attributes)
Publishes payload to specified topic. If topic doesn't exists it creates topic and then sends payload to that topic
Response: Promise aws-sdk standard publishTopic response with additional topicName property
let ASR = ;let SNSService = ASRSNS;let snsService = "us-west-2";//Params//topicName: string value//payload: JSON object only//subject: Optional parameter to be used as the "Subject" line when the message is delivered to email endpoints.// Subjects must be ASCII text that begins with a letter, number, or punctuation mark; must not include line breaks// or control characters; and must be less than 100 characters long//phoneNumber: Optional parameter. The phone number to which you want to deliver an SMS message//attributes: Optional parameter. JSON object only. Sets the message attributes of the message so that the messages can be filtered by the attribute.// example, attributes: { "subscriberEmail": "subscriber1@email.com" }. If the topic subscriber assigns filter policy for 'subscriberEmail' attribute and// the attribute matches, then the subscriber receives the message.snsService;
SQS Service:
-
getQueueUrl(queueName)
Gets queue url if exists else creates new queue with the provided queue name
Response: Promise aws-sdk standard response
let ASR = ;let SQSService = ASRSQS;let region = "us-west-2";let sqsService = region;sqsService; -
readMessage(queueUrl, maxNumberOfMessagesToRead)
Reads message from Queue. If maxNumberOfMessagesToRead (range 1 to 10) is not valid it defaults to 10.
Response: Promise aws-sdk standard response with additional QueueUrl property
let ASR = ;let SQSService = ASRSQS;let region = "us-west-2";let sqsService = region;sqsService; -
deleteMessage(queueUrl, receiptHandle)
Delete message from Queue.
Response: Promise aws-sdk standard response
let ASR = ;let SQSService = ASRSQS;let region = "us-west-2";let sqsService = region;sqsService; -
processMessage(queueName, sqsConfig)
Reads message from Queue, Sends message to destination topic, Deletes message from Queue and returns delete message response
- Any failures in processing will leave the message in queue for safety and should be manually fixed.
- Once message maxRetryAttempt is reached, it will be sent to failure topic instead of trigger topic (destination topic)
Response: Promise delete SQS message aws-sdk standard response in array.
Note:
- Messages can only be processed if it is send to queue using SNS service.
- And the message payload format is of type JSON.
let ASR = ;let SQSService = ASRSQS;let region = "us-west-2";let sqsService = region;let queueName = "queue-name"; //Required. Should be a valid queue namelet maxNumberOfMessagesToRead = 6; //Required. Can be any number between 1 to 10.let readConfigFromMessage = true; //Accepts either true/false.//If true, config values are retrieved from message body. It is assumed that asrConfig (as below) will get sent in message.//Example: message.asrConfig = {// triggerTopicName: "trigger-topic-name",// failureTopicName: "failure-topic-name",// maxRetryAttempts: 2// }//If false, config values are retrieved from sqsConfig which is passed as parameter//Example: sqsConfig : {// triggerTopicName: "trigger-topic-name",// failureTopicName: "failure-topic-name",// maxRetryAttempts: 2// }let sqsConfig =triggerTopicName: "trigger-topic-name" //requiredfailureTopicName: "failure-topic-name" //requiredmaxRetryAttempts: 2 //optional. Defaults to 1 if not provided;sqsService;