Amazon Kinesis Data Analytics Integration
A fully managed service to help you process streaming data in real time using standard SQL. With Site24x7's AWS integration you can collect, visualize and alert on metrics to effectively manage your kinesis data analytics application.
Setup and configuration
- If you haven't done it already, enable access to your AWS resource by creating Site24x7 as an IAM user or by creating a cross-account IAM role between your account and Site24x7's AWS account. Learn more.
- Next, In the Integrate AWS Account page, please make sure the Kinesis Analytics checkbox is selected in the Services to be discovered field. Learn more.
Policies and Permissions
Please ensure the following read level actions are present in the policy document assigned to the Site24x7 IAM entity. Learn more.
- "kinesisanalytics:ListApplications",
- "kinesisanalytics:ListTagsForResource",
- "kinesisanalytics:DescribeApplication
Polling frequency
Site24x7 queries the CloudWatch API as per the poll frequency set (1 minute to a day), to collect performance metrics. Learn more.
IT Automations
You can add automations for the AWS services supported by Site24x7. Log in to Site24x7 and go to Admin > IT Automation Templates (+) > Add Automation Templates. Once automations are added, you can schedule them to be executed one after the other.
You can now start or stop data analytics application using Amazon Kinesis Data Analytics automations.
To view data
- Log in to the Site24x7 console, click on AWS > Monitored AWS account and select Kinesis Analytics from the drop down.
- From the list of monitored applications, choose the application for which you want to access metrics for.
- The various graphs applicable for your analytics application are displayed in the Application Details tab.
Supported metrics
The following metrics are collected:
| Attribute | Description | Data type | Statistics |
|---|---|---|---|
| Bytes | Measures the number of bytes read (per input stream) or written (per output stream). | Bytes | Sum |
| Millis behind latest | Measures how far behind from the current time an application is reading from the streaming source. | Milliseconds | Average |
| Records | Measures the number of records read (per input stream) or written (per output stream). | Count | Sum |
| Success | Gets incremented by 1 for every successfull delivery to the destination, configured for your application. | Count | Sum |
| Input processing duration | Measures the time taken for each Lambda function invocation performed by Kinesis Data Analytics. | Milliseconds | Average |
| Input Processing OkRecords | Measures the number of records returned by a Lambda function that were marked with Ok status. | Count | Sum |
| Input processing OkBytes | Measures the sum of bytes of the records returned by a Lambda function that were marked with Ok status. | Bytes | Sum |
| Input processing dropped records | Measures the time taken from when the complete fragment data is received and archived. | Milliseconds | Average |
| Input processing processing failed records | Measures the number of records returned by a Lambda function that were marked with ProcessingFailed status. | Count | Sum |
| Input processing success | Measures the number of successful Lambda invocations by Kinesis Data Analytics. | Count | Sum |
| Lambda delivery OkRecords | Measures the number of records returned by a Lambda function that were marked with Ok status. | Count | Sum |
| LambdaDelivery. delivery failedRecords | Measures the number of records returned by a Lambda function that were marked with DeliveryFailed status. | Count | Sum |
| LambdaDelivery.Duration | The time taken for each Lambda function invocation performed by Kinesis Data Analytics. | Milliseconds | Average |
| Attribute | Description |
|---|---|
| Application Name | Displays the name for the Kinesis Data Analytics application. |
| Application ARN | Shows the Amazon Resource Name for the application. |
| Application Status | Shows the current status for the application. |
| Version ID | Displays the current application version ID. |
| Creation Time | Shows the time stamp when the application was created. |
| Last Updated Time | Shows the time stamp when the application was last updated. |
| Application Code | Shows the application code that you provided to perform data analysis. |
| Application Description | Shows the description of the application. |
| Logging Option ID | Displays the ID of the CloudWatch logging option description. |