Recipes for observability solutions at AWS

Overview

AWS o11y recipes

See aws-observability.github.io/aws-o11y-recipes/.

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.

Writing recipes

To write recipes we're using MkDocs with the Material theme. MkDocs is a static site generator, converting the Markdown files you edit to static HTML pages which are then served as via GitHub pages

Local preview

In order to locally preview the recipes site you need to have MkDocs installed, for example on macOS you can use brew install mkdocs. Further, we depend on the Material theme and some p[lugins you can install as follows:

pip install mkdocs-material mkdocs-awesome-pages-plugin mkdocs-macros-plugin

To generate a local preview do:

$ mkdocs serve
INFO     -  Building documentation...
INFO     -  Cleaning site directory
...

Now head over to http://127.0.0.1:8000/aws-o11y-recipes/ where you should find the local preview of the recipes site.

If you are looking for formatting tips, check out the Material theme reference.

Note the following when writing a recipe:

  • The recipes are located in the docs/recipes/.
  • The name of the recipe follows in general the dimensions, for example, ec2-eks-metrics-go-adot-ampamg.md indicates a recipe for an EKS on EC2 setup, showing how to ingest metrics from a Go app into AMP and consume in AMG.
  • If you have any supporting files such as YAML manifests or scripts, create a directory with the same name as the recipe Markdown file, so in above case you would find a directory docs/recipes/ec2-eks-metrics-go-adot-ampamg/ that contains YAML files used in the recipes.
  • Put all your images, be it a screen shot or the like, into the docs/images directory.
  • When you create a new recipe, don't forget to link it from one of the top-level pages found in the docs/ directory. For example, above recipe you could add to amg.md, amp.md, and eks.md.

IMPORTANT Before you send in a PR, make sure that the local preview with mkdocs serve renders OK, that is, all images are shown and the rest of the formatting, such as code, displays as you would expect.

Publishing

Once you PR the repo, we will review and test the recipes and the merge of your PR kicks of a GitHub action that publishes your recipe automatically.

Comments
  • Alertmanagerfile

    Alertmanagerfile

    Hi Team,

    Submitting the request for the following issue :

    Recipe for Auto-scaling-amazon-ec2-using-amazon-managed-service-for-prometheus-and-alert-manager #52

    Sathish : [email protected]

    opened by arumusat 7
  • ho11y App's DockerFile is missing GOPROXY=Direct and GIT package. So Docker Build Fails.

    ho11y App's DockerFile is missing GOPROXY=Direct and GIT package. So Docker Build Fails.

    ho11y App's Dockerfile is missing the follow statements so the docker build fails for goproxy wilth following error

    go: github.com/bitly/[email protected]: git init --bare in /go/pkg/mod/cache/vcs/460f3a74af59eb3467f80d2023ccd36e4efe950fa38720c01021f858760cc232: exec: "git": executable file not found in $PATH
    The command '/bin/sh -c CGO_ENABLED=0 go build -a -ldflags '-extldflags "-static"'' returned a non-zero code: 1
    

    Please add the below after line 1 in ho11y's Dockerfile to get through this error :

     RUN go env -w GOPROXY=direct
     RUN apk add --no-cache git
    
    opened by elamaran11 3
  • Cloudwatch metrics streams missing tags dimensions

    Cloudwatch metrics streams missing tags dimensions

    As an Observability engineer, I would like to be able to filter metric results using resource tags to be able to have group elements form the same resource example: cpu utilization of all ec2 instances with the tag team:teamX

    opened by maksimize 2
  • Filter metrics by tag

    Filter metrics by tag

    Please review this recipe: "Using Amazon CloudWatch Metrics explorer to aggregate and visualize metrics filtered by resource tags" @mhausenblas

    opened by tomaszwaws 1
  • Cost and Usage Report (CURE) Dashboard for Amazon Managed Grafana

    Cost and Usage Report (CURE) Dashboard for Amazon Managed Grafana

    Issue #, if available:

    The Cost and Usage Report (CURE) Dashboard is a customizable Amazon Managed Grafana dashboard which provides a summary view as well as granular service level views for EC2, EBS Storage, Data Transfer and RDS. DevOps and IT Organizations will find the detailed data applicable to deep dive cost and fine tune cost efficiency.

    This repo folder will be used for references in the corresponding blog.

    Description of changes:

    • AMG Dashboard JSON
    • AMG Dashboard gif

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    opened by awsdabra 0
  • Bump vm2 from 3.9.9 to 3.9.11 in /sandbox/CWMetricStreamExporter/cdk

    Bump vm2 from 3.9.9 to 3.9.11 in /sandbox/CWMetricStreamExporter/cdk

    Bumps vm2 from 3.9.9 to 3.9.11.

    Release notes

    Sourced from vm2's releases.

    3.9.11

    New Features

    https://github.com/patriksimek/vm2/commit/58478a58c6f4af3c54faf4117ed5ab72d2cc8cd5: Add option require.strict to allow to load required modules in non strict mode.

    Fixes

    https://github.com/patriksimek/vm2/commit/d9a7f3cc995d3d861e1380eafb886cb3c5e2b873: Security fix.

    3.9.10

    New Features

    61d240f69cc02974be27c7582fee2defd0e6c7a8: Add uptime to process.

    Fixes

    e3e573fdc99a98a9c7db026e4c40474eb78cab4a: Security fix. 245da82dcdfa67031e065fd7c7a7348b5e21f2b8: Fix inspect showProxy.

    Changelog

    Sourced from vm2's changelog.

    v3.9.11 (2022-08-28)

    [new] Add option require.strict to allow to load required modules in non strict mode.
    [fix] Security fix.

    v3.9.10 (2022-07-05)

    [new] Add uptime to process.
    [fix] Security fix.
    [fix] Fix inspection with showProxy.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • New Blog - Monitoring Workspaces Using AMP and AMG

    New Blog - Monitoring Workspaces Using AMP and AMG

    This is PR to house scripts for our upcoming blog on "Monitoring Amazon Workspaces using Amazon Managed Service for Prometheus and Amazon Managed Grafana". Please approve this PR to help us move forward with the blog process. We have automation and cleanup scripts with a README.md file with all information.

    opened by elamaran11 0
  • Updated mkdocs installation steps using pip installer

    Updated mkdocs installation steps using pip installer

    Updated mkdocs installation steps using pip installer

    Tested on two Cloud9 environments (one with default python3.7 , another upgraded to python3.7) and local Mac laptop

    @mhausenblas

    opened by tomaszwaws 0
  • Update metrics-explorer-filter-by-tags.md

    Update metrics-explorer-filter-by-tags.md

    @mhausenblas: Updated Metrics Explorer as there were formatting differences between material and the default github theme and the way material handles html tags

    opened by tomaszwaws 0
  • Bump async from 3.2.0 to 3.2.4 in /sandbox/CWMetricStreamExporter/cdk

    Bump async from 3.2.0 to 3.2.4 in /sandbox/CWMetricStreamExporter/cdk

    Bumps async from 3.2.0 to 3.2.4.

    Changelog

    Sourced from async's changelog.

    v3.2.4

    • Fix a bug in priorityQueue where it didn't wait for the result. (#1725)
    • Fix a bug where unshiftAsync was included in priorityQueue. (#1790)

    v3.2.3

    • Fix bugs in comment parsing in autoInject. (#1767, #1780)

    v3.2.2

    • Fix potential prototype pollution exploit

    v3.2.1

    Commits
    Maintainer changes

    This version was pushed to npm by hargasinski, a new releaser for async since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 0
  • Nit updates on AppRunner recipes content

    Nit updates on AppRunner recipes content

    Description

    Nit updates on AppRunner recipes content. Fix one broken link.

    Test Detail

    [email protected] aws-o11y-recipes % mkdocs serve
    INFO     -  Building documentation...
    INFO     -  Cleaning site directory
    INFO     -  The following pages exist in the docs directory, but are not
                included in the "nav" configuration:
                  - recipes/amg-athena-plugin.md
                  - recipes/amg-automation-tf.md
                  - recipes/amg-google-auth-saml.md
                  - recipes/amg-redshift-plugin.md
                  - recipes/amp-alertmanager-terraform.md
                  - recipes/ec2-eks-metrics-go-adot-ampamg.md
                  - recipes/fargate-eks-metrics-go-adot-ampamg.md
                  - recipes/fargate-eks-xray-go-adot-amg.md
                  - recipes/lambda-cw-metrics-go-amp.md
                  - recipes/monitoring-hybridenv-amg.md
                  - recipes/servicemesh-monitoring-ampamg.md
    INFO     -  Documentation built in 0.35 seconds
    INFO     -  [12:54:06] Watching paths for changes: 'docs', 'mkdocs.yml'
    INFO     -  [12:54:06] Serving on http://127.0.0.1:8000/aws-o11y-recipes/
    INFO     -  [12:54:12] Browser connected:
                http://127.0.0.1:8000/aws-o11y-recipes/
    
    opened by yimipeng 0
  • What Go version should be used for CWMetricsStreamExporter?

    What Go version should be used for CWMetricsStreamExporter?

    When I do a go build -o main main.go on https://github.com/aws-observability/aws-o11y-recipes/blob/main/sandbox/CWMetricStreamExporter/lambda/main.go I get the following error on a Mac with go 1.18.2:

    » go build -o main main.go                                                  
    # command-line-arguments
    ./main.go:82:15: cannot use currentLabels (variable of type []*prompb.Label) as type []prompb.Label in struct literal
    ./main.go:246:15: cannot use timeseries (variable of type []*prompb.TimeSeries) as type []prompb.TimeSeries in struct literal
    -----------------------------------------------------------------------------------------------------------------------------
    
    opened by wattdave 2
  • Recipe for Auto-scaling-amazon-ec2-using-amazon-managed-service-for-prometheus-and-alert-manager

    Recipe for Auto-scaling-amazon-ec2-using-amazon-managed-service-for-prometheus-and-alert-manager

    Submitting an issue to convert https://aws.amazon.com/blogs/mt/auto-scaling-amazon-ec2-using-amazon-managed-service-for-prometheus-and-alert-manager/ in the Observability recipes

    opened by arumusat 2
  • Integrating AMP as a destination for Cloudwatch metric streams and/or kinesis firehose

    Integrating AMP as a destination for Cloudwatch metric streams and/or kinesis firehose

    opened by maksimize 1
  • EKS Observability Accelerator

    EKS Observability Accelerator

    Turn Introducing Amazon EKS Observability Accelerator into a recipe and:

    1. Make sure that https://github.com/aws-ia/terraform-aws-eks-blueprints/pull/617 is merged first
    2. Call out how to deal with Error: could not download chart: issue (this is a known TF Helm provider issue, one needs to manually helm repo remove xxx and helm repo update to fix it)
    opened by mhausenblas 0
  • Add recipe for AWS managed Grafana workspace URL to be pointing to a custom URL/Domain

    Add recipe for AWS managed Grafana workspace URL to be pointing to a custom URL/Domain

    Add recipe for AWS managed Grafana workspace URL to be pointing to a custom URL/Domain. I have been trying this with cloudfront sitting with an alternate domain name but keep getting the error corresponding relay isn't found It will be nice if someone can add that as it be useful for other people too

    opened by hassanejaz 3
  • Add “AWSAppRunner” icon in “Compute Unit” dimension row

    Add “AWSAppRunner” icon in “Compute Unit” dimension row

    Please kindly help adding “AWS App Runner” icon in “Compute Unit” dimension row: https://github.com/aws-observability/aws-o11y-recipes/blob/main/docs/images/o11y-space.png

    opened by yimipeng 0
Integrate AWS EKS Anywhere cluster with AWS Services

This article provides step-by-step instruction on integrating AWS EKS Anywhere with AWS Services so the applications running on customer data center can securely connect with these services.

Hari Ohm Prasath 2 Mar 6, 2022
Apis para la administracion de notifiaciones, utilizando servicios como AWS SNS y AWS SQS

notificacion_api Servicio para envío de notificaciónes por difusión en AWS SNS Especificaciones Técnicas Tecnologías Implementadas y Versiones Golang

Universidad Distrital Francisco José de Caldas 0 Jan 7, 2022
AWS credential_process utility to assume AWS IAM Roles with Yubikey Touch and Authenticator App TOPT MFA to provide temporary session credentials; With encrypted caching and support for automatic credential refresh.

AWS credential_process utility to assume AWS IAM Roles with Yubikey Touch and Authenticator App TOPT MFA to provide temporary session credentials; With encrypted caching and support for automatic credential refresh.

Ari Palo 18 Oct 27, 2022
A package for access aws service using AWS SDK for Golang

goaws ?? A package for access aws service using AWS SDK for Golang Advantage with goaws package Example for get user list IAM with AWS SDK for Golang

Muhammad Ichsanul Fadhil 1 Nov 25, 2021
Simple CRUD API written in Go, built using AWS SAM tool and using the AWS' infrastructure.

tutor-pet API Simple CRUD API written in Go, built using AWS SAM tool and using the AWS' infrastructure. Macro architecture: Code architecture: Pre-Re

Lucas Ferreira 3 Aug 17, 2022
Aws-parameter-bulk - Export AWS SSM Parameter Store values in bulk to .env files

aws-parameter-bulk Utility to read parameters from AWS Systems Manager (SSM) Par

Adam Malik 18 Oct 18, 2022
null 2 Feb 7, 2022
Aws-cognito-demo-go - Source code for AWS Cognito in Go

AWS Cognito Demo in Go Source code for YouTube series, AWS Cognito in Go - https

null 2 Aug 31, 2022
Una prueba técnica: Servicio Golang REST API local, sobre Docker, gRPC, AWS Serverless y sobre Kubernetes en AWS EC2

Una prueba técnica: Servicio Golang REST API local, sobre Docker, gRPC, AWS Serverless y sobre Kubernetes en AWS EC2

Emilio del Cañal Calleja 4 May 7, 2022
Aws-cdk-go-examples - Example projects using the AWS CDK by Golang

aws-cdk-go-examples Example projects using the AWS CDK by Golang Useful commands

null 0 Aug 23, 2022
Bitfield - A collection of code samples, listings, and solutions to challenges from the book For the Love of Go

For the Love of Go - code listings This is a collection of code samples, listing

Zack Proser 1 Jan 10, 2022
AWS SDK for the Go programming language.

AWS SDK for Go aws-sdk-go is the official AWS SDK for the Go programming language. Checkout our release notes for information about the latest bug fix

Amazon Web Services 8k Nov 19, 2022
Simple tool to search tagged resources between all AWS resouces

Welcome to Cloud Inventory Tags ?? Simple tool to search tagged resources around all AWS Account Installation MacOS / OSX

Matheus Fidelis 16 Jan 26, 2022
Lambda stack to turn off and destroy all resources from your personal AWS Account to avoid billing surprises

AWS, Turn off my Account, please Lambda stack to turn off and destroy all resources from your personal AWS Account to avoid billing surprises Resource

Matheus Fidelis 67 Oct 25, 2022
Assume AWS IAM roles from GitHub Actions workflows with no stored secrets

AWS IAM roles for GitHub Actions workflows Background and rationale GitHub Actions are a pretty nice solution for CI/CD. Where they fall short is inte

Glass Echidna 171 Feb 12, 2022
rpCheckup is an AWS resource policy security checkup tool that identifies public, external account access, intra-org account access, and private resources.

rpCheckup - Catch AWS resource policy backdoors like Endgame rpCheckup is an AWS resource policy security checkup tool that identifies public, externa

Gold Fig Labs Inc. 146 Nov 4, 2022
efsu is for accessing AWS EFS from your machine without a VPN

efsu: VPN-less access to AWS EFS efsu is for accessing AWS EFS from your machine without a VPN. It achieves this by deploying a Lambda function and sh

Glass Echidna 42 Mar 11, 2022
lambda-go-api-proxy makes it easy to port APIs written with Go frameworks such as Gin to AWS Lambda and Amazon API Gateway.

aws-lambda-go-api-proxy makes it easy to run Golang APIs written with frameworks such as Gin with AWS Lambda and Amazon API Gateway.

Amazon Web Services - Labs 735 Nov 23, 2022
Use AWS SQS as a clipboard to copy and paste across different systems and platforms

sqs_clipboard Use AWS SQS as a clipboard to copy and paste across different systems and platforms. Clipboard contents are encrypted in transit and at

John Taylor 22 Oct 16, 2022