ssdt - Survey security.txt files

Overview

ssdt - Survey security.txt files

A program to quickly survey security.txt files found on the Alexa Top 1 Million websites. The program takes about 15 hours to run over a 1.5Mbit residential DSL connection. It could go much faster over high-speed links.

$ ps -p 165199 -o etime
  ELAPSED
  15:06:42

Sample output

{"website" ["contacts"] "expires"}
{"github.com" ["https://hackerone.com/github"] ""}
{"google.com" ["https://g.co/vulnz" "mailto:[email protected]"] ""}
{"facebook.com" ["https://www.facebook.com/whitehat/report/"] ""}
{"linkedin.com" ["mailto:[email protected]" "https://www.linkedin.com/help/linkedin/answer/62924"] ""}
{"cloudflare.com" ["https://hackerone.com/cloudflare" "mailto:[email protected]" "https://www.cloudflare.com/abuse/"] "sat, 20 mar 2021 13:24:05 -0700"}

Build the program

$ make

Run the program

$ ./ssdt -hosts top-1m-alexa.csv 2> err.txt > out.txt

Remove invalid security.txt entries

$ grep -v "\[\]" out.txt

Count results

$ grep -v "\[\]" out.txt | wc -l

Notes

  • You may need to adjust the nofile limit in /etc/security/limits.conf before running ssdt. Otherwise, you may exceed the open file limit.
  • Read my blog post about why I wrote this program.
You might also like...
Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format

Go-Hex Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format Dump Hashes ----

🔍 A jq-like tool that queries files via glob.

🔍 fq A jq-like tool that queries files via glob. ✅ Prerequisites Go 1.17+ jq (installed and on PATH) 📦 Installation $ go get github.com/siketyan/fq

Package macho implements access to and creation of Mach-O object files.

go-macho [WIP] 🚧 Package macho implements access to and creation of Mach-O object files. Why 🤔 This package goes beyond the Go's debug/macho to: Cov

A utility library to do files/io/bytes processing/parsing in file-system or network.

goreader A utility library to do files/io/bytes processing/parsing in file-system or network. These features are really common to be implemented for a

Go-library that facilitates the usage of .env files

Goenv Golang-library that facilitates the use of .env files. Installation go get github.com/fabioelizandro/goenv Usage Place a .env file in the root

Backme - A backup files organizer

backme --- A backup files organizer Quite often big files (like database dumps)

Small tool for splitting files found in a path into multiple groups

Small tool for splitting files found in a path into multiple groups. Usefull for parallelisation of whatever can be paralleled with multiple files.

mesh-kridik is an open-source security scanner that performs various security checks on a Kubernetes cluster with istio service mesh and is leveraged by OPA (Open Policy Agent) to enforce security rules.
mesh-kridik is an open-source security scanner that performs various security checks on a Kubernetes cluster with istio service mesh and is leveraged by OPA (Open Policy Agent) to enforce security rules.

mesh-kridik Enhance your Kubernetes service mesh security !! mesh-kridik is an open-source security scanner that performs various security checks on a

This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.
This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.

This small Docker project is the easiest way to send notifications directly via .txt files to services like: Gotify, Telegram, SMTP (Email) or Webhook.

Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert
Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert

DeepAlert DeepAlert is a serverless framework for automatic response of security alert. Overview DeepAlert receives a security alert that is event of

Web-Security-Academy - Web Security Academy, developed in GO

Web-Security-Academy - Web Security Academy, developed in GO

grobotstxt is a native Go port of Google's robots.txt parser and matcher library.

grobotstxt grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port Preserve

A CoreDNS plugin to serve temporary TXT records for validation purposes (eg. Let's Encrypt DNS-01)

temptxt Name temptxt - serves TXT records for validation purposes (eg. ACME DNS-01 challenge) updated through a HTTP api. Description The temptxt plug

DNStxt-exp - 一个提供查询 TXT 记录的 DNS 服务利用工具。例如:可配合 Windows 下的 certutil 工具传输小文件(64KB)

DNStxt-exp 一个提供查询 TXT 记录的 DNS 服务利用工具。例如:可配合 Windows 下的 certutil 工具传输小文件(64KB) HE

Txt-lsp - A toy project with Language Server Protocol (LSP)

txt-lsp txt-lsp is a toy project where I play around with Language Server Protoc

Service that calls uzma24/project1 service, takes input from .txt file and prints JSON output returned from the service.

Service that calls uzma24/project1 service, takes input from .txt file and prints JSON output returned from the service. Program can take large input files.

GoLang ads.txt scraper

Collects and parses ads.txt GoLang program scrapes sites for ads.txt and stores its significant details to PostgreSQL database. Give it a file with CS

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Owner
null
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Patrick D'appollonio 197 Nov 22, 2022
A golang library to validate and format swiss social security numbers

s3n is a golang library to validate and format swiss social security numbers (aka. AVS in french and AHV in german).

Julien M'Poy 0 Nov 15, 2021
a tool for creating exploited media files for discord

Discord-Exploits A program for creating exploited media files for discord written in Go. Usage discord-exploits is a command line utility, meaning you

schmenn 219 Dec 29, 2021
A tool to check problems about meta files of Unity

A tool to check problems about meta files of Unity on Git repositories, and also the tool can do limited autofix for meta files of auto-generated files.

DeNA 63 Nov 23, 2022
libraries for various programming languages that make it easy to generate per-process trace files that can be loaded into chrome://tracing

chrometracing: chrome://tracing trace_event files The chrometracing directory contains libraries for various programming languages that make it easy t

Google 23 Oct 6, 2022
A program for extracting information from chrome session files.

What A tool for dumping the contents of chrome session files. Why? Most tools for reading SNSS files (chrome session files) are either broken or outda

null 42 Nov 23, 2022
Small utility to allow simpler, quicker testing of parsing files in crowdsec

cs_parser_test Small utility to allow simpler, quicker testing of parsing files in crowdsec Usage $ sudo cs_parser_test -t syslog /var/log/mail.log N

david reid 3 Jul 13, 2021
a sharded store to hold large IPLD graphs efficiently, packaged as location-transparent attachable CAR files, with mechanical sympathy

DAG store This README will be populated soon. In the meantime, please refer to the design document.

Filecoin 33 Oct 31, 2022
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.

LogAnalyzer Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory

Johnson Simon 6 May 31, 2022
Helm plugin to reference value files packaged in dependency charts

Helm Octopus Plugin This Helm plugin allows to reference packaged value files (other than the default values.yaml). Install helm plugin install https:

Softonic 3 Sep 23, 2021