ssdt - Survey security.txt files

Overview

ssdt - Survey security.txt files

A program to quickly survey security.txt files found on the Alexa Top 1 Million websites. The program takes about 15 hours to run over a 1.5Mbit residential DSL connection. It could go much faster over high-speed links.

$ ps -p 165199 -o etime
  ELAPSED
  15:06:42

Sample output

{"website" ["contacts"] "expires"}
{"github.com" ["https://hackerone.com/github"] ""}
{"google.com" ["https://g.co/vulnz" "mailto:[email protected]"] ""}
{"facebook.com" ["https://www.facebook.com/whitehat/report/"] ""}
{"linkedin.com" ["mailto:[email protected]" "https://www.linkedin.com/help/linkedin/answer/62924"] ""}
{"cloudflare.com" ["https://hackerone.com/cloudflare" "mailto:[email protected]" "https://www.cloudflare.com/abuse/"] "sat, 20 mar 2021 13:24:05 -0700"}

Build the program

$ make

Run the program

$ ./ssdt -hosts top-1m-alexa.csv 2> err.txt > out.txt

Remove invalid security.txt entries

$ grep -v "\[\]" out.txt

Count results

$ grep -v "\[\]" out.txt | wc -l

Notes

  • You may need to adjust the nofile limit in /etc/security/limits.conf before running ssdt. Otherwise, you may exceed the open file limit.
  • Read my blog post about why I wrote this program.
Issues
  • Heads up

    Heads up

    there is a bunch of tumblr duplicates (80+) in the out file :)

    great project!

    opened by Impostor-syndrome 0
  • bleeding edge canary file

    bleeding edge canary file

    might make sense to leverage some aspect of signing from this Canary standard.

    https://github.com/canarytail/

    opened by egberts 0
Bitwise AND on two byte-slices using SIMD instructions

This package provides a vectorised function which performs bitwise AND operation on all pairs of elements in two byte-slices. It detects CPU instruction set and chooses the available best one (AVX512, AVX2, SSE2).

Wei Shen 4 Jul 20, 2021
A Golang tool to whitelist ASN's based on organization name

A Golang tool to whitelist ASN's based on organization name. This works by providing a list of ASN org names. This tool uses goPacket to monitor incoming traffic, capturing the IP's and checking the IP to see if it is a part of a whitelisted ASN. If it is not, it blocks that connection and future connections using iptables.

JP 10 Jul 14, 2021
a tool for creating exploited media files for discord

Discord-Exploits A program for creating exploited media files for discord written in Go. Usage discord-exploits is a command line utility, meaning you

schmenn 194 Jul 22, 2021
A tool to check problems about meta files of Unity

A tool to check problems about meta files of Unity on Git repositories, and also the tool can do limited autofix for meta files of auto-generated files.

DeNA 38 Jul 14, 2021
Discover internet-wide misconfigurations while drinking coffee

netz ?? ?? The purpose of this project is to discover an internet-wide misconfiguration of network components like web-servers/databases/cache-service

null 259 Jun 24, 2021
A tool to filter URLs by parameter count or size

GoFilter A tool to filter URLs by parameter count or size. This tool requires unique sorted URL list. For example: cat hosts.txt | sort -u > sorted &&

Ayberk ESER 6 Jun 25, 2021
Versatile Go code generator.

Generis Versatile Go code generator. Description Generis is a lightweight code preprocessor adding the following features to the Go language : Generic

SenseLogic 27 Jun 13, 2021
gProfiler combines multiple sampling profilers to produce unified visualization of what your CPU

gProfiler combines multiple sampling profilers to produce unified visualization of what your CPU is spending time on, displaying stack traces of your processes across native programs1 (includes Golang), Java and Python runtimes, and kernel routines.

Granulate 332 Jul 21, 2021
Maintain a lower-bitrate copy of a music library in sync with the main copy.

msync Maintain a lower-bitrate copy of your music library, in sync with the main copy.

Chris Dzombak 16 Feb 1, 2021
A tool and library for using structural regular expressions.

Structural Regular Expressions sregx is a package and tool for using structural regular expressions as described by Rob Pike (link).

Zachary Yedidia 22 Jun 18, 2021
OCI Registry As Storage

OCI Registry As Storage ?? This project is currently under active development. The API may and will change incompatibly from one commit to another. ??

OCI Registry As Storage (ORAS) 13 Jul 22, 2021
Customisable and automated HTTP header injection

headi Customisable and automated HTTP header injection. Example run from the HTB machine Control: InsecureSkipVerify is not currently configured, if y

mlcsec 101 Jul 8, 2021
A Dero service to sell Eth for Dero

ETH Seller - a Dero Service This is a Dero service for the Stargate R2 testnet, written for the dARCH 2021 Event 0.5 competition

null 3 May 4, 2021
Listing git repository from URL/User/Org

This tool is available when the repository, such as github, is included in the bugbounty scope. Sometimes specified as an org name or user name rather than a specific repository, you can use this tool to extract url from all public repositories included in the org/user.

HAHWUL 33 Jul 22, 2021