A fast and easy to use URL health checker ⛑️ Keep your links healthy during tough times

Overview

AreYouOK?

A minimal, fast & easy to use URL health checker

cat areyouok logo

Go report card License CodeQL Analysis Twitter: bhupeshimself areyouok-v1.0.0-demo

Who is AreYouOk made for ?

  • OSS Package Maintainers πŸ“¦οΈ :
    With packages comes documentation which needs to be constantly updated & checked for dead/non-functioning URLs.
  • Tech Bloggers ✍️ :
    If you are someone who writes countless tutorials & owns the source for your website, Use areyouok to make sure your blogs don't contain any non-functioning URLs.
  • Literally anyone who wants to check a bunch of URLs for being dead ☠️ or not, SEO experts?

With time AreYouOk can evolve to analyze URLs over a remote resource as well, send your ideas ✨️ through Discussions

Installation

  • Linux

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-linux-amd64
    # or
    wget -q https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-linux-amd64
  • MacOS

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-darwin-amd64
    # or
    wget -q https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-darwin-amd64
  • Windows

    curl -LJO https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-windows-amd64.exe
    # or
    wget -q https://github.com/Bhupesh-V/areyouok/releases/latest/download/areyouok-windows-amd64.exe

Check installation by running areyouok -v

$ mv areyouok-darwin-amd64 areyouok
$ areyouok -v
v1.0.0

Download builds for other architectures from releases

Usage

AreYouOk provides 3 optional arguments followed by a directory path (default: current directory)

  1. -t type of files to scan for links
  2. -i list of files or directories to ignore for links (node_modules, .git)
  3. -r type of report to generate

Some example usages:

  • Analyze all HTML files for hyperlinks. The default directory path is set to current directory.

    areyouok -t=html Documents/my-directory

    There is not limitation on file type, to analyze json files use -t=json. The default type is set to md (markdown)

  • To ignore certain directories or file-types use the ignore -i flag.

    areyouok -i=_layouts,.git,_site,README.md,build.py,USAGE.md
  • By default AreYouOk outputs analyzed data directly into console. To generate a report, use the -r flag

    areyouok -i=_layouts,.git,_site,README.md,build.py,USAGE.md -r=html ~/Documents/til/

    Currently supported report formats are: json, txt, html & github.

    Different report types provide different levels of information which is briefly summarized below:

    1. JSON (report.json)
      The JSON report can be used for other computational tasks as required (E.g emailing dead urls to yourself)

      {
         "http://127.0.0.1:8000/": {
                 "code": "",
                 "message": "Get \"http://127.0.0.1:8000/\": dial tcp 127.0.0.1:8000: connect: connection refused",
                 "response_time": ""
         },
         "http://freecodecamp.org": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "5.44s"
         },
         "http://ogp.me/": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "3.60s"
         },
         "http://prnbs.github.io/projects/regular-expression-parser/": {
                 "code": "200",
                 "message": "OK",
                 "response_time": "0.25s"
         },
         "https://bhupeshv.me/30-Seconds-of-C++/": {
                 "code": "404",
                 "message": "Not Found",
                 "response_time": "3.84s"
         },
         ...
      }
    2. Text (report.txt)
      The text format just lists the URLs which were not successfully fetched. Useful if you just want dead urls. Text report also puts the no.of hyperlinks analyzed along with total files & total reponse time.

      74 URLs were analyzed across 31 files in 21.69s
      
      Following URLs are not OK:
      
      http://freecodecamp.org`
      http://127.0.0.1:8000/
      https://drive.google.com/uc?export=view&id=`
      https://drive.google.com/file/d/
      https://drive.google.com/uc?export=view&id=$get_last
      https://github.com/codeclassroom/PlagCheck/blob/master/docs/docs.md
      https://bhupeshv.me/30-Seconds-of-C++/
      

      Note that the total time would vary according to your internet speed & website latency.

    3. HTML (report.html)
      The html report is the most superior formats of all & can be used to have a visual representaion of analyzed links.
      Below is demo of how this HTML report looks like, you can see it live report-latest

    4. GitHub (report.github)
      The github report format is well suited if you are utilizing Github Actions. The format generated is largely HTML, compatible with github's commonmark markdown renderer.
      Below is a demo of a Github Action which reports the analyzed URLs through github issues. Here is a demo link

      demo-action

Development

Prerequisites

  1. Clone the repository.
    git https://github.com/Bhupesh-V/areyouok.git
  2. Run tests.
    go test -v
  3. Format & Lint the project.
    gofmt -w areyouok.go && golint areyouok.go

Projects Using AreYouOk

πŸ“ Changelog

See the CHANGELOG.md file for details.

☺️ Show your support

Support me by giving a ⭐️ if this project helped you! or just Twitter URL

Donate using Liberapay Support on ko-fi

πŸ“ License

Copyright Β© 2020 Bhupesh Varshney.
This project is MIT licensed.

πŸ‘‹ Contributing

Please read the CONTRIBUTING file for the process of submitting pull requests to us.

Contributors ✨

Thanks goes to these wonderful people (emoji key):


MaDDogx

πŸ›

This project follows the all-contributors specification. Contributions of any kind welcome!

Comments
  • UTF Icons don't show up in HTML report

    UTF Icons don't show up in HTML report

    Describe the bug UTF icons don't show up in the generated HTML report.

    To Reproduce Steps to reproduce the behavior:

    1. Run the binary
    2. Serve the generated HTML report.
    3. No fancy icons 😒

    Expected behavior Need muh UTF icons.

    Screenshots image

    Desktop (please complete the following information):

    • OS: Windows
    • Browser: N/A
    • Version: N/A

    Smartphone (please complete the following information):

    • Device: N/A
    • OS: N/A
    • Browser: N/A
    • Version: N/A

    Additional context N/A

    bug 
    opened by Animesh-Ghosh 4
  • Modularise code

    Modularise code

    Right now the logic for areyouok contains in one single big file. It would be nice to have some structure in place. We can follow the 12 factor app pattern here

    opened by Bhupesh-V 0
  • fix go automated tests

    fix go automated tests

    Right now the go setup action is failing to install the latest unstable release 1.16

    the issue seems to be in the version label not being understood https://github.com/Bhupesh-V/areyouok/blob/c1b22b7dbb93b2b45fbbe51223ab93226615af66/.github/workflows/go.yml#L16-L20

    See https://github.com/actions/setup-go/issues/63

    help wanted good first issue tooling 
    opened by Bhupesh-V 0
  • Throws a `999` Status Error Code for LinkedIn links

    Throws a `999` Status Error Code for LinkedIn links

    Describe the bug While checking for broken links, areyouok throws a 999 Error Status code for URLs pointing towards LinkedIn.

    EDIT: Just found out, even URLs pointing towards Twitter returns a 404 Not Found Error. Here's an example issue thread for reference.

    To Reproduce Steps to reproduce the behavior:

    1. Create an example README.md file with the a link to a LinkedIn profile.
    2. Run areyouok against the newly created README.md file (which contains the LinkedIn URL).
    3. Check the generated report for a 999 Error Status Code.

    Expected behavior areyouok should've returned a 200 OK Status Code.

    Screenshots image

    Desktop (please complete the following information):

    • OS: Ubuntu 20.04
    • areyouok version: v1.1.0 built on (26 Jan 2021)
    bug 
    opened by Jarmos-san 2
  • socket: too many open files

    socket: too many open files

    Describe the bug there is a limit to how many file descriptors can be open at a time. See https://stackoverflow.com/questions/32325343/go-tcp-too-many-open-files-debug

    To Reproduce Steps to reproduce the behavior:

    1. Run Areyouok on file with greater than 200 links (For e.g awesome-go README)
    2. A lot of URLs are reported as above error

    Expected behavior All URLs should be correctly fetched

    Screenshots If applicable, add screenshots to help explain your problem.

    Desktop (please complete the following information):

    • OS: [e.g. Windows 10]
    • areyouok version: [latest]

    Additional context Use buffered channels, and add a limit for it (use ulimit -n)

    bug 
    opened by Bhupesh-V 0
  • support for line numbers in github & html report format

    support for line numbers in github & html report format

    Is your feature request related to a problem? Please describe. Right now the links appear in list of tables for the GitHub report format. It would be nice if we can link to the exact line number where the link exists.

    Describe the solution you'd like A new parser kinda thing will be required to find the line numbers of exact regexp matches. Its an interesting and fun problem to solve

    opened by Bhupesh-V 0
Owner
Bhupesh Varshney
FOSS Enthusiast β€’ Tech Writer β€’ Developer β€’ Building cool stuff every day | My idea are expressed through my software
Bhupesh Varshney
High-Performance Shortlink ( Short URL ) app creator in Golang. For privacy reasons, you may prefer to host your own short URL app and this is the one to use.

About The Project Shortlink App in Golang Multiple Node based Architecture to create and scale at ease Highly performant key-value storage system Cent

null 130 Nov 9, 2022
A Go-based program to find links from the list of Js files.

linkinjs - A Go based program to find links from list of Js files quickly Installation go get -u github.com/rc4ne/linkinjs Efficient Usage with other

Sachin Sharma 3 Sep 30, 2022
PHP security vulnerabilities checker

Local PHP Security Checker The Local PHP Security Checker is a command line tool that checks if your PHP application depends on PHP packages with know

Fabien Potencier 975 Nov 26, 2022
gosec - Golang Security Checker

Inspects source code for security problems by scanning the Go AST.

Secure Go 6.4k Nov 19, 2022
log4jshell vulnerability checker tool

Description log4j-checker tool helps identify whether a certain system is running a vulnerable version of the log4j library. Download and run the tool

null 1 Dec 20, 2021
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Optimus Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality

Open Data Platform 670 Nov 17, 2022
PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. You can use PHP like functions in your app, module etc. when you add this module to your project.

PHP Functions for Golang - phpfuncs PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. Y

Serkan Algur 52 Aug 26, 2022
An easy-to-use XChaCha20-encryption wrapper for io.ReadWriteCloser (even lossy UDP) using ECDH key exchange algorithm, ED25519 signatures and Blake3+Poly1305 checksums/message-authentication for Go (golang). Also a multiplexer.

Quick start Prepare keys (on both sides): [ -f ~/.ssh/id_ed25519 ] && [ -f ~/.ssh/id_ed25519.pub ] || ssh-keygen -t ed25519 scp ~/.ssh/id_ed25519.pub

null 25 Nov 9, 2022
Easy to use cryptographic framework for data protection: secure messaging with forward secrecy and secure data storage. Has unified APIs across 14 platforms.

Themis provides strong, usable cryptography for busy people General purpose cryptographic library for storage and messaging for iOS (Swift, Obj-C), An

Cossack Labs 1.6k Nov 21, 2022
An easy-to-use SHA-1 hash-cracker written in Golang.

wrench - An easy-to-use SHA-1 hash-cracker. Wrench is an SHA-1 hash-cracker that relies on wordlists for comparing hashes, and cracking them. Before W

null 4 Aug 30, 2021
Easy-to-use Fortnite Launcher for DLL Injection & SSL-Bypass

Easy-to-use Fortnite Launcher for DLL Injection & SSL-Bypass

Ali Hashemi 16 Nov 26, 2022
Nuclei is a fast tool for configurable targeted vulnerability scanning based on templates offering massive extensibility and ease of use.

Fast and customisable vulnerability scanner based on simple YAML based DSL. How β€’ Install β€’ For Security Engineers β€’ For Developers β€’ Documentation β€’

ProjectDiscovery 10.6k Nov 19, 2022
null 7 Nov 9, 2022
Use golang.org/x/vuln to scan your dependencies

vulnz - Use golang.org/x/vuln to scan your dependencies vulnz uses golang.org/x/vuln to scan the dependencies in your go.mod. Install go install githu

Miki Tebeka 2 Oct 10, 2022
Ah shhgit! Find secrets in your code. Secrets detection for your GitHub, GitLab and Bitbucket repositories: www.shhgit.com

shhgit helps secure forward-thinking development, operations, and security teams by finding secrets across their code before it leads to a security br

Paul 3.6k Nov 23, 2022
Encrypt your files or notes by your GPG key and save to MinIO or Amazon S3 easily!

Super Dollop Super Dollop can encrypt your files and notes by your own GPG key and save them in S3 or minIO to keep them safe and portability, also yo

Nedim AKAR 58 Jul 11, 2022
A fast port scanner written in go with a focus on reliability and simplicity. Designed to be used in combination with other tools for attack surface discovery in bug bounties and pentests

Naabu is a port scanning tool written in Go that allows you to enumerate valid ports for hosts in a fast and reliable manner. It is a really simple to

ProjectDiscovery 2.8k Nov 19, 2022