A high performance go implementation of Wappalyzer Technology Detection Library

Related tags

wappalyzergo
Overview

wappalyzergo

A high performance port of the Wappalyzer Technology Detection Library to Go. Inspired by https://github.com/rverton/webanalyze.

Features

  • Very simple and easy to use, with clear code base.
  • Normalized regexes + auto-updating database of wappalyzer fingerprints.
  • Optimized for performance, parsing html manually for best speed.

Using go get

$ GO111MODULE=on go get github.com/projectdiscovery/wappalyzergo/cmd/update-fingerprints

After this command wappalyzergo library source will be in your current go.mod.

Example

Usage Example:

package main

import (
	"fmt"
	"io/ioutil"
	"log"
	"net/http"

	wappalyzer "github.com/projectdiscovery/wappalyzergo"
)

func main() {
	resp, err := http.DefaultClient.Get("https://www.hackerone.com")
	if err != nil {
		log.Fatal(err)
	}
	data, _ := ioutil.ReadAll(resp.Body) // Ignoring error for example

	wappalyzerClient, err := wappalyzer.New()
	fingerprints := wappalyzerClient.Fingerprint(resp.Header, data)
	fmt.Printf("%v\n", fingerprints)

	// Output: map[Acquia Cloud Platform:{} Amazon EC2:{} Apache:{} Cloudflare:{} Drupal:{} PHP:{} Percona:{} React:{} Varnish:{}]
}
Issues
  • Fingerprints automation

    Fingerprints automation

    opened by ehsandeep 0
  • Add version detection too for wappalyzer fingerprint database

    Add version detection too for wappalyzer fingerprint database

    Type: Enhancement Type: Question 
    opened by Ice3man543 1
  • How does updating fingerprint works

    How does updating fingerprint works

    Hi Team,

    As always another major and interesting opensource project from you all :fire:

    One of the first thing that I noticed that https://github.com/rverton/webanalyze downloads the fingerprints from the wappalyzer repo and save them locally and also has a update flag which can be used to update the same.

    But looks like this project is hard coding them in https://github.com/projectdiscovery/wappalyzergo/blob/master/fingerprints_data.go which is weird considering you have to update the code every time their is an update in the wappalyzer fingerprint which is quite frequent.

    Also the original file https://github.com/AliasIO/wappalyzer/blob/master/src/technologies.json is of 22181 lines where as even if we beautify https://github.com/projectdiscovery/wappalyzergo/blob/master/fingerprints_data.go its of 10861 lines only. Would love to understand why is this difference, are you ignoring non useful tags from the list ?

    I did see this https://github.com/projectdiscovery/wappalyzergo/blob/master/cmd/update-fingerprints/main.go But how does this works if am using this in my code like below, does it tries to update the fingerprint every time wappalyzer.New() is being called ? Can we manually invoke this update part using something like wappalyzer.update() only once before reusing the same wappalyzerClient.

          for _, url range urls{
    	resp, err := http.DefaultClient.Get(url)
    	if err != nil {
    		log.Fatal(err)
    	}
    	data, _ := ioutil.ReadAll(resp.Body) // Ignoring error for example
    
    	wappalyzerClient, err := wappalyzer.New()
    	fingerprints := wappalyzerClient.Fingerprint(resp.Header, data)
    	fmt.Printf("%v\n", fingerprints)
           }
    

    -- Regards, @bugbaba

    Priority: Medium Type: Question 
    opened by bugbaba 2
Releases(v0.0.7)
Owner
ProjectDiscovery
Security Through Intelligent Automation
ProjectDiscovery
Path to a Software Architect

Contents What is a Software Architect? Levels of Architecture Typical Activities Important Skills (1) Design (2) Decide (3) Simplify (4) Code (5) Docu

Justin Miller 7.1k Jul 27, 2021
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

English ∙ 日本語 ∙ 简体中文 ∙ 繁體中文 | العَرَبِيَّة‎ ∙ বাংলা ∙ Português do Brasil ∙ Deutsch ∙ ελληνικά ∙ עברית ∙ Italiano ∙ 한국어 ∙ فارسی ∙ Polski ∙ русский язы

Donne Martin 139.2k Jul 23, 2021
Object detection on multiple datasets with an automatically learned unified label space.

An object detector trained on multiple large-scale datasets with a unified label space; Winning solution of ECCV 2020 Robust Vision Challenges.

Xingyi Zhou 259 Jul 21, 2021
Go implementation of the yolo v3 object detection system

Go YOLO V3 This repository provides a plug and play implementation of the Yolo V3 object detection system in Go, leveraging gocv. Prerequisites Since

Wim Spaargaren 41 Jul 8, 2021
A curated list of Awesome Go performance libraries and tools

Awesome Go performance Collection of the Awesome™ Go libraries, tools, project around performance. Contents Algorithm Assembly Benchmarks Compiling Co

Oleg Kovalov 227 Jul 6, 2021
A quick tour (or reminder) of Go performance tools

Basics of benchmarking, profiling and tracing with Go Introduction This documentation gives an overview of possibilities offered by go tooling to meas

Samuel GAY 220 Apr 11, 2021
Go Training Class Material :

Go Training Review our different courses and material To learn about Corporate training events, options and special pricing please contact: William Ke

Ardan Labs 9.4k Jul 18, 2021
Golang implementation of the Paice/Husk Stemming Algorithm

##Golang Implementation of the Paice/Husk stemming algorithm This project was created for the QUT course INB344. Details on the algorithm can be found

Aaron Groves 26 Nov 28, 2019
🚀 fgprof is a sampling Go profiler that allows you to analyze On-CPU as well as Off-CPU (e.g. I/O) time together.

?? fgprof - The Full Go Profiler fgprof is a sampling Go profiler that allows you to analyze On-CPU as well as Off-CPU (e.g. I/O) time together. Go's

Felix Geisendörfer 1.8k Jul 19, 2021
Ensembles of decision trees in go/golang.

CloudForest Google Group Fast, flexible, multi-threaded ensembles of decision trees for machine learning in pure Go (golang). CloudForest allows for a

Ryan Bressler 687 Jul 17, 2021
A Naive Bayes SMS spam classifier written in Go.

Ham (SMS spam classifier) Summary The purpose of this project is to demonstrate a simple probabilistic SMS spam classifier in Go. This supervised lear

Dan Wolf 10 Apr 28, 2021
A Go library implementing an FST (finite state transducer)

vellum A Go library implementing an FST (finite state transducer) capable of: mapping between keys ([]byte) and a value (uint64) enumerating keys in l

bleve 45 Jul 20, 2021
A recommender system service based on collaborative filtering written in Go

Language: English | 中文 gorse: Go Recommender System Engine Build Coverage Report GoDoc RTD Demo gorse is an offline recommender system backend based o

Zhenghao Zhang 3.6k Jul 27, 2021
Go Machine Learning Benchmarks

Benchmarks of machine learning inference for Go

Nikolay Dubina 15 Jun 17, 2021