urlsh is URL shortener application built on Go language.

Overview

urlsh

urlsh is URL shortener application built on Go language.

It does not use external libraries except the gorm for postgres database and minimal redis module redigo.

It registers itself as Go module github.com/adhocore/urlsh (however it has not been submitted to Go package registry for public usage).

URLSH

Usage

Just visit urlssh.xyz. You can also integrate API for programmatic usage. Read below for self hosting, API integration and/or contributing to urlsh.


Getting source

git clone [email protected]:adhocore/urlsh.git
cd urlsh

Configuring

It should be configured using env variables.

Please check .env.example for available variables and explanation.

DATABASE_URL is always required and is string of the following form:

DATABASE_URL=postgres://user:[email protected]:port/dbname

When running urlsh with docker-compose, the preferred way, DATABASE_URL is automatically set from POSTGRES_* variables.

Please note that urlsh does not ship with .env loader so to run it in bare metal, one needs to use export KEY=VALUE or source .env manually.

If REDIS_URL is provided with redis://host:port, then it will cache popular urls, the popularity threshold is hard coded to minimal value in common.constant. The cache will sync when short url is deleted and sometimes when expired. The hit counters will however sync in realtime.

Setting up docker

To set up dockerized urlsh, run the commands below:

# first time only
cp .example.env .env

# change auth token for admin if you want in `.env` file
# APP_ADMIN_TOKEN=

docker-compose up

After a few seconds, you should be able to browse to localhost:2000.

Testing

For running tests,

docker-compose exec urlsh sh -c "APP_ENV=test go test ./..."

# for coverage
docker-compose exec urlsh sh -c "APP_ENV=test go test -cover ./..."

APP_ENV=test is not required but ensures that tests are run against clone database with name prefixed by test_. Normally test db is already prepared if you use docker-compose.


API Endpoints

GET /status

Status route for health/status check.

Response payload

{
    "status": 200,
    "message": "it works"
}

POST /api/urls

Creates a new short code for given URL.

Request example

  • url: string, required, http/https/ftp only
  • expires_on: string, optional, utc date yyyy-mm-dd hh:mm:ss, default=9999-01-01
  • keywords: array of strings, 2-25 chars each, max 10 keywords
{
    "url": "http://somedomain.com/some/very/long/url",
    "expires_on": "",
    "keywords": ["key", "word"]
}

Response example

{
    "status": 200,
    "short_code": "qaFxz",
    "short_url": "http://localhost:2000/qaFxz"
}

If env var APP_ALLOW_DUPE_URL is set to 0 or empty, then trying to shorten same URL again will return status 409 and payload will contain existing short_code. However if existing short_code is deleted, it will be shortened as usual.


GET /{shortCode}

Redirects the shortcode to original long URL.

Response payload

In case short code exists it responds with 301 redirect.

If the short code is expired or deleted, it responds like so:

{
    "status": 410,
    "message": "requested resource is not available"
}

GET /api/admin/urls

Authentication

Token required in Authorization header like so:

Authorization: Bearer 

Request query

The query params are optional.

page=
short_code=
keyword=

Examples:

  • /api/admin/urls?short_code=somecode
  • /api/admin/urls?page=1&keyword=something

Response example

Response contains multiple matching url object inside urls array.

{
    "status": 200,
    "urls": [
        {
            "short_code": "X5JkFd",
            "origin_url": "http://somedomain.com/some/very/long/url",
            "hits": 1,
            "is_deleted": false,
            "expires_on": "9999-01-01T00:00:00Z"
        }
    ]
}

DELETE /api/admin/urls

Authentication

Token required in Authorization header like so:

Authorization: Bearer 

Request query

Query param short_code is requied.

Example: /api/admin/urls?short_code=somecode

Response example

If delete success:

{
    "status": 200,
    "deleted": true
}

If the code does not exist:

{
    "status": 404,
    "message": "the given short code is not found"
}

Using postman

urlsh comes with postman collection and environment to aid manual testing of endpoints.

Open the postman app, click Import at top left, select Folder and drag/choose postman folder of this repo. You may need to adjust the token in postman urlsh env if you have configured APP_ADMIN_TOKEN.

The collection comes with post/pre request hooks for requests so you can just run the endpoints one after another in postman UI.

For redirect request, you have to disable postman follow redirects from Settings > General > Automatically follow redirects.

License

Please check license file.

Issues
  • Error

    Error "invalid memory address or nil pointer dereference" using docker-compose

    Hi! This seems like an awesome service, especially for self-hosters like myself.

    Running urlsh using docker-compose, the server and its requirements all seem to run fine, but when I click the "Shorten" button I receive a HTTP 500 error with the following message:

    {"message":"runtime error: invalid memory address or nil pointer dereference","status":500}
    

    I'm not seeing any logs in the docker-compose run, even when launching using APP_ENV=dev.

    What could be causing this issue? Any way I can debug it?

    Is it possible the error message is acutally coming from Docker (and not from urlsh)?

    Thanks.

    opened by nyancow 6
  • Support redis cache

    Support redis cache

    closes #22

    • cache popular urls whose hits exceed certain threshold (hardcoded in constant.go)
    • on delete, inactive status is synced to cache
    • configure using APP_CACHE_HOST=host:port env var.
    opened by adhocore 0
Releases(v0.2.0)
  • v0.2.0(May 7, 2021)

  • v0.1.2(May 4, 2021)

    Features

    • Middleware: Add recover middleware (Jitendra Adhikari) 3bcbcce

    Internal Refactors

    • Middleware: Switch type usage (Jitendra Adhikari) 624659e
    • Router: Wrap handler with recover (Jitendra Adhikari) 17cd64d
    • Orm: Use panic (Jitendra Adhikari) c16c881

    Miscellaneous

    • Common: Add server error (Jitendra Adhikari) b3cadf9

    Builds

    • Docker: No CGO in alpine container (Jitendra Adhikari) 1856cb5
    • Docker: Env map should be .env only (Jitendra Adhikari) 080e65c
    • Docker: Use alpine images (Jitendra Adhikari) 88bc5bf
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Apr 14, 2021)

  • v0.1.0(Apr 13, 2021)

    v0.1.0 (2021-04-13)

    Features

    • Cmd.urlshc: Add urlshc command (Jitendra Adhikari) 7f30ea5
    • Service.url: Check url reach (host) if configured (Jitendra Adhikari) ca97005
    • Request.url: Append host to input (Jitendra Adhikari) 79c845c
    • Tmpl: Toggle check on copy from history row (Jitendra Adhikari) 5833b70
    • Add /banner route and resource for og:image (Jitendra Adhikari) ec7ed9c
    • Tmpl: Add og tags (Jitendra Adhikari) f9f4a1d
    • *****: Add favicon and robots router and template (Jitendra Adhikari) 72a55fa
    • Tmpl: Add copy btn in history row, simplify dynamic row (Jitendra Adhikari) aa6bfdc
    • Tmpl: Add history table, rename copy button (Jitendra Adhikari) e7367cb
    • Controller: Show UI in homepage (Jitendra Adhikari) 3d1f396
    • Tmpl: Add home html template (Jitendra Adhikari) 0fc94ad
    • Import.service: Make cache deactivation async, increment hits to compare threshold (Jitendra Adhikari) 97fdfe3
    • Controller: Make hits increment async (Jitendra Adhikari) 193af34
    • Service.url: Make cache deactivation async (Jitendra Adhikari) bd6fb24
    • Controller: Add x-cached header for cached data (Jitendra Adhikari) eadf345
    • Service.client: Lookup return flag to signify cached value (Jitendra Adhikari) ce0ed85
    • Cache: Add redis cache with lookup, persist and deactivate feature (Jitendra Adhikari) e781d68
    • Controller: Add endpoint to serve short url (Jitendra Adhikari) df2d6a9
    • Service.url: Add service to lookup origin & inc hits by shortcode (Jitendra Adhikari) 79ac77a
    • Common: Add shortcode regex (Jitendra Adhikari) 040db1d
    • Router: Register short url delete endpoint (Jitendra Adhikari) c12f42a
    • Controller: Add delete endpoint (Jitendra Adhikari) 79455d4
    • Common: Add no short code error (Jitendra Adhikari) 31e2f43
    • Service.url: Add service to delete short url (Jitendra Adhikari) 24e8d7d
    • Common: Add auth token errors (Jitendra Adhikari) e7fa252
    • Middleware: Add auth middleware to validate header token for admin (Jitendra Adhikari) 11f06d9
    • Router: Register admin list/search controller (Jitendra Adhikari) 7ec1ac1
    • Controller: Add admin list/search endpoint controller (Jitendra Adhikari) 136666e
    • Common: Add data not found error (Jitendra Adhikari) 98c1160
    • Service.url: Add service to list urls by filter for admin (Jitendra Adhikari) be7bff1
    • Request: Add UrlFilter definition with offset getter for search (Jitendra Adhikari) 70dc8d4
    • Response.body: Add Merge helper (Jitendra Adhikari) e6f8e12
    • Controller: Add client api controller for short url (Jitendra Adhikari) b729830
    • Common: Add short code length constant (Jitendra Adhikari) 6c93eae
    • Service: Add short url creation service (Jitendra Adhikari) ba6e13b
    • Request: Add short url create request struct with validator (Jitendra Adhikari) d291da2
    • Common: Add errors to be used in validation throughout (Jitendra Adhikari) dbf74f6
    • Util: Add random length helper util (Jitendra Adhikari) dbbf1ad
    • Orm: Add postgres connection handler for gorm (Jitendra Adhikari) aca8fc1
    • Add DateLayout const, use time.UTC instead of common.UTC (Jitendra Adhikari) f8f53a8
    • Model: Add url model (Jitendra Adhikari) 8f0af8a
    • Model: Add keyword model (Jitendra Adhikari) 8462de3
    • Controller: Add not found handler (Jitendra Adhikari) e8437d5
    • Router: Add route registrar with multi endpoints handler (Jitendra Adhikari) 153a45a
    • Main: Listen and serve request, register default route (Jitendra Adhikari) 1020674
    • Controller: Add frontend controller (Jitendra Adhikari) 0955f36
    • Response: Add json response writer (Jitendra Adhikari) 5894f71
    • Init module (Jitendra Adhikari) 8e796d3

    Bug Fixes

    • Typo (Jitendra Adhikari) 4c11604
    • Service.url: Use generic error (Jitendra Adhikari) 034ae99
    • Cache: Redis uses different env (Jitendra Adhikari) 6038ced
    • Request: Validate url with regex (Jitendra Adhikari) 3bc274c
    • Request: Exclude pattern (Jitendra Adhikari) e27a8b8
    • Tmpl: Use short href (Jitendra Adhikari) d1c067e
    • Request: Blacklist local urls (Jitendra Adhikari) d37cfc5
    • Tmpl: Hide copy btn on error (Jitendra Adhikari) 6e08e41
    • Cache: User h is not usable (Jitendra Adhikari) ef8e844
    • Init-db: Make executable (Jitendra Adhikari) 48ca0cf
    • Main: Use port dynamically (Jitendra Adhikari) 5233f98
    • Cache.redis: A;ways populate cached model (Jitendra Adhikari) e7f265d
    • Service.url: Ignore empty short code early (Jitendra Adhikari) d6439aa
    • Service.url: Select deleted field (Jitendra Adhikari) 998d8c3
    • Service.url: When checking dupe, exclude deleted (Jitendra Adhikari) dc99646
    • Middleware: Use req.URL.Path for portability (Jitendra Adhikari) 860b16d

    Internal Refactors

    • Move tmpl/ to assets/ (Jitendra Adhikari) 65bc2a0
    • Service.url: Make validateURLInput public (Jitendra Adhikari) 85c9194
    • Validate url len, keyword char, rephrase url filter error msg (Jitendra Adhikari) 6bc3661
    • *****: Use 2000 as default port (Jitendra Adhikari) 01f58ab
    • Tmpl: Add source code url, fix per w3c validator (Jitendra Adhikari) 2c0a8c6
    • Use PORT instead of APP_PORT (Jitendra Adhikari) 5c014f9
    • Orm,cache: Use DATABASE_URL and REDIS_URL env instead (Jitendra Adhikari) 8910662
    • *****: Use 301 permanent redirect instead (Jitendra Adhikari) 2619adc
    • Postman: Move "/" to "/status" (Jitendra Adhikari) 42f5da3
    • Router: Move "/" to "/status" (Jitendra Adhikari) 9ee7a87
    • Cache.redis: Use pool instead (Jitendra Adhikari) cc24a94
    • Cache: Support redis user/pass (Jitendra Adhikari) 0ad1754
    • Controller: Adapt to changed service layer (with cache) (Jitendra Adhikari) c4edc2e
    • Service.url: Integrate with cache layer for delete (Jitendra Adhikari) df1ad3f
    • Service.url: Integrate with cache layer for lookup (Jitendra Adhikari) 9bdc002
    • Config.env: Add some hints (Jitendra Adhikari) bbe5596
    • Orm: Use test db for APP_ENV=test (Jitendra Adhikari) 6fefad1
    • Router: Refactor to handle ServeShortUrl request (Jitendra Adhikari) a6d8a1f
    • Main: Attach mux handler from router (Jitendra Adhikari) 8fa951b
    • Router: Use mux instead to make it middleware friendly (Jitendra Adhikari) 4f44825
    • Service.url: Add doc, always use offset (Jitendra Adhikari) 22f50e7
    • Controller.client: Simplify with body.Merge (Jitendra Adhikari) 8899f07
    • Request.url: Cleanup and simplify GetExpiresOn (Jitendra Adhikari) cc24550
    • Main: Switch to router instead (Jitendra Adhikari) e314a4a

    Miscellaneous

    • Minor tweak homepage html (Jitendra Adhikari) f912681
    • Use short url to source code (Jitendra Adhikari) 79db201
    • Timeout 1s (Jitendra Adhikari) 41e9fbf
    • Add check conf (Jitendra Adhikari) 68bc576
    • No tag autocomplete (Jitendra Adhikari) d94c26c
    • Fix typo (Jitendra Adhikari) 78bf1ff
    • Tmpl: Use nbsp (Jitendra Adhikari) 4e7b2cb
    • Config: Fix db host (Jitendra Adhikari) bc4871b
    • Config: Add DATABASE_URL and REDIS_URL (Jitendra Adhikari) bf17b7a
    • Common: Normalize error msg (Jitendra Adhikari) b7f93e6
    • Cleanup (Jitendra Adhikari) c34d86a
    • Docker cache (Jitendra Adhikari) ec6d9a8
    • Flush redis before test (Jitendra Adhikari) b3a0610
    • Add gha test (Jitendra Adhikari) 75fca1a
    • Gh codeowner (Jitendra Adhikari) 6c23bac
    • Common: Popular website hits count (Jitendra Adhikari) 1804fe9
    • Config: Redis cache host:port (Jitendra Adhikari) 4cbb672
    • Deps: Add redigo module (Jitendra Adhikari) e37f192
    • Gha lint (Jitendra Adhikari) 36e8942
    • Config: Document APP_PORT (Jitendra Adhikari) 2916f87
    • Postman: Add collection and environment (Jitendra Adhikari) f630b8b
    • Config: Add env example config (Jitendra Adhikari) 83a3dd6
    • Git: Add .env to ignore (Jitendra Adhikari) ceab82c
    • Deps: Add postgres driver and gorm (Jitendra Adhikari) 544705a
    • Cs: Add editorconfig config (Jitendra Adhikari) 97b5312
    • Add gitignore (Jitendra Adhikari) cc2d8f1

    Documentations

    • Add urlshc terminal client (Jitendra Adhikari) 25c8a31
    • Add screenshot (Jitendra Adhikari) d682cb4
    • Update readme (Jitendra Adhikari) b7100c3
    • Move "/" to "/status" (Jitendra Adhikari) 86d1933
    • About redis cache (Jitendra Adhikari) bb8ead9
    • Cache: Add method docs (Jitendra Adhikari) 925bab0
    • About test env and cov (Jitendra Adhikari) 696ca9b
    • Create url short code param and behavior (Jitendra Adhikari) 0379809
    • Postman redirect (Jitendra Adhikari) 26c8efc
    • Readme: Add installation/setup/testing steps, endpoint docs and postman (Jitendra Adhikari) e7fe6c6
    • *****: Add method docs for all source (Jitendra Adhikari) ed3277f
    • Init repo (Jitendra Adhikari) 9c9ad4a

    Builds

    • Docker: APP_DB_DSN is no longer needed (Jitendra Adhikari) 249e4f6
    • Docker: Add redis cache (Jitendra Adhikari) 744d5eb
    • Docker: Add docker-compose (Jitendra Adhikari) d2b182f
    • Docker: Add init-test-db (Jitendra Adhikari) 5832a09
    Source code(tar.gz)
    Source code(zip)
Owner
Jitendra Adhikari
ISTJ. Thinker. Solver. Creator.
Jitendra Adhikari
A URL shortener using http://is.gd/ and the Go programming language (http://golang.org/)

goisgd A simple command line URL shortener using http://is.gd/. Getting the Code go get github.com/NickPresta/GoURLShortener Usage Import this librar

Nick Presta 22 Apr 6, 2022
A simple URL shortener application.

url-shortener A simple URL shortener application. Features The service supports 2 data stores (in-memory and PostgreSQL). The -db parameter is used to

Demid Fedorov 0 Dec 9, 2021
a url shortener made using go and redis

__ | |--.--.--.----.-----. .-----.-----. | <| | | _|-- __|__| _ | _ | |__|__|_____|__| |_____|__|___

André Kelpe 76 Dec 10, 2020
A barebones URL Shortener implementation in Go using Gin and MySQL. Also features a basic frontend.

URL Shortener in Go This is a barebones URL Shortener implementation in Go using the Gin web framework and MySQL. Also features a basic frontend. Loca

Shreyas Gupta 6 Dec 22, 2021
Go URL shortener

Golang Short URL To build the short link engine by Go. Development Environment: Ubuntu 20.04 Go 1.17.1 Requirement Console app, receive “URL” from any

Siong-Ui Te (戴上為) 0 Nov 4, 2021
Yandex practicum go url shortener service

go-musthave-shortener-tpl Шаблон репозитория для практического трека «Go в веб-р

Dindon 0 Feb 26, 2022
Url Shortener Api For Golang

Url Shortener Api Using Go Endpoints Get and redirect to long url GET: "/:id" Cr

Lutfi Andriyanto 1 May 29, 2022
Simple URL shortener written in Golang

Alviss Introduction Simple URL shortener project, written in Golang. Setup and r

Amir Iravanimanesh 21 Aug 3, 2022
A sample url shortener app to test Keploy integration capabilities

Example URL Shortener A sample url shortener app to test Keploy integration capabilities Installation git clone https://github.com/keploy/example-url-

null 5 Mar 12, 2022
API from AnonURL website, an anonymous URL shortener

API from AnonURL website, an anonymous URL shortener

null 0 Jan 23, 2022
Cfurlshort - Url Shortener service with golang

Url Shortener This respository contains the code for my url shortener service. D

null 0 Jan 26, 2022
Goshort - Just another simple url shortener to use with curl

qnd.be Just another simple url shortener to use with curl. Kinda like 0x0.st. Ju

rune 0 Jan 29, 2022
OMUS - One More URL Shortener

OMUS OMUS - One More URL Shortener; For now - planned only API. Functions: Gener

Vladislav 2 Feb 12, 2022
Short url backend - Go musthave shortener tpl

go-musthave-shortener-tpl Шаблон репозитория для практического трека «Go в веб-р

Dmitriy 0 Feb 15, 2022
The 'Scalable' URL shortener

Chinnaswamy The 'Scalable' URL shortener Running tests To run end to end tests: CHINNASWAMY_TEST_HOST="<host:port>" go test -count=1 ./integration_tes

nilenso 2 May 21, 2022
urlhunter is a recon tool that allows searching on URLs that are exposed via shortener services such as bit.ly and goo.gl.

a recon tool that allows searching on URLs that are exposed via shortener services

Utku Sen 1.2k Aug 1, 2022
GoogleBookAPI is built on top of flogo, a flow based application.

GoogleBookAPI Example GoogleBookAPI is built on top of flogo, a flow based application. Upon launch for first time, the application creates a topic go

Abhishek K 0 Nov 19, 2021
This codebase was created to demonstrate a fully fledged fullstack application built with Golang/Echo including CRUD operations, authentication, routing, pagination, and more.

This codebase was created to demonstrate a fully fledged fullstack application built with Golang/Echo including CRUD operations, authentication, routing, pagination, and more.

Nayef Haidir 2 Mar 22, 2022
Go-watchdog - a web application observability tool built for Go

Go-watchdog is a web application observability tool built for Go, it exposes a status endpoint for application services like databases, caches, message-brokers, mails and storages.

salem ododa 4 Jul 11, 2022