๐ŸŒ• Server application for storing doujinshi, manga, art collections and other galleries with API and user control. Written in Go.

Related tags

Network server
Overview

Mangatsu

Still at very experimental stage. Expect lots of breaking changes.

๐ŸŒ• Server application for storing doujinshi, manga, art collections and other galleries with API and user control. Written in Go.

๐Ÿ“ฐ CHANGELOG | โค CONTRIBUTING | ๐ŸŽฏ TODO

๐Ÿ“Œ Installation and usage

๐Ÿณ Docker setup (recommended)

  • Coming soon for both the server and the web app

๐Ÿ’ป Local setup

  • Coming soon

๐Ÿ“Œ Clients

๐Ÿ–ฅ Web interface

๐Ÿ“ฑ Tachiyomi extension for Android

  • Coming soon

๐Ÿ“Œ Features

  • Organizing and tagging local (and remote with tools like rclone) collections of manga, doujinshi and other art

    • Mangatsu will never do any writes inside the archive location.
    • Supports ZIP (or CBZ), RAR (or CBR) and plain image (png, jpg, jpeg, webp, gif, tiff, bmp) files.
      • 7z, PDF and video support is planned.
  • Metadata parsing from filenames, JSON/TXT files (inside or beside the archive). More to come.

  • API-access to the collection and archives

    • Extensive filtering, sorting and searching capabilities.
    • Additional features for registered users such as tracking reading progress and adding favorite groups.
  • User access control

    • Private: only logged-in users can access the collection and archives (registration can be disabled).
    • Restricted: users need a global passphrase to access collection and its galleries.
    • Public: anyone can access (only read) collection and its galleries.
  • Users with roles (admin, member, viewer) and indefinite login sessions with the option to log out or delete them

  • Local cache and thumbnail support

๐Ÿ“Œ Usage

๐Ÿ“ Configuration

Options inside the .env file:

  • MTSU_INITIAL_ADMIN_NAME=admin
  • MTSU_INITIAL_ADMIN_PW=admin321
    • Credentials for the initial admin user.
  • MTSU_HOSTNAME=localhost
  • MTSU_PORT=5050
    • Hostname and port for the server.
  • MTSU_BASE_PATHS=freeform;/home/user/unstructured-manga;;structured:/home/user/structured-manga
    • Root paths to the collection and archives. Relative or absolute paths are accepted.
    • First specify the type of the directory (freeform or structured) and then the path separated by a semicolon: ;
    • Multiple paths can be separated by a double-semicolon: ;;
  • MTSU_DATA_PATH=../data
    • Location of the data dir which includes the SQLite db and cache for gallery images and thumbnails. Relative or absolute paths are accepted.
  • MTSU_VISIBILITY=public
    • public: anyone can access the collection and its galleries
    • restricted: users need a global passphrase to access collection and its galleries
    • private: only logged-in users can access the collection and its galleries
  • MTSU_RESTRICTED_PASSPHRASE=secret321
    • Passphrase to access the collection and its galleries.
    • Only used when VISIBILITY is set to restricted.
  • MTSU_REGISTRATIONS=false
    • Whether to allow user registrations. If set to false, only admins can create new users. Currently, only affects the API path /register. Has no effect in the frontend.
  • MTSU_JWT_SECRET=secret123
    • Secret value for signing JWT tokens for login sessions in the backend.

๐Ÿ“‚ Directory structure

Multiple root directories are supported. I suggest creating a structured format for proper long-running manga, and a freeform structure for doujinshi and other art collections. Examples follow:

  • Freeform: galleries can be up to three levels deep. Good for doujinshi, one-shots and other more unstructured collections.
    • External JSON metadata files have to be placed in the same level as the gallery archive. Preferably having the same name as the gallery archive. If no exact match is found, filename close enough will be used instead.
    • Option to create categories according to the 1st level of the directory structure. In the example, doujinshi and art would be created as categories. The last lonely archive would be uncategorized.
๐Ÿ“‚ freeform
โ”œโ”€โ”€ ๐Ÿ“‚ doujinshi
โ”‚       โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“‚ oppai
โ”‚       โ”‚     โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“ฆ [Group (Artist)] Ecchi Doujinshi.cbz
โ”‚       โ”‚     โ””โ”€โ”€โ”€โ”€ ๐Ÿ“„ [Group (Artist)] Ecchi Doujinshi.json
โ”‚       โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“ฆ (C99) [Group (Artist)] elfs.zip
โ”‚       โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“„ (C99) [Group (Artist)] elfs.json
โ”‚       โ””โ”€โ”€โ”€โ”€ ๐Ÿ“ฆ (C88) [kroup (author, another author)] Tankoubon [DL].zip  (JSON or TXT metafile inside)
โ”œโ”€โ”€ ๐Ÿ“‚ art
โ”‚       โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“‚ [Artist] Pixiv collection
โ”‚       โ”‚     โ”œโ”€โ”€โ”€โ”€ ๐Ÿ–ผ๏ธ 0001.jpg
โ”‚       โ”‚     โ”œโ”€โ”€โ”€โ”€...
โ”‚       โ”‚     โ””โ”€โ”€โ”€โ”€ ๐Ÿ–ผ๏ธ 0300.jpg
โ”‚       โ”œโ”€โ”€โ”€โ”€ ๐Ÿ“ฆ art collection y.rar
โ”‚       โ””โ”€โ”€โ”€โ”€ ๐Ÿ“„ art collection y.json
โ””โ”€โ”€ ๐Ÿ“ฆ (C93) [group (artist)] Lonely doujinshi (Magical Girls).cbz
  • Structured: galleries follow a strict structure. Good for long-running manga (shounen, seinen etc).
    • Manga -> Volumes -> Chapters, Manga -> Volumes or Manga -> Chapters
    • Galleries' Series is set to the name of the 1st level directory except for galleries in the root directory.
๐Ÿ“‚ structured
โ”œโ”€โ”€ ๐Ÿ“• Manga 1
โ”‚       โ”œโ”€โ”€ ๐Ÿ“ฆ Volume 1.cbz
โ”‚       โ”œโ”€โ”€ ๐Ÿ“ฆ Volume 2.cbz
โ”‚       โ”œโ”€โ”€ ๐Ÿ“ฆ Volume 3.cbz
โ”‚       โ””โ”€โ”€ ๐Ÿ“ฆ Volume 4.zip
โ”œโ”€โ”€ ๐Ÿ“˜ Manga 2
โ”‚       โ””โ”€โ”€ ๐Ÿ“‚ Vol. 1
โ”‚           โ”œโ”€โ”€ ๐Ÿ“ฆ Chapter 1.zip
โ”‚           โ”œโ”€โ”€ ๐Ÿ“ฆ Chapter 2.zip
โ”‚           โ””โ”€โ”€ ๐Ÿ“ฆ Chapter 3.rar
โ”œโ”€โ”€ ๐Ÿ“— Manga 3
โ”‚       โ”œโ”€โ”€ ๐Ÿ“ฆ Chapter 1.zip
โ”‚       โ”œโ”€โ”€ ...
โ”‚       โ””โ”€โ”€ ๐Ÿ“ฆ Chapter 30.rar
โ””โ”€โ”€ ๐Ÿ“ฆ One Shot Manga.rar
Comments
  • Parse or save the UserAgent of login requests to have a meaningful session name

    Parse or save the UserAgent of login requests to have a meaningful session name

    Either

    • Save the raw user agent as the session name and let client handle the parsing
    • OR
    • Parse the UserAgent and save the result

    https://github.com/Mangatsu/server/blob/d5e49ae09508adf12ee31939a6d642b238e64da7/pkg/api/user.go#L74

    enhancement server web 
    opened by CrescentKohana 1
  • Migrate `db.go`, `library.go` to goqu

    Migrate `db.go`, `library.go` to goqu

    Fixes #14

    As said in the beginning of library.go:

    // FIXME: it won't work yet!! CombinedLibrary has to be a plain struct,
    // something like this:
    //
    // 	type User struct {
    // 		FirstName string `db:"first_name"`
    // 		LastName  string `db:"last_name"`
    //	}
    

    We need to create "model" structs with column names in db tags.

    And also for now, newLibrary doesn't handle the case when an ID exists. I haven't tested the code because we don't have those annotated model structs, as said earlier.

    opened by karmek-k 1
  • Migrate gallery.go and validations.go to goqu

    Migrate gallery.go and validations.go to goqu

    This one might be the most challenging part of the goqu rework.

    • GetGalleries function is very elaborate and prone to bugs if not converted carefully.
    • UpdateGallery can probably be refactored to be much shorter and better written overall. The Jet library made it hard to do partial updates.
    • NeedsUpdate isn't used anywhere currently and can be pretty much ignored
    goqu-rework 
    opened by CrescentKohana 1
  • Initialize database and database user on first launch if needed

    Initialize database and database user on first launch if needed

    Initialize database and database user on first launch if needed. Also an environmen var to disable this completely.

    For example for MySQL/MariaDB it'd be:

    CREATE USER '<USERNAME>'@'<ADDRESS>' IDENTIFIED BY '<PASSWORD>';
    CREATE DATABASE IF NOT EXISTS <DATABASE_NAME> CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
    GRANT ALL PRIVILEGES on <DATABASE_NAME>.* to '<USERNAME>'@'<ADDRESS>';
    FLUSH privileges;
    

    but in Go.

    The utf8mb4 COLLATE utf8mb4_unicode_ci helps with support for Japanese, Chinese etc. characters.

    goqu-rework priority 
    opened by CrescentKohana 0
  • TODO list

    TODO list

    Web TODO here

    Short term

    • [x] #12
    • [ ] #5
    • [ ] #1
    • [ ] #2
    • [ ] #3
    • [ ] #4
    • [ ] #7
    • [x] #26
    • [ ] Better plain file support and general improvements for the scanner and tagger
    • [ ] More testing especially for the server (db, parsers)
    • [x] Allow disabling internal file server to allow for external file serving (like NGINX)
    • [x] Group structured manga with the same series names

    Long term

    • [ ] #6
    • [ ] Improvements to cache policy (e.g. max size)
    • [ ] Rework sessions to utilize JWT refresh tokens (expire in 5-15 minutes or so)?
    • [ ] Flutter (or something else) mobile app frontend
    opened by CrescentKohana 0
Releases(v0.4.3)
  • v0.4.3(Apr 17, 2022)

  • v0.4.2(Apr 16, 2022)

    Added

    • Show internal server error messages in console
    • API endpoint for gallery count /galleries/count
    • Seed parameter for shuffling gallery results

    Fixed

    • Setting a custom SQLite db filename
    • Updating galleries internally
    • Updating translated and native titles

    Changed

    • Return only structured & "non-null Series" galleries when grouping

    Docs

    • Update preview images
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Apr 15, 2022)

  • v0.4.0(Apr 15, 2022)

    Added

    • Periodically prune the gallery cache of old entries
      • Time to live can be configured via MTSU_CACHE_TTL environment variable. Defaults to 336h (14 days)
      • Utilizes mutex to prevent reading and deleting the same entry at the same time
    • Environmental variable to disable migrations MTSU_DB_MIGRATIONS=false
    • Environment variable to override the default database name: MTSU_DB_NAME=mangatsu
    • Embedded migrations to binary

    Changed

    • Updated dependencies

    Docs

    • Cross out 7z in README as it's not supported yet
    • Fix mistakes in example.env and ENVIRONMENTALS.md
    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Mar 24, 2022)

  • v0.3.0(Mar 20, 2022)

    Added

    • Support for returning galleries grouped by Series from the API
    • Hath ([email protected]) and EHDL meta text file parsers
    • Support for calling all metadata parsers through the API
    • Validations when updating gallery

    Fixed

    • GetTags SQL query

    Changed

    • Updated dependencies
    • Updated Go to 1.18
    • Harden title language name parsing (list of supported languages)
    • Disallow empty namespaces or names in tags
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Feb 27, 2022)

    Added

    • UpdateGallery API method

    Fixed

    • Error when a single (e.g. Gallery) result is empty
    • Returning favorite groups could include empty groups

    Changed

    • Renamed TitleShort column to TitleTranslated
    • Upgraded packages

    Removed

    • Artists and Circle columns from Gallery
      • Not needed as the same can be achieved by using the artist and circle namespace tags
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Feb 3, 2022)

  • v0.1.1(Feb 3, 2022)

  • v0.1.0(Jan 30, 2022)

    Initial release

    • Multiple root directories with two layouts: freeform and structured
    • Scanning manga and other collections
      • Supports ZIP (or CBZ), RAR (or CBR), 7z and plain image (png, jpg, jpeg, webp, gif, tiff, bmp) files
    • Metadata parsing from filenames and x styled JSON files inside the archive or besides it
    • API
      • Extensive filtering, sorting and searching capabilities
      • Additional features for registered users such as tracking reading progress and adding favorite groups
    • User access control
      • Private: only logged-in users can access the collection and archives (public registration disabled by default)
      • Restricted: users need a global passphrase to access collection and its galleries
      • Public: anyone can access (only read) collection and its galleries
      • Assignable roles (admin, member, viewer)
      • Session management
    • Local cache and thumbnail support
      • File server can be disabled to allow web servers like NGINX to handle the files
    Source code(tar.gz)
    Source code(zip)
Owner
null
๐Ÿˆ๐Ÿ“ฆ nyaa.si client library for Go. Fetch Anime, Manga, Music and more torrents

?? ?? go-nyaa nyaa.si client library for Go Built on top of: gofeed - search using RSS colly - scrap torrent details page Original idea: ejnshtein/nya

Ilya Revenko 25 Sep 23, 2022
Go Http Proxy with Authentication, Schedule Control, and Portal Control

goproxy Go Http Proxy with Authentication, Schedule Control, and Portal Control Why this tool? You may need to restrict my kids's youtube watch time i

Wu Shilin 2 Mar 27, 2022
ScriptTiger 20 Sep 23, 2022
concurrent caching proxy and decoder library for collections of PMTiles

go-pmtiles A caching proxy for the serverless PMTiles archive format. Resolves several of the limitations of PMTiles by running a minimalistic, single

Protomaps 45 Jan 2, 2023
MPD client inspired by ncmpcpp written in GO with builtin cover art previews.

goMP MPD client inspired by ncmpcpp written in GO demo.mp4 Roadmap Add Functionality to Sort out most played songs Add a config parser Image Previews

Aditya Kurdunkar 46 Jan 1, 2023
Tapestry is an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs.

Tapestry This project implements Tapestry, an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate

Han Cai 1 Mar 16, 2022
App with CRUD for user, with palindrome checker for user's first and last name

Run db container first, so that app does not connect to db while db has not started yet docker-compose up -d db docker-compose up -d app CRUD endpoint

Vinh Tran 0 Dec 9, 2021
The server-pubsub is the main backend of DATAVOC project that manages all the other web-server modules of the same project such as the processor

server-pubsub The server-pubsub is the main backend of DATAVOC project that manages all the other web-server modules of the same project such as the p

null 0 Dec 3, 2021
API server for BFE control plane

API Server ๆ˜ฏ BFE ๆŽงๅˆถ้ขๆ ธๅฟƒๆจกๅ—๏ผŒๅฎŒๆˆ้…็ฝฎ็š„ๅฝ•ๅ…ฅใ€ๅญ˜ๅ‚จๅ’Œๅฏผๅ‡บใ€‚ ๆŽงๅˆถ้ข็ป„ไปถ ๅ›พ1๏ผšๆŽงๅˆถ้ข็ป„ไปถ BFEๆŽงๅˆถ้ขๅŒ…ๅซๅฆ‚ไธ‹็ป„ไปถ๏ผš API Server: ๅฏนๅค–ๆไพ›Open APIๆŽฅๅฃ๏ผŒๅฎŒๆˆBFE้…็ฝฎ็š„ๅ˜ๆ›ดใ€ๅญ˜ๅ‚จๅ’Œไธ‹ๅ‘ Dashboard: ็ฎก็†ๆŽงๅˆถๅฐ๏ผŒ็”จไบŽBFE้›†็พค็š„ๅฏ่ง†ๅŒ–็ฎก็† ไป“ๅบ“ๅœฐๅ€ๅœจ

null 30 Jul 20, 2022
server-to-server sync application, written in go/golang.

svcpy: server to server copy a basic server-to-server copy application. on a single binary, it can be a server or a client. example usage: on the serv

Mert Akengin 0 Nov 4, 2021
A suite of tools for NFT generative art.

nftool A suite of tools for NFT generative art. Features Traits/Attributes/Properties Generation Configure custom rarity Generate collection attribute

Aleph Retamal 150 Dec 15, 2022
Archiving Street Art in a distributed manner

Graffiti Repository provides a tool for graffiti community to keep track and preserving street artwork in their areas. Features Distributed storage Th

Roman Blanco 4 Oct 27, 2022
Headscale - An open source, self-hosted implementation of the Tailscale control server

Headscale - An open source, self-hosted implementation of the Tailscale control server

Juan Font 9.9k Dec 29, 2022
This app brings the Matterbridge binary to your server to connect Nextcloud Talk with other chat services

This app brings the Matterbridge binary to your server to connect Nextcloud Talk with other chat services

Nextcloud 27 Sep 12, 2022
Transparent TLS and HTTP proxy serve and operate on all 65535 ports, with domain regex whitelist and rest api control

goshkan Transparent TLS and HTTP proxy serve & operating on all 65535 ports, with domain regex whitelist and rest api control tls and http on same por

Sina Ghaderi 11 Nov 5, 2022
Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser.

reciva-web-remote Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser. Usage This progr

null 6 May 3, 2022
Uses the Finger user information protocol to open a TCP connection that makes a request to a Finger server

Finger Client This client uses the Finger user information protocol to open a TCP connection that makes a request to a Finger server. Build and Run Ru

Linda Xiao 0 Oct 7, 2021
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Spiral Scout 6.9k Jan 1, 2023
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Spiral Scout 6.1k Dec 9, 2021