Read the text of memes, then inject that text into the image as searchable metadata.

Overview

Make Meme Text Searchable

I have an extensive set of memes I've been collecting since the early days of Flickr. #icanhascheeseburger

It's a pain in the ass to not be able to search my memes (now stored in Apple’s Photos.app) to find what I'm looking for when I need it.

I had something for this.

This project uses Amazon Rekognition to read the text from the images, then go-exif to write the text into the image metadata as a caption/description. Photo apps and services should be able to parse and index this data, making your images (memes, really) searchable by the text that's in the image.

General Program Flow

  1. Read the binary image data into memory.

  2. Rekognition only supports PNG and JPEG formats, so…

    1. If the image is already a PNG or JPEG, skip to the next step.

    2. If the image is a GIF, WEBP, or HEIF format, convert the in-memory representation of the image to JPEG format.

  3. Submit the (PNG or JPEG) bytes of the image to Rekognition.

  4. Get back the results. Merge, deduplicate, and munge the resulting text matches, whatever they are. Words don't always come back in the right order, so think of them less as a sentence and more of a collection of keywords.

    Wait a minute, I had something for this.

    …might become…

    a for had i minute something this wait

  5. Using the EXIF library, write these words into the file (the one we read into memory), into the ImageDescription EXIF field of the metadata.

  6. Optionally, we can:

    1. Overwrite the original file with an updated description (destructive).

    2. Read the file from one location, and write an updated copy to a new location (non-destructive).

      1. This new location can even be a different format, such as JPEG or PNG. Whatever Go's standard library supports.

Why Go?

Go (aka, "Golang") compiles down to a static binary, and is stupidly fast. It can also compile to WebAssembly, which means it can run in Node.js or web browsers. It also has the fastest boot time on AWS Lambda (more-or-less tied for first place with Rust), so creating a POST endpoint should be easy as well.

Someday, I want to learn how to develop mobile apps so that I can solve this user problem. Compiled Go code can be called from Android and iOS.

Progress

Library

  • Importable with go get.
  • Handles image bytestreams and decoding.
  • Handles converting the image to JPEG before passing to Rekognition.
  • Sends data to Rekognition.
  • Parses the results from Rekognition into words.
  • Converts the image bytestream to JPEG before sending to Rekognition.
  • Preserves any existing EXIF data.
  • Writes the words into the EXIF data.

CLI Tool

  • Supports reading a file.
  • Supports reading a directory.
  • Supports reading a glob.
  • Supports verbose logging.
  • Supports AWS credentials as environment variables.
  • Supports AWS credentials as a profile reference.
  • Supports -v.
  • Supports -vv and -vvv.
  • Supports -q.
  • Supports outputting a copy to a new directory.
  • Supports outputting a copy in a new format.
  • Supports writing the Rekognition results into EXIF data at all.
  • Supports status updates for jobs.
  • Supports an index of already-processed images to facilitate restarting a failed queue.

Usage

Library

Incomplete example. Error handling removed for brevity.

import "github.com/skyzyx/make-meme-text-searchable/meme"

func main() {
    // Open the file as am io.Reader.
    r, _ := os.Open("./images/paris-airport.heic")

    // Read the io.Reader, decode the image, then re-encode the image data as JPEG format.
    buf, _, _, _ := meme.ReadImage(r, meme.DefaultJPEGQuality)

    // Pass the image data to AWS Rekognition.
    results, _ := meme.DetectText(ctx, &awsConfig, buf)

    // Sanitize, de-dupe, remove punctuation, and sort the resulting words.
    words := meme.GetSanitizedText(results)

    // Write the string of words back to the image into the EXIF ImageDescription field.
    _ := meme.WriteImageDescription(r, words)
}

CLI

(Brainstorming) Something like…

meme-text [--report=TEXT|JSON] [--out=FILE] [--outdir=DIR] [--outformat=GIF|HEIC|JPG|PNG|WEBP] [--quiet] [--verbose] [--force] INPUT...
  • INPUT is one or more files, directories of files, or globs of files. Supports: GIF, HEIC, JPEG, PNG, WEBP. Also works with STDIN.
  • --report will write data to STDOUT in the specified format.
  • --quiet will silence all output.
  • --verbose maybe be specified up to 3 times, with increasing levels of verbosity. The default value is equivalent to WARNING. -v, -vv, and -vvv are equivalent to INFO, DEBUG, and TRACE (respectively).
  • --force disables any interactive prompts.

Web UI

This should be relatively simple to write as long as people can drag-and-drop/upload their images into the webpage, then provide an email address to send the results to (asychronously). A simple desktop app is also possible — maybe with Electron, Wails, or Tauri?

Things to read

Things I need to read and understand. Apparently writing EXIF data can be non-trivial.

Owner
Ryan Parman
Builder of digital things used by millions. Proud dad. Xennial. Gamer. Ambivert. Curious. Perpetual learner. Not a coffee drinker. Get vaccinated.
Ryan Parman
Hntoebook - Converts the best HN stories to .mobi format to be read using an e-reader

HN to E-Book What? This program converts the best HN stories to .mobi format to

Abishek Muthian 76 May 6, 2022
💾 Wolke API is the API behind Wolke image storage and processing aswell as user management

?? Wolke API Wolke API is the API behind Wolke image storage and processing aswell as user management Deploying To deploy Wolke Bot you'll need podman

wolke.casa 1 Dec 21, 2021
Dedugo - Simple duplicate image finder for golang

Dedugo Image Duplicate Finder De-duplicate in Go (get it?) Summary This simple p

Mike Lloyd 1 May 5, 2022
Convert JPEG images from S3 bucket to BMP, GIF, PNG into another bucket

aws-lambda Convert JPEG images from S3 bucket to BMP, GIF, PNG into another bucket Setup two buckets jpeg-images for source jpeg images converted-jpeg

Volodymyr Tochytskyi 0 Feb 13, 2022
Ydb-go-yc-metadata - Helpers to connect to YDB inside yandex-cloud using metadata service

ydb-go-yc-metadata helpers to connect to YDB inside yandex-cloud using metadata

YDB Platform 2 Apr 19, 2022
View reddit memes and posts from ur terminal with golang and webscraping

goddit View reddit memes and posts from your terminal with golang and webscraping Installation run the following commands on your terminal to install

Swapnadeep Som 11 Feb 22, 2021
Lightweight go web server that provides a searchable directory index.

autoindex Lightweight go web server that provides a searchable directory index. Optimized for handling large numbers of files (100k+) and remote file

Niels AD 20 May 9, 2022
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers

k8s-vault-webhook is a Kubernetes admission webhook which listen for the events related to Kubernetes resources for injecting secret directly from sec

Opstree Container Kit 111 Apr 28, 2022
Hotdog is a set of OCI hooks used to inject the Log4j Hot Patch into containers.

Hotdog Hotdog is a set of OCI hooks used to inject the Log4j Hot Patch into containers. How it works When runc sets up the container, it invokes hotdo

null 33 Apr 26, 2022
Script to inject Aliucord into a Discord ipa.

Aliucord-Patcher Script to patch Discord's ipa with a custom build of hermes, apply a custom icon and apply changes to Info.plist. Usage: go run cmds/

Zoey 3 Jan 15, 2022
Image - This repository holds supplementary Go image librariesThis repository holds supplementary Go image libraries

Go Images This repository holds supplementary Go image libraries. Download/Insta

null 0 Jan 5, 2022
k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically. k8s-image-swapper is a mutating webhook for Kubernetes

Enrico Stahn 300 May 1, 2022
Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Packiffer is a lightweight cross-platform networking toolkit that let you sniff/analyze/inject/filter packets.

Massoud Asadi 52 Apr 13, 2022
K8s_dns_chaos: enables inject DNS chaos in a Kubernetes cluster for Chaos Engineering

k8s_dns_chaos Name k8s_dns_chaos - enables inject DNS chaos in a Kubernetes cluster for Chaos Engineering. Description This plugin implements the Kube

邢凯 0 Dec 12, 2021
Simple-read-file - Example of how to read file in Go

simple-read-file This repository contains a simple example of how to read file i

Yessy Purnama 0 Jan 11, 2022
Litestream-read-replica-demo - A demo application for running live read replication on fly.io with Litestream

Litestream Read Replica Demo A demo application for running live read replicatio

Ben Johnson 59 May 10, 2022
Golang wrapper for Exiftool : extract as much metadata as possible (EXIF, ...) from files (pictures, pdf, office documents, ...)

go-exiftool go-exiftool is a golang library that wraps ExifTool. ExifTool's purpose is to extract as much metadata as possible (EXIF, IPTC, XMP, GPS,

null 112 May 16, 2022
ID3, MP4 and OGG/FLAC metadata parsing in Go

MP3/MP4/OGG/FLAC metadata parsing library This package provides MP3 (ID3v1,2.{2,3,4}) and MP4 (ACC, M4A, ALAC), OGG and FLAC metadata detection, parsi

David Howden 377 May 9, 2022
You had one job, or more then one, which can be done in steps

Leprechaun Leprechaun is tool where you can schedule your recurring tasks to be performed over and over. In Leprechaun tasks are recipes, lets observe

Strahinja 86 May 7, 2022
Compact, sortable and fast unique IDs with embedded metadata.

A spec for unique IDs in distributed systems based on the Snowflake design, i.e. a coordination-based ID variant. It aims to be friendly to both machi

null 62 Mar 21, 2022
QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enable you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to CSV file

Mohamed Shapan 100 Dec 22, 2021
Get any cryptocurrencies ticker and trade data in real time from multiple exchanges and then save it in multiple storage systems.

Cryptogalaxy is an app which will get any cryptocurrencies ticker and trade data in real time from multiple exchanges and then saves it in multiple storage systems.

Pavan Shetty 94 Apr 28, 2022
DipDup plugin for selective metadata indexing

DipDup metadata indexer DipDup service for indexing contract and token metadata. Based on TzKT indexer. For start synchronization of DipDup state TzKT

DipDup 13 Mar 25, 2022
Download Vimeo videos and retrieve metadata in Go.

vimego Download Vimeo videos and retrieve metadata. Largely based on yashrathi's vimeo_downloader. Installing go get github.com/raitonoberu/vimego Ple

Denis 6 Feb 10, 2022
A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes

sigurlscann3r A web application attack surface mapping tool. It takes in a list of urls then performs numerous probes Resources Features Installation

Signed Security 8 Nov 5, 2021
A tiny Go library + client for downloading Youtube videos. The library is capable of fetching Youtube video metadata, in addition to downloading videos.

A tiny Go library + client (command line Youtube video downloader) for downloading Youtube videos. The library is capable of fetching Youtube video metadata, in addition to downloading videos. If ffmpeg is available, client can extract MP3 audio from downloaded video files.

Kunal Diwan 3 Apr 30, 2022
A youtube library for retrieving metadata, and obtaining direct links to video-only/audio-only/mixed versions of videos on YouTube in Go.

A youtube library for retrieving metadata, and obtaining direct links to video-only/audio-only/mixed versions of videos on YouTube in Go. Install go g

José Pastor 3 Oct 22, 2021
A simple Hasura client for query and metadata APIs

Hasura Client A Hasura client for schema APIs https://hasura.io/docs/latest/graphql/core/api-reference/index.html Usage Create client client := hasura

Toan Nguyen 0 Oct 31, 2021