Plant environment data sensor aggregation / UI server.

Overview

Build Status Go Report Card GoDoc blog

asdf

What?

A pet project to design a simple garden sensor and custom server for aggregating data and providing a simple user interface.

  • Nginx gateway for scaling, static file hosting, and SSL termination
  • Golang API backend
  • ReactJS single page web app
  • Postgres database
  • Docker driven integration tests and deployment
  • More buzzwords as I can add them in, sorry no Haskell in the foreseeable future

Why?

The intent is to merge my interests in sustainability, backend system software development, embedded programming, and electronics tinkering while providing the opportunity to practice with programming techniques I may not get to use as much as I like during my day-job.

Where?

Running @ https://freyr.erdmanczyk.com

![login](repo_files/Screenshot 2016-05-09 23.17.39.png "Freyr") ![latest](repo_files/Screenshot 2016-05-09 23.17.48.png "Freyr") ![chart](repo_files/Screenshot 2016-05-09 23.18.04.png "Freyr")

How?

Want to run this on your own machine? There's a bit of setup; follow the steps below:

Install prerequisites

You will first need to install the prerequisite tools:

  1. Go
  2. Docker
  3. Docker-Compose
  4. Webpack (optional, see below)

When possible, I highly reccomend installing using a package manager such as homebrew for mac.

Generate a Google Oauth Id/Secret

These are necessary for Google oauth login within the app to work. Generating these and configuring Google oauth is beyond the scope of this how-to, but this guide is a good comprehensive walk-through.

You don't need to worry about updating any code in the repo, you just need to worry about:

  • Creating your Google 'app'
  • Generating your id/secret
  • Adding your allowed callbacks
    • The Golang server tells Google where to redirect users when they've authorized, Google rejects url's you haven't configured as allowed.
    • For running locally you will need a url with localhost or <docker_vm_ip> (if running docker in a VM) as the domain name
    • For production, of course, you will need a url with your server's domain name.

Generate configuration

This app uses environment variables for configuration in containers. Docker-Compose loads these from .env files. I didn't include these with the repo because I didn't want to encourage cloning, running, and then forgetting you're running with potentially insecure values.

Instead, I created a tool to manage config.

After you've setup your $GOPATH, etc. take the following steps:

  1. Clone this repo to $GOPATH/src/github.com/serdmanczyk/freyr
  2. cd to cmd/init_freyr/ in the project and run go build
  3. ./init_freyr takes some flags for config values, run with -h to see them. With the flags provided it will automatically generate needed config files, including the nginx configuration.

Generate a server key/certificate

  1. Run ./gencert.sh in cmd/init_freyr/ to generate a key/certificate. It uses openssl, so you will need that installed.

OpenSSL will prompt you for some values, you can goof around on these if you want. For good practice make sure Common Name (e.g. server FQDN or YOUR name) matches the domain name and/or ip (e.g. docker VM ip) you will use to connect to your server.

These are just for local testing, and modern browsers such as chrome will warn it can't verify its authenticity. Once you're in production I highly reccomend using something such as letsencrypt to obtain a verified CA signed certificate.

A note on 'make'

From this point forth I'll be using a Makefile in the repo with common commands for convenience (especially for me, not just other users). If you're not in an environment with make available you can reference the file for the convenienced commands.

Build ReactJS frontend with webpack (optional)

If you plan to make modifications to the frontend, you will need webpack to run this step. If you just want to get up and running you can skip this step as there is a pre-generated bundle.js in the repo.

  1. From project root run make webpack. This will build a 'production' bundle.js file from the React .jsx files.

Run tests

  1. From project root run make buildgo then make test, this will run several commands building go executables and containers as well as running integration and acceptance tests.

If tests pass this verifies that all systems are compiling and functioning correctly together. We are ready to run the app!

Run the app!

There are two ways to run the app: production and dev mode. Production builds all dependencies into the container so it can be shipped to separate machines. Dev mounts in local directories for things such as static web files to enable live reloading and speed up container spin-up/spin-down.

Note if you are running docker in a VM (e.g. mac), you will need to use your docker VM ip:<port> to access the app.

Development

  1. Run make rundev
  2. Navigate to https://<localhost/docker_vm_ip>:8443 in your browser!

Bonus: Because this mounts in directories for static files, if you are editing React files you can run webpack --watch in the nginx/static directory and it will hot reload any changes you make. This was you don't have to constantly rebuild the containers.

Note: dev nginx is configured to listen on 8080/8443 (HTTP/HTTPS). Because of this http->https redirects will fail, because https will by default redirect to 443.

'Production'

  1. Run make buildstatic then make runstatic
  2. Navigate to https://<localhost/docker_vm_ip> in your browser!

To deploy to a server, you will need to run the generate conf steps again but with your server's hostname.

Etymology

Freyr (Fray-ur), the Norse god associated with fertility, sunshine, and fair weather. The CLI testing tool is named after Surtr, the fire giant who battles Freyr to his death at Ragnarök.

The symbol is the Futhark ᚠ (Fehu) (Norse 'F') imposed on a leaf, resembling the leaf's veins. Seen above, it takes the place of the 'F' in Freyr.

Issues
  • Error when running docker-compose build

    Error when running docker-compose build

    ERROR: Couldn't find env file: /home/oren/p/Freyr/.env
    

    Is that's the correct way to run the project? Also the demo page (https://freyr.erdmanczyk.com/latest) shows no data.

    opened by oren 6
  • Travis CI build is failing

    Travis CI build is failing

    The command "sudo apt-get -o Dpkg::Options::="--force-confnew" install -y docker-engine=${DOCKER_VERSION}" failed and exited with 100 during .
    

    It appears the command to install docker is outdated; need to research and update.

    opened by serdmanczyk 1
  • Navigating to url's when not logged in will show blank page

    Navigating to url's when not logged in will show blank page

    The React app doesn't currently verify the user is authenticated on all pages, leading to some buggy behavior when trying to access pages while not logged in. Research making more intuitive.

    To repro, navigate to https://freyr.erdmanczyk.com/latest when not logged in.

    opened by serdmanczyk 1
Releases(v2.0)
  • v2.0(Aug 22, 2017)

    Migration to Traefik

    Benefits:

    • Dynamically watches container labels and can route subdomains to new projects on the fly
    • Automatically renews LetsEncrypt certificates

    Why drop Nginx?

    Nginx may be the proper enterprise choice, but using Traefik for my hobby projects will save me an enormous amount of time (read: almost no more time jiggering reverse-proxy configuration).

    I can focus almost entirely on code now.

    Also Changed

    • Dropped Travis-CI. Not worth the upkeep for solo project (I just run tests local, verify on host)
    • Docker compose files re-organized
    • Added job scheduling backend via bifrost project
    • Added ability to post many readings at once (schedules query-able background job via bifrost)
    Source code(tar.gz)
    Source code(zip)
  • v1.0(Aug 22, 2017)

Owner
Steven James Erdmanczyk Jr
Steven James Erdmanczyk Jr
Extract structured data from web sites. Web sites scraping.

Dataflow kit Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You

Dmitry Narizhnykh 539 Aug 10, 2022
A golang framework helps gopher to build a data visualization and admin panel in ten minutes

the missing golang data admin panel builder tool. Documentation | 中文介绍 | DEMO | 中文DEMO | Twitter | Forum Inspired by laravel-admin Preface GoAdmin is

GoAdmin 6.2k Aug 12, 2022
Gazer system is used to track data.

Gazer System - 凝视系统 这是一个用于长期追踪数据变化的工具。 我们常需要定时针对某些链接进行爬取,以获取历史数据,这个系统为此而生。这是一组无状态的服务,意味着它可以简单地水平扩展。它提供了如下几个功能: 自动代理:使用洋葱路由进行代理,无需自行维护 IP 池,也可以绕过大多数基于

Jannchie 8 Jul 25, 2022
An application for generating Microsoft Word resumes from JSON Resume data files

ResumeFodder NOTE: Primary development has moved over to GitLab: https://github.com/andrzejressel/ResumeFodder. If you're reading this on GitHub, then

​Andrzej Ressel 1 Nov 25, 2021
Uploads data to the Storj ecosystem and tries to download induvidual pieces from different servers to measure latency

Storj longtail This application uploads data to the Storj ecosystem and tries to download induvidual pieces from different servers to measure latency.

Elek, Márton 0 Oct 28, 2021
A simple RESTful API for storing and retrieving data during MRI image reconstructions

MRD Storage Server The MRD Storage Server provides a simple RESTful API for storing and retrieving data during MRI image reconstructions. It supports:

null 0 Mar 18, 2022
A simple app that reads NBMiner status REST API data and sends it to InfluxDB

NBMiner Reporter A simple Go app that reads NBMiner status REST API data and sends it to InfluxDB. Usage Using the reporter is quite easy, specially i

null 3 Feb 8, 2022
GoAdmin is a toolkit to help you build a data visualization admin panel for your golang app.

the missing golang data admin panel builder tool. Documentation | 中文文档 | 中文介绍 | DEMO | 中文DEMO | Twitter | Forum Inspired by laravel-admin Preface GoAd

null 1 Nov 25, 2021
Rental-api - A RESTful-API that allows developers to connect to data about rental properties

Rentals-API is a RESTful-API that allows developers to connect to data about rental properties.

null 0 Jan 24, 2022
IP geolocation web server

freegeoip NOTE: as of April 2018 this repository is no longer active. Please visit https://github.com/apilayer/freegeoip/ for the current version. Thi

Alexandre Fiori 4.9k Aug 12, 2022
A quick and easy password protected web server for your files. httpfolder makes downloading/uploading files from your current working directory easy, even for fairly large files.

httpfolder A quick and easy password protected web server for your files. httpfolder makes downloading/uploading files from your current working direc

null 14 Dec 13, 2021
Go Server/API boilerplate using best practices DDD CQRS ES gRPC

Go Server/API boilerplate using best practices DDD CQRS ES gRPC

Rafał Lorenz 646 Aug 11, 2022
An RRDtool api server that exports a subset of rrdtool commands over http(s).

rrdsrv An RRDtool api server that exports a subset of rrdtool commands over http(s). Be sure to checkout the grafana plugin. Demo video. Usage Usage o

null 7 Aug 10, 2022
Server and bookmarklet to download files via youtube-dl directly from your browser

gropple A web service and bookmarklet to download videos with a single click. Pre-requisites a passing familiarity with the command line youtube-dl (p

Justin Hawkins 28 Aug 10, 2022
Compose sample application Go server with an Nginx proxy and a Postgres database

Compose sample application Go server with an Nginx proxy and a Postgres database Project

null 1 Apr 7, 2022
Very simple REST server written in Go.

Simple Web API written in Go For practice and learning sake. .env is a dummy file with no real credentials. Endpoints GET /books - get all books; GET

null 1 Oct 18, 2021
A simple RESTful server with golang

A simple RESTful server Setup local development environment cp develop.env .env docker-compose up -d docker-compose exec mysql mysql -u root --passwor

null 0 Oct 28, 2021
REST server to build and install multicloud environments based on shared.configV2

builder A REST server to build and install multicloud environments based on shared.configV2 json For configuration, please see the Documentation Below

null 0 Nov 2, 2021
A basic example of a go web server with a react frontend

GO-React starter This is a basic example of a go web server with a react frontend. It uses the go fiber framework Getting started Running locally Clon

Zidian Lyu 0 Nov 16, 2021