Publish Your GIS Data(Vector Data) to PostGIS and Geoserver

Overview

Go Report Card GitHub license GitHub issues Coverage Status Build Status Documentation GitHub forks GitHub stars Twitter

GISManager

Publish Your GIS Data(Vector Data) to PostGIS and Geoserver

  • How to install:
    • go get -v github.com/hishamkaram/gismanager
  • Usage:
    • testdata folder content:
      ./testdata/
      ├── neighborhood_names_gis.geojson
      ├── nested
      │   └── nyc_wi-fi_hotspot_locations.geojson
      ├── sample.gpkg
      
    • create ManagerConfig instance:
      manager:= gismanager.ManagerConfig{
        Geoserver: gismanager.GeoserverConfig{WorkspaceName: "golang", Username: "admin", Password: "geoserver", ServerURL: "http://localhost:8080/geoserver"},
        Datastore: gismanager.DatastoreConfig{Host: "localhost", Port: 5432, DBName: "gis", DBUser: "golang", DBPass: "golang", Name: "gismanager_data"},
        Source:    gismanager.SourceConfig{Path: "./testdata"},
        logger:    gismanager.GetLogger(),
      }
      
    • get Supported GIS Files:
      files, _ := gismanager.GetGISFiles(manager.Source.Path)
      for _, file := range files {
        fmt.Println(file)
      }
      
      • output:
        <full_path>/testdata/neighborhood_names_gis.geojson
        <full_path>/testdata/nested/nyc_wi-fi_hotspot_locations.geojson
        <full_path>/testdata/sample.gpkg
        
    • read files and get layers Schema:
        for _, file := range files {
          source, ok := manager.OpenSource(file, 0)
          if ok {
            for index := 0; index < source.LayerCount(); index++ {
              layer := source.LayerByIndex(index)
              gLayer := gismanager.GdalLayer{
                Layer: &layer,
              }
              fmt.Println(layer.Name())
              for _, f := range gLayer.GetLayerSchema() {
                fmt.Printf("\n%+v\n", *f)
              }
            }
          }
        }
      
      • output sample:
        neighborhood_names_gis
        
        {Name:geom Type:POINT}
        
        {Name:stacked Type:String}
        
        {Name:name Type:String}
        
        {Name:annoline1 Type:String}
        
        {Name:annoline3 Type:String}
        
        {Name:objectid Type:String}
        
        {Name:annoangle Type:String}
        
        {Name:annoline2 Type:String}
        
        {Name:borough Type:String}
        ...
        
    • add your gis data to your database:
        for _, file := range files {
            source, ok := manager.OpenSource(file, 0)
            targetSource, targetOK := manager.OpenSource(manager.Datastore.BuildConnectionString(), 1)
            if ok && targetOK {
              for index := 0; index < source.LayerCount(); index++ {
                layer := source.LayerByIndex(index)
                gLayer := gismanager.GdalLayer{
                  Layer: &layer,
                }
                newLayer, postgisErr := gLayer.LayerToPostgis(targetSource, manager, true)
                if postgisErr != nil {
                  panic(postgisErr)
                }
                logger.Infof("Layer: %s added to you database", newLayer.Name())
              }
            }
        }
      
      • output:
        INFO[14-10-2018 17:28:37] Layer: neighborhood_names_gis added to you database 
        INFO[14-10-2018 17:28:38] Layer: nyc_wi_fi_hotspot_locations added to you database 
        INFO[14-10-2018 17:28:38] Layer: hwy_patrol added to you database
        
    • update the previous code to publish your postgis layers to geoserver
       for _, file := range files {
         source, ok := manager.OpenSource(file, 0)
         targetSource, targetOK := manager.OpenSource(manager.Datastore.BuildConnectionString(), 1)
         if ok && targetOK {
           for index := 0; index < source.LayerCount(); index++ {
             layer := source.LayerByIndex(index)
             gLayer := gismanager.GdalLayer{
               Layer: &layer,
             }
             if newLayer, postgisErr := gLayer.LayerToPostgis(targetSource, manager, true); newLayer.Layer != nil || postgisErr != nil {
               ok, pubErr := manager.PublishGeoserverLayer(newLayer)
               if pubErr != nil {
                 logger.Error(pubErr)
               }
               if !ok {
                 logger.Error("Failed to Publish")
               } else {
                 logger.Info("published")
               }
             }
      
           }
         }
       }
      
      • output:
       INFO[14-10-2018 17:37:07] url:http://localhost:8080/geoserver/rest/workspaces/golang  Status=404  
       ERRO[14-10-2018 17:37:07] No such workspace: 'golang' found            
       INFO[14-10-2018 17:37:07] url:http://localhost:8080/geoserver/rest/workspaces  Status=201  
       INFO[14-10-2018 17:37:07] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis?quietOnNotFound=true  Status=404  
       INFO[14-10-2018 17:37:07] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores  Status=201  
       ERRO[14-10-2018 17:37:07] {"featureType":{"name":"neighborhood_names_gis","nativeName":"neighborhood_names_gis"}} 
       INFO[14-10-2018 17:37:07] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis/featuretypes  Status=201  
       INFO[14-10-2018 17:37:07] published                                    
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang  Status=200  
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis?quietOnNotFound=true  Status=200  
       ERRO[14-10-2018 17:37:08] {"featureType":{"name":"nyc_wi_fi_hotspot_locations","nativeName":"nyc_wi_fi_hotspot_locations"}} 
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis/featuretypes  Status=201  
       INFO[14-10-2018 17:37:08] published                                    
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang  Status=200  
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis?quietOnNotFound=true  Status=200  
       ERRO[14-10-2018 17:37:08] {"featureType":{"name":"hwy_patrol","nativeName":"hwy_patrol"}} 
       INFO[14-10-2018 17:37:08] url:http://localhost:8080/geoserver/rest/workspaces/golang/datastores/gis/featuretypes  Status=201  
       INFO[14-10-2018 17:37:08] published 
      
      • done check you geoserver or via geoserver rest api url http://localhost:8080/geoserver/rest/layers.json :
         {
           "layers": {
             "layer": [..., {
               "name": "golang:hwy_patrol",
               "href": "http:\/\/localhost:8080\/geoserver\/rest\/layers\/golang%3Ahwy_patrol.json"
             }, {
               "name": "golang:neighborhood_names_gis",
               "href": "http:\/\/localhost:8080\/geoserver\/rest\/layers\/golang%3Aneighborhood_names_gis.json"
             }, {
               "name": "golang:nyc_wi_fi_hotspot_locations",
               "href": "http:\/\/localhost:8080\/geoserver\/rest\/layers\/golang%3Anyc_wi_fi_hotspot_locations.json"
             }]
           }
         }
        

Todo:

  • backup postgis as geopackage
Issues
  • Compile Issues

    Compile Issues

    go get -v github.com/hishamkaram/gismanager

    ../../../../pkg/mod/github.com/hishamkaram/[email protected]/layer.go:131:29: cannot use index (type int) as type int64 in argument to layer.Layer.Feature ../../../../pkg/mod/github.com/hishamkaram/[email protected]/utils.go:104:20: invalid method expression archiver.Zip.Open (needs pointer receiver: (*archiver.Zip).Open)

    go version go version go1.17.1 darwin/amd64

    Thanks, Migael

    opened by migaelhartzenberg 0
Owner
Hisham waleed karam
A highly focused software developer. Organised, methodical and a keen eye for detail results in solid coding and trustworthy software programmes.
Hisham waleed karam
Go package to quick and easy create json data in geojson format.

#GEOJSON Go package to easy and quick create datastructure which can be serialized to geojson format INSTALLATION $ go get github.com/kpawlik/geojson

Kris Pawlik 50 Jun 6, 2022
General purpose library for reading, writing and working with OpenStreetMap data

osm This package is a general purpose library for reading, writing and working with OpenStreetMap data in Go (golang). It has the ability to read OSM

Paul Mach 213 Jun 23, 2022
A library provides spatial data and geometric algorithms

Geoos Our organization spatial-go is officially established! The first open source project Geoos(Using Golang) provides spatial data and geometric alg

spatial-go 528 Jun 29, 2022
Go (golang) wrapper for GDAL, the Geospatial Data Abstraction Library

------------- About ------------- The gdal.go package provides a go wrapper for GDAL, the Geospatial Data Abstraction Library. More information about

null 223 Jul 1, 2022
Encoding and decoding GeoJSON <-> Go

go.geojson Go.geojson is a package for encoding and decoding GeoJSON into Go structs. Supports both the json.Marshaler and json.Unmarshaler interfaces

Paul Mach 205 Jun 24, 2022
Package kml provides convenince methods for creating and writing KML documents.

go-kml Package kml provides convenience methods for creating and writing KML documents. Key Features Simple API for building arbitrarily complex KML d

Tom Payne 67 May 31, 2022
Package polyline implements a Google Maps Encoding Polyline encoder and decoder.

go-polyline Package polyline implements a Google Maps Encoding Polyline encoder and decoder. Encoding example func ExampleEncodeCoords() { coords :=

Tom Payne 68 Jun 17, 2022
Types and utilities for working with 2d geometry in Golang

orb Package orb defines a set of types for working with 2d geo and planar/projected geometric data in Golang. There are a set of sub-packages that use

Paul Mach 545 Jun 27, 2022
Real-time Geospatial and Geofencing

Tile38 is an open source (MIT licensed), in-memory geolocation data store, spatial index, and realtime geofence. It supports a variety of object types

Josh Baker 8.2k Jul 3, 2022
Publish Your GIS Data(Vector Data) to PostGIS and Geoserver

GISManager Publish Your GIS Data(Vector Data) to PostGIS and Geoserver How to install: go get -v github.com/hishamkaram/gismanager Usage: testdata fol

Hisham waleed karam 45 Jun 16, 2022
geoserver is a Go library for manipulating a GeoServer instance via the GeoServer REST API.

Geoserver geoserver Is a Go Package For Manipulating a GeoServer Instance via the GeoServer REST API. How to install: go get -v gopkg.in/hishamkaram/g

Hisham waleed karam 74 Jun 16, 2022
geoserver is a Go library for manipulating a GeoServer instance via the GeoServer REST API.

Geoserver geoserver Is a Go Package For Manipulating a GeoServer Instance via the GeoServer REST API. How to install: go get -v gopkg.in/hishamkaram/g

Hisham waleed karam 74 Jun 16, 2022
Publish posts to your personal facebook profile.

FB Post Publish posts to your personal facebook profile. Usage Get list of codes for 2FA. then copy them and paste them in txt/codes.txt use the comma

Mohamed Samir 2 Jan 13, 2022
publish a tree-like data structure from a backend to a front-end

tree-publish publish a tree-like data structure from a backend to a front-end. It needs to be a tree in order to publish the data as JSON document. If

Lutz Behnke 0 Dec 20, 2021
An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy

Click to take a quick look at our demos! Image search Chatbots Chemical structure search Milvus is an open-source vector database built to power AI ap

The Milvus Project 11.1k Jun 26, 2022
ready-to-use RTSP / RTMP server and proxy that allows to read, publish and proxy video and audio streams

rtsp-simple-server is a simple, ready-to-use and zero-dependency RTSP / RTMP server and proxy, a software that allows users to publish, read and proxy live video and audio streams. RTSP is a specification that describes how to perform these operations with the help of a server, that is contacted by both publishers and readers and relays the publisher's streams to the readers.

Alessandro Ros 3.3k Jun 29, 2022
Rasterx is an SVG 2.0 path compliant rasterizer that can use either the golang vector or a derivative of the freetype anti-aliaser.

rasterx Rasterx is a golang rasterizer that implements path stroking functions capable of SVG 2.0 compliant 'arc' joins and explicit loop closing. Pat

Steven R Wiley 97 Jun 11, 2022
Cairo in Go: vector to SVG, PDF, EPS, raster, HTML Canvas, etc.

Canvas is a common vector drawing target that can output SVG, PDF, EPS, raster images (PNG, JPG, GIF, ...), HTML Canvas through WASM, and OpenGL. It h

Taco de Wolff 986 Jun 22, 2022
Static bit vector structures in Go

teivah/bitvector Overview A bit vector is an array data structure that compactly stores bits. This library is based on 5 static different data structu

Teiva Harsanyi 72 Jan 24, 2022
Vald. A Highly Scalable Distributed Vector Search Engine

Vald is a highly scalable distributed fast approximate nearest neighbor dense vector search engine.

Vector Data as a Service 946 Jun 28, 2022
Weaviate is a cloud-native, modular, real-time vector search engine

Weaviate is a cloud-native, real-time vector search engine (aka neural search engine or deep search engine). There are modules for specific use cases such as semantic search, plugins to integrate Weaviate in any application of your choice, and a console to visualize your data.

SeMI Technologies 2.5k Jun 24, 2022
Weaviate is a cloud-native, modular, real-time vector search engine

Weaviate is a cloud-native, real-time vector search engine (aka neural search engine or deep search engine). There are modules for specific use cases such as semantic search, plugins to integrate Weaviate in any application of your choice, and a console to visualize your data.

SeMI Technologies 2.5k Jun 24, 2022
GoVector is a vector clock logging library written in Go.

GoVector is a vector clock logging library written in Go. The vector clock algorithm is used to order events in distributed systems in the absence of a centralized clock. GoVector implements the vector clock algorithm and provides feature-rich logging and encoding infrastructure.

Distributed clocks 163 Jun 18, 2022
Red team tool that emulates the SolarWinds CI compromise attack vector.

SolarSploit Sample malicious program that emulates the SolarWinds attack vector. Listen for processes that use the go compiler Wait for a syscall to o

TestifySec 18 Feb 11, 2022
A Go package converting a monochrome 1-bit bitmap image into a set of vector paths.

go-bmppath Overview Package bmppath converts a monochrome 1-bit bitmap image into a set of vector paths. Note that this package is by no means a sophi

tunabay 2 Mar 22, 2022
IntSet - Integer based Set based on a bit-vector

IntSet - Integer based Set based on a bit-vector Every integer that is stored will be converted to a bit in a word in which its located. The words are

Jakob Möller 0 Feb 2, 2022
Generate vector tiles for the entire planet on relatively low spec hardware.

Sequentially Generate Planet Mbtiles Sequentially generate and merge an entire planet.mbtiles vector tileset on low memory/power devices for free. com

Jack Bizzell 18 Jun 24, 2022
High performance, distributed and low latency publish-subscribe platform.

Emitter: Distributed Publish-Subscribe Platform Emitter is a distributed, scalable and fault-tolerant publish-subscribe platform built with MQTT proto

emitter 3.3k Jun 26, 2022
command-line tool to publish, subscribe, and process messages for AMQP 0.9.1 compliant message brokers

Bunny A BSD licenced, go-powered CLI tool for publishing and subscribing to RabbitMQ

SkunkWerks GmbH 3 Sep 11, 2021