💧 Visual Data Preparation (VDP) is an open-source tool to seamlessly integrate Vision AI with the modern data stack

Overview

Instill AI - Visual Data Preparation Made for All

Website | Community | Blog

Get Early Access


Visual Data Preparation (VDP) is an open-source tool to streamline the end-to-end visual data processing pipeline:

  1. Ingest unstructured visual data from data sources such as data lakes or IoT devices;
  2. Transform visual data to meaningful structured data representations by Vision AI models;
  3. Load the structured data into warehouses, applications, or other destinations.

The goal of VDP is to seamlessly bring Vision AI into the modern data stack with a standardised framework. Check our blog post Missing piece in modern data stack: visual data preparation on how this tool is proposed to streamline unstructured visual data processing across different stakeholders.

Table of contents

Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.

How VDP works

The core concept of VDP is pipeline. A pipeline is an end-to-end workflow that automates a sequence of tasks to process visual data. Each pipeline consists of three ordered components:

  1. data source: where the pipeline starts. It connects the source of image and video data to be processed.
  2. model: a deployed Vision AI model to process the ingested visual data and generate structured outputs
  3. data destination: where to send the structured outputs

Based on the mode of a pipeline, it will ingest and process the visual data, send the outputs to the destination every time the trigger event occurs.

We use data connector as a general term to represent data source and data destination. Please find the supported data connectors here.

Quick start

Download and run VDP locally

Execute the following commands to start pre-built images with all the dependencies:

$ git clone https://github.com/instill-ai/vdp.git && cd vdp

# Build instill/vdp:dev local development image
$ make build

# Launch all services.
$ make all

⚠ī¸ Downloading the Triton server image will take a while, but it should be just a one-time effort.

Run the samples to trigger an object detection pipeline

We provide sample codes on how to build and trigger an object detection pipeline. Run it with the local VDP:

$ cd examples-go

# Download a YOLOv4 ONNX model for object detection task (GPU not required)
$ curl -o yolov4-onnx-cpu.zip https://artifacts.instill.tech/vdp/sample-models/yolov4-onnx-cpu.zip

# [optional] Download a test image or use your own images
$ curl -o dog.jpg https://artifacts.instill.tech/dog.jpg

# Deploy the model
$ go run deploy-model/main.go --model-path yolov4-onnx-cpu.zip --model-name yolov4

# Test the model
$ go run test-model/main.go --model-name yolov4 --test-image dog.jpg

# Create an object detection pipeline
$ go run create-pipeline/main.go --pipeline-name hello-pipeline --model-name yolov4

# Trigger the pipeline by using the same test image
$ go run trigger-pipeline/main.go --pipeline-name hello-pipeline --test-image dog.jpg

Create a pipeline with your own models

Please follow the guideline "Prepare your own model to deploy on VDP ". Based on the above sample codes, you can deploy a prepared model and create your own pipeline.

Clean up

To clean up all running services:

$ make prune

Documentation

The gRPC protocols in protobufs provide the single source of truth for the VDP APIs. To view the generated OpenAPI spec on http://localhost:3000:

$ make doc

Community support

For general help using VDP, you can use one of these channels:

  • GitHub (bug reports, feature requests, project discussions and contributions)
  • Discord (live discussion with the community and the Instill AI Team)

License

See the LICENSE file for licensing information.

Issues
  • docs(pipeline): add SYNC and ASYNC diagram and section for connectors

    docs(pipeline): add SYNC and ASYNC diagram and section for connectors

    Because

    • we need a document for users to get to know more about how the pipeline works

    This commit

    • add SYNC and ASYNC diagram and section for connectors
    • close #52
    opened by bochengyang 7
  • Wrong model type

    Wrong model type

    The model type is mistakenly set as tensorrt. https://github.com/instill-ai/vdp/blob/814ea12e94b7ebb4656c7208f5c160c646f8523a/examples-go/deploy-model/main.go#L65

    bug 
    opened by pinglin 5
  • test: update integration test

    test: update integration test

    Because

    • need integration test to make sure feature work correctly

    This commit

    • add integration test for model-backend and update for pipeline-backend
    • closes #67
    • closes #69

    Limitation: error code is not covered in this PR. It will be added during working this ticket

    Note: docker image for model-backend is dev which should update when have a new release

    opened by Phelan164 4
  • chore: add pipeine-backend integration test and refactor example and docker-compose

    chore: add pipeine-backend integration test and refactor example and docker-compose

    Because

    • we need an integration test for our ci
    • there is a wrong depend configuration in docker-compose
    • refactor example to be more self-contained

    This commit

    • add condition when waiting for pg_sql ready
    • add pipeline integration test by k6
    • add argument when creating pipeline and correct README.md
    opened by bochengyang 3
  • [doc]: There has no make build command

    [doc]: There has no make build command

    Issue

    • According to the quick start of the repo: if you want to develop locally, you could do the following
    $ git clone https://github.com/instill-ai/vdp.git && cd vdp
    
    # Build instill/vdp:dev local development image
    $ make build
    
    # Launch all services.
    $ make all
    

    But actually, there has no make build command in the Makefile

    https://github.com/instill-ai/vdp/blob/main/Makefile#L15

    Only exist make dev and make all.

    Solution

    Please update the readme quick start guideline.

    documentation 
    opened by EiffelFly 2
  • [release] v0.1.4-alpha

    [release] v0.1.4-alpha

    Protobufs

    • [x] https://github.com/instill-ai/protobufs/issues/17
    • [x] https://github.com/instill-ai/protobufs/issues/33

    model-backend

    • [x] https://github.com/instill-ai/model-backend/issues/44
    • [x] https://github.com/instill-ai/model-backend/issues/33
    • [x] https://github.com/instill-ai/model-backend/issues/45
    • [x] https://github.com/instill-ai/model-backend/issues/30

    pipeline-backend

    • [x] https://github.com/instill-ai/pipeline-backend/issues/32
    • [x] https://github.com/instill-ai/pipeline-backend/issues/33

    vdp

    • [x] Use the latest docker images in vdp
    next-release 
    opened by xiaofei-du 2
  • Quick start sample code bug

    Quick start sample code bug

    Deploy the model

    If I run the following script in quick start

    # Deploy the model
    go run deploy-model/main.go --model-path yolov4-onyx-cpu.zip --model-name yolov4
    

    I get a model with model name yolov4 with 1 version.

    2022/02/21 00:16:13 model has been created, the response is: id:1 name:"yolov4" full_Name:"local-user/yolov4" cv_task:DETECTION versions:{version:1 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402547 nanos:80792000} updated_at:{seconds:1645402547 nanos:80807000}}

    The response missed the status of version 1 of the model.

    If I run the above script the second time, I get a model with model name yolov4 with 2 versions.

    bash 2022/02/21 00:20:07 model has been created, the response is: id:1 name:"yolov4" full_Name:"local-user/yolov4" cv_task:DETECTION versions:{version:1 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402547 nanos:80792000} updated_at:{seconds:1645402625 nanos:977017000} status:ONLINE} versions:{version:2 model_id:1 description:"YoloV4 for object detection" created_at:{seconds:1645402779 nanos:961661000} updated_at:{seconds:1645402779 nanos:961692000}}

    The response only includes the status of version 1, but no status of version 2.

    Test the model

    # Test the model
    go run test-model/main.go --model-name yolov4 --test-image dog.jpg --model-version 2
    

    Get response

    2022/02/21 00:24:15 error when triggering predict: rpc error: code = Code(400) desc = {"status":400,"title":"PredictModel","detail":"Model is offline"}
    

    Note: shouldn't we use status code 422 instead of 400 for the above scenario?

    But when GET /models/yolov4

    {
        "id": 1,
        "name": "yolov4",
        "full_Name": "local-user/yolov4",
        "cv_task": "DETECTION",
        "versions": [
            {
                "version": 1,
                "model_id": 1,
                "description": "YoloV4 for object detection",
                "created_at": "2022-02-21T00:15:47.080792Z",
                "updated_at": "2022-02-21T00:20:09.486272Z",
                "status": "ONLINE"
            },
            {
                "version": 2,
                "model_id": 1,
                "description": "YoloV4 for object detection",
                "created_at": "2022-02-21T00:19:39.961661Z",
                "updated_at": "2022-02-21T00:20:09.486272Z",
                "status": "ONLINE"
            }
        ]
    }
    

    The response shows both model versions are online.

    bug hot-fix 
    opened by xiaofei-du 2
  • docs: refactor doc structure

    docs: refactor doc structure

    Because

    • The root README.md is too long and needs to be restructured

    This commit

    • replace trigger mechanism of a data source with pipeline mode concept and add the docs in docs/pipeline-mode.md.
    • move prepare-you-own-model doc to docs/model.md
    • add check-yaml in pre-commit
    • update redoc service name to redoc_openapi

    The pipeline mode is determined by the combination of data source and destination. It describes how an end-to-end pipeline processes its workload.

    opened by xiaofei-du 2
  • chore(main): release 0.2.4-alpha

    chore(main): release 0.2.4-alpha

    :robot: I have created a release beep boop

    0.2.4-alpha (2022-08-01)

    Product Updates

    fix console bugs 🐛

    • fix keep redirecting back to onboarding form issue
    • fix Airbyte form default value issue
    • fix Toggle Field and Upload file Field issue
    • adapt new design-system component props structure
    • fix the wrong connector state
    • remove default scroll behaviour on type=number input

    VDP

    Miscellaneous Chores

    Console (0.7.0-alpha)

    Features

    Bug Fixes


    This PR was generated with Release Please. See documentation.

    autorelease: tagged 
    opened by droplet-bot 1
  • chore(main): release 0.2.3-alpha

    chore(main): release 0.2.3-alpha

    :robot: I have created a release beep boop

    0.2.3-alpha (2022-07-29)

    Product Updates

    features ✨

    • annotate destination data name with batch index and timestamp

    fix bugs 🐛

    • fix ArtiVC model import, destination creation and pipeline display issue
    • improve management backend error message
    • fix pipeline destination data loss issue

    VDP

    Miscellaneous Chores

    Console (0.6.3-alpha)

    Bug Fixes

    • fix get pipeline bug on model page (#174) (5e8116d)
    • fix ArtiVC model creation issue (#178) (d8ef4e0)
    • fix can't correctly create Airbyte destination (#185) (732844d)
    • fix destination cache not remove and create http & grpc issue (#184) (8c35d95)
    • fix onboarding form (#182) (a944e1d)

    Mgmt-backend (0.2.4-alpha)

    Bug Fixes

    Pipeline-backend (0.7.0-alpha)

    Features

    • add data association with dst conn (9233429)

    Connector-backend (0.5.0-alpha)

    Features

    • add data association with pipeline (c3236f7)

    Bug Fixes

    • fix dst data duplicate and lost issue (eb7ad97), closes #25

    This PR was generated with Release Please. See documentation.

    autorelease: tagged 
    opened by droplet-bot 1
  • chore(main): release 0.2.2-alpha

    chore(main): release 0.2.2-alpha

    :robot: I have created a release beep boop

    0.2.2-alpha (2022-07-20)

    Product Updates

    fix bugs 🐛🐛🐛

    • ⚡ī¸ speed up creating model operation
    • better console form handling
    • improve output format for models that are not labelled with any task
    • handle inference for images with RGBA channels

    VDP

    Miscellaneous Chores

    Console (0.6.0-alpha)

    Console update

    • Update sidebar design
    • Refactor lots of types, make our codebase much more consistent
    • Fix tracking and collect user usage data issue
    • Fix Airbyte nested fields validation and not correct value issue

    Features

    Bug Fixes

    • fix action issue (#172) (c02e4ad)
    • fix Airbyte nested form validation issue (#171) (1101acc)
    • fix description field is duplicated when create pipeline (#165) (7ef7b1a)

    Connector-backend (0.4.5-alpha)

    Bug Fixes

    • fix airbyte emitted_at timestamp (3eca9d6)
    • fix wrong state when connector check failed (92367e1)
    • remove airbyte namespace (bd7baa4)

    Model-backend (0.6.3-alpha)

    Features

    • speed up creating model operation (#115) (6930140)
    • standardize the output of the unspecified task(7fcd867)

    Bug Fixes

    • fix client stream server recv wrong file length interval (#143) (0e06f7c)
    • post process for unspecified task output (ad88068)
    • trigger image with 4 channel (#141) (7445f5f)

    Pipeline-backend (0.6.5-alpha)

    Bug fixes

    complete trigger async binary file for REST multipart (5003e5c)


    This PR was generated with Release Please. See documentation.

    autorelease: tagged 
    opened by droplet-bot 1
  • chore: add vdp protocol file

    chore: add vdp protocol file

    Because

    • we want to standardise Computer Vision (CV) tasks using JSON schema

    This commit

    • add the VDP protocol file that includes JSON schema for model batch outputs of all supported CV tasks
    opened by xiaofei-du 0
  • Support async model inference

    Support async model inference

    At the moment, a pipeline in async mode still relies on sync model inference and only asynchronously write to the destination. We should make the model-backend trigger endpoints use Temporal workflow. The pipeline-backend should implement Temporal workflow to request model-backend async inference as well.

    enhancement 
    opened by pinglin 0
  • Support filter feature for all backends

    Support filter feature for all backends

    Currently, only pipeline-backend supports search or filter endpoints (AIP-160) (See https://github.com/instill-ai/vdp/issues/109). We need to implement the same function for all backends

    • [ ] Connector-backend @pinglin
    • [ ] Model-backend @Phelan164
    • [ ] Mgmt-backend @xiaofei-du
    enhancement 
    opened by xiaofei-du 0
  • Resource state change safety check

    Resource state change safety check

    Similar to #109, if a resource (source/model-instance/destination) is online

    • if the resource is used by any online pipeline, we should not allow the operation to change the resource state from ✅ to ⛔.
    • if the resource is used by pipelines that are not online, we should allow the operation to change the resource state.

    Related backends:

    • [ ] connector-backend: @pinglin
    • [ ] model-backend: @Phelan164
    enhancement 
    opened by xiaofei-du 0
Releases(v0.2.4-alpha)
  • v0.2.4-alpha(Aug 1, 2022)

    0.2.4-alpha (2022-08-01)

    Product Updates

    fix console bugs 🐛

    • fix keep redirecting back to onboarding form issue
    • fix Airbyte form default value issue
    • fix Toggle Field and Upload file Field issue
    • adapt new design-system component props structure
    • fix the wrong connector state
    • remove default scroll behaviour on type=number input

    VDP

    Miscellaneous Chores

    Console (0.7.0-alpha)

    Features

    Bug Fixes

    Source code(tar.gz)
    Source code(zip)
  • v0.2.3-alpha(Jul 29, 2022)

    0.2.3-alpha (2022-07-29)

    Product Updates

    features ✨

    • annotate destination data name with batch index and timestamp

    fix bugs 🐛

    • fix ArtiVC model import, destination creation and pipeline display issue
    • improve management backend error message
    • fix pipeline destination data loss issue

    VDP

    Miscellaneous Chores

    Console (0.6.3-alpha)

    Bug Fixes

    • fix get pipeline bug on model page (#174) (5e8116d)
    • fix ArtiVC model creation issue (#178) (d8ef4e0)
    • fix can't correctly create Airbyte destination (#185) (732844d)
    • fix destination cache not remove and create http & grpc issue (#184) (8c35d95)
    • fix onboarding form (#182) (a944e1d)

    Mgmt-backend (0.2.4-alpha)

    Bug Fixes

    Pipeline-backend (0.7.0-alpha)

    Features

    • add data association with dst conn (9233429)

    Connector-backend (0.5.0-alpha)

    Features

    • add data association with pipeline (c3236f7)

    Bug Fixes

    • fix dst data duplicate and lost issue (eb7ad97), closes #25
    Source code(tar.gz)
    Source code(zip)
  • v0.2.2-alpha(Jul 20, 2022)

    0.2.2-alpha (2022-07-20)

    Product Updates

    fix bugs 🐛🐛🐛

    • ⚡ī¸ speed up creating model operation
    • better console form handling
    • improve output format for models that are not labelled with any task
    • handle inference for images with RGBA channels

    VDP

    Miscellaneous Chores

    Console (0.6.0-alpha)

    Console update

    • Update sidebar design
    • Refactor lots of types, make our codebase much more consistent
    • Fix tracking and collect user usage data issue
    • Fix Airbyte nested fields validation and not correct value issue

    Features

    Bug Fixes

    • fix action issue (#172) (c02e4ad)
    • fix Airbyte nested form validation issue (#171) (1101acc)
    • fix description field is duplicated when create pipeline (#165) (7ef7b1a)

    Connector-backend (0.4.5-alpha)

    Bug Fixes

    • fix airbyte emitted_at timestamp (3eca9d6)
    • fix wrong state when connector check failed (92367e1)
    • remove airbyte namespace (bd7baa4)

    Model-backend (0.6.3-alpha)

    Features

    • speed up creating model operation (#115) (6930140)
    • standardize the output of the unspecified task(7fcd867)

    Bug Fixes

    • fix client stream server recv wrong file length interval (#143) (0e06f7c)
    • post process for unspecified task output (ad88068)
    • trigger image with 4 channel (#141) (7445f5f)

    Pipeline-backend (0.6.5-alpha)

    Bug fixes

    complete trigger async binary file for REST multipart (5003e5c)

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-alpha(Jul 12, 2022)

    0.2.1-alpha (2022-07-12)

    Product Updates

    • đŸ”ĨđŸ”ĨđŸ”Ĩ New YOLOv4 vs. YOLOv7 live demo
    • Add optional description field when creating models
    • Better console page title
    • Support default model card for models without README.md

    VDP

    Miscellaneous Chores

    Console (0.5.0-alpha)

    Features

    Bug Fixes

    • use instill owned icons when interact with instill owned destination (#150) (d466a09)

    Model-backend (v0.6.0-alpha)

    Miscellaneous Chores

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-alpha(Jul 8, 2022)

    0.2.0-alpha (2022-07-08)

    Product Updates

    A lot of gooooodies đŸŦ 🍰 🧁 to share with the community

    • đŸĨŗ Add new management server mgmt-backend for user management
    • 🎊 Add new connector server connector-backend for pre-built source/destination connector management
      • new HTTP connector
      • new gRPC connector
    • đŸ”Ĩ Now you can import models from GitHub, Hugging Face, or remote storage managed by DVC or ArtiVC
    • 🎉 New vision task: besides image classification and object detection, VDP supports keypoint detection
    • 🎮 New Console UI to enable youenabling you to use VDP without writing code
    • 🚀 Build pipelines for diverse scenarios
      • SYNC pipeline for real-time inference
      • ASYNC pipeline for on-demand workload

    VDP

    Features

    Bug Fixes

    • fix connector-backend-worker in docker-compose-dev (c5d7260)
    • fix OpenAPI swagger service (#104) (2e3fd90)
    • fix temporal dynamicconfig (57edcde)

    Mgmt-backend (v0.2.3-alpha)

    Features

    • add reporter for usage collection (320d9a4)
    • add retrieve all user function (da78893)

    Bug Fixes

    • support CORS (https://github.com/instill-ai/mgmt-backend/issues/3) (5fb1439)
    • use cors package to replace naive implementation (ae49c1c)
    • add tlsenabled in usage backend configuration (dd8843a)
    • init config before logger (455ef7a)
    • init config first (4e3dea6)
    • put client connection in main (3ff4821)
    • refactor usage collection (c6a41f6)
    • specify time duration unit (27b77d2)
    • check existing user by id (dd862a4)
    • change port from 8080 to 8084 (364f155)

    Connector-backend (v0.4.3-alpha)

    Features

    Bug Fixes

    • fix det jsonschema for empty case (cd4f102)
    • fix duration configuration bug (4a4111c)
    • fix usage collection (c19cf9b)
    • fix usage disbale logic (664660d)
    • fix usage-backend non-tls dial (457a74d)
    • remove worker debug volume (acaa01c)
    • return 422 delete error msg (5498494)
    • fix error handler middleware (2fc9487)
    • skip JSON Schema snake_case convert (c19cc53), closes https://github.com/instill-ai/connector-backend/issues/17

    Model-backend (v0.6.0-alpha)

    Features

    • add credential definition (https://github.com/instill-ai/model-backend/issues/109) (92d3391)
    • support artivc (https://github.com/instill-ai/model-backend/issues/102) (b8e21a4)
    • support creating a HuggingFace model (https://github.com/instill-ai/model-backend/issues/113) (1577d87)

    Bug Fixes

    • fix config path (a8cf2c0)
    • regexp zap logger with new protobuf package (8b9c463)
    • bug usage storage (https://github.com/instill-ai/model-backend/issues/103) (975fdc1)
    • fix duration configuration bug (ee4a310)
    • init config before logger (9d3fb4a)
    • status code when deploy model error (https://github.com/instill-ai/model-backend/issues/111) ([31d3f11](https://github.com/instill-ai/model- backend/commit/31d3f11ba04ee59b12521b8e0dd724849a81b94f))
    • update model definitions and tasks in usage collection (https://github.com/instill-ai/model-backend/issues/100) ([c593087](https://github.com/instill-ai/model- backend/commit/c5930870595c5d280d7db005a711c0cc9bff802c))
    • wrong logic when checking user account and service account (7058db6)
    • model definition in list model and missing zero in output (https://github.com/instill-ai/model-backend/issues/121) (a90072d)

    Pipeline-backend (v0.6.3-alpha)

    Features

    • add cors support (c847912)
    • add async pipeline trigger (6d39b0a)
    • add filter for list pipeline (ffe8856)
    • add usage collection (7e71744)
    • support trigger multi model instances (e3d4263)

    Bug Fixes

    • fix create pipeline recipe resource name (bab3eaa)
    • fix duration configuration bug (998eafa)
    • fix usage collection (243e7a1)
    • fix usage disbale logic (962823b)
    • fix usage-backend non-tls dial (b864df3)
    • close https://github.com/instill-ai/pipeline-backend/issues/56 ([c627a53](https://github.com/instill-ai/pipeline- backend/commit/c627a539999d65bc96ac6f88e2bd203548c34825))
    • fix empty description update (2579f2e)

    Console (v0.3.3-alpha)

    Features

    • add ArtiVC Icon for model definition (#18) (b5250db)
    • add delete connector, pipeline service (#9) (526ff70)
    • add DISABLE_USAGE_COLLECTION args config (#45) (7f9352d)
    • add GitHub action related to docker hub (#11) (5e7569a)
    • add mock server and render method of react-query (#12) (80d89d7)
    • add mock source service handler (#13) (21ea1ee)
    • add onboarding e2e test (#20) (c952b73)
    • add pipeline e2e test (#44) (0acc846)
    • add playwright (#14) (01beb24)
    • add project docker-compose (#2) (14ab822)
    • add release please action (#21) (2321bab)
    • add tailwind plugin to support intelligence picking up custom style and refactor classnames (#16) (62d853f)
    • add test model instance service (#15) (a367577)
    • add unit-test action (#17) (d57d265)
    • implement statefull toggle and adapt new prop (#49) (9358247)
    • make sidebar collapsable (#8) (ecf8cd3)
    • let user add backend related env with docker-compose environment (#62) (507534a)
    • add ModelInstanceReadmeCard (#76) (28d3927)
    • add script to download airbyte's icons (#66) (96dd22e)
    • support ArtiVC (#64) (1f1fd11)

    Bug Fixes

    Source code(tar.gz)
    Source code(zip)
  • v0.1.6-alpha(Apr 3, 2022)

  • v0.1.5-alpha(Mar 22, 2022)

  • v0.1.4-alpha(Mar 22, 2022)

  • v0.1.3-alpha(Feb 24, 2022)

  • v0.1.2-alpha(Feb 21, 2022)

    0.1.2-alpha (2022-02-21)

    Project Updates

    • A full local development tooling with docker-compose
    • Preliminary model serving for multiple Deep Learning frameworks
    • Pipeline with sync mode to handle blocking data flow
    • Go example codes

    Bug Fixes

    • add name in update model api (#41) (35b7e31)
    • docker-compose: change correct dependency for temporal (#60) (eb98049)
    • update model backend without specify version when creating model (#63) (f4cf161)
    • wrong type in sample model prediction (#39) (523d5f8)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1-alpha(Feb 13, 2022)

  • v0.1.0-alpha(Feb 11, 2022)

    0.1.0-alpha (2022-02-11)

    Features

    • add provisioned visual-data-pipeline from docker-compose (0ec790b)
    • initial commit (2916d75)
    • refactor the name to visual data preparation and add model-backend into docker-compose (#7) (d24c0ef)
    • refactor to adopt koanf configuration management (51eba89)

    Bug Fixes

    Source code(tar.gz)
    Source code(zip)
Owner
Instill AI
Visual data preparation made for all - Empower modern data stack, tapping the value of unstructured visual data with our open source community.
Instill AI
An open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developersAn open-source, distributed, cloud-native CD (Continuous Delivery) product designed for developers

Developer-oriented Continuous Delivery Product âŖ English | įŽ€äŊ“中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? How to

null 0 Oct 19, 2021
Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd data, and intelligent diagnosis.

Kstone 中文 Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd

TKEStack 555 Aug 4, 2022
go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data.

go-opa-validate go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data. Installation Usage Cont

chenk 5 Feb 5, 2022
The Elastalert Operator is an implementation of a Kubernetes Operator, to easily integrate elastalert with gitops.

Elastalert Operator for Kubernetes The Elastalert Operator is an implementation of a Kubernetes Operator. Getting started Firstly, learn How to use el

null 20 Jun 28, 2022
Plugin for Helm to integrate the sigstore ecosystem

helm-sigstore Plugin for Helm to integrate the sigstore ecosystem. Search, upload and verify signed Helm Charts in the Rekor Transparency Log. Info he

sigstore 36 Aug 8, 2022
🔮 ✈ī¸ to integrate OPA Gatekeeper's new ExternalData feature with cosign to determine whether the images are valid by verifying their signatures

cosign-gatekeeper-provider To integrate OPA Gatekeeper's new ExternalData feature with cosign to determine whether the images are valid by verifying i

sigstore 41 Jun 25, 2022
Vilicus is an open source tool that orchestrates security scans of container images(docker/oci) and centralizes all results into a database for further analysis and metrics.

Vilicus Table of Contents Overview How does it work? Architecture Development Run deployment manually Usage Example of analysis Overview Vilicus is an

Ederson Brilhante 77 Jun 21, 2022
Open Source runtime tool which help to detect malware code execution and run time mis-configuration change on a kubernetes cluster

Kube-Knark Project Trace your kubernetes runtime !! Kube-Knark is an open source tracer uses pcap & ebpf technology to perform runtime tracing on a de

Chen Keinan 31 Jul 27, 2022
Horusec is an open source tool that improves identification of vulnerabilities in your project with just one command.

Table of contents 1. About 2. Getting started 2.1. Requirements 2.2. Installation 3. Usage 3.1. CLI Usage 3.2. Using Docker 3.3. Older versions 3.4. U

ZUP IT INNOVATION 772 Aug 5, 2022
:rocket: Modern cross-platform HTTP load-testing tool written in Go

English | 中文 Cassowary is a modern HTTP/S, intuitive & cross-platform load testing tool built in Go for developers, testers and sysadmins. Cassowary d

Roger Welin 603 Aug 9, 2022
k6 is a modern load testing tool for developers and testers in the DevOps era.

k6 is a modern load testing tool, building on our years of experience in the load and performance testing industry. It provides a clean, approachable scripting API, local and cloud execution, and flexible configuration.

k6 17.5k Aug 14, 2022
Bubbly is an open-source platform that gives you confidence in your continuous release process.

Bubbly Bubbly - Release Readiness in a Bubble Bubbly emerged from a need that many lean software teams practicing Continuous Integration and Delivery

Valocode 34 Jul 27, 2022
Open Source runtime scanner for Linux containers (LXD), It performs security audit checks based on CIS Linux containers Benchmark specification

lxd-probe Scan your Linux container runtime !! Lxd-Probe is an open source audit scanner who perform audit check on a linux container manager and outp

Chen Keinan 14 May 16, 2022
KubeCube is an open source enterprise-level container platform

KubeCube English | 中文文æĄŖ KubeCube is an open source enterprise-level container platform that provides enterprises with visualized management of Kuberne

KubeCube IO 289 Aug 9, 2022
Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron is an open source software delivery workflow for kubernetes written in go.

Devtron Labs 2.3k Aug 12, 2022
TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative.

TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative. TriggerMesh allows you to declaratively define event flows between sources and targets as well as add even filter, splitting and processing using functions.

TriggerMesh 322 Aug 6, 2022
Fleet - Open source device management, built on osquery.

Fleet - Open source device management, built on osquery.

Fleet Device Management 883 Aug 11, 2022