基于 Goland + Gin + Logrus+ OpenCv 4 + WebSocket + YoloV4 + Redis + Vue 3 的直播流(支持RTSP、RTMP、FLV、HLS)在线场景智能识别系统

Overview

基于 Goland + Gin + OpenCv 4 + WebSocket + YoloV4 + Redis + Logrus + Vue 3 + TypeScript

目前市面上AI在Go中的应用和相关使用较少,Go这门语言是我比较喜欢的;另外就是自己在AI部门从事多年该项工作,但是没有接触这块比较具体的工作,还有比较重要的一点就是为了简化AI识别这块的使用,所以就想自己动手尝试一下。本系统支持直播流(RTSP、RTMP、FLV、HLS) 在线场景智能识别,通过WebSocket的交互方式,跟Web端进行实时关键帧和识别结果的传输。目前AI识别这块使用的是YoloV4开源模型,在很多特定场景下会出现误识别的情况,如何提高识别率这是后续的问题了,本系统仅供参考学习使用。

识别效果

视频处理效果

出入口识别视频效果

图像处理效果

出入口识别 出入口识别 出入口识别

OpenCv在Go中如何使用

  1. 在windows或者linux下需要自行编译,需要下载CMake和mingw64对源码编译。

  2. 我是使用gocv下的win_build_opencv.cmd进行编译的,该文件需要根据不同系统的要求进行相应修改,我电脑使用的是win10, 所以我编译的是基于windows的cpu版本,本来想使用CUDA版本,折腾了好几天最后没弄成功。最好使用CUDA版本进行AI处理, 否则处理速度和性能会差很多。

  3. opencv-4.5.x和opencv-contrib-4.5.x我是手动下载并且解压的,然后放到C盘的opencv目录下进行编译,编译时间大概需要20分钟。

  4. 如果使用opencv打开网络视频流,需要下载opencv_videoio_ffmpeg_64.dll,记得名称需要改成编译版本一样的,否则会出错。

YoloV4在Go中遇到的问题总结

  1. yolov4官方提供的模型可以识别80种物体,可以通过提高置信度和非极大抑制值的数值过滤掉一些误识别的结果。

  2. 如果使用cpu版本来识别,处理单张图片的时间要好几秒,如果想要提高识别时间,可以修改gocv.BlobFormImage的 image.Pt(416, 416),改成image.Pt(128, 128),最低可以改成64,但是会降低识别精度,改成128处理单张图片时间 大概在200到300毫秒,如果是用CUDA,416处理单张图片时间可以在200到300毫秒。

  3. yolov4的模型有250MB,需要到官网去下载。

  4. 由于演示的视频会看到关键信息,已对视频右下角做了高斯滤波的模糊处理。

WebSocket与Web端的传输性能问题

  1. 试验了多种分辨率与Web端传输效果,目前640*480处理性能较好,一旦分辨率太高,前端接收不及时,会造成网络堵塞, 内存爆满或者浏览器卡顿奔溃的现象。

  2. 除了使用websocket之外,还可以使用nginx-rtmp进行推流,但是需要在调用ffmpeg推流时,需要把关键帧转成mjpeg格式, 若使用yuv格式会导致推流后的视频流异常。

基于Vue 3版本的Web端

使用了Vue 3 + Vite + TypeScript + WebSocket进行了web端的开发,感兴趣的请查看vue3-live-stream-ai, 如果想查看简单的调试版,请查看web目录下的index.html。

目前存在问题

  1. 目前识别的处数统计是不对的,会有大量重复,后续会加入卡尔曼滤波算法,该算法可以对重复的处数进行准确的统计。

  2. 后续会支持在linux系统下,采用cuda架构进行AI计算处理。

You might also like...
Automated penetration and auxiliary systems, providing XSS, XXE, DNS log, SSRF, RCE, web netcat and other Servers,gin-vue-admin
Automated penetration and auxiliary systems, providing XSS, XXE, DNS log, SSRF, RCE, web netcat and other Servers,gin-vue-admin

Simple DNS log Server,easy to ACME DNS challenge log easy send to elasticsearch https://github.com/hktalent/DNS_Server go4Hacker Automated penetration

YSHOP-GO基于当前流行技术组合的前后端RBAC管理系统:Go1.15.x+Beego2.x+Jwt+Redis+Mysql8+Vue 的前后端分离系统,权限控制采用 RBAC,支持数据字典与数据权限管理,支持动态路由等

YSHOP-GO 后台管理系统 项目简介 YSHOP-GO基于当前流行技术组合的前后端RBAC管理系统:Go1.15.x+Beego2.x+Jwt+Redis+Mysql8+Vue 的前后端分离系统,权限控制采用 RBAC,支持数据字典与数据权限管理,支持动态路由等 体验地址: https://go

Gin-errorhandling - Gin Error Handling Middleware is a middleware for the popular Gin framework

Gin Error Handling Middleware Gin Error Handling Middleware is a middleware for

Parse and generate m3u8 playlists for Apple HTTP Live Streaming (HLS) in Golang (ported from gem https://github.com/sethdeckard/m3u8)

go-m3u8 Golang package for m3u8 (ported m3u8 gem https://github.com/sethdeckard/m3u8) go-m3u8 provides easy generation and parsing of m3u8 playlists d

Parser and generator of M3U8-playlists for Apple HLS. Library for Go language. :cinema:

M3U8 This is the most complete opensource library for parsing and generating of M3U8 playlists used in HTTP Live Streaming (Apple HLS) for internet vi

ready-to-use RTSP / RTMP server and proxy that allows to read, publish and proxy video and audio streams
ready-to-use RTSP / RTMP server and proxy that allows to read, publish and proxy video and audio streams

rtsp-simple-server is a simple, ready-to-use and zero-dependency RTSP / RTMP server and proxy, a software that allows users to publish, read and proxy live video and audio streams. RTSP is a specification that describes how to perform these operations with the help of a server, that is contacted by both publishers and readers and relays the publisher's streams to the readers.

🎥 Proxy livestreams from websockets to external RTMP endpoints.

pxy pxy is a Go server that routes incoming livestream data from websockets to an external RTMP endpoint. This project is a work in progress, I'll upd

Prismplus - Prism+ lets you multicast your rtmp stream to multiple destinations
Prismplus - Prism+ lets you multicast your rtmp stream to multiple destinations

prism+ Use at your own risk! It has worked for us.. but very much alpha quality!

hls converter.

hlsconv hls converter. 外部依赖 ffmpeg: 4.2.1 使用方式 linix/macos # 转换单个文件 bin/hlsconv -i video/in.mp4 -o outvideo/ # 批量转换文件 bin/hlsconv -i video/ -o outvide

Muxer - This repo is designed to trancode RTMP streams from the ingester and then push them to be muxed
Muxer - This repo is designed to trancode RTMP streams from the ingester and then push them to be muxed

VidersMuxer This repo is designed to trancode RTMP streams from the ingester and

Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具,满足用户非常灵活的同步、迁移需求。
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具,满足用户非常灵活的同步、迁移需求。

RedisShake is mainly used to synchronize data from one redis to another. Thanks to the Douyu's WSD team for the support. 中文文档 English tutorial 中文使用文档

Go package for computer vision using OpenCV 4 and beyond.
Go package for computer vision using OpenCV 4 and beyond.

GoCV The GoCV package provides Go language bindings for the OpenCV 4 computer vision library. The GoCV package supports the latest releases of Go and

使用 Golang+Chrome+OpenCV 破解腾讯滑块验证码
使用 Golang+Chrome+OpenCV 破解腾讯滑块验证码

一、背景 滑块验证码是一项人机识别技术,操作简单,真人体验好,机器识别效果也不差,可以有效防止脚本做任务,增加机器脚本薅羊毛的难度。但其破解也相对简单,这里演示一个Demo,以了解。通过 OpenCV 匹配找出滑块位置,计算出滑动距离,然后模拟 js 鼠标事件,在 Chrome 控制台执行脚本,完成

Pokemon Unite scoreboard HUD and extra tools running over captured game feeds using the OpenCV video processing API and Client/Server architecture.
Pokemon Unite scoreboard HUD and extra tools running over captured game feeds using the OpenCV video processing API and Client/Server architecture.

unite Pokemon Unite scoreboard HUD and extra tools running over captured game feeds using the OpenCV video processing API. Client (OBS Live) Server Ar

go websocket client for unit testing of a websocket handler

wstest A websocket client for unit-testing a websocket server The gorilla organization provides full featured websocket implementation that the standa

Proxy is a high performance HTTP(S) proxies, SOCKS5 proxies,WEBSOCKET, TCP, UDP proxy server implemented by golang. Now, it supports chain-style proxies,nat forwarding in different lan,TCP/UDP port forwarding, SSH forwarding.Proxy是golang实现的高性能http,https,websocket,tcp,socks5代理服务器,支持内网穿透,链式代理,通讯加密,智能HTTP,SOCKS5代理,黑白名单,限速,限流量,限连接数,跨平台,KCP支持,认证API。 Awesome WebSocket CLient - an interactive command line client for testing websocket servers
Awesome WebSocket CLient - an interactive command line client for testing websocket servers

Awesome WebSocket CLient - an interactive command line client for testing websocket servers

Encrypted-websocket-chat - Encrypted websocket chat using golang

Encrypted websocket chat First version written in python This version should be

Websocket-chat - A simple websocket chat application
Websocket-chat - A simple websocket chat application

WebSocket Chat App This is a simple chat app based on websockets. It allows user

Owner
pfhds
pfhds
Go bindings for OpenCV / 2.x API in gocv / 1.x API in opencv

Go OpenCV binding A Golang binding for OpenCV. OpenCV 1.x C API bindings through CGO, and OpenCV 2+ C++ API (GoCV) through SWIG. Disclaimer This is a

go-opencv 1.3k Sep 27, 2022
beego-vue-admin基于当前流行技术组合的前后端RBAC管理系统:Go1.15.x+Beego2.x+Jwt+Redis+Mysql8+Vue 的前后端分离系统,权限控制采用 RBAC,支持数据字典与数据权限管理,支持动态路由等

beego-vue-admin 后台管理系统 项目简介 beego-vue-admin基于当前流行技术组合的前后端RBAC管理系统:Go1.15.x+Beego2.x+Jwt+Redis+Mysql8+Vue 的前后端分离系统,权限控制采用 RBAC,支持数据字典与数据权限管理,支持动态路由等 体验

null 118 Sep 19, 2022
Go-logging-logrus - Learn how to log management in golang with logrus

Learn how to logging in golang with logrus How to run this project git clone htt

Vandy Ahmad 0 Jan 19, 2022
golang library for mux and demux file containers such as mpeg-ts,mpeg-ps,flv

gomedia mpeg-ts,mpeg-ps,flv muxer/demuxer mpeg-ts muxer mpeg-ts demuxer mpeg-ps muxer mpeg-ps demuxer flv muxer flv demuxer mpeg-ps will be done in th

yaping 76 Sep 27, 2022
Implementation of the test task, chat in the goland language

Implementation of the test task, chat in the goland language

Dmitriy Fofanov 1 Dec 5, 2021
Gjg - Go jump goland tool

GJG go-jump-goland tool Allows you to launch quickly the Goland IDE and open pro

null 3 Mar 14, 2022
Opinionated Go starter with gin for REST API, logrus for logging, viper for config with added graceful shutdown

go-gin-starter An opinionated starter for Go Backend projects using: gin-gonic/gin as the REST framework logrus for logging viper for configs Docker f

Udaya Prakash 67 Aug 8, 2022
Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo.

go-chat Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo. Why Eu gostaria de aprender algumas ferramenta

Kevin Souza 0 Jul 14, 2022
mogutouERP 是一个前后端分离的微型进存销系统,采用 Gin + Vue 开发

mogutouERP 蘑菇头进存销管理系统 mogutouERP 是一个前后端分离的微型进存销系统,采用 Gin + Vue 开发。

徐旭 207 Sep 23, 2022
A ToDoList Demo based on Vue+Gin+Gorm+mysql

bubble清单 一个基于gin+gorm开发的练手小项目,通过该项目可初识go web开发该有的姿势。 前端页面基于vue和ElementUI开发,对前端不熟悉的童鞋可直接下载templates和static文件夹下的内容使用。 使用指南 下载 [email protected]:mao888/GoWe

胡超 6 Mar 16, 2022