A simple library to extract video and audio frames from media containers (based on libav).

Overview

Reisen GoDoc

A simple library to extract video and audio frames from media containers (based on libav, i.e. ffmpeg).

Dependencies

The library requires libav components to work:

  • libavformat
  • libavcodec
  • libavutil
  • libswresample
  • libswscale

For Arch-based Linux distributions:

sudo pacman -S ffmpeg

For Debian-based Linux distributions:

sudo apt install libswscale-dev libavcodec-dev libavformat-dev libswresample-dev libavutil-dev

For macOS:

brew install libav

Installation

Just casually run this command:

go get github.com/zergon321/reisen

Usage

Any media file is composed of streams containing media data, e.g. audio, video and subtitles. The whole presentation data of the file is divided into packets. Each packet belongs to one of the streams and represents a single frame of its data. The process of decoding implies reading packets and decoding them into either video frames or audio frames.

The library provides read video frames as RGBA pictures. The audio samples are provided as raw byte slices in the format of AV_SAMPLE_FMT_DBL (i.e. 8 bytes per sample for one channel, the data type is float64). The channel layout is stereo (2 channels). The byte order is little-endian. The detailed scheme of the audio samples sequence is given below.

Audio sample structure

You are welcome to look at the examples to understand how to work with the library. Also please take a look at the detailed tutorial.

Issues
  • Support for Windows

    Support for Windows

    Hello, I have just discovered this repo. Is there any plan to support shared library(by calling .dll, .so, .dylib) and support for windows ? There was a WIP project that has not been updated over a year. I tried it but it has some audio skipping issues. Thank you for this work

    help wanted 
    opened by rookiesH 23
  • Re-enable macOS support

    Re-enable macOS support

    Thanks for the great project. I used the tagged version but wanted to contribute so switched to master, where I had some trouble:

    On master the project no longer works on darwin, but it does in 0.1.1. The change for a windows fix moved audio.go (and video.go) to a -windows and -linux suffix, but there is no -darwin so compile fails.

    opened by andydotxyz 6
  • panic: buffered: len(pix) was 8294400 but must be 3686400

    panic: buffered: len(pix) was 8294400 but must be 3686400

    Hi! Thank you for this great project! I've tried your player code with your test mp4 and it works as intended. But when I try running it with any other mp4 video file I consistently get the error panic: buffered: len(pix) was 8294400 but must be 3686400. I noticed the factor between those two numbers is exactly 2.25. Any idea what the issue might be here? Thank you!

    help wanted 
    opened by boriwo 6
  • Cant run go get reisen. ERROR: could not determine kind of name for C.av_bsf_receive_packet and C.av_bsf_send_packet

    Cant run go get reisen. ERROR: could not determine kind of name for C.av_bsf_receive_packet and C.av_bsf_send_packet

    Getting errors when I go get reisen using go get github.com/zergon321/reisen

    using brew install libav did not work for me, so tried building ffmpeg from source using brew install ffmpeg --build-from-source which installed all the library and header files, now i get all the following libraries in /usr/local/include and /usr/local/lib

    • libavformat
    • libavcodec
    • libavutil
    • libswresample
    • libswscale

    but I’m getting the following error after running go get github.com/zergon321/reisen ../../go/pkg/mod/github.com/zergon321/[email protected]/media.go:207:12: could not determine kind of name for C.av_bsf_receive_packet

    ../../go/pkg/mod/github.com/zergon321/[email protected]/media.go:199:12: could not determine kind of name for C.av_bsf_send_packet

    I can confirm that all dependencies are install correctly. please whats happing and can anything be done to resolve this, thank you

    opened by onos9 4
  • drift between video and audio stream in example player

    drift between video and audio stream in example player

    Another question after spending more time with the video player example.

    When playing longer videos I notice a slight drift between the video and the audio stream over time. This becomes even more pronounced when I add some extra image processing. I think I understand what the issue is: Essentially the video frame processing and the audio playing take place on their own go routines without any synchronization between them. So I guess the video processing could fall behind, while the audio processing keeps up? I understand that the player is just a very simple example, so this behavior is probably not surprising. Nevertheless any recommendation how to synchronize audio and video better would be great.

    Also, just for my better understanding, I noticed that media.ReadPacket() will return audio and video packets at random for all open streams. Most of the time video and audio packets will alternate perfectly although every once in a while I see 2 video packets for 1 audio packet. What is the rational here in terms of stream synchronization?

    help wanted 
    opened by boriwo 2
  • interesting observation regarding framerates

    interesting observation regarding framerates

    I found a small issue with your player example code:

    videoFPS, _ := media.Streams()[0].FrameRate()
    spf := 1.0 / float64(videoFPS)
    frameDuration, err := time.ParseDuration(fmt.Sprintf("%fs", spf)) 
    

    You are ignoring the denominator here which is fine for your sample video but it should be considered for the general case.

    But it gets more interesting yet. Some videos come with valid frame rate of 29.97. In this case the nominator will be 30000 and the denominator will be 1001 (which again is correct). However, if I play a video with that frame rate I observe the increasing drift between video and audio I reported in a previous issue. It gets better: If I manually force the frame rate to 30 instead of 29.97 the drift between audio and video is gone!

    I observed the same for videos with 23.98 frame rate: There is a drift unless I force the player to play at 24 frames per second.

    Currently, I'm working with mp4 snippets I download from youtube. Of course there is a possibility that the metadata is wrong in my mp4s but I don't think so. Other video players play those videos correctly - but then again those other players may do a better job syncing audio and video and ignore the possibly flawed frame rate data in my videos.

    Anyways, just wanted to bring this to your attention and see if you have any insight why I have to "round up" 29.97 to 30 and 23.98 to 24. Thanks!

    opened by boriwo 1
  • good job ,found another project may be useful .

    good job ,found another project may be useful .

    nice work , simple and clean . i have just read your article on Medium , learned your findings . i used to play with this Go Media Framework . it seems a full-featured binding , but not very active . wanna take a look ?

    opened by c1ngular 0
  • OSX example fails with panic: time: invalid duration

    OSX example fails with panic: time: invalid duration "NaNs"

    tried multiple files, same error.

    go run main.go
    Duration: 2m30.102176s
    Format name: mov,mp4,m4a,3gp,3g2,mj2
    Format long name: QuickTime / MOV
    MIME type: 
    Number of streams: 2
    
    panic: time: invalid duration "NaNs"
    
    goroutine 1 [running]:
    main.handleError(...)
            <omitted>/main.go:142
    main.main()
            <omitted>/main.go:28 +0x11bf
    exit status 2
    ------
    Duration: 4m0.140416s
    Format name: matroska,webm
    Format long name: Matroska / WebM
    MIME type: audio/webm,audio/x-matroska,video/webm,video/x-matroska
    Number of streams: 2
    
    panic: time: invalid duration "NaNs"
    
    goroutine 1 [running]:
    main.handleError(...)
           <omitted>/main.go:142
    main.main()
            <omitted>/main.go:28 +0x11bf
    exit status 2
    

    I ran libav as the README noted:

    🍺  /usr/local/Cellar/libav/12.3_8: 15 files, 32.4MB
    ==> Running `brew cleanup libav`...
    Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
    Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
    ==> No outdated dependents to upgrade!
    ==> Checking for dependents of upgraded formulae...
    Disable this behaviour by setting HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK.
    Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
    ==> Reinstalling 1 dependent with broken linkage from source:
    libav
    Warning: libav has been deprecated because it is not maintained upstream!
    
    opened by xackery 4
Releases(v0.1.4)
  • v0.1.4(Feb 23, 2022)

  • v0.1.3(Jan 2, 2022)

    It's now possible to apply bitstream filters to streams. Bitstream filters serve to transform encoded packet data into a different form. New methods of the baseStream struct are ApplyFilter, RemoveFilter and Filter. See the player example for usage.

    The full list of bitstream filters and their parameters is located here.

    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Dec 26, 2021)

    New features are available:

    • bundling on Windows added;
    • fixed Mac OS compatibility for some functions (thanks to @andydotxyz );
    • added the Rewind method for streams;
    • added methods to access the contents of the packet.

    Please create an issue if you encounter an error (especially if the error is related to compatibility).

    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Sep 14, 2021)

    New features:

    • helper functions to get only video streams of the media or only audio streams of the media;
    • scaling - an opportunity to receive video frames in a resolution different from the original one;
    • interpolation algorithms enumeration; it's now possible to choose an interpolation algorithm for video frames scaling.
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Sep 1, 2021)

    Available features:

    • read some information about media files;
    • read some information about streams;
    • read packets of media data;
    • decode packets containing video frames as RGBA pictures;
    • decode packets containing audio frames as AV_SAMPLE_FMT_DBL stereo byte sequences.
    Source code(tar.gz)
    Source code(zip)
Owner
NightGhost
I'm a Golang programmer. I like playing with Docker containers and different databases, have gamedev as a hobby.
NightGhost
LiveKit - Open source, distributed video/audio rooms over WebRTC

LiveKit is an open source project that provides scalable, multi-user conferencing over WebRTC. It's designed to give you everything you need to build real time video/audio capabilities in your applications.

LiveKit 2.4k May 10, 2022
Go-video-preview-ffmpeg-wrapper - A simple helper wrapper to generate small webm video previews using ffmpeg, useful for web previews.

Go-video-preview-ffmpeg-wrapper A simple helper wrapper to generate small webm video previews using ffmpeg, useful for web previews. Getting Started u

Robert van Alphen 0 Jan 5, 2022
SlideXtract - A tool to help extract slides from a video file.

SlideXtract A tool to help extract slides from a video file. Slides are output in the out folder. Features I didn't find any other piece of code that

Soumitra Shewale 1 Feb 13, 2022
[WIP] a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg software

About a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg

Mohammed Al Ashaal 8 May 4, 2022
Go bindings for libVLC and high-level media player interface

Go bindings for libVLC 2.X/3.X/4.X and high-level media player interface. The package can be useful for adding multimedia capabilities to applications

Adrian-George Bostan 299 May 12, 2022
golang library for mux and demux file containers such as mpeg-ts,mpeg-ps,flv

gomedia mpeg-ts,mpeg-ps,flv muxer/demuxer mpeg-ts muxer mpeg-ts demuxer mpeg-ps muxer mpeg-ps demuxer flv muxer flv demuxer mpeg-ps will be done in th

yaping 11 May 7, 2022
Go Media Framework

Go FFmpeg Bindings Installation Prerequisites Current master branch supports all major Go versions, starting from 1.6. Build/install FFmpeg build last

alex 758 May 9, 2022
Videncode - Media Encoder (with ffmpeg)

Videncode - Media Encoder (with ffmpeg) Powered by yellyoshua (With2 easy steps) - Build JSON with folder of videos > Process the videos to the new fo

Yoshua Lopez 2 Oct 25, 2021
Frf-media-download - This tool downloads all the files you have uploaded to FreeFeed

FreeFeed Media Downloader This tool downloads all the files you have uploaded to

Давид Мзареулян 0 Jan 29, 2022
A tool to stream videos📺 directly into VLC media player just by its name from terminal.

PeerWatch A tool to stream videos directly into the VLC media player. Download Download the tool from here: Windows Linux win-x64 linux-x64 Building Y

Ab 0 Feb 12, 2022
lmmp3 is a little golang library that download a video from youtube, and convert it to a mp3 file using ffmpeg

lmmp3 lmmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg You need to have installed ffmpeg in your syst

pai 8 Jan 11, 2022
👾 Annie is a fast, simple and clean video downloader built with Go.

?? Annie is a fast, simple and clean video downloader built with Go. Installation Prerequisites Install via go install Homebrew (macOS only) Arch Linu

Rohan 0 Dec 27, 2021
Go4vl is Go library for working with the Video for Linux API (V4L2) natively, without any C bindings.

go4vl A Go library for working with the Video for Linux user API (V4L2). Gov4l hides all the complexities of working with V4L2 and exposes idiomatic G

Vladimir Vivien 72 May 13, 2022
ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically process video and manipulate subtitles.

% FFCOMMANDER(1) ffcommander 2.39 % Mikael Hartzell (C) 2018 % 2021 Name ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically pro

Mikael Hartzell 2 May 9, 2022
Project to get Youtube video descriptions and search those videos as required

FamPayProject Project to get Youtube video descriptions and search those videos as required Prerequisities Postgres DB for persisting data Youtube Dat

null 0 Nov 5, 2021
golang function that download a video from youtube, and convert it to a mp3 file using ffmpeg

echedwnmp3 echedwnmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg example package main import(echedwn

pai 4 Dec 7, 2021
Personal video streaming server.

tube This is a Golang project to build a self hosted "tube"-style video player for watching your own video collection over HTTP or hosting your own ch

davy wybiral 221 May 1, 2022
Short video direct link acquisition 短视频直连获取工具

Glink 短视频去水印一键解析应用 Short video direct link acquisition 短视频直连获取工具 Glink是一款基于go语言开发的短视频解析应用,前端使用vue+argon主题,后端使用go-fiber框架,支持web在线模式、客户端模式。

佰阅 113 May 5, 2022
Quik.do is a video conferencing tool.

Quik.do is a video conferencing tool.

Bora Tanrıkulu 108 Apr 18, 2022