Parse and demux MPEG Transport Streams (.ts) natively in GO

Overview

GoReportCard GoDoc Travis Coveralls

This is a Golang library to natively parse and demux MPEG Transport Streams (ts) in GO.

WARNING: this library is not yet production ready. Use at your own risks!

Installation

To install the library use the following:

go get -u github.com/asticode/go-astits/...

Before looking at the code...

The transport stream is made of packets.
Each packet has a header, an optional adaptation field and a payload.
Several payloads can be appended and parsed as a data.

                                           TRANSPORT STREAM
 +--------------------------------------------------------------------------------------------------+
 |                                                                                                  |
 
                       PACKET                                         PACKET
 +----------------------------------------------+----------------------------------------------+----
 |                                              |                                              |
 
 +--------+---------------------------+---------+--------+---------------------------+---------+
 | HEADER | OPTIONAL ADAPTATION FIELD | PAYLOAD | HEADER | OPTIONAL ADAPTATION FIELD | PAYLOAD | ...
 +--------+---------------------------+---------+--------+---------------------------+---------+
 
                                      |         |                                    |         |
                                      +---------+                                    +---------+
                                           |                                              |
                                           +----------------------------------------------+
                                                                DATA

Using the library in your code

WARNING: the code below doesn't handle errors for readability purposes. However you SHOULD!

// Create a cancellable context in case you want to stop reading packets/data any time you want
ctx, cancel := context.WithCancel(context.Background())

// Handle SIGTERM signal
ch := make(chan os.Signal, 1)
signal.Notify(ch, syscall.SIGTERM)
go func() {
    <-ch
    cancel()
}()

// Open your file or initialize any kind of io.Reader
f, _ := os.Open("/path/to/file.ts")
defer f.Close()

// Create the demuxer
dmx := astits.New(ctx, f)
for {
    // Get the next data
    d, _ := dmx.NextData()
    
    // Data is a PMT data
    if d.PMT != nil {
        // Loop through elementary streams
        for _, es := range d.PMT.ElementaryStreams {
                fmt.Printf("Stream detected: %d\n", es.ElementaryPID)
        }
        return
    }
}

Options

In order to pass options to the demuxer, look for the methods prefixed with Opt and add them upon calling New:

// This is your custom packets parser
p := func(ps []*astits.Packet) (ds []*astits.Data, skip bool, err error) {
        // This is your logic
        skip = true
        return
}

// Now you can create a demuxer with the proper options
dmx := New(ctx, f, OptPacketSize(192), OptPacketsParser(p))

CLI

This library provides a CLI that will automatically get installed in GOPATH/bin on go get execution.

List streams

$ astits -i <path to your file> -f <format: text|json (default: text)>

List packets

$ astits packets -i <path to your file>

List data

$ astits data -i <path to your file> -d <data type: eit|nit|... (repeatable argument | if empty, all data types are shown)>

Features and roadmap

  • Parse PES packets
  • Parse PAT packets
  • Parse PMT packets
  • Parse EIT packets
  • Parse NIT packets
  • Parse SDT packets
  • Parse TOT packets
  • Parse BAT packets
  • Parse DIT packets
  • Parse RST packets
  • Parse SIT packets
  • Parse ST packets
  • Parse TDT packets
  • Parse TSDT packets
Comments
  • Add muxer

    Add muxer

    I've created a MPEG-TS muxer based on your demuxer library, as requested in #10 .

    I must say here that I had no intention for muxer to support everything demuxer supports. What I'm trying to do is to solve my tasks with the library (I guess that's how open source works anyway). So it's not full-featured, but it still helps a lot.

    TODOs and limitations:

    1. Most important: It doesn't really support PSI sections yet. Though it knows how to write tables, each table must fit in one TS packet. Mostly the limitation is there because I don't see the easy and clean way to do it now. And I don't really need it right now.
    2. Only PAT and PMT tables are supported. No SDT, NIT, EIT, etc. Though it won't be hard to add the support if needed.
    3. Muxer doesn't support more than one program (it's not hard to do). Number of elementary streams is not restricted.

    Additionally while working on the muxer I added some little things here and there, most notably:

    1. autoDetectPacketSize now knows how to work with bufio.Reader correctly
    2. ~~packetPool.add got an option flushOnPIDChange to overcome behaviour when one-packet tables doesn't get returned to demuxer at once.~~
    3. I've changed CRC32 API slightly to be able to update it incrementally.

    Sorry for the big pull request, but I don't see a way to do it in smaller parts :)

    opened by barbashov 14
  • Add a muxer

    Add a muxer

    i.e. I might want to read in a stream, modify it and output the stream.

    In fact, I did just that in a project I'm working on where I take the m2ts stream from libbluray, and add in language descriptors (from the buray clipinfo) and then serialize it out.

    enhancement help wanted 
    opened by sjpotter 13
  • Incorrect calculation of duration compared to ffprobe.

    Incorrect calculation of duration compared to ffprobe.

    Hey. Thanks for the project)

    I need to get the duration of the file, but I’m not getting the correct value compared to ffprobe.

    Here is an example of my code and file videoAndAudio.zip

    package main
    
    import (
    	"context"
    	"flag"
    	"fmt"
    	"os"
    	"os/signal"
    	"syscall"
    	"time"
    
    	astits "github.com/asticode/go-astits"
    )
    
    var (
    	tsfile  = flag.String("f", "", "path to ts file")
    	vebrose = flag.Bool("vebrose", false, "vebrose duration")
    )
    
    func main() {
    	flag.Parse()
    
    	// Create a cancellable context in case you want to stop reading packets/data any time you want
    	ctx, cancel := context.WithCancel(context.Background())
    
    	// Handle SIGTERM signal
    	ch := make(chan os.Signal, 1)
    	signal.Notify(ch, syscall.SIGTERM)
    	go func() {
    		<-ch
    		cancel()
    	}()
    
    	durationBasedPTS, err := GetChunkFileDurationPTS(ctx, *tsfile, *vebrose)
    	if err != nil {
    		fmt.Printf("Failed to open file: %v\n", err)
    		return
    	}
    	fmt.Println("Chunk duration PTS based:", durationBasedPTS.Nanoseconds())
    	durationBasedPCR, err := GetChunkFileDurationPCR(ctx, *tsfile, *vebrose)
    	if err != nil {
    		fmt.Printf("Failed to open file: %v\n", err)
    		return
    	}
    	fmt.Println("Chunk duration PCR based:", durationBasedPCR.Nanoseconds())
    }
    
    func GetChunkFileDurationPTS(ctx context.Context, filename string, vebrose bool) (time.Duration, error) {
    	var (
    		firstPTSDuration time.Duration
    		lastPTSDuration  time.Duration
    	)
    
    	// Open your file or initialize any kind of io.Reader
    	f, err := os.Open(filename)
    	if err != nil {
    		return 0, err
    	}
    	defer f.Close()
    	// Create the demuxer
    	dmx := astits.New(ctx, f)
    	for {
    		d, err := dmx.NextData()
    		if err != nil {
    			if err == astits.ErrNoMorePackets {
    				duration := lastPTSDuration - firstPTSDuration
    				return duration, nil
    			}
    			// Если есть ошибка и это не индикатор окончания данных, то
    			// возвращаем саму ошибку
    			return 0, err
    		}
    
    		if d.PES != nil {
    			ptsDuration := d.PES.Header.OptionalHeader.PTS.Duration()
    			if vebrose {
    				fmt.Println(ptsDuration.Nanoseconds())
    			}
    			if firstPTSDuration == 0 {
    				firstPTSDuration = ptsDuration
    			} else {
    				lastPTSDuration = ptsDuration
    			}
    		}
    	}
    }
    
    func GetChunkFileDurationPCR(ctx context.Context, filename string, vebrose bool) (time.Duration, error) {
    	var (
    		firstPCRDuration time.Duration
    		lastPCRDuration  time.Duration
    	)
    
    	// Open your file or initialize any kind of io.Reader
    	f, err := os.Open(filename)
    	if err != nil {
    		return 0, err
    	}
    	defer f.Close()
    	// Create the demuxer
    	dmx := astits.New(ctx, f)
    	for {
    		d, err := dmx.NextPacket()
    		if err != nil {
    			if err == astits.ErrNoMorePackets {
    				duration := lastPCRDuration - firstPCRDuration
    				return duration, nil
    			}
    			// Если есть ошибка и это не индикатор окончания данных, то
    			// возвращаем саму ошибку
    			return 0, err
    		}
    		// fmt.Println(d.AdaptationField.)
    		if d.Header.HasAdaptationField {
    			if d.AdaptationField.HasPCR {
    				pcrDuration := d.AdaptationField.PCR.Duration()
    				if vebrose {
    					fmt.Println(pcrDuration.Nanoseconds())
    				}
    				if firstPCRDuration == 0 {
    					firstPCRDuration = pcrDuration
    				} else {
    					lastPCRDuration = pcrDuration
    				}
    			}
    		}
    	}
    }
    
    

    example response:

    $ ffprobe -v 8 -hide_banner -of json -show_format  testdata/videoAndAudio.ts | jq .format.duration
    "5.052000"
    
    $ go run main.go -f testdata/videoAndAudio.ts
    Chunk duration PTS based: 4766666666
    Chunk duration PCR based: 4520000000
    

    Am I doing something wrong?
    Why do PCR and PTS values differ from each other, and ffprobe?

    bug 
    opened by jidckii 13
  • Muxer: increase Continuity Counter of PAT and PMT

    Muxer: increase Continuity Counter of PAT and PMT

    Hello, this library allowed rtsp-simple-server to convert seamlessly from RTSP to HLS during the last months, hence a big thank you from the entire community.

    As you may already know, HLS is a video/audio streaming format that uses MPEG-TS segments, containing H264 and AAC.

    The only change i had to do to generate valid HLS was hacking a couple of functions in order to increase the continuity counter of PAT and PMT after every delivery.

    Although the specification is not clear enough about the continuity counter of PAT and PMT, when it is not increased, VLC starts printing these kind of errors:

    [00007f80ec001180] ts demux error: libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 0
    [00007f80ec001180] ts demux error: libdvbpsi error (PSI decoder): TS duplicate (received 0, expected 1) for PID 4096
    

    At the same time, hls.js (JS library that converts from MPEG-TS to browser-compatible fMP4) is unable to generate a valid video stream.

    PAT and PMT buffers are reused in order to save RAM.

    opened by aler9 8
  • Decode packet encoded in ts from socket

    Decode packet encoded in ts from socket

    Hello,

    I would like to decode some packets encoded in H.264/AAC data encapsulated in MPEG2/TS container coming from tcp socket.

    How can I proceed using your library ?

    Is that possible ? If not, it can be a good improvement ! Thanks

    question 
    opened by AhmedX6 8
  • Performance optimization: using NextBytesNoCopy where possible

    Performance optimization: using NextBytesNoCopy where possible

    (For this to merge, https://github.com/asticode/go-astikit/pull/9 should be merged before.)

    I've changed a bunch of places in demuxer to use NextByteNoCopy instead of NextByte. I can't say it's night and day difference, but still it's a nice little improvement :) Benchmark results follow.

    Before:

    goos: darwin
    goarch: arm64
    pkg: github.com/asticode/go-astits
    BenchmarkParsePATSection-8   	12254799	        97.23 ns/op	      72 B/op	       7 allocs/op
    BenchmarkParsePESData
    BenchmarkParsePESData/without_header-8         	17240697	        69.27 ns/op	      56 B/op	       4 allocs/op
    BenchmarkParsePESData/with_header-8            	 4049978	       295.6 ns/op	     304 B/op	      18 allocs/op
    BenchmarkParsePMTSection-8                     	 4220971	       282.7 ns/op	     632 B/op	      15 allocs/op
    BenchmarkParsePSIData-8                        	  141775	      8532 ns/op	    5321 B/op	     169 allocs/op
    BenchmarkParseDescriptor
    BenchmarkParseDescriptor/AC3-8                                	 8806401	       136.0 ns/op	     288 B/op	       6 allocs/op
    BenchmarkParseDescriptor/ISO639LanguageAndAudioType-8         	 9917152	       121.1 ns/op	     288 B/op	       6 allocs/op
    BenchmarkParseDescriptor/MaximumBitrate-8                     	10645449	       113.2 ns/op	     260 B/op	       6 allocs/op
    BenchmarkParseDescriptor/NetworkName-8                        	 9511978	       126.6 ns/op	     280 B/op	       6 allocs/op
    BenchmarkParseDescriptor/Service-8                            	 8413123	       141.9 ns/op	     336 B/op	       7 allocs/op
    BenchmarkParseDescriptor/ShortEvent-8                         	 7699062	       156.6 ns/op	     344 B/op	       8 allocs/op
    BenchmarkParseDescriptor/StreamIdentifier-8                   	11814889	       101.9 ns/op	     254 B/op	       5 allocs/op
    BenchmarkParseDescriptor/Subtitling-8                         	 4409683	       271.7 ns/op	     412 B/op	      15 allocs/op
    BenchmarkParseDescriptor/Teletext-8                           	 5119393	       234.3 ns/op	     370 B/op	      11 allocs/op
    BenchmarkParseDescriptor/ExtendedEvent-8                      	 5383294	       222.2 ns/op	     416 B/op	      11 allocs/op
    BenchmarkParseDescriptor/EnhancedAC3-8                        	 8112375	       147.8 ns/op	     304 B/op	       6 allocs/op
    BenchmarkParseDescriptor/Extension-8                          	 6975066	       171.7 ns/op	     352 B/op	       8 allocs/op
    BenchmarkParseDescriptor/Component-8                          	 8063560	       148.5 ns/op	     324 B/op	       7 allocs/op
    BenchmarkParseDescriptor/Content-8                            	 7474143	       161.0 ns/op	     290 B/op	       8 allocs/op
    BenchmarkParseDescriptor/ParentalRating-8                     	 7275045	       164.7 ns/op	     320 B/op	       8 allocs/op
    BenchmarkParseDescriptor/LocalTimeOffset-8                    	 2556608	       468.0 ns/op	     392 B/op	      13 allocs/op
    BenchmarkParseDescriptor/VBIData-8                            	 6399276	       187.6 ns/op	     326 B/op	       9 allocs/op
    BenchmarkParseDescriptor/VBITeletext-8                        	 7012162	       170.5 ns/op	     320 B/op	       8 allocs/op
    BenchmarkParseDescriptor/AVCVideo-8                           	10797754	       111.3 ns/op	     264 B/op	       5 allocs/op
    BenchmarkParseDescriptor/PrivateDataSpecifier-8               	10777999	       112.2 ns/op	     260 B/op	       6 allocs/op
    BenchmarkParseDescriptor/DataStreamAlignment-8                	11817196	       101.8 ns/op	     254 B/op	       5 allocs/op
    BenchmarkParseDescriptor/PrivateDataIndicator-8               	10707218	       112.0 ns/op	     260 B/op	       6 allocs/op
    BenchmarkParseDescriptor/UserDefined-8                        	12038341	       100.3 ns/op	     256 B/op	       5 allocs/op
    BenchmarkParseDescriptor/Registration-8                       	 8706368	       137.6 ns/op	     292 B/op	       7 allocs/op
    BenchmarkParseDescriptor/Unknown-8                            	 9858508	       121.5 ns/op	     288 B/op	       6 allocs/op
    BenchmarkParseDescriptor/Extension#01-8                       	 8554720	       140.0 ns/op	     304 B/op	       7 allocs/op
    BenchmarkParsePacket-8                                        	 4242537	       281.3 ns/op	     448 B/op	      15 allocs/op
    BenchmarkDemuxer_NextData-8   	     	     	     	     	  179368	        6088 ns/op	    6410 B/op	     188 allocs/op
    

    After:

    goos: darwin
    goarch: arm64
    pkg: github.com/asticode/go-astits
    BenchmarkParsePATSection-8      14293448            83.59 ns/op       64 B/op          5 allocs/op
    BenchmarkParsePESData
    BenchmarkParsePESData/without_header-8          20300631            58.82 ns/op       52 B/op          3 allocs/op
    BenchmarkParsePESData/with_header-8              5215488           229.1 ns/op       280 B/op         10 allocs/op
    BenchmarkParsePMTSection-8                       5153559           232.2 ns/op       618 B/op          9 allocs/op
    BenchmarkParsePSIData-8                           150284          7986 ns/op        5049 B/op        115 allocs/op
    BenchmarkParseDescriptor
    BenchmarkParseDescriptor/AC3-8                                   9969420           119.7 ns/op       284 B/op          4 allocs/op
    BenchmarkParseDescriptor/ISO639LanguageAndAudioType-8           11080315           108.2 ns/op       284 B/op          4 allocs/op
    BenchmarkParseDescriptor/MaximumBitrate-8                       14564409            82.07 ns/op      252 B/op          3 allocs/op
    BenchmarkParseDescriptor/NetworkName-8                          11150904           108.1 ns/op       276 B/op          4 allocs/op
    BenchmarkParseDescriptor/Service-8                               9185078           130.7 ns/op       328 B/op          5 allocs/op
    BenchmarkParseDescriptor/ShortEvent-8                            8483942           141.8 ns/op       344 B/op          6 allocs/op
    BenchmarkParseDescriptor/StreamIdentifier-8                     15024884            79.83 ns/op      249 B/op          3 allocs/op
    BenchmarkParseDescriptor/Subtitling-8                            5325788           226.1 ns/op       398 B/op          9 allocs/op
    BenchmarkParseDescriptor/Teletext-8                              5553873           215.2 ns/op       366 B/op          9 allocs/op
    BenchmarkParseDescriptor/ExtendedEvent-8                         5627692           209.0 ns/op       416 B/op          9 allocs/op
    BenchmarkParseDescriptor/EnhancedAC3-8                           9031544           132.8 ns/op       300 B/op          4 allocs/op
    BenchmarkParseDescriptor/Extension-8                             7636591           157.5 ns/op       352 B/op          6 allocs/op
    BenchmarkParseDescriptor/Component-8                             9726552           123.7 ns/op       320 B/op          5 allocs/op
    BenchmarkParseDescriptor/Content-8                               9240913           129.0 ns/op       283 B/op          5 allocs/op
    BenchmarkParseDescriptor/ParentalRating-8                        8024780           150.0 ns/op       316 B/op          6 allocs/op
    BenchmarkParseDescriptor/LocalTimeOffset-8                       2829170           424.7 ns/op       376 B/op          7 allocs/op
    BenchmarkParseDescriptor/VBIData-8                               6965751           172.3 ns/op       322 B/op          7 allocs/op
    BenchmarkParseDescriptor/VBITeletext-8                           7676050           156.5 ns/op       315 B/op          6 allocs/op
    BenchmarkParseDescriptor/AVCVideo-8                             13757174            87.12 ns/op      256 B/op          3 allocs/op
    BenchmarkParseDescriptor/PrivateDataSpecifier-8                 14734185            82.31 ns/op      252 B/op          3 allocs/op
    BenchmarkParseDescriptor/DataStreamAlignment-8                  14950507            80.34 ns/op      249 B/op          3 allocs/op
    BenchmarkParseDescriptor/PrivateDataIndicator-8                 14538712            82.22 ns/op      252 B/op          3 allocs/op
    BenchmarkParseDescriptor/UserDefined-8                          14633461            82.71 ns/op      252 B/op          3 allocs/op
    BenchmarkParseDescriptor/Registration-8                         11132430           108.7 ns/op       284 B/op          4 allocs/op
    BenchmarkParseDescriptor/Unknown-8                              11541582           104.9 ns/op       284 B/op          4 allocs/op
    BenchmarkParseDescriptor/Extension#01-8                          9737190           123.8 ns/op       300 B/op          5 allocs/op
    BenchmarkParsePacket-8                                           5243634           229.4 ns/op       416 B/op          9 allocs/op
    BenchmarkDemuxer_NextData-8   	     	     	     	     	  206743	        5514 ns/op	    6082 B/op	     122 allocs/op
    
    opened by barbashov 5
  • Q: aac and h264 codecs

    Q: aac and h264 codecs

    In here https://github.com/asticode/go-astits/issues/9 you have said to use data.PMT.ElementaryStreams to retrieve information about tracks, essentially, and data.PES.Data to get the actual data, which I am used to from https://github.com/nareix/joy4.

    But it seems you have no support for h264 video and very small for audio - I can see you have defined only few audio stream types:

    // Stream types
    const (
    	StreamTypeLowerBitrateVideo          = 27 // ITU-T Rec. H.264 and ISO/IEC 14496-10
    	StreamTypeMPEG1Audio                 = 3  // ISO/IEC 11172-3
    	StreamTypeMPEG2HalvedSampleRateAudio = 4  // ISO/IEC 13818-3
    	StreamTypeMPEG2PacketizedData        = 6  // ITU-T Rec. H.222 and ISO/IEC 13818-1 i.e., DVB subtitles/VBI and AC-3
    )
    

    so I wonder why not provide the packet parser as well, ie. https://github.com/nareix/joy4/tree/master/codec, since mepgts is mostly used with h264 anyway?

    Also what is the difference between data.PES.Header.OptionalHeader.PTS, data.PES.Header.OptionalHeader.DTS and data.PES.Header.OptionalHeader.ESCR?

    I would assume that one is the duration "timestamp" of the packet on the media timeline, the other is the duration of the packet itself and the third is the duration since the stream begun(ie. not the data stream, but the connection).

    question 
    opened by ivanjaros 5
  • End of PES payload is truncated for certain streams

    End of PES payload is truncated for certain streams

    I'm trying to extract timed ID3 data, which contains a JSON payload, but the end of every PES payload seems to be truncated by a few bytes. This also happens when attempting to extract the AAC audio (results in a corrupted file), the H264 stream seems to extract fine though.

    I've compared this to extracting the data using ffmpeg (ffmpeg -i test.ts -map 0:2 -c copy -f data ID3.bin), ffmpeg actually has it's own issues in that it doesn't output the magic number at the start of the stream for whatever reason, but it seems to correctly extract the payload otherwise.

    Here is an example ts file to use, and here is the extracted ID3 stream using astits vs ffmpeg. You can see the first payload astits extracted ends in "transc_s":15694516, while ffmpeg is "transc_s":1569451686155}

    You can see the code I'm using to extract the PES payload here: https://gist.github.com/Hakkin/6cab34dcf004494d436025a1846633c0 It's possible I'm doing something wrong in my code, but it seems fairly straightforward, so I'm not sure.

    bug 
    opened by Hakkin 5
  • PES parsing fails with out of range index

    PES parsing fails with out of range index

    Using the provided example program in master, I get an error trying to dump PES info.

    ERRO[0000] astits: fetching data failed: astits: getting next data failed: astits: building new data failed: astits: parsing PES data failed: astits: getting next bytes failed: astits: slice length is 15922, offset 15935 is invalid
    

    This seems to have broke @https://github.com/asticode/go-astits/commit/8d58895ff21f7636d41169fad8e71d93de72fff7.

    Here's an example ts that triggers this if you need it.

    The command I'm running is astits data -d all -i test.ts

    bug 
    opened by Hakkin 5
  • Logging

    Logging

    Hello, I am getting strange descriptors in https://github.com/asticode/go-astits/blob/master/descriptor.go#L1239 in my stream and your package spam messages from https://github.com/asticode/go-astits/blob/master/descriptor.go#L1398. Can we refactor message logging in one of the following ways:

    • pass an instance of the logger (interface) and log in to it. There is a disadvantage that several pure functions will have one more logger argument.
    • do not log this message at all, then, however, we will lose this information

    What do you think about it?

    P.S. I can create pull request, if you agree with me)

    question 
    opened by kravtsov-dima 4
  • parseDescriptors panics

    parseDescriptors panics

    I have a test file that produces a panic in parseDescriptors from current master:

    $ astits -i test.ts -v
    DEBU[0000] Fetching data...
    DEBU[0000] astits: unlisted descriptor tag 0x6
    DEBU[0000] astits: unlisted descriptor tag 0xf
    DEBU[0000] astits: unlisted descriptor tag 0xfe
    DEBU[0000] astits: unlisted descriptor tag 0xf
    DEBU[0000] astits: unlisted descriptor tag 0xfe
    DEBU[0000] astits: unlisted descriptor tag 0xf
    DEBU[0000] astits: unlisted descriptor tag 0xfe
    DEBU[0000] astits: unlisted descriptor tag 0xf
    DEBU[0000] astits: unlisted descriptor tag 0x90
    DEBU[0000] astits: unlisted descriptor tag 0xfe
    DEBU[0000] astits: unlisted descriptor tag 0xf
    ...
    DEBU[0000] astits: unlisted descriptor tag 0xfe         
    DEBU[0000] astits: unlisted descriptor tag 0xf          
    DEBU[0000] astits: unlisted descriptor tag 0xfe         
    DEBU[0000] astits: unlisted descriptor tag 0xf          
    DEBU[0000] astits: unlisted descriptor tag 0x90         
    DEBU[0000] astits: unlisted descriptor tag 0xfe         
    DEBU[0000] astits: unlisted descriptor tag 0xf          
    DEBU[0000] astits: unlisted descriptor tag 0x90         
    DEBU[0000] astits: unlisted descriptor tag 0xfe         
    DEBU[0000] astits: unlisted descriptor tag 0xf
    panic: runtime error: index out of range
    
    goroutine 1 [running]:
    github.com/asticode/go-astits.parseDescriptors(0xc420420240, 0xb8, 0xb8, 0xc4200538b8, 0x5, 0xc42029d940, 0x4)
    	/home/chalupecky/src/github.com/asticode/go-astits/descriptor.go:791 +0x10b2
    github.com/asticode/go-astits.parsePMTSection(0xc420420240, 0xb8, 0xb8, 0xc4200538b8, 0x11e, 0x411b1c, 0x10)
    	/home/chalupecky/src/github.com/asticode/go-astits/data_pmt.go:51 +0x1a2
    github.com/asticode/go-astits.parsePSISectionSyntaxData(0xc420420240, 0xb8, 0xb8, 0xc4200538b8, 0xc4200c7780, 0xc4201893e0, 0x11e, 0x1)
    	/home/chalupecky/src/github.com/asticode/go-astits/data_psi.go:318 +0x57c
    github.com/asticode/go-astits.parsePSISectionSyntax(0xc420420240, 0xb8, 0xb8, 0xc4200538b8, 0xc4200c7780, 0x11e, 0x4)
    	/home/chalupecky/src/github.com/asticode/go-astits/data_psi.go:261 +0xc2
    github.com/asticode/go-astits.parsePSISection(0xc420420240, 0xb8, 0xb8, 0xc4200538b8, 0x524477, 0xc4200bc008, 0xc4200b8030, 0xc4200538c6)
    	/home/chalupecky/src/github.com/asticode/go-astits/data_psi.go:120 +0x174
    github.com/asticode/go-astits.parsePSIData(0xc420420240, 0xb8, 0xb8, 0xc420420201, 0xb8, 0xb8)
    	/home/chalupecky/src/github.com/asticode/go-astits/data_psi.go:93 +0x104
    github.com/asticode/go-astits.parseData(0xc42000e1f0, 0x1, 0x1, 0x0, 0xc4200bc008, 0xc4200b8030, 0x0, 0x409130, 0xc4201893c2, 0xea3e6e4c, ...)
    	/home/chalupecky/src/github.com/asticode/go-astits/data.go:58 +0x453
    github.com/asticode/go-astits.(*Demuxer).NextData(0xc4200be000, 0xc420053c60, 0xc4201893c2, 0xc420053cf0)
    	/home/chalupecky/src/github.com/asticode/go-astits/demuxer.go:128 +0x156
    main.programs(0xc4200be000, 0xc420010200, 0x673b20, 0xc4200b4000, 0x0, 0x0)
    	/home/chalupecky/src/github.com/asticode/go-astits/astits/main.go:238 +0x31f
    main.main()
    	/home/chalupecky/src/github.com/asticode/go-astits/astits/main.go:79 +0x2dc
    
    opened by vladimir-ch 4
  • Implement ATSC standard as well

    Implement ATSC standard as well

    I'm trying to read the program listings from the channel data stream from a HDHomerun. For now, I'm just using curl to pull from the device into a file. I am taking the sample app and just reporting on any EIT packets. My app finds none. If I use dvbtee to dump out data, I definitely get schedule listings from the same file. Looking for some guidance around retrieving the program listings. Is the EIT packet the correct one? Is that something that the library supports? (The checkbox indicates it should...)

    Thanks!

    opened by a2geek 7
  • Major optimization 6x faster

    Major optimization 6x faster

    RTP to MPEGTS benchmark.
    Before: 33556858 ns/op  4763101 B/op
    After:   5169437 ns/op  3881852 B/op
    

    I wrote this a few months ago and have been using it in production for weeks on end without issues. I no longer have a use for mpegts library, so I figured you could have it. There may be some small breaking changes, but all tests pass.

    Changes:

    • Use icza/bitio for bit manipulation
    • Dynamic CRC32 calculation
    • Don't use fmt.Sprintf when parsing timestamp
    opened by Curid 1
  • Handle truncated PES packets

    Handle truncated PES packets

    Simple fix for #35

    Unfortunately this is breaking some tests which are creating and operating on malformed packets.

    --- FAIL: TestParsePESData (0.00s)
        --- FAIL: TestParsePESData/with_header (0.00s)
            data_pes_test.go:407:
                	Error Trace:	data_pes_test.go:407
                	Error:      	Not equal:
                	            	expected: &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61}, Header:(*astits.PESHeader)(0x1381690)}
                	            	actual  : &astits.PESData{Data:[]uint8{0x64, 0x61, 0x74, 0x61, 0x73, 0x74, 0x75, 0x66, 0x66}, Header:(*astits.PESHeader)(0xc0000534a0)}
    
                	            	Diff:
                	            	--- Expected
                	            	+++ Actual
                	            	@@ -1,4 +1,4 @@
                	            	 (*astits.PESData)({
                	            	- Data: ([]uint8) (len=4) {
                	            	-  00000000  64 61 74 61                                       |data|
                	            	+ Data: ([]uint8) (len=9) {
                	            	+  00000000  64 61 74 61 73 74 75 66  66                       |datastuff|
                	            	  },
                	            	@@ -62,3 +62,3 @@
                	            	   }),
                	            	-  PacketLength: (uint16) 67,
                	            	+  PacketLength: (uint16) 9,
                	            	   StreamID: (uint8) 1
                	Test:       	TestParsePESData/with_header
    --- FAIL: TestParseData (0.00s)
        data_test.go:47:
            	Error Trace:	data_test.go:47
            	Error:      	Not equal:
            	            	expected: []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073770)}
            	            	actual  : []*astits.DemuxerData{(*astits.DemuxerData)(0xc000073720)}
    
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -24,4 +24,4 @@
            	            	   PES: (*astits.PESData)({
            	            	-   Data: ([]uint8) (len=4) {
            	            	-    00000000  64 61 74 61                                       |data|
            	            	+   Data: ([]uint8) (len=9) {
            	            	+    00000000  64 61 74 61 73 74 75 66  66                       |datastuff|
            	            	    },
            	            	@@ -85,3 +85,3 @@
            	            	     }),
            	            	-    PacketLength: (uint16) 67,
            	            	+    PacketLength: (uint16) 9,
            	            	     StreamID: (uint8) 1
            	Test:       	TestParseData
    FAIL
    FAIL	github.com/asticode/go-astits	0.008s
    FAIL
    

    Specifically, the with_header PES test advertises a packet length of 67 even though it only has 9 bytes of data:

    https://github.com/asticode/go-astits/blob/473f66c0d7ff8e35f155c629471b214018b61fe0/data_pes_test.go#L258-L265

    https://github.com/asticode/go-astits/blob/473f66c0d7ff8e35f155c629471b214018b61fe0/data_pes_test.go#L329-L332

    I tried to fix these tests but they make a lot of assumptions and reuse fixtures like pesWithHeaderBytes() which creates quite the rabbit hole.

    opened by tmm1 6
  • demuxer fails when corrupted PES packet advertises incorrect length

    demuxer fails when corrupted PES packet advertises incorrect length

    I have a sample that looks like this:

    $ ./astits-probe packets -i id3.ts
    2022/01/21 12:07:04 Fetching packets...
    2022/01/21 12:07:04 PKT: 0
    2022/01/21 12:07:04   Continuity Counter: 4
    2022/01/21 12:07:04   Payload Unit Start Indicator: true
    2022/01/21 12:07:04   Has Payload: true
    2022/01/21 12:07:04   Has Adaptation Field: false
    2022/01/21 12:07:04   Transport Error Indicator: false
    2022/01/21 12:07:04   Transport Priority: false
    2022/01/21 12:07:04   Transport Scrambling Control: 0
    2022/01/21 12:07:04 PKT: 480
    2022/01/21 12:07:04   Continuity Counter: 4
    2022/01/21 12:07:04   Payload Unit Start Indicator: true
    2022/01/21 12:07:04   Has Payload: true
    2022/01/21 12:07:04   Has Adaptation Field: false
    2022/01/21 12:07:04   Transport Error Indicator: false
    2022/01/21 12:07:04   Transport Priority: false
    2022/01/21 12:07:04   Transport Scrambling Control: 0
    2022/01/21 12:07:04 PKT: 502
    2022/01/21 12:07:04   Continuity Counter: 13
    2022/01/21 12:07:04   Payload Unit Start Indicator: true
    2022/01/21 12:07:04   Has Payload: true
    2022/01/21 12:07:04   Has Adaptation Field: false
    2022/01/21 12:07:04   Transport Error Indicator: false
    2022/01/21 12:07:04   Transport Priority: false
    2022/01/21 12:07:04   Transport Scrambling Control: 0
    2022/01/21 12:07:04 PKT: 502
    2022/01/21 12:07:04   Continuity Counter: 14
    2022/01/21 12:07:04   Payload Unit Start Indicator: true
    2022/01/21 12:07:04   Has Payload: true
    2022/01/21 12:07:04   Has Adaptation Field: true
    2022/01/21 12:07:04   Transport Error Indicator: false
    2022/01/21 12:07:04   Transport Priority: false
    2022/01/21 12:07:04   Transport Scrambling Control: 0
    2022/01/21 12:07:04   Adaptation Field: &{AdaptationExtensionField:<nil> DiscontinuityIndicator:false ElementaryStreamPriorityIndicator:false HasAdaptationExtensionField:false HasOPCR:false HasPCR:false HasTransportPrivateData:false HasSplicingCountdown:false Length:80 IsOneByteStuffing:false StuffingLength:79 OPCR:<nil> PCR:<nil> RandomAccessIndicator:false SpliceCountdown:0 TransportPrivateDataLength:0 TransportPrivateData:[]}
    2022/01/21 12:07:04 PKT: 502
    2022/01/21 12:07:04   Continuity Counter: 15
    2022/01/21 12:07:04   Payload Unit Start Indicator: false
    2022/01/21 12:07:04   Has Payload: true
    2022/01/21 12:07:04   Has Adaptation Field: true
    2022/01/21 12:07:04   Transport Error Indicator: false
    2022/01/21 12:07:04   Transport Priority: false
    2022/01/21 12:07:04   Transport Scrambling Control: 0
    2022/01/21 12:07:04   Adaptation Field: &{AdaptationExtensionField:<nil> DiscontinuityIndicator:false ElementaryStreamPriorityIndicator:false HasAdaptationExtensionField:false HasOPCR:false HasPCR:false HasTransportPrivateData:false HasSplicingCountdown:false Length:82 IsOneByteStuffing:false StuffingLength:81 OPCR:<nil> PCR:<nil> RandomAccessIndicator:false SpliceCountdown:0 TransportPrivateDataLength:0 TransportPrivateData:[]}
    

    which fails to demux with this error:

    $ ./astits-probe data -i id3.ts
    2022/01/21 12:09:22 Fetching data...
    2022/01/21 12:09:22 astits: fetching data failed: astits: getting next data failed: astits: building new data failed: astits: parsing PES data failed: astits: fetching next bytes failed: astikit: slice length is 184, offset 285 is invalid
    

    http://0x0.st/oo1F.bin

    opened by tmm1 2
Owner
Quentin Renard
Freelance | Senior backend developer (GO)
Quentin Renard
Go4vl is Go library for working with the Video for Linux API (V4L2) natively, without any C bindings.

go4vl A Go library for working with the Video for Linux user API (V4L2). Gov4l hides all the complexities of working with V4L2 and exposes idiomatic G

Vladimir Vivien 128 Nov 17, 2022
Parse and generate m3u8 playlists for Apple HTTP Live Streaming (HLS) in Golang (ported from gem https://github.com/sethdeckard/m3u8)

go-m3u8 Golang package for m3u8 (ported m3u8 gem https://github.com/sethdeckard/m3u8) go-m3u8 provides easy generation and parsing of m3u8 playlists d

Tan Quang Ngo 102 Nov 19, 2022
Package implement reading and writing popular playlist formats: PLS, ASX, M3U, XSPF and others.

go-playlist ⚠️ WARNING The API is not stable yet and can change. Package playlist implement reading and writing popular playlist formats: PLS, ASX, M3

Roman Kulish 0 Oct 14, 2021
ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically process video and manipulate subtitles.

% FFCOMMANDER(1) ffcommander 2.39 % Mikael Hartzell (C) 2018 % 2021 Name ffcommander - An easy frontend to FFmpeg and Imagemagick to automatically pro

Mikael Hartzell 2 May 9, 2022
golang library to read and write various subtitle formats

libgosubs Golang library to read and write subtitles in the following formats Advanced SubStation Alpha v4 SRT TTML v1.0 - This is based on the spec p

Wargarblgarbl 20 Sep 27, 2022
Go bindings for libVLC and high-level media player interface

Go bindings for libVLC 2.X/3.X/4.X and high-level media player interface. The package can be useful for adding multimedia capabilities to applications

Adrian-George Bostan 325 Nov 23, 2022
Parser and generator of M3U8-playlists for Apple HLS. Library for Go language. :cinema:

M3U8 This is the most complete opensource library for parsing and generating of M3U8 playlists used in HTTP Live Streaming (Apple HLS) for internet vi

Alexander I.Grafov 978 Nov 17, 2022
Plays videos using Prometheus and Grafana, e.g. Bad Apple.

prometheus_video_renderer Plays videos using Prometheus and Grafana, e.g. Bad Apple. Modes Currently 3 different modes are supported. Bitmap The bitma

Jacob Colvin 91 Nov 15, 2022
RTSP 1.0 client and server library for the Go programming language

RTSP 1.0 client and server library for the Go programming language

Alessandro Ros 323 Nov 21, 2022
falco is VCL parser and linter optimized for Fastly.

falco falco is VCL parser and linter optimized for Fastly. Disclaimer This is a VCL parser, but dedicated to Fastly's VCL (version 2.x), so we don't c

Yoshiaki Sugimoto 62 Nov 14, 2022
Desktop application to download videos and playlists from youtube by simply copying its url.

tubemp3 Desktop application to download videos and playlists from youtube by simply copying its url. You just need to run tubemp3 and copy (CTRL + C)

Fernando Guevara Sanchez 13 Oct 25, 2022
A simple library to extract video and audio frames from media containers (based on libav).

Reisen A simple library to extract video and audio frames from media containers (based on libav, i.e. ffmpeg). Dependencies The library requires libav

NightGhost 73 Nov 15, 2022
Project to get Youtube video descriptions and search those videos as required

FamPayProject Project to get Youtube video descriptions and search those videos as required Prerequisities Postgres DB for persisting data Youtube Dat

null 0 Nov 5, 2021
Synthetic media is a realistic transformation of audio and video using artificial intelligence.

Synthetic media is a realistic transformation of audio and video using artificial intelligence.

null 1 Nov 20, 2021
golang function that download a video from youtube, and convert it to a mp3 file using ffmpeg

echedwnmp3 echedwnmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg example package main import(echedwn

pai 4 Dec 7, 2021
lmmp3 is a little golang library that download a video from youtube, and convert it to a mp3 file using ffmpeg

lmmp3 lmmp3 is a function that download a video from youtube, and convert it to a mp3 file using ffmpeg You need to have installed ffmpeg in your syst

pai 8 Aug 12, 2022
👾 Annie is a fast, simple and clean video downloader built with Go.

?? Annie is a fast, simple and clean video downloader built with Go. Installation Prerequisites Install via go install Homebrew (macOS only) Arch Linu

Rohan 1 Jun 1, 2022
[WIP] a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg software

About a very simple, tiny and intuitive ffmpeg wrapper with a cli interface for inspecting & transforming media files supported by the original ffmpeg

Mohammed Al Ashaal 10 Oct 21, 2022
golang library for mux and demux file containers such as mpeg-ts,mpeg-ps,flv

gomedia mpeg-ts,mpeg-ps,flv muxer/demuxer mpeg-ts muxer mpeg-ts demuxer mpeg-ps muxer mpeg-ps demuxer flv muxer flv demuxer mpeg-ps will be done in th

yaping 105 Nov 20, 2022
Transport to allow go-libp2p applications to natively use i2p for communication

I2P Transport for go-libp2p This library can be used to build go-libp2p applications using the i2p network. Look at transport_test.go for example usag

null 2 Sep 15, 2022