MPEG - Motion Pictures Expert Group

Index

General


MPEG

ITU-T (joint) ISO-IEC (standards) part name
layer free public
standard
doc implementation application



JTC SC WG number title name



ISO/IEC ITU MPEG technologies Tutorials and seminars
MPEG Visions /
programari maquinari



1 (Information Technology)
24 (Computer graphics and image processing)

GKS, PHIGS, CGM, VRML, X3D














29 (Coding of audio, picture, multimedia and hypermedia) 1 (Coding of still pictures)
03
(11544)
JBIG-1














04
(10918)
JPEG














10
(14492)
JBIG-2














12
(14495)
JPEG-LS














14
(15444)
JPEG 2000














16
(16485)
MRC














20
(24800)
JPSearch














11 (Coding of moving pictures and audio) (MPEG) 11172 MPEG-1 Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s 1 Systems  - Program streams











2 Video for CD-I











3 Audio
I










II










III (mp3)









4 Conformance

x








5 Technical report - Software simulation

x








13818 MPEG-2
Generic coding of moving pictures and associated audio information 1 Systems (TS, PES)
(amdt. 6 "Transport of scalable video over ITU-T Rec H.220/ ISO/IEC 13818-1": AVC_video_descriptor)


x



ts-muxer

H.262 (doc) VCEG
2 Video (profiles/levels)


07/95

ffmpeg




3 Audio











4 Conformance

x








5 Technical report

x








6 DSM-CC





programari / software





7 AAC











8












9 RTI











10 Conformance Extensions for DSMCC











11 IPMP on MPEG2











14496 (c)
MPEG-4
Coding of audio-visual objects 1 Sytems










2 Visual (profiles/levels) SP






H.263 (doc)

ASP

01/05






3 Audio
(AAC)










4 Conformance

x








5 Reference software

x








6 DMIF










7 Optimized software

x








8 MPEG-4 on IP











9 Reference hardware








H.264 (1,2)
(doc)
VCEG
10 AVC (profiles/levels)


11/07
03/10 04/17
corrigendum

JVT SVC (Annex G)







MVC (Annex H)







11 Scene Description and Application Engine (BIFS
MPEG-J)












12 ISO Base Media File Format


x



(MP4, 3GP, MJP2, MP21, AVC)



AMD  1: General Improvements including hint tracks, metadata support and sample groups











AMD 3: DASH (sidx, ssix), RTP reception hint












13 IPMP Extensions











14 MP4 File format




(extension of part 12)





15 Advanced Video Coding File Format
  •  (amd. 2008: includes SVC)
  • amdt 2: avc3





(extension of part 12)





16 AFX (3D)










17 Streaming Text Format











18 Font Compression and Streaming











19 Synthesized Streams











20 LASeR

x








SAF










21 MPEG-J Graphical Framework eXtension (GFX) (3D)










22 Open Font Format











23 Symbolic Music Representation (SMR)











24 Audio and systems interaction











25 3D Graphics Compression Model










26
Audio conformance











27
3D Graphics conformance











28
Composite Font












29
Web Video Coding












30
Timed Text and Other Visual Overlays in ISO Base Media File Format











15938 (c)
MPEG-7
Multimedia content description interface 1 Systems




programari / software






2 DDL









3 Visual









4 Audio









5 Multimedia description schemes









6 Reference software

x






7 Conformance testing

x






8 Extraction and use of description









9 MPEG-7 Profiles









10 Schema Definition

x






11 Profile schemas

x






21000 (c)

MPEG-21

Multimedia framework 1 Technical Report

x

programari / software





2 DID

x






3 DII









4 IPMP









5 Rights Expression Language









6 Rights Data Dictionary









7 DIA









8 Reference Software

x






9 File Format





MP21Box



10 DIP









11 Evaluation Methods for Persistent Association









12 Test Bed for MPEG-21 Resource Delivery









13 SVC









14 Conformance Testing









15 Event Reporting









16 Binary Format









17 Fragment Identification of MPEG Resources









18 DIS









23000 MPEG-A Multimedia Application Format - MAF 1 Purpose of MAFs











2 Music player AF











3 Photo player AF











4 Musical slide show AF











5 Media streaming AF











6 Professional archiver AF











7 Open access AF











8 Portable video AF











9 Digital Multimedia Broadcasting - DMB AF











10 Video surveillance AF











11 Stereoscopic video AF











23001 MPEG-B MPEG systems technologies 1
Binary MPEG format for XML











2
Fragment Request Units











3
XML IPMP messages











4
Codec configuration representation











5
Bitstream Syntax Description Language (BSDL)











6
Dynamic Adaptive Streaming over HTTP (DASH) (moved to 23009)


DIS







7
Common encryption in ISO base media file format











8
Coding-independent code-points











9
Common Encryption for MPEG-2 Transport Streams











23002 MPEG-C MPEG video technologies 1 Accuracy requirements for implementation of integer-output 8x8 inverse discrete cosine transform











2 Fixed-point 8x8 inverse discrete cosine transform and discrete cosine transform











3 Representation of auxiliary video streams and supplemental information (2D+Depth)











4
Video tool library











23003 MPEG-D MPEG audio technologies 1
MPEG Surround











2
Spatial Audio Object Coding (SAOC)











3
Unified Speech and Audio Coding (USAC)











23004 MPEG-E Multimedia Middleware 1
Architecture











2
Multimedia application programming interface (API)











3
Component model











4
Resource and quality management











5
Component download











6
Fault management











7
System integrity management











8
Reference software











23008
MPEG-H
High Efficiency Coding and Media Delivery in Heterogeneous Environments 1
MMT MPEG Media Transport







H.265 / H.NGVC  / ...
JCT
2
HEVC (High Efficiency Video Coding)









3
Audio













(MPEG-J)














23006
MPEG-M
MPEG extensible middleware (MXM)
Multimedia Service Platform Technologies
1
MXM architecture and technologies










2
MXM API











3
MXM Reference software










4
MXM Protocols








H.721 Q.13 / SG16

ITU-T: The IPTV Terminal (Basic Model)


03/09





?
Advanced IPTV Terminal










(Mobile IPTV Terminal)












23007
MPEG-U
Rich media user interfaces (widgets)
1
Widgets






MPEG-U with GPAC (GPAC)





2
Advanced User Interaction Interface









3
Conformance and reference software









23005 MPEG-V Information exchange with virtual worlds (Metaverse)
Media Context and Control
1 Architecture
















2 Control information










3
Sensory information










4
Virtual world object characteristics










5
Data formats for interaction devices










6
Common types and tools










7
Conformance and reference software


























23009


DASH
Dynamic adaptive streaming over HTTP (formerly 23001-6)


1
Media presentation description and segment formats












2
Conformance and reference software for DASH

w12167






3
Implementation Guidelines of DASH

w12168






12 13522 MHEG
5 MHEG-5











JTC SC WG number title name



ISO/IEC ITU

programari maquinari
ITU-T (joint) ISO-IEC (standards) part name
layer free public
standard

doc implementation application


  • MPEG technologies (summary, 1-pager, white paper, presentation)



    standards
    1. Media Coding






    2D video coding MPEG-1 video MPEG-1/2
    MPEG-2 Video MPEG-2/2
    MPEG-4 Visual (rect) MPEG-4/2
    Shape coding
    Advanced video coding MPEG-4/10
    Scalable video coding MPEG-4/10
    Multi-view coding
    MPEG-4/10
    High performance video coding

    Decoder representation Reconfigurable video coding

    Coding tool repository

    3D video coding Auxiliary video data
    MPEG-C/3 ?
    3D video coding

    Audio coding ...

    2D graphic coding Texture coding

    2D mesh coding

    3D graphic coding Face and body animation

    3D mesh coding

    AFX

    Synthetic audio coding ...

    Text coding ...

    Font coding
    Font compression and streaming

    Open Font Format

    Music coding
    ...

    Media Context and control
    Control information

    Sensory information

    Virtual object characteristics

    Media value chains


    2. Composition coding
    Composition coding
    Binary format for scenes

    Audio BIFS

    BIFS for digital audio

    Lightweigth scene representation
    MPEG-4/20
    Presentation and modification od structured information

    3. Description coding
    Description technologies
    Description definition language

    MPEG-7 schemas

    Video description
    Low level descriptions

    High level descriptions

    Overview

    Visual description tools

    Image and video signature

    Audio description
    Low level descriptions
    High level descriptions
    Multimedia description
    Multimedia description schemes

    4. Systems support
    Multiplexing and synchronization
    MPEG-1

    MPEG-2
    MPEG-4
    Signaling
    DSM-CC user to user

    DSM-CC user to network

    DMIF

    5. IPMP
    General
    MPEG technologies for DRM

    Identification technologies
    MPEG-2 copyright identifier

    Object content information

    Digital Item identification
    MPEG-21/3
    Rights expression technologies
    ...

    Persitent association technologies
    ...

    Access technologies


    6. Digital Item
    Digital item technologies
    Digital item declaration
    MPEG-21/2
    Digital item processing

    C++ bindings

    Session mobility

    Event reporting

    Schema files

    Digital item representation

    Resources in Digital Items
    Digital item adaptation

    Fragment identification for MPEG resources

    7. Transport and file format
    Transport of media streams
    Program stream
    MPEG-1/1
    Transport stream
    MPEG-2/1
    M4Mux
    MPEG Media Transport

    Media file formats
    ISO base media file formats
    MPEG-4/12
    MPEG-4 file format
    MPEG-4/14
    AVC file format
    MPEG-4/15
    SVC file format
    MPEG-4/15
    MVC file format

    Digital item file format
    MPEG-21/9
    Transport of digital items
    Digital item streaming

    8. User interaction
    User interaction
    Widgets
    MPEG-U
    Advanced user interaction

    9. Multimedia architecture
    Terminal architecture
    ...

    Application programming interfaces
    MPEG-J

    MPEG-J GFX
    MPEG-4/21
    M3W multimedia API

    MXM API
    MPEG-M
    Terminals
    ...

    10. Application formats
    Application formats
    Music player
    MPEG-A/2
    Photo player
    MPEG-A/3
    Musical slide show
    MPEG-A/4
    Media streaming
    MPEG-A/5
    Professional archiving
    MPEG-A/6
    Open access
    MPEG-A/7
    Portable video
    MPEG-A/8
    Digital multimedia broadcasting
    MPEG-A/9
    Video surveillance
    MPEG-A/10
    Stereoscopic video
    MPEG-A/11
    Interactive music

    11. Generic media technologies
    XML technologies
    Binary MPEG format for XML
    MPEG-B/1
    Signal processing technologies
    Generic inverse DCT specification
    MPEG-C/1
    Fixed point implementation of DCT/IDCT
    MPEG-C/2
    Bitstream technologies
    ...

    12. Protocols
    Protocols
    MXM protocols
    MPEG-M
    13. Reference implementations
    Reference software


    Reference hardware description


    14. Conformance
    MPEG-1

    MPEG-1/4
    MPEG-2
    MPEG-2/4
    MPEG-4
    MPEG-4/4
    MPEG-7
    MPEG-7/7
    MPEG-21
    MPEG-21/14
    MPEG-A

    MPEG-B

    MPEG-C

    MPEG-D

    MPEG-E

    MPEG-M

    MPEG-U

    MPEG-V

    15. Maintenance
    MPEG-1

    MPEG-2

    MPEG-4

    MPEG-7

    MPEG-21

    MPEG-A

    MPEG-B

    MPEG-C

    MPEG-D

    MPEG-E

    MPEG-M

    MPEG-U

    MPEG-V


  • Estŕndards / Standards
    • ISO/IEC freely available standards
    • WD
      Working Draft
      CD Comitee Draft
      FCD Final Comitee Draft
      FDIS Final Draft International Standard
      IS
      International Standard
      PDAM Proposed Amendment
      FPDAM Final Proposed Draft Amendment
      FDAM Final Draft Amendment
      AMD
      Amendment
      DCOR Draft Corrigenda
      COR
      Corrigenda
      PDTR Proposed Draft Technical Report
      DTR Draft Technical Report
      TR
      Technical Report


    • 3m

      4m

      2m

      publish
      Specification
      CD
      ->
      FCD
      -> FDIS
      -> IS

      Amendment
      PDAM
      -> FPDAM
      -> FDAM
      -> AMD

      Corrigenda
      -
      -> DCOR
      -> -
      -> COR

      Technical report
      PDTR
      -> -
      -> DTR
      -> TR

    • m: input documents; w:output documents
  • Maquinari
  • Programari / Software
  • Taules SI
    • PAT
    • PMT
    • NIT

MPEG-2 (ISO/IEC 13818)

  • MPEG-2
    • MPEG-2: Overview of the systems layer (pdf) (BBC Research)
    • MPEG headers
    • part 1: Systems
      • programme #1
        ES (0xa6) PES
        (PES packets)

        SP-TS (single programme TS)
        (transport packet: 188 bytes)
        • PCR#1 (every 0.1s)
        TS
        ES (0x65) PES
        programme #2
        ES (0xa7) PES SP-TS
        • PCR#2 (every 0.1s)
        ES (0x66) PES
      • For a given ES:
        access units (compressed data)


        AU#1

        AU#1
        AU#1
        AU#1

        AU#1

        AU#1
        AU#2

        AU#2 ...
        PES packets

        H
        P
         
        P

        P

        P

        H
        P


        P

        P


        P
        ...
        transport packets (H+P: 188 bytes)
        H
        P
        H
        P
        H
        P
        H
        P
        H
        P
        H
        A
        P
        H
        P
        H
        A
        P
        ...
      • H: header (header of first PES packet for an access unit can carry its PTS)
        P: payload
        A: adaptation field (carries PCR)
      • Transport Stream
      • Program-specific information (PSI) (wp)
      • 2.4.3.6 PES (wp)
        • table 2.22 2-18: stream_id
        • table 2.34 2-29: stream_type
        • PES packet header:
          • 0
            0
            0
            0
            0
            0
            0
            0
            PES-packet start code prefix
            start code
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            0
            1
            x
            x
            x
            x
            x
            x
            x
            x
            stream_id
            x
            x
            x
            x
            x
            x
            x
            x
            PES-packet length

            x
            x
            x
            x
            x
            x
            x
            x

            1
            0
            x
            x
            x
            x
            x
            x
            flags 1

            P
            D
            x
            x
            x
            x
            x
            x
            flags 2

            x
            x
            x
            x
            x
            x
            x
            x
            PES header data length


            presentation timestamp PTS (if present)


            decoding timestamp DTS (if present)


            further optional fields

      • Transport packet
        • Header:
          • 0
            1
            0
            0
            0
            1
            1
            1
            sync byte (=0x47)
            TEI
            PUSI
            TP
            PID

            PID

            TSC
            AFC
            CC

      • Clock references and timestamps

        • clock
          sample (27MHz, 42-bit)
          carried in (every 0.1s)
          Program Stream
          System clock
          System clock reference (SCR)

          Transport Stream
          Programme clock
          Programme clock reference (PCR)
          adaptation fields of transport packets
        • Timestamps (90kHz, 33-bit) (carried in PES header, every 0.7s)
          • Presentation Time Stamp (PTS)
          • Decoding Time Stamp (DTS)
    • part 2: video
      • video sequence
        • sequence header: aspect ratio (AR), [AFD]
        • [additional sequence header]
        • group of pictures (GOP)
          • group_of_pictures_header: [AFD]
          • picture: I, B, P
            • picture header: [AFD]
            • slice
              • macroblock (16x16 pixels): 4 Y blocks + 1 Cr block + 1 Cb block
                • block (8x8 pixels)
        • end of sequence
  • A beginners guide for MPEG-2 standard
  • MPEG-2 encoder chip (Sony)
  • max br
    (bps)
    max samples
    maximum video bit rate (bps)
    80M 1920x1080@30
    1920x1152@25
    1280x720@60
    levels high
    80M 300M

    100M
    60M 1440x1080@30
    1440x1152@25
    1280x720@30
    high-1440
    60M 80M
    60M 80M
    15M 720x480@30
    720x576@25
    352x288@15
    320x240@24
    main 15M
    384k
    15M 50M 15M
    20M
    4M 352x240@30
    352x288@25
    352x288@30
    176x144@15
    low 96k 4M
    4M


    simple main 422 SNR spatial high
    profiles
    frames I x x x x x x
    P x x x x x x
    B
    x x x x x
    YCbCr 4:2:0 x x
    x x x
    4:2:2

    x

    x
    scalable SNR


    x x x
    spatial



    x x
  • ISO_639_language_descriptor (used in PMT) (dvbsnoop source code: dvbstrAudio_TYPE) (ISO/IEC 13818-1 2.6.19)
    • lang: ISO 639-2 (Alpha-3) (added by DVB)
    • audio_type
      used for
      0x00
      undefined main audio
      0x01
      clean effects
      no language
      0x02
      hearing impaired Clean Audio
      0x03
      visual impaired commentary
      0x04 - 0xff
      reserved

  • stream_id (used by PES) (ISO/IEC 13818-1, 2.4.3.6, table 2-18) (completed by DVB: TS 101 154, 4.1.6)
    binary
    hex

    TS 101 154, 4.1.6
    1011 1100 0xbc
    program_stream_map
    1011 1101 0xbd
    private_stream_1 AC-3, Enhanced AC-3, DTS
    1011 1110 0xbe padding_stream
    1011 1111 0xbf private_stream_2
    110x xxxx
    0xc0 - 0xdf
    ISO/IEC 13818-3 or ISO/IEC 11172-3 or ISO/IEC 13818-7 or ISO/IEC
    14496-3 audio stream number x xxxx
    MPEG-4 AAC, MPEG-4 HE AAC, MPEG-4 HE AAC v2 a
    1110 xxxx
    0xe0 - 0xef
    ITU-T Rec. H.262 | ISO/IEC 13818-2 or ISO/IEC 11172-2 or ISO/IEC
    14496-2 video stream number xxxx
    0xea: VC-1
    1111 0000
    0xf0 ECM_stream

    1111 0001
    0xf1 EMM_stream

    1111 0010
    0xf2 ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Annex A or ISO/IEC 13818-
    6_DSMCC_stream

    1111 0011
    0xf3 ISO/IEC_13522_stream

    1111 0100
    0xf4 ITU-T Rec. H.222.1 type A

    1111 0101
    0xf5 ITU-T Rec. H.222.1 type B

    1111 0110
    0xf6 ITU-T Rec. H.222.1 type C

    1111 0111
    0xf7 ITU-T Rec. H.222.1 type D

    1111 1000
    0xf8 ITU-T Rec. H.222.1 type E

    1111 1001
    0xf9 ancillary_stream

    1111 1010
    0xfa ISO/IEC14496-1_SL-packetized_stream

    1111 1011
    0xfb ISO/IEC14496-1_FlexMux_stream

    1111 1100 ... 1111 1110

    reserved data stream

    1111 1111
    0xff program_stream_directory


  • stream_type (used in PMT) (used by PES) (ISO/IEC 13818-1, 2.4.3.6, table 2-29) (dvbsnoop source code: dvbstrStream_TYPE)


    TS 101 154, 4.1.6
    0x00 ITU-T | ISO/IEC Reserved
    0x01 ISO/IEC 11172-2 Video (MPEG-1 video)

    0x02 ITU-T Rec. H.262 | ISO/IEC 13818-2 Video or ISO/IEC 11172-2 constrained parameter video stream
    0x03 ISO/IEC 11172-3 Audio (MPEG-1 audio)

    0x04 ISO/IEC 13818-3 Audio (MPEG-2 audio)

    0x05 ITU-T Rec. H.222.0 | ISO/IEC 13818-1 private_sections
    0x06 ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data AC-3, Enhanced AC-3, DTS
    0x07 ISO/IEC 13522 MHEG
    0x08 ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Annex A DSM CC
    0x09 ITU-T Rec. H.222.1
    0x0a ISO/IEC 13818-6 type A
    0x0b ISO/IEC 13818-6 type B
    0x0c ISO/IEC 13818-6 type C
    0x0d ISO/IEC 13818-6 type D
    0x0e ISO/IEC 13818-1 auxiliary
    0x0f ISO/IEC 13818-7 Audio with ADTS transport sytax
    0x10 ISO/IEC 14496-2 Visual
    0x11 ISO/IEC 14496-3 Audio with LATM transport syntax as def. in ISO/IEC 14496-3/AMD1 MPEG-4 AAC
    MPEG-4 HE AAC
    MPEG-4 HE AAC v2 a
    0x12 ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in PES packets
    0x13 ISO/IEC 14496-1 SL-packetized stream or FlexMux stream carried in ISO/IEC 14496 sections
    0x14 ISO/IEC 13818-6 DSM-CC synchronized download protocol
    0x15 Metadata carried in PES packets using the Metadata Access Unit Wrapper
    0x16 Metadata carried in metadata_sections
    0x17 Metadata carried in ISO/IEC 13818-6 (DSM-CC) Data Carousel
    0x18 Metadata carried in ISO/IEC 13818-6 (DSM-CC) Object Carousel
    0x19 Metadata carried in ISO/IEC 13818-6 Synchronized Download Protocol using the Metadata Access Unit Wrapper
    0x1a IPMP stream (defined in ISO/IEC 13818-11, MPEG-2 IPMP)
    0x1b AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video
    0x1c-0x7e ITU-T Rec. H.222.0 | ISO/IEC 13818-1 Reserved
    0xd1 Dirac video
    0x7f IPMP stream
    0x08-0xff User Private

MPEG-4 (ISO/IEC 14496)

  • MPEG-4
    • MPEG-4 demo applet
    • MPEG-4 (IEEE Spectrum article)
    • MPEG-4 Tutorial (GPAC)
    • MPEG-4 Part 3: AAC (Advanced Audio Coding)
      • Codec parameter (MPEG-4 audio):
        • AAC LC
          mp4a.40.2
          HE-AAC
          mp4a.40.5
          HE-AAC v2
          mp4a.40.29

    • MPEG-4 Part 10: AVC-SVC-MVCH.264
    • H.264 profiles / levels (ITU-T REC H.264-201704: A.2 Profiles, A.3 Levels) (profile-idc: Annex A) (SVC profiles and levels: G.10) (MVC profiles and levels: H.10) (italic text are parameters to JSVM reference software configuration files)
      samples (max fps)

      maximum video bit rate (bps) level_idc
      8192
      7680
      4096
      3680
      3616
      2560
      2048
      1920
      1600
      1408
      1280 (HD)
      1024
      800
      720 (SD)
      704
      400 352
      320
      176
      144 128
      4320
      4096
      4320
      2304 2160 2048 1536 1536 1920 1536 1088 1024 1088 1200 1152 960 1024 960 720 768 600 576 480 576 480 224 576 480 288 240 240 176 144 80 96



       16:9  
       4Kx2K

      2.35:1
       16VGA
       4XGA
       2Kx1080
       2Kx1K
       1080HD
       4SVGA
       16CIF
       525 16SIF
      SXGA
      4VGA
      720p HD
      XGA
      SVGA
      625SD
       525SD
       4CIF
      525 4SIF

      625 HHR
      525 HHR
      CIF 525 SIF
      QVGA
      QCIF
      SQCIF
      120.9
      127.5
      128.9
      300.0 300.0
      300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0 300.0
      300.0 300.0 300.0
      300.0 300.0 300.0 300.0 300.0
      300.0

      300.0

      levels
      6.2









      60.4


































      6.1









      30.2


































      6.0












































      5.2












      26.7
      28.4
      30.0





      120.5





















      5.1
      (4K)
      240M 300M 720M 960M
      33






      26.7
      30.7
      67.8 72.0 72.3





















      5.0 135M 168,75M 405M 540M
      32










      60.0
      64.0
      (60)






















      4.2 (1080p) 50M 62,5M
      FF
      150M 200M

      2A











      30.0 30.1




      68.3















      4.1 (1080i) 29
































      4.0 (HD) 20M 25M
      EE
      60M 80M

      28
















      42.2
      60.0















      3.2
      (720p60)
      25M 20


















      30.0

      66.7 80.0











      3.1
      (720p30)
      14M 17,5M 42M 56M
      1F





















      25.0 30.0


      51.1 61.4






      3.0
      (SD)
      10M
      DBDB
      10M 10M
      D
      12,5M
      D
      30M 40M
      1E





















      12.5 15.0


      25.6 30.7






      2.2 4M 5M 12M 16M

      16


























      25.0 30.0






      2.1 15




























      30.0
      36.0
      (30)




      2.0 2M
      CC
      2M 2M 2,5M 6M 8M
      14

































      1.3 768k 960k 2304k 3072k
      0D




























      15.2
      20.0
      (15)




      1.2 384k
      BB
      384k 384k 480k 1152k 1536k
      0C




























      7.5
      10.0
      30.3

      1.1 192k 240k 576k 768k
      0B
































      15.0
      30.9 1b 128k
      A
      128k 128k 160k 384k 512k


































      1.0 64k 80k 192k 256k
      0A


      AVC
      42 (66)
      58 (88)
      4D (77)
      64 (100)
      6E (110)
      7A (122)
      F4 (244)
      2C (44)
      profile_idc
      SVC
      53 (83)


      56 (86)




      MVC multiview


      76 (118)




      stereo


      80 (128)






      [scalable]
      BASELINE
      EXTENDED MAIN [scalable[intra], stereo, multiview]
      HIGH
      HIGH10 [intra]
      HIGH 4:2:2 [intra]
      HIGH 4:4:4 [intra]
      CAVLC 4:4:4 intra

      FRExt


      scalability
      partition
      prediction MB quantisation slices CA transform
      profiles






      arbitrary slice order


      x x












      redundant slices

      x x










      flexible macroblock ordering



      x x














      4x4 x x x x











      CAVLC (lower complexity)
      (SymbolMode=0)

      x x x x










      I slices


      x
      x
      x
      x











      P slices

      x x x x










      SP and SI slices


      x







      data partioning







      x












      B slices


      x (constraint_set5_flag)
      x (constraint_set5_flag) x (constraint_set5_flag)







      weighted prediction
      (WeightedPrediction, WeightedBiprediction)






      x x x











      CABAC (high complexity)
      (SymbolMode=1)

      (certain levels)

      x x









      quantisation matrix





      x












      8x8
      (Enable8x8Transform=1)
      (certain levels)

      x




      layer spatial ratio








      1:1
      1.5:1
      2:1


      unrestricted





    • profile-level-id (sdp), codecs (mpd)

      • byte
        bit
        value
        meaning
        ffmpeg H.264
        1st
        profile_idc



        -profile:v ...
        2nd
        profile-iop constraint_set0_flag 0
        (must be 0 when profile_idc is: 44, 100, 110,
        122, or 244)
        ?
        1
        • all constraints in A.2.1 (baseline)

        constraint_set1_flag 0
        (must be 0 when profile_idc is: 44, 100, 110,
        122, or 244)

        1
        • all constraints in A.2.2 (main)

        constraint_set2_flag 0
        (must be 0 when profile_idc is: 44, 100, 110,
        122, or 244)

        1
        • all constraints in A.2.3 (extended)

        constraint_set3_flag 0


        1


        constraint_set4_flag 0


        1


        constraint_set5_flag 0


        1


        reserved_zero_2bits 0


        reserved_zero_2bits 0


        3rd
        level_idc


        -level:v ...

      • 1st byte: profile_idc (ITU-T REC H.264-200711: AVC: A.2; SVC: G.10; MVC: H.10):
        profile
        AVC
        SVC
        MVC
        scalable
        multiview stereo
        CAVLC 4:4:4 intra 2C (44)


        baseline


        42 (66) 53 (83)


        main


        4D (77)


        extended


        58 (88)


        high



        64 (100) 56 (86)
        76 (118)
        80 (128)

        intra



        10


        6E (110)


        intra



        4:2:2


        7A (122)


        intra



        4:4:4

        F4 (244)


        intra



      • 2nd byte: profile-iop
        • H.264 04/2017: 7.4.2.1.1 Sequence parameter set data semantics
        • Examples:
          • c0
          • e0
      • examples:
        • profile-level-id
          (sdp)
          codecs
          (mpd)

          meaning of profile-level-id



          profile_idc profile-iop level_idc
          42c00d avc1.42c00d

          42: baseline
          c0: 110- ----
          0d: 1.3
          42c01e avc1.42c01e

          1e: 3.0
          42c01f avc1.42c01f

          1f: 3.1
          42c028 avc1.42c028

          28: 4.0
          42e0..
          constrained baseline e0: 111- ----
          4200..
          baseline

          4d00..
          main 4d: main

          640c..
          constrained high 64: high

          6400..
          high

          33: 5.1
    • ETSI TS 102 005 (IP):
      • Annex A.3.1.5 "Summary of key tools and parameter ranges for capability A to E IRDs"
        • constraint_set1_flag
      • Annex B: "TS 102 005 usage in DVB-IP datacast"
        • H264/AVC resolutions and capabilities (A,B,C,D,DB,E)
        • SVC
          capability H.264/AVC SVC VC-1 resolution IPDC resolution
          profile level profile level scalability profile level
          SNR spatial
          A baseline 1b



          simple LL 176x144@15 176x144@15
          128x96@30
          144x80@30
          B 1.2 scalable baseline 1.2 x - ML 176x144@30 (VC-1)
          320x240@24 (VC-1)
          352x288@15
          176x144@30
          320x240@15
          320x176@15
          C 2.0 2.0 - advanced L0 352x288@30 320x240@30
          320x176@30
          400x224@30
          DB 3.0 3.0 2-layer

          720x576@25
          720x576@30

          D main scalable high advanced L1 480x320@30 (SVC)
          640x352@30 (SVC)
          640x480@30 (SVC)
          800x448@30 (SVC)
          E high 4.0 4.0 3-layer L3 1280x720@50/60
          1920x1080@25/30

          F 4.2 4.2

          1920x1080@60
    • ISO Base Media File Format (MPEG-4 part 12)
      • Info
      • Extensions (wp)

        description
        owner
        brand
        encryption
        3GPP



        3GPP2



        AVC FF


        iso6 (avc3)

        CFF
        Common File Format
        UltraViolet / DECE
        ccff

        DVB-FF
        DVB File Format
        DVB
        dvbr1
        dvbt1


        MP4
        MPEG
        mp41
        mp42

        [cenc]
        OMA DCF
        OMA
        odcf x
        OMA PDCF
        opf2 x
        PIFF Protected Interoperable File Format
        Microsoft
        piff
        x
      • Stream Access Points (SAP) (ISO/IEC 14496-12: Annex I)
        • SAP types:
          type

          name
          info
          1
          Tept = Tdec = Tsap =Tptf
          Closed GOP random access point
          accepted by DASH-AVC/264
          2
          Tept = Tdec = Tsap <Tptf
          3
          Tept < Tdec = Tsap <=Tptf Open GOP random access point

          4
          Tept <= Tptf < Tdec = Tsap
          Gradual Decoding Refresh (GDR)

          5
          Tept = Tdec < Tsap

          6
          Tept < Tdec < Tsap


    • MP4 (MPEG-4 part14) / AVC FF (MPEG-4 part 15) (extensions of ISOBMFF)
      • Formats de fichiers MPEG (C.Concolato)
      • AAC Audio and the MP4 Media Format
      • iso6 / avc3 (inband)
      • atoms
        • moov: metadata (timescale, duration, display...)
          • mvhd
          • trak
            • mdia
            • mdia
            • ...
          • trak
          • ...
        • mdat: data
        • ...
      • Programari / Software
      • Fragmentació / Fragmentation
        • Descŕrrega progressiva / Progressive download
        • stream-m: Fragment sizes
        • Eines / Tools

        • description
          creation

          info
          limitations


          MP4Box
          ffmpeg
          Bento4
          MP4Box -info
          regular file
          1. mdat
          2. moov





          progressive download file
          1. moov
          2. mdat
          (any)
          • -movflags faststart
          • (fragmentation)

          File suitable for progressive download (moov before mdat)

          interleaved

          -inter ms



          fragmented
          1. moov (can be forced to be empty)
          2. moof (frag#1)
            1. traf (track #1)
            2. traf (track #2)
          3. mdat (frag#1)
          4. moof (frag#2)
            1. traf (track #1)
            2. traf (track #2)
          5. mdat (frag#2)
          6. ...
          -frag ms
          • -movflags empty_moov+frag_keyframe
          • -frag_duration duration_ľs
          • -frag_size size_bytes
          mp4fragment

          fragmented with separated tracks
          ?
          • -movflags separate_moof


          • HbbTV TS 102 796 B.3.1: "The movie fragment box ('moof') shall contain only one track fragment box ('traf')"
          splitting
          1. chunk#1.mp4
          2. chunk#2.mp4
          3. ...
          -split seconds
          segment
          mp4split


          dash
          • mp4 file (init)
            1. moov
          • m4s file (frag#1)
            1. moof (frag#1)
            2. mdat (frag#1)
          • m4s file (frag#2)
            1. moof (frag#2)
            2. mdat (frag#2)
          • ...
          -dash ms

          mp4-dash



      • The MP4 Registration Authority (MP4RA)
        • Box: meta...
        • Brands: mp41, mp42, 3gp4, 3gp5...
        • Codecs: avc1, mp4a, mp4s, mp4v... (VLC, MP4Box -info) (see a superset at fourcc.org)
        • Track references
        • handler / group: sdsm, soun, vide, hint, mp21... (MP4Box -info)
        • objectTypeIndication / streamType:

          streamType
          (sdp: a=fmtp:...streamType=)
          (<saf:...Header>)
          1 3 4 5
          ObjectDescriptor Scene Description Visual Audio
          objectTypeIndication (*)
          (sdp: a=fmtp:...objectType)
          (<saf:...Header>)




          1
          MPEG4Systems1 Object Descriptor Stream BIFS Stream version 1

          2
          MPEG4Systems2
          BIFS Stream version 2

          9 0x09 LASeR?
          generated by
          • Laser Java reference software: bin2xml (xml file)
          • GPAC mp4_streamer: sdp
          • Darwin: sdp


          32 0x20 MPEG4Visual

          MPEG-4 Visual Stream
          64 0x40 MPEG4Audio


          MPEG-4 Audio Stream

          0x60 MPEG2VisualSimple

          MPEG-2 Visual Stream
          97 0x61 MPEG2VisualMain



          0x62 MPEG2VisualSNR



          0x63 MPEG2VisualSpatial



          0x64 MPEG2VisualHigh



          0x65 MPEG2Visual422



          0x66 MPEG2AudioMain


          MPEG-2 Audio Stream / Part 7 -AAC

          0x67 MPEG2AudioLowComplexity



          0x68 MPEG2AudioScaleableSamplingRate



          0x69 MPEG2AudioPart3


          MPEG-2 Audio Stream Part 3
          107 0x6b MPEG1Audio



          108 0x6c JPEG

          JPEG Visual stream
          109 0x6d PNG

          PNG Visual stream

          0xde Vorbis




          0xdf Theora



    • Relacions d'aspecte / Aspect ratios

MPEG-7 (ISO/IEC 15938)

MPEG-21 (ISO/IEC 21000)

MMT (MPEG Media Transport / MPEG Modern Transport / Modern Media Transport / Multi Media Transport)

MPEG-DASH (ISO/IEC 23009)

  • Info
  • Profiles:
    MPEG-DASH

    HbbTV
    DVB-DASH GPAC


    urn MPD@type
    1.1
    1.5 (Annex B) 2.0 (Annex E)
    ETSI TS 103 285 - A168
    --profile
    HTTP  streaming

    full
    urn:mpeg:dash:profile:full:2011




    urn:dvb:dash:profile:dvb-dash:2014
    full

    ISO base media FF
    onDemand
    urn:mpeg:dash:profile:isoff-on-demand:2011
    static
    each media presentation is a single segment


    onDemand, dashavc264:onDemand
    MP4Box ...  -single-segment ...
    live
    urn:mpeg:dash:profile:isoff-live:2011



    urn:hbbtv:dash:profile:isoff-live:2012

    live, dashavc264:live
    main
    urn:mpeg:dash:profile:isoff-main:2011
    dynamic




    main

    MPEG-2 TS
    main
    urn:mpeg:dash:profile:mp2t-main:2011







    simple
    urn:mpeg:dash:profile:mp2t-simple:2011





    simple

  • HTTP streaming
  • HTTP streaming software
  • DRM
    • 5.8.4.1 ContentProtection
      • urn:mpeg:dash:mp4protection:2011
  • Data model (ISO/IEC 23009-1:2012(E), Figure 3)
    • Media Presentation Description
      Period #1
      Adaptation set (video 1)
      Representation (bit rate 1)
      Segment #1
      (6 seconds: -dash 6000)
      sidx
      (-subsegs-per-sidx 4)
      subsegment #1 (whole number of AU)
      (whole number of complete movie fragments)
      movie fragment #1
      (500ms: -frag 500)
      moof
      mdat
      ...
      mdat

      movie fragment #2
      movie fragment #3

      subsegment #2
      subsegment #3
      subsegment #4
      sidx subsegment #5
      subsegment #6
      subsegment #7
      subsegment #8

      Segment #2 (6s)

      Representation (bit rate 2)

      Adaptation set (audio 1)
      Adaptation set (audio 2)

      Period #2

      ISOBMF structure
    • Segment index (pdf)


      • GPAC MP4Box option
        -
        SS1
        SS2
        SS3
        SS4
        SS5
        SS6
        -subsegs-per-sidx -1
        simple
        SI1
        SS1
        SS2
        SS3
        SS4
        SS5
        SS6
        -subsegs-per-sidx 0
        hierarchical
        SI1
        SI2
        SS1
        SS2
        SI3
        SS3
        SS4
        SI4
        SS5
        SS6
        -subsegs-per-sidx 2
        daisy-chain
        SI1
        SS1
        SS2
        SI2
        SS3
        SS4
        SI3
        SS5
        SS6
        -subsegs-per-sidx 2
        -daisy-chain

    • onDemand profile: each media presentation is a single segment (GPAC: -single-segment)
    • ISO/IEC 23009-1:2012(E)
      • p.10: "For example, in the very simplest cases each Segment or Subsegment begins with a random access point and the boundaries of Segments or Subsegments are aligned across the Representations of an Adaptation Set. In this case, switching Representation involves playing to the end of a (Sub)Segment of one Representation and then
        playing from the beginning of the next (Sub)Segment of the new Representation."
      • p.11: "For live services, however, Segments become available with time as the content is produced. The Media Presentation Description may be updated regularly to reflect changes in the presentation over time, for example Segment URLs for new segments may be added to the MPD and those
        for old, no longer available Segments may be removed. However, if Segment URLs are described using a template, this updating may not be necessary except for some redundancy/failover cases."
      • 8.4 ISO Base media file format live profile: "Note that despite the profile is optimized for live services, the MPD@Type attribute may be set to 'static' to distribute non-live content, for example in case a live Media Presentation is terminated, but kept available as On-Demand service."
  • MPD
    • mime-type
      • application/dash+xml
    • MPEG-DASH MPD Validator (ITEC uni-kla)
    • Codec description (codecs=...)
    • MPD
      MP4Box option
      SegmentBase
      -single-segment
      SegmentList
      -segment-name
      SegmentTemplate
      -url-template


    • urn:mpeg:DASH:schema:MPD:2011
      urn:mpeg:DASH:schema:MPD:2011
      urn:mpeg:mpegB:schema:DASH:MPD:DIS2011
      urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009
      MP4Client


      -
      x
      VLC


      x
      -

      • MPD
        xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011"
        profiles="urn:mpeg:dash:profile:isoff-main:2011"
        type="static"
        mediaPresentationDuration="PT1H37M38S"
        minBufferTime="PT10.0S"
        • BaseURL
          • http://www-itec.uni-klu.ac.at/ftp/datasets/mmsys12/RedBullPlayStreets/redbull_10s/
        • Period  start="PT0S"
          • AdaptationSet bitstreamSwitching="true"
            • Representation id="0" codecs="avc1" mimeType="video/mp4" width="320" height="240" startWithSAP="1" bandwidth="100031"
              • SegmentBase
                • Initialization sourceURL="redbull_10sec_100kbit/redbull_100kbit_dash.mp4"
              • SegmentList duration="10"
                • SegmentURL media="redbull_10sec_100kbit/redbull_10sec1.m4s"
                • SegmentURL media="redbull_10sec_100kbit/redbull_10sec2.m4s"
                • ...
            • Representation id="1" codecs="avc1" mimeType="video/mp4" width="320" height="240" startWithSAP="1" bandwidth="148267"
              • SegmentBase
                • Initialization sourceURL="...mp4"
              • SegmentList  duration="10"
                • SegmentURL ...
                • SegmentURL ...
                • ...
            • ...
            • Representation  id="16" codecs="avc1" mimeType="video/mp4" width="1920" height="1080" startWithSAP="1" bandwidth="5927049"
              • SegmentBase
                • Initialization sourceURL="redbull_10sec_6000kbit/redbull_6000kbit_dash.mp4"
              • SegmentList duration="10"
                • SegmentURL media="redbull_10sec_6000kbit/redbull_10sec1.m4s"
                • SegmentURL media="redbull_10sec_6000kbit/redbull_10sec2.m4s"
                • ...
      • MPD
        xmlns="urn:mpeg:DASH:schema:MPD:2011"
        profiles="urn:mpeg:dash:profile:isoff-live:2011"
        xmlns:mas="urn:marlin:mas:1-0:services:schemas:mpd"
        type="static"
        mediaPresentationDuration="PT14M48S"
        minBufferTime="PT3.00S"
        • Period
          • AdaptationSet mimeType="audio/mp4"
            • ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc"
            • ContentProtection schemeIdUri="urn:uuid:..."
              • mas:MarlinContentIds
                • mas:MarlinContentId
                  • urn:marlin:kid:...
            • Representation bandwidth="137568" id="audio"
              • SegmentList duration="3000" timescale="1000"
                • Inititalisation  sourceURL="audio/init.mp4"
                • SegmentURL media="audio/seg-0000.m4f"
                • SegmentURL media="audio/seg-0001.m4f"
                • SegmentURL ...
          • AdaptationSet mimeType="video/mp4" startWithSAP="1"
            • ContentProtection schemeIdUri="urn:mpeg:dash:mp4protection:2011" value="cenc"
            • ContentProtection schemeIdUri="urn:uuid:..."
              • mas:MarlinContentIds
                • mas:MarlinContentId
                  • urn:marlin:kid:...
            • Representation bandwidth="248784" id="video.1"
              • SegmentList duration="3000" timescale="1000"
                • Inititalisation sourceURL="video/1/init.mp4"
                • SegmentURL media="video/1/seg-0000.m4f"
                • SegmentURL media="video/1/seg-0001.m4f"
                • ...
            • Representation bandwidth="610512" id="video.2"
              • SegmentList duration="3000" timescale="1000"
                • Inititalisation sourceURL="video/2/init.mp4"
                • SegmentURL media="video/2/seg-0000.m4f"
                • SegmentURL media="video/2/seg-0001.m4f"
                • ...
            • Representation ...
          • AdaptationSet...
      • MPD
        xmlns:xsi="http://www.w3.org/2001/XMLSchema"
        xmlns="urn:mpeg:mpegB:schema:DASH:MPD:DIS2011"
        xsi:schemaLocation="urn:mpeg:mpegB:schema:DASH:MPD:DIS2011"
        profiles= "urn:mpeg:mpegB:profile:dash:isoff-basic-on-demand:cm"
        type="OnDemand"
        mediaPresentationDuration="PT5M0.08S"
        minBufferTime="PT10.00S"
        • Period
          • Group segmentAlignmentFlag="true" mimeType="video/3gpp"
            • Representation  bandwidth="249967"
              • SegmentInfo duration="PT10.00S"
                • InitialisationSegmentURL sourceURL="toto_v200.3gp"
                • Url sourceURL="toto_v200_1.3gs"
                • Url sourceURL="toto_v200_2.3gs"
                • ...
            • Representation bandwidth="430314"
              • SegmentInfo duration="PT10.00S"
                • InitialisationSegmentURL sourceURL="toto_v400.3gp"
                • Url sourceURL="toto_v400_1.3gs"
                • Url sourceURL="toto_v400_2.3gs"
                • ...
            • Representation...
      • MPD
        xmlns="urn:3GPP:ns:PSS:AdaptiveHTTPStreamingMPD:2009"
        type="OnDemand"
        • ProgramInformation
          • Title
        • Period start="PT0S" duration="PT0H2M29.08S"
          • Representation mimeType="video/mp4" width="480" height="272" sampleRate="22050" numChannels="2" lang="und" startWithRAP="false" bandwidth="1209572"
            • SegmentInfo duration="PT10.00S"
              • InitialisationSegmentURL sourceURL="toto_dash.mp4"
              • Url sourceURL="seg1.m4s"
              • Url sourceURL="seg2.m4s"
              • ...


      • MPD
        xmlns="urn:mpeg:DASH:schema:MPD:2011"
        profiles="urn:mpeg:dash:profile:isoff-live:2011"
        type="static"
        mediaPresentationDuration="PT14M48S"
        minBufferTime="PT3.00S"
        • Period
          • AdaptationSet mimeType="audio/mp4"
            • Representation bandwidth="137568" id="audio">
              • SegmentTemplate duration="3000" initialisation="audio/init.mp4" media="audio/seg-$Number%04d$.m4f" startNumber="0" timescale="1000"
          • AdaptationSet mimeType="video/mp4" startWithSAP="1"
            • Representation bandwidth="248784" id="video.1"
              • SegmentTemplate duration="3000" initialisation="video/1/init.mp4" media="video/1/seg-$Number%04d$.m4f" startNumber="0" timescale="1000"
            • Representation bandwidth="610512" id="video.2"
              • SegmentTemplate duration="3000" initialisation="video/2/init.mp4" media="video/2/seg-$Number%04d$.m4f" startNumber="0" timescale="1000"
            • ...
          • AdaptationSet...


      example
      RedBullPlayStreets_10s_isoffmain_DIS_23009_1_v_2_1c2_2011_08_30.mpd
      it
      mtbike.mpd
      bbb_dash.mpd

MPEG-H (ISO/IEC 23008)

  • Part 2: HEVC / H.265

http://www.francescpinyol.cat/mpeg.html
Primera versió: 22 de febrer de 2008 / First version: 22nd February 2008
Darrera modificació: 25 de gener de 2024 / Last update: 25th January 2024

Valid HTML 4.01!

Cap a casa / Back home.