AI Video Compression and Neural Codecs

Executive Summary

The exponential growth of video traffic across streaming platforms, video conferencing, surveillance, AR/VR, autonomous vehicles, and cloud gaming has created increasing demand for more efficient video compression technologies. Traditional codecs such as H.264/AVC, H.265/HEVC, AV1, and VVC achieve compression through carefully engineered signal-processing techniques. While these codecs have improved steadily over decades, their gains are becoming incremental and increasingly expensive to realize.

Machine Learning (ML)-based video codecs, also known as Neural Video Codecs (NVCs) or Learned Video Compression (LVC), represent a new paradigm. Instead of manually designing prediction, transform, quantization, and entropy coding algorithms, ML codecs learn optimal compression strategies directly from data using deep neural networks.

1. Introduction

1.1 Traditional video codec

A traditional video codec is a video compression system that uses handcrafted algorithms and predefined mathematical models to reduce the size of video data while preserving visual quality. It employs fixed coding techniques such as motion estimation, motion compensation, transform coding (e.g., DCT), quantization, entropy coding, and in-loop filtering, which are designed and optimized by experts rather than learned from data. Traditional codecs follow standardized compression methods to ensure interoperability across devices and applications. Examples: MPEG-2, H.264/AVC, H.265/HEVC, H.266/VVC, and AV1.

1.2 Video Compression technologies using Neural/ML-Based Video Codecs/Compression

An AI-based video codec, also known as a machine learning-based or neural video codec, is a video compression system that uses artificial intelligence and deep learning models to learn efficient representations of video data directly from training datasets. It employs neural networks such as CNNs, autoencoders, RNNs, and transformers for tasks like prediction, motion estimation, entropy modeling, and video reconstruction, enabling improved compression efficiency and visual quality.

Examples: DVC (Deep Video Compression), FVC (Feature-space Video Compression), and DCVC (Deep Contextual Video Compression).

2. Evolution of Neural/ML-Based Video Codecs/Compression

Figure 2. Evolution of various Neural-Video-CodingFigure

3. Proposed Solution: Technology Overview

Machine learning (ML)-based video codecs integrate multiple deep learning techniques to improve compression efficiency, reconstruction quality, and rate-distortion performance. The key enabling technologies are described below

4. Neural Networks in Video Coding

Neural networks or machine learning can be incorporated into video coding using two primary approaches: Neural Network-Enhanced Hybrid Video Coding and End-to-End Learned Video Compression.

5. Flowchart for ML based Video Codec:

The encoding-side and decoding-side flowcharts of the proposed ML-based video codec are presented below to illustrate the overall video compression and reconstruction process.

5.1 Encoding Side Flow Chart

5.2 Decoding Side Flow Chart

6. Benefits of Neural/ML-Based Video Codecs/Compression

7. Challenges of Neural/ML-Based Video Codecs/Compression

8. IP Activity in Neural/ML-Based Video Codecs/Compression

As part of our analysis of patent activity in Neural/ML-Based Video Codecs/Compression, an IP landscape study was conducted to identify patents related to this technology. A total of 2244 patent applications were analyzed and 640 patent or patent applications were selected for final analysis.

8.1 Relevant keywords, synonyms and classes used for search

Machine Learning Video Codec, ML-based video coding, AI video codec, intelligent video coding, machine- learning-assisted compression, Neural codec, AI codec, deep learning codec, neural compression codec, DNN- based codec, AI-powered codec, machine learning codec, neural video compression
Machine Learning, ML, Artificial Intelligence (AI), Deep Learning, Neural Network Learning, Adaptive Learning, Automated Learning, AI based, Neural network-based, Deep neural network, DNN-based, Data-Driven Learning
Auto-Encoders, Neural autoencoder, deep autoencoder, convolutional autoencoder (CAE), variational autoencoder (VAE), neural encoder-decoder, latent representation model
Implicit Neural Representations (INR), Neural implicit representation, Neural field representation, Coordinate- based neural representation, Continuous neural representation, Neural scene representation, Implicit neural coding
Neural Loop Filtering, Learned loop filter, Neural in-loop filter, AI-based loop filtering, Deep neural filter, Learned reconstruction filter, Neural post-processing filter
Learned Video Compression, End-to-end video compression, neural video compression, AI-based compression, neural compression
Neural Motion Estimation, Learned motion estimation, AI motion estimation, neural motion prediction, deep motion estimation, feature motion estimation, intelligent motion modelling
Neural Entropy Coding, Learned entropy model, neural entropy model, neural arithmetic coding, entropy prediction network, spatial-temporal entropy model
Neural Transform Coding, Learned transform, adaptive transform coding, neural transform, autoencoder transform, learned synthesis transform, feature transform coding
Relevant patent classification includes G06T9/002, G06N3/08, H04N19/117, G06N3/0455, H04N19/90, H04N19/124, G06N3/045, and H04N19/172.

8.2 Graphical Analysis from the identified patents (640 patent applications):

Figure 9 shows the distribution of patents across the top 12 priority countries, highlighting the major research and development (R&D) centers for Neural Video Codec/Compression technology. China (CN) leads with 263 patents, followed by the United States (US) with 234 patents. India ranks fifth with 24 patents, reflecting its emerging role in research and innovation in this domain.

Figure 10 illustrates the patenting trend over the period 2017–2026. Both Patent filings and granted patents exhibit a steady upward trend from 2017 to approximately 2024, indicating sustained innovation activity and technological advancement. However, after 2024, the patent filing trend shows a noticeable decline toward 2026. This decrease may be attributed to changes in patent filing strategies, reporting delays, or the time lag between patent submission and publication.

Figure 11 presents the distribution of patents by legal status. Among the total patents analyzed, 296 (46%) are granted, 149 (23%) are pending, and 195 (31%) are classified as dead.

Figure 12 illustrates the leading patent assignees in the field of Neural Video Codec/Compression technology. Qualcomm Inc. ranks first with 50 patents, followed by Sharp Corporation and InterDigital CE Patent Holdings SAS

9. Market and Commercial Outlook

9.1 AI Video Compression Market Outlook 2025-2034

The global AI Video Compression market was valued at USD 4.2 billion in 2025 and is projected to reach USD 21.5 billion by 2034, growing at a CAGR of 19.8%. This growth is driven by the increasing demand for video streaming, social media content, and high-resolution multimedia applications. AI-based compression techniques significantly reduce file sizes while maintaining video quality, resulting in lower bandwidth and storage requirements. The widespread adoption of 5G, edge computing, cloud services, and AI-powered video analytics is further accelerating market growth across industries such as healthcare, surveillance, enterprise collaboration, and digital media.

Source:

https://marketintelo.com/report/ai-video-compression-market

9.2 Key Market Players

The following are some of the key market players operating in the field of Neural/ML-based Video Codec technologies.

10. Applications Area & Future Outlook

Machine learning (ML)-based video codecs have wide-ranging applications across various domains. Some of the key application areas are listed below:

ML-based video codecs are expected to become a key component of next-generation multimedia systems by providing higher compression efficiency, adaptive processing, and improved perceptual quality compared with conventional codecs. With the rapid growth of 4K/8K streaming, 5G/6G networks, cloud gaming, AR/VR, autonomous systems, and AI-driven video applications, the demand for intelligent compression technologies will continue to increase. Future neural codecs are likely to be integrated into consumer devices, cloud platforms, and edge computing systems, enabling real-time, low-latency, and energy-efficient video delivery.

11. Conclusion

Neural and machine learning-based video codecs are transforming video compression through improved efficiency, adaptability, and visual quality. Techniques such as VAEs, INRs, learned motion estimation, neural entropy models, and end-to-end neural codecs enable significant bitrate reduction and enhanced compression performance, with solutions such as DCVC-RT

Growing R&D and patent activity, particularly in the USA and China, highlight the rapid innovation in this field. Increasing demand for video streaming, 5G, and immersive media is expected to drive adoption, despite challenges related to computational complexity and standardization.

Going forward, advances in adaptive AI models, hardware optimization, and standardization are expected to improve the scalability and practical deployment of neural codecs, establishing them as an important technology for next-generation digital media.

How Pitch Scientific supports in Neural/ML-Based Video Codecs/Compression

Pitch Scientific combines expertise in Neural/ML-based Video Codec technologies with IP intelligence to help clients navigate emerging video compression technologies, including AI-driven neural codecs, deep learningbased video compression, learned video representations, and next-generation hybrid codec architectures, alongside established standards such as HEVC/H.265, AV1, VVC/H.266, EVC, LCEVC, and AV2.

We support clients across the neural video codec lifecycle through:

Prior Art & Patentability Searches:Identify relevant patents and prior art to assess the novelty and patentability of innovations in neural codec architectures, machine learning-based compression algorithms, and AI-driven video encoding and decoding
Freedom-to-Operate (FTO) Analysis:Evaluate potential patent risks associated with developing and commercializing neural video codecs, deep learning-based compression models, and AI-enabled video processing solutions.
Patent Landscape & Competitor Analysis:Map global patent portfolios, key players, emerging technologies, and innovation trends across neural video compression, learned codecs, and machine learning-based multimedia processing.
IP Intelligence & Strategic Insights: Deliver actionable intelligence on patent filing trends, technology evolution, key innovators, competitor R&D activities, patent ownership, and emerging market opportunities to support strategic planning, technology investments, and IP decision-making.
White-Space Analysis:Identify technology gaps and emerging opportunities in neural codec architectures, learned compression models, content-adaptive encoding, generative compression, and AIassisted video reconstruction to guide future R&D and patent filing strategies.
Patent Claim Mapping & Technology Benchmarking: Analyze patent claims against neural codec architectures, compression algorithms, machine learning models, and video encoding/decoding implementations to assess technology coverage, identify potential overlaps, and benchmark competing solutions
Patent Portfolio Strategy & Commercialization Support:Help clients strengthen patent portfolios, identify potential licensing opportunities, evaluate relevant patent assets, and develop IP strategies for commercializing next-generation neural video compression technologies
Ongoing Patent Monitoring:Track new patent filings, grants, emerging neural codec architectures, competitor activities, and developments in AI-driven video compression to help clients stay informed about the evolving technology landscape

By combining deep technical expertise in neural video codecs with strategic IP intelligence, Pitch Scientific helps clients reduce IP risks, strengthen patent portfolios, identify innovation and commercialization opportunities, and accelerate the development of next-generation AI-driven video compression solutions.