Video is integral to digital experiences. Whether end-users are scrolling through social media, binging content on their connected TV, or sweating it out to an online fitness class, streaming now plays a central role in driving online engagement. 

But building video into products and services is tough. Businesses need robust streaming infrastructure to store, encode, manage, deliver, and analyze video content. Plus, most dev teams have expertise in their company’s core competency rather than back-end video technology. 

That’s where online video platforms (OVPs), video streaming application programming interfaces (APIs), and software development kits (SDKs) come into play.

Think of OVPs as all-in-one solutions. They offer a comprehensive suite of tools to manage your entire video workflow, from ingestion to analytics. These are ideal for businesses needing a user-friendly platform with minimal development effort. But if your requirements go beyond simply uploading and sharing video content, OVPs may be a poor fit.

APIs, on the other hand, provide granular control. They act as messengers, allowing you to integrate specific video functionalities like playback, encoding, or analytics into your existing applications. APIs are perfect for developers seeking the flexibility to develop advanced applications, without having to start from scratch.

Finally, SDKs are pre-built toolkits designed for integrating specific video features into mobile and web applications. They save development time by offering all the building blocks for a specialized language or task — like deploying your HTML5 player on Roku. 

SDKs are often used in conjunction with APIs and OVPs. For this reason, it’s not always a question of OVP vs. API vs. SDK, but rather which combination of technologies is best for your business.

Acronyms abound in the alphabet soup that is video streaming. But don’t worry. In this guide to OVPs, APIs, and SDKs, we define each term and explore which option is best depending on your use case. From there, we recommend the top products in each category for business leaders and software developers alike.

Technical requirements for deploying online video

Before getting into it, let’s nail down the capabilities and features needed to integrate video into your product and look at how OVPs, APIs, and SDKs support these requirements.

Encoding and transcoding 

Encoding and transcoding are often used interchangeably, but they refer to two distinct steps. Encoding involves converting RAW video into a compressed digital format directly after the video source is captured, while transcoding employs a digital-to-digital conversion process to prepare and optimize video content for distribution to end users.

Most online video content has been both encoded and transcoded before it reaches viewers. These processes are what make it possible to deliver bulky video over the internet and ensure smooth playback across a variety of devices. 

Transcoding is a critical capability that’s supported by all major OVPs and APIs. What differs, though, is how advanced and flexible different platforms’ transcoding features are. Most OVPs take a one-size-fits-all approach. This means the video bitrate, frame rate, and other technical parameters are predefined and all streams are prepared in the same way.

APIs, however, offer more control over transcoding configurations without having to access a dashboard. This allows developers to configure encoding settings and use a variety of protocols and codecs. The process of uploading videos is also automated with APIs, whereas OVPs generally require manual uploads through the interface. Finally, some encoding solutions offer per-title encoding/transcoding capabilities. 

With per-title encoding, the settings are customized to each video. We designed the Bitmovin Per-Title Encoding solution to automatically analyze the complexity of every file and create the ideal adaptive bitrate (ABR) ladder depending on the content. This ensures high-quality viewing experiences and efficient data usage by creating dynamic bitrate ladders on a case-by-case basis. The player can then select from multiple bitrates based on network and computing resources available. 

One additional item is that you can deploy Bitmovin’s VOD and Live encoders on your infrastructure within any major cloud provider using Bitmovin’s Cloud Connect feature. This helps maintain the highest cost efficiency and use Bitmovin’s infrastructure through its managed service.

Read our Video Encoding Streaming Technology Guide to learn more.

Storage

Video accounts for the majority of the internet’s traffic. As such, it’s no surprise that CDN and storage bills make up the biggest operating expenses for OTT providers. The best way to minimize these costs is through technologies like per-title encoding, so you’ll want to consider how different components of your workflow impact one another when evaluating OVPs and APIs.

A Forrester study found that Bitmovin customers running their encoding in the cloud saw a 355% ROI over a three-year period.

Other factors to think about that impact storage costs include the anticipated volume, geographic distribution, and integration efforts. Many OVPs offer built-in storage solutions as part of their platform to simplify management. This provides a centralized storage system within the platform, but it’s difficult to tailor it to your specific storage requirements.

Streaming video APIs offer a more customizable approach to storage, including the ability to integrate with popular cloud storage providers like AWS, Microsoft Azure, and Google Cloud. This means developers can adapt the approach based on their scalability and geographic redundancy needs, and also optimize storage costs based on their existing workflows.

Distribution

Video delivery comes next, which is made possible by Content Delivery Networks (CDNs) like AWS, Akamai, Microsoft Azure, and Google Cloud Marketpalce. These networks of interconnected servers ensure efficient video distribution across the world. 

Most OVPs have multiple CDNs built-in, whereas APIs often give users the flexibility to deliver streams on their own CDN. With Bitmovin, you can do either, ensuring both customization and easy workflow configuration. 

Playback

Video players are essential components of streaming platforms, giving viewers control over what they watch, which devices they watch it on, and when the content plays. Players also tie everything together, making player control critical to the workflow.

HTML5 players can be built from scratch using an open-source option or deployed and customized using a solution like the Bitmovin Player. The same goes for deploying native players for iOS and Android. Going with a pre-built option provides access to advanced features like adaptive bitrate playback, DRM support, monetization capabilities, and interactive playback controls. 

Software development kits (SDKs) also play a major role in streamlining support for a range of devices and mobile applications by providing platform-specific integration tools. This helps organizations scale their solution and ensure a high-quality viewing experience for their audience without requiring significant development time. 

OVPs always have integrated video players as part of their platform, but they may lack the flexibility and customization required for branding or integrating unique playback features.

Read our Ultimate Guide to Video Players to learn more.

Analytics

Even the most straightforward streaming workflows have hiccups. As such, insight into video performance and quality of experience is a must. Organizations need the ability to pinpoint issues before they impact their audience, gain actionable insight into viewer behavior, and optimize resource utilization with visibility across the video streaming pipeline.

OVPs typically provide basic metrics like views, watch time, and completion rate. Some take this even further with heatmaps and click-through rates. For deeper insight, though, APIs are the way to go. 

With API access, you can gain insights into a wider range of data points, including:

• Error tracking
• Stream performance
• Advertising metrics
• Viewer demographics
• And more.

With Bitmovin’s Analytics, organizations can actively track more than 200 data points in real time and see how their streams compare to industry benchmarks. They can view performance within the Bitmovin Dashboard or utilize the Analytics API to get more granular insights which can then be pushed to major data aggregator platforms, such as Grafana, Looker Studio, AWS S3, and others for a more holistic view.

Online Video Platforms (OVPs)

Now that we’ve explored the primary requirements of video streaming — encoding and transcoding, storage, distribution, playback, and analytics — let’s dive into online video platforms (OVPs) and the best options for businesses.  

What is an OVP?

Online video platforms, or OVPs, are the prefabricated homes of video streaming. They act as turnkey solutions for managing, distributing, and monetizing online video content — eliminating the need for technical expertise or third-party integrations.

With an OVP, you get it all. The content management system (CMS), HTML5 video player for web-based devices, native players for mobile experiences (sometimes), and monetization tools are built in. This is great for businesses that want an effortless solution, but customization can be limited. It’s difficult to tailor OVPs to unique business models or existing workflows. As such, OVPs are better suited for building your business’s online presence across a dedicated channel, simple video workflows that don’t need to be fine-tuned, and hosting small content libraries on your website. 

OVP benefits

OVPs act as an all-in-one streaming platform for businesses with limited developer expertise and straightforward requirements. The benefits include:

• Turnkey solution: If you’re looking to host an online streaming event, embed content on your website, or use video for employee communications, OVPs are the quickest way to get started.
• Low cost of entry: OVPs are affordable and sometimes even free. They are also a great way to test interest among your user base before investing developer resources into building out a more comprehensive solution.

OVP cons

Because OVPs are designed for simple streaming workflows, businesses are limited to the tools and capabilities built into these platforms. This means that if you’re trying to build something specific, like an esports platform or fitness app, you’d be better off with an API. 

• Limited functionality: Advanced features and specific functionalities like low-latency streaming, VR & 360, and ad insertion are often missing from OVPs.
• Lacking control: Because OVPs control every step of the streaming workflow — including the encoding technology, CDN, and player — businesses using OVPs don’t have the same control over their infrastructure. 
• Missing insight: OVPs offer basic analytics capabilities. However, businesses requiring detailed insight into viewer behavior and stream performance would be better off with a streaming analytics API.

What to look for in an OVP

If an OVP makes sense for your business or video project, you’ll want to evaluate the following aspects of selecting a vendor:

1. Ease of use: Convenience is the name of the game with OVPs, so you’ll want to pick something with an intuitive user interface (UI). The goal is to streamline tasks like uploading, managing, and distributing video content for non-technical users.
2. Feature set: Essential features like uploading and sharing content, embedding video on your website, and the ability to stream live content should be table stakes when comparing providers. From there, in-depth analytics, the ability to customize the viewing experiences, and advanced encoding capabilities help set some OVPs apart.
3. Scalability and flexibility: If you’re planning for growth, you’ll want to choose an OVP that can scale with your business. Some OVPs offer APIs to accommodate future expansion, eliminating the need to migrate to a more flexible solution. 
4. Reliability and performance: Assess the uptime guarantees, server stability, and service level agreements of each option. Additionally, look for features like adaptive bitrate streaming, integration across multiple CDNs, and global delivery capabilities to ensure smooth playback for viewers worldwide.
5. Security measures: Content protection is key for use cases like corporate communications or streaming premium content to subscribed viewers. Encryption, access controls, and digital rights management (DRM) all help to this end.
6. Customer support: Assess the level of customer support provided by the OVP vendor, including tutorials, technical assistance, and dedicated account management. Look for a vendor that offers responsive support channels and comprehensive resources to help you maximize the capabilities of the platform.

Best OVPs for businesses

You’re likely familiar with leading brands in the OVP space. YouTube, Vimeo, and Bitmovin’s Streams are three popular examples. Here’s a look at how they compare. 

YouTube

As one of the most recognizable names in online video, YouTube needs no introduction. The platform allows users to upload, view, share, and comment on videos within its platform. Businesses can also use YouTube to embed videos on their website, but YouTube branding and advertisements make this a less-than-ideal application. Monetizing the content is also restrictive, as is content security. 

Most businesses use YouTube as a way to build their online presence rather than support their video infrastructure. For that reason, it’s often categorized as a social media channel rather than as an OVP. 

Marketers and businesses shouldn’t sleep on YouTube. However, creating video-powered products and services requires taking a different approach.

Vimeo

Vimeo offers tools for making, managing, and sharing videos. The platform lets businesses and creators host virtual events, webinars, and other marketing-focused experiences. It also provides the functionality to live stream to multiple social channels and websites simultaneously.

Vimeo offers limited monetization tools and often drives traffic to vimeo.com rather than your business’s website. That said, the platform delivers ad-free experiences and more end-user customization options than YouTube.

Marketers looking for a simple way to embed video on their website and live stream across multiple platforms may want to give Vimeo a try. But if you’re serious about building native video experiences that live entirely on your owned digital properties, you’ll want a more business-oriented OVP like Bitmovin’s Streams. 

Bitmovin’s Streams

Bitmovin’s Streams helps simplify streaming for businesses, serving as a single platform for live and on-demand encoding, CDN delivery, video playback, analytics, security, and more. As an end-to-end solution that’s built for the cloud, it eliminates the complexity of building your streaming infrastructure in-house. 

Features supported by Streams include:

• Flexible video uploading and encoding for on-demand content
• Live streaming and transcoding
• Drag-and-drop Player customization
• Simple sharing and easy-to-use embedding
• In-depth analytics
• WordPress plugin for quick integration
• Content protection with Signed URLs and domain restrictions

Streams also has a simple API for organizations looking for greater control, which brings us to the next section.

“Streams is one of our most important launches to date because it helps new media companies deliver high-quality streams to audiences simply and efficiently. New media companies typically have smaller developer teams that don’t have the time and capacity to get familiar with the complexities of video streaming. Therefore, there is a clear market need for a straightforward, low- or no-code solution like Streams that removes the complexity of video streaming to deliver content at speed and scale.

Demand for video streaming has grown at an incredible rate in recent years, all of which has been underpinned by extraordinary technological advancements. However, there now needs to be a greater focus on making innovations work in a simpler, more user-friendly way so video streaming can truly become ubiquitous, to enable everyone to build video products on the same level of quality and experience as the big names like Netflix.”

– Stefan Lederer (CEO, Bitmovin)

Video Streaming APIs

APIs, or application programming interfaces, are essential tools in every developer’s toolkit. They provide the flexibility to develop advanced apps while hiding the complexity behind the scenes. Here’s a look at the role they play in the world of audio and video streaming.

What is a video streaming API?

Video streaming APIs connect developers to streaming platforms like Bitmovin using code. Unlike traditional user interfaces (UI) found on video platforms like YouTube, APIs offer programmatic access to a wide range of features and capabilities, empowering developers to build customized streaming experiences tailored to their specific needs.

Without video APIs, businesses looking to deploy unique and innovative video applications would have to start from scratch. In this way, APIs speed things up significantly. Many developers elect to use video APIs to support a wide range of functionality for creating, customizing, and controlling video workflows. 

Some platforms that offer APIs can also be managed via a no-code UI. This is a great middle ground. While the UI might not provide the same level of control and customization, API access is available should the business need it. 

APIs impose almost no limits on the external services and functionality that you can integrate into your application while speeding up development through access to core services like encoding and playback.  

How do video APIs work?

Video APIs act as intermediaries, facilitating communication between the developer’s application and the underlying streaming infrastructure. In doing so, APIs hide the intricacies of online video distribution, letting developers focus on the products they’re building. 

Here’s what takes place behind the scenes when using a video streaming API.

1. Establishing communication: Video APIs create channels for developers to interact with the underlying video streaming platform. These channels typically operate over HTTP or HTTPS protocols, allowing for secure data transmission.
2. Authentication and authorization: Before accessing the functionalities offered by the streaming video API, developers need to authenticate themselves and obtain appropriate authorization. This is often achieved through the issuance of API keys or tokens, which verify the identity of the requesting user.
3. Requesting services and data: From there, developers can use video APIs to request various services and data from the streaming platform. This may include tasks such as uploading video content, initiating encoding or transcoding processes, retrieving playback URLs, or fetching analytics metrics.
4. Processing requests: The video API then processes these requests by interfacing with the backend infrastructure of the streaming platform. This involves executing the requested operations, such as encoding/transcoding video files into multiple formats, storing content in designated locations, or generating playback manifests.
5. Handling responses: After processing requests, video APIs generate responses containing the results of the requested operations. These responses are returned to the developers in a standard data format like JSON or XML. 
6. Monitoring and management: Video APIs often include functionalities for monitoring and managing video assets and workflows. This may involve querying the status of ongoing encoding jobs, adjusting playback settings dynamically, or accessing real-time analytics data to gain insights into viewer behavior.
7. Ensuring reliability and performance: Video APIs prioritize reliability and performance to ensure smooth and uninterrupted video streaming experiences. Mechanisms for fault tolerance, load balancing, and adaptive bitrate delivery help handle varying levels of demand and end-user bandwidth and mitigate potential disruptions.

Types of video streaming APIs

Streaming APIs are often broken out by the specific capabilities they support. As such, you may hear references to more nuanced services like a live video streaming API or video analytics API. The names are self-explanatory, but let’s touch on how they compare.

VOD encoding APIs

Video-on-demand (VOD) encoding APIs take source files and convert them into adaptive streaming formats like MPEG-DASH and HLS for adaptive bitrate delivery. They also create thumbnails, subtitles, and other metadata. But that’s not all. When using Bitmovin’s encoder, you benefit from per-title encoding capabilities, multi-codec streaming, and HDR support as well.

Capabilities to look for in a VOD encoding API:

• Multi-codec and format support
• Adaptive bitrate delivery (ABR)
• Per-title (also called content-aware) encoding
• Thumbnail and metadata generation
• Cloud-based processing
• Advanced features like DRM protection and ad insertion
• Integrations with your existing cloud storage, CMS, or analytics platforms

Live streaming APIs

Broadcasting live video online is no simple task. Unlike video-on-demand (VOD) encoding APIs, which focus on processing pre-recorded content, live streaming APIs facilitate the real-time transmission of video content to viewers as it happens. These workflows often use a contribution encoder like OBS or Videon EdgeCaster, as well as a live streaming API like Bitmovin. 

Whether broadcasting live events, webinars, or gaming streams, these APIs empower developers to deliver high-quality live video content reliably and efficiently. To keep lag at a minimum, you’ll want to find a live streaming API with support for low-latency protocols like RTMP or SRT.

Capabilities to look for in a live streaming API:

• Low-latency protocols like RTMP, SRT, and WebRTC
• Support for primary and backup inputs with automatic failovers
• Integration with popular contribution encoders like OBS, Wirecast, etc.
• Integration with interactivity tools like chat and polling
• Security and DRM
• Live recording and archiving
• Analytics and insights

“Bitmovin’s flexible and customizable technology has enabled us to solve one of our unique broadcasting challenges: to seamlessly generate a vast amount of parallel live video feeds and present them to the user in the highest quality, wherever they are in the world. 

Znipe.TV’s unique technology of broadcasting time-synchronized video stream of multiple angles sets new demand on a transcoder service, which Bitmovin delivers with their fantastic technical roadmap. To achieve the unique Znipe.TV viewing experience, we chose Bitmovin’s encoding to handle the video transcoding so that we can focus on what we do best, providing world-class entertainment for fans globally, live and on demand.”

– Erik Åkerfeldt (CEO & Co-founder, Znipe.TV)

Playback APIs

Playback APIs, also called client-side video APIs, allow developers to interact with a video player’s core functionality. This includes creating video player instances, controlling playback, or loading new sources. A video player API can also be used to monitor the state of a video player and receive notifications when certain playback events occur.

While some video player APIs differ across platforms, we designed the Bitmovin Player API’s to provide platforms with a unified development experience across Web/HTML5, Android, iOS, and Roku.

Capabilities to look for in a player API:

• Cross-platform compatibility and SDKs for different devices
• Customizable video player UI
• Advanced playback features like subtitles and playback speed control
• Adaptive bitrate support
• Offline playback support
• Integration with analytics platforms
• Security features
• Testing solutions to ensure quality playback

Analytics APIs

Video analytics APIs provide extensive customization over data architecture and how it’s presented. The Bitmovin Analytics API, for instance, allows developers to export raw datasets to cloud storage buckets and further enrich their insight with information collected by other providers. 

With analytics APIs, developers gain access to real-time monitoring and reporting capabilities. Whether the goal is to detect playback errors, identify trends, or monitor audience engagement during live events, these APIs enable timely decision-making and proactive intervention should any hiccups occur.

Capabilities to look for in an analytics API:

• Data capture across an extensive range of data points
• Real-time insights and reporting
• Data customization and export
• Integration with existing analytics platforms

Considerations when comparing video APIs

We’ve covered the capabilities needed for specific parts of the workflow, but what about general considerations that apply to all streaming APIs? Here’s a look at key considerations.

Flexible deployment

Development teams shouldn’t be retrained to specific hardware or cloud services. As such, you’ll want to look for video encoding and playback APIs that are decoupled from any underlying technology. Finding video infrastructure solutions that can be deployed anywhere prevents vendor lock-in and boosts agility.

If you’re already running applications in the cloud, then finding products that can run on your existing resources often makes sense. You’ll also gain more control over costs and commitments by finding a video streaming solution that’s available on your existing cloud providers’ marketplaces.

Bitmovin’s solutions are available on AWS Marketplace, Azure Marketplace, and Google Cloud Marketplace. We also offer the flexibility to utilize your pool of resources on Google Cloud or use our solution on Akamai Cloud.

Comprehensive functionality

Video streaming workflows have a lot of moving parts. So we’d suggest finding a video API that offers coverage across every step, from encoding to playback to analytics. It’s also imperative to assess your specific needs — such as low-latency streaming, ad insertion, and advanced UI styling — before landing on a vendor. 

Check out our extensive library of interactive demos and API examples for a peak at the functionality our platform supports.

Customization

One of the main benefits of going with a video API over an OVP is the extensive customization it will allow. This is especially important at customer touchpoints like the video player. The ability to adjust the appearance and add interactive elements to the player UI will help deliver the differentiated experience you’re aiming to build. 

That said, when speed-to-market is a priority, you don’t want to start from scratch. Finding a player API that can be tweaked without having to build the entire interface is a great middle ground.

Integration effort

The ease of integrating video capabilities into your workflow will impact your development timeline and the associated costs. Teams looking to get their services to market ASAP may be better off with a turnkey OVP than a video API. But, if the flexibility of a video API is non-negotiable, then you’ll want to find a solution with extensive developer tools. Launching cross-platform video experiences is already complex, which is why builder-centric resources are worth their weight in gold. 

Any vendor worth your investment should offer:

• Documentation
• Support for popular programming languages
• Development guides
• Code samples
• Community forums
• 24/7 technical support and SLAs
• Automated testing solutions
• Dedicated software developer kits (SDKs)

OVPs vs. streaming APIs

To wrap up the last two sections, here’s a table summarizing the key differences between OVPs and APIs:

Video streaming SDKs

An SDK is a set of software-building resources tailored to a specific platform (like Roku) or scripting language (like Python). 

What is an SDK?

An SDK, or software development kit, is a set of developer-centric tools designed for a specific hardware, operating system, or language. These pre-packaged kits are made up of libraries, APIs, documentation, and code samples — essentially everything required to make a developer’s life easier.

What is a streaming SDK?

Video streaming SDKs help businesses accomplish specific tasks, such as mobile encoding or video playback on gaming consoles. While SDKs aren’t required to support this functionality, they abstract away much of the complexity and provide developers with specialized tools catered to a need.

Common video SDK Examples

Popular streaming SDKs are tailored to address the unique requirements of different platforms and use cases. Here are some of the SDKs video engineers rely on.

Encoding SDKs

Say you’re looking to build a user-generated content (UGC) mobile app that enables users to stream live video within your platform. For this, you’d need a mobile encoding SDK to convert the raw video files into a compressed streaming format for transport over the internet. 

Technologies like Streamaxia OpenSDK and the Larix Broadcaster SDK support these capabilities by encoding live content into contribution protocols like RTMP, SRT, and RTSP.

Alternatively, imagine you’re building a fitness broadcast platform like Classpass that lets users stream on-demand workout videos. Integrating the Bitmovin API directly into your platform would ensure high-quality playback for viewers and cost-effective storage for your organization. However, implementing this into your existing technology stack could require writing and maintaining code specific to the API’s structure and functionalities. 

With encoding SDKs catering to specific languages like Java, Ruby, and Python, developers can rely on pre-written methods for interacting with the Bitmovin Player API — significantly speeding up time to market.

Bitmovin offers dedicated SDKs for the following programming languages:

• PHP
• C#
• Python
• Node.js / JavaScript
• Java
• Ruby
• Go

Video player SDKs

Consider all the devices that we stream content on today. End users demand the same experience across mobile screens, web browsers, and smart TVs. All of these environments have different requirements, though, which translates to countless hours of development time. 

Using a player SDK catered to each device alleviates this challenge. Video player SDKs make it easy to deploy your solution everywhere viewers are tuning in while ensuring flawless playback across screens. 

They do so by providing the app development tools required to embed video players into specific devices, making it simple for developers to create, control, and monitor the video player experience.

Bitmovin offers dedicated SDKs for the many devices out there, including:

• Android and Android TV SDK
• Flutter SDK
• Gaming Consoles SDK
• HTML5 Web SDK
• iOS and tvOS SDK
• React Native SDK
• Roku SDK
• Smart TV SDK

Did you know…

Deploying Bitmovin’s Player on 2 or more SDKs enables customers to reach an additional 200,000 viewers monthly. Moreover, utilizing it on both iOS and Android platforms can save over 600 hours in player maintenance annually.

Learn more.

Streaming APIs vs. video SDKs

In most cases, developers use APIs and SDKs in tandem. So when considering video encoding, player, and analytics solutions, you’ll want to find developer-centric partners like Bitmovin that provide robust APIs and SDKs, extensive documentation, Github repositories, and community forums to speed up buildout.

Here’s a summary of how APIs and SDKs compare in terms of required expertise, development effort, and customization.

Conclusion

If you’re looking to add video to your service or application, you’re going to need an OVP, API, SDK, or a combination of all three. 

Here at Bitmovin, we use YouTube as a marketing channel and the Streams UI as a tool to quickly go live and share on-demand content on our website. These OVPs are great for tasks like uploading and sharing video content.

Companies looking to build innovative video platforms need more features than an OVP can provide. There’s always the option to develop bespoke solutions in-house, but it can get expensive. Plus, time to market matters. And by selecting ready-to-use streaming solutions that integrate with your existing ecosystem, businesses can speed things up.  

APIs and SDKs provide the perfect middle ground of speed and customization. That’s why we offer developer-centric video infrastructure solutions backed by API coverage across the video workflow.  Our extensive library of streaming APIs, VOD and Live Encoders, Player SDKs, and real-time Analytics simplifies building and optimizing without constraints.

Whether you need an end-to-end video platform backed by a simple API or a combination of components (such as an HTML5 player, cloud-based encoding, live encoding, or video analytics solution), we provide the development solution required to power the future of online video. 

Find out how Bitmovin’s streaming products, APIs, and SDKs can give you a competitive edge. Start your trial today.

The post Video Platforms, Video Streaming APIs, and SDKs Explained appeared first on Bitmovin.

Welcome to our encoding definition and adaptive bitrate guide.

This article is for anyone seeking a way into the world of Video Technology and Development, or for those of you looking for a quick refresher on the key terms that define the industry.

You’ll learn exactly what encoding is and some of the most important factors within the encoding process.

Let’s get started. 

What is a Codec?

A codec is a device or a program intended to compress raw media files (ex: video, audio, and/or subtitles). There are multiple variations of codecs for each file format; common examples of video codecs include: H.264, HEVC, VP9 and AV1.

For audio there are: AAC, MP3 or Opus. A few essential codecs are visible in the image below:

The purpose of a codec is to efficiently and quickly transfer, store or play back a file on a device. The process of compressing these raw or uncompressed files into a codec is known as encoding. 

What is Encoding?

Encoding is the process of converting a raw video file (codec) into a compatible, compressed and efficient digital format. The new compressed file is capable of distribution across the web and playback in mobile or TV players. 

For example: A multimedia conglomerate could be tasked with distributing OTT content like Game of Thrones to a commuter’s mobile device in a region that may have slower internet/data speeds. This transmission would therefore require a lot of back-end communications and encodes; where distributing an individual episode at the highest quality (recording quality of cameras), would be highly inefficient & expensive.

A solution is to run these ultra high quality videos through a video encoder during the processing phase. This  packages the requested video files in a way that will lose minimum quality during the transmission, otherwise known as “semi-lossless compression”. 

From a technical perspective, an example of encoding would be the delivery of a single uncompressed RGB 16-bit frame, with a size of 12.4MB to a Monochrome 8-bit frame with a size of 3.11MB.

If you are reading this from Europe – the standard is 25 frames per second (FPS), whereas videos in the US run at 29.97 FPS. So, for 60 seconds of video at 24 frames per second an encoding software would bring the total size of the video file down from 17.9GB to 2.9GB.

However, 3GB for 60 seconds of video may still be too much to stream from your phone while you are attempting to watch something on the bus to work, so further optimization is needed. 

What is Transcoding?

A more complex variation of encoding is transcoding, the process of converting one codec to another (or the same) codec. Both decoding & encoding are necessary steps to achieving a successful transcode.

Transcoding is a standard practice for online video – the process of compressing an already compressed file. Therefore enabling consumers to access higher quality experiences at significantly lower costs of distribution.

In other words, more steps are necessary to deliver that high quality video to multiple devices. Additionally, an encoder can implement frame size reductions to maximize the perceived quality of your average consumer.

So, how does one further compress a data file?

Using a command line interface, encoders like Bitmovin, who provide both API and GUI encoder products, analyze and process all inputted video files.

Depending on which resolution is needed in the output file, a different video codec is used. The best video codec is one that encodes video for the specific resolution and format that optimizes for best perceived quality in the smallest possible size.

One of the standard metrics of measurement for video quality is the peak signal-to-noise ratio (PSNR): the comparison of “good data” against how much noise there is within the file; the higher the number, the better.

PSNR is measured using decibels (like in sound) and 80db is typically a good magnitude for quality.

However, not all video files are equal, sports and dynamic videos are significantly larger in size and complexity than your average cartoon. As a result, encoders like Bitmovin utilize a further customizable solution, per-title encoding, which tunes each compression to achieve maximum quality, minimum size, and at the lowest cost.

What is Bitrate?

Having learned the definitions of Encoding and Transcoding and how they affect content quality, the next step is defining the basis of measurement for speed and cost in media transmission.

The industry standard, Bitrates, are calculated (and charged) based on the number of bits per second that can be transmitted along a digital network. The higher amount of Bits that can be processed per second are indicative of a faster and higher quality transfer – however this usually comes at a higher cost.

All available bitrates and resolutions that video (and audio) segments  are encoded in, as well as their server locations are referenced in a text file defined by either the DASH or HLS protocols. These manifest files (.mpd for DASH, .m4u8 for HLS) are fed into a player; which protocol is used depends entirely on the device capabilities of the consumer.

Bitrate gives a value of how much of the video file (in bits) can we process over time while the video is playing back. However, it doesn’t always make sense to transfer the highest quality to every user and every device.

There are some who will consume the content on a cellular network while in motion (like for our friendly aforementioned commuter) and others who will consume that same content on a 4K TV with a fibre optic connection.

In addition, that same user may start viewing the content on the 4K TV and continue en route to their office on a mobile phone with a 3G network. 

Encoding & Bitrates in Action

During an encode, video and audio components are split (a reference is kept for the decode) in 1 second segments; the segment length can be arbitrary, but the maximum is 10 seconds.

Each of these video segments can be saved in a different quality (and frame size) by a video encoder.

The quality and size of the output video is set by selecting a bitrate by a distributing service. In a perfect world, the service provider will select the perfect bitrate for each video to be transferred to the end user that will avoid stuttering or buffering.

You can find a chart of the standard bitrate ladder below as compared to the ladder for Bitmovin’s Per-Title Encoding solution:

Latest in Encoding Tech: VVC and VP9 codecs

The latest state of the art encoding tech is Versatile Video Coding (VVC); an improvement over Next Gen Open Video or VP9 codec (2013). VVC improves the prediction of the parts (blocks) in a frame by looking at other neighboring blocks and comparing them to what they behaved like before the encode/transcode.

Factors that play into how the VVC function include: the motion of the block with respect to all others (or motion compensation), changes of the block from how it looked in the past, and a prediction of how it will look like in the future (temporal motion prediction).

Future of Video: Common Media Application Format (CMAF) 

The future of streaming is driven by CMAF, an encoding method that splits a video file into small chunks.

These chunked files are instantly playable by a consumer, unlike segmented files which need to be fully downloaded before playing.

Think of a flaky connection: high lags with long buffer times, just to download 10 seconds of video. CMAF aims to solve flaky videos with a Common Encryption format to ease the deployment of Digital Rights Management technologies.

We hope you found this encoding definition and adaptive bitrate guide useful.  If you did, please don’t be afraid to share it on your social networks!

More video technology guides and articles:

• Back to Basics: Guide to the HTML5 Video Tag 
• What is a VoD Platform? A comprehensive guide to Video on Demand (VOD)
• HEVC vs VP9: Modern codecs comparison
• What is the AV1 Codec?

The post Encoding Definition and Adaptive Bitrate: Video Compression Basics appeared first on Bitmovin.

How to use the Simple Encoding API

The Simple Encoding API will help you to create an encoding that will fit a vast array of use cases with only a single endpoint. You can either use any of Bitmovin’s SDKs or directly call the endpoints with tools like Postman. For additional information on how to add the API key to the request can be found here: https://bitmovin.com/docs/encoding/tutorials/get-started-with-the-bitmovin-api.
With a single endpoint in mind, the only things that you need to set up are the inputs and the outputs. 
The first thing you will need is input. The most common input types are supported: S3, GCS, Azure Blob Storage, Akamai NetStorage, HTTP(S), and (S)FTP.
You can choose between four different input types: audio, video, subtitles, and closed captions. If you don’t specifically state an input type for a file, it will be assumed to contain a video track and an optional audio track.
Multiple audio and subtitles/closed captions inputs can be used but only one video input is possible. Specifying a language is mandatory for subtitles and closed captions; as a best practice, we also recommend specifying the language for audio inputs.

Simple Encoding Output

Let’s define an input. 

"inputs":[
   {
    "url": "https://ftp.halifax.rwth-aachen.de/blender/demo/movies/ToS/ToS-4k-1920.mov"
   }
]

Next, we are going to define an output. The Simple Encoding API supports S3, GCS, Azure Blob Storage, and Akamai NetStorage as output, further details can be found in our documentation. One thing to consider is if the created files need to be private or public. If the files are public the output has to be able to support that.
Next, let’s set up a private S3 output.

"outputs": [
        {
            "url": "s3://your/output/path",
            "credentials": {
                "accessKey": "accessKey",
                "secretKey": "secretKey"
            }
        }
    ]

Now you only need to choose a name for your encoding job and put the inputs and the outputs into one JSON. It will look somewhat like this:

{
    "name": "simple_encoding_name",
    "inputs": [
        {
            "url": "https://ftp.halifax.rwth-aachen.de/blender/demo/movies/ToS/ToS-4k-1920.mov"
        }
    ],
    "outputs": [
        {
            "url": "your/output/path",
            "credentials": {
                "accessKey": "accessKey",
                "secretKey": "secretKey"
            }
        }
    ],
}

With the JSON completed the last thing to do is to send it to the API via this endpoint.

POST https://api.bitmovin.com/v1/encoding/simple/jobs/vod

The response will look like this

{
    "id": "854a2a86-9028-4e33-863b-d602a6bac24b",
    "status": "CREATED",
    "encodingId": null,
    "inputs": [
        {
            "url": "https://ftp.halifax.rwth-aachen.de/blender/demo/movies/ToS/ToS-4k-1920.mov"
        }
    ],
    "outputs": [
        {
            "url": "your/output/path",
        }
    ],
    "createdAt": 2022-01-17T14:26:54Z,
    "modifiedAt": 2022-01-17T14:26:54Z,
    "name": "simple_encoding_name"
}

Now that the start call is fully set up, the last thing to do is wait for the encoding to finish.
The endpoint will show the current status of your encoding

GET /encoding/simple/jobs/ vod/{simple_encoding_job_id}
{
    "id": "854a2a86-9028-4e33-863b-d602a6bac24b",
    "status": "FINISHED",
    "encodingId": "5b71dcf1-5f92-4534-b530-47706003e7a4",
    "inputs": [
        {
            "url": "https://ftp.halifax.rwth-aachen.de/blender/demo/movies/ToS/ToS-4k-1920.mov"
        }
    ],
    "outputs": [
        {
            "url": "your/output/path",
            "makePublic": true
        }
    ],
    "name": "simple_encoding_name"
}

Once the status is set to “finished” the encoding is done.
The Simple Encoding API can be accessed by our SDKs as well.

Simple Encoding API output

Once the simple encoding has finished, it will write the output to your specified location. The encoding will create segmented video files as well as HLS and DASH manifests. The encoding will furthermore create thumbnails, sprites, subtitles, closed captions, and audio if your input contains them.

We know that starting your video encoding can be hard, thus we aimed to keep this API as simple as possible while still providing you with everything you might need for your workflow. We will continue to improve our Simple Encoding API, so if you have any suggestions for improvements or how we can make it more suitable for your use case, please let us know!

How to Troubleshoot the Simple Encoding API & FAQs

Error Messages

“No API key found in request”
API requests must be authenticated using your API key.  This is found in the dashboard under your name (top right/Account settings).  This is placed as the X-Api-Key in the header of your POST request.  We suggest using a tool such as POSTMAN to create the POST request during testing 
Could not determine the scheme
Output URLs follow the standard URL structure provided by the storage providers.   These are written above as “your/output/path” however must also include the scheme identifier:   for google this is gcs://<bucket-name>/folder,  for amazon this is s3://<bucket-name>/folder,  for ftp ftp://__ etc.    Failure to put the <something>:// will result in this error message.

Questions

I want to the Simple Encoding API,  but I don’t have an API key:
The easiest way is to sign up for a trial via the bitmovin homepage bitmovin.com /  Try for free (top right)
I have submitted the POST request,  the response says “SUCCESS” but nothing is appearing in the dashboard:
There may be something wrong in credentials or file locations.

1. Copy the “id”  found in the first line of the response
2. Query it with GET /encoding/simple/jobs/vod/{simple_encoding_job_id}  (where simple_encoding_job_id is the id you just copied)
3. This will provide more thorough details of the simple-encoding-job.

What’s the difference between simple_encoding_job_id (referenced above) and Encoding ID (in the dashboard)? 
Once a simple-encode-id-job is successfully submitted an encodeId will be assigned.  You can find the encodeId for a particular simple_encoding_job_id via GET /encoding/simple/jobs/vod/{simple_encoding_job_id} and looking for encodeId in the response.
How to add different input types to the Simple Encoding request?
You can view all input types and how to use them in our documentation here: https://bitmovin.com/docs/encoding/articles/simple-encoding-api
Try out our new Simple Encoding API for yourself by signing up for a trial today.

Video technology guides and articles

• Back to Basics: Guide to the HTML5 Video Tag 
• What is a VoD Platform? A comprehensive guide to Video on Demand (VOD)
• Video Technology [2022]: Top 5 video technology trends
• HEVC vs VP9: Modern codecs comparison
• What is the AV1 Codec?
• Video Compression: Encoding Definition and Adaptive Bitrate
• What is adaptive bitrate streaming
• MP4 vs MKV: Battle of the Video Formats
• AVOD vs SVOD; the “fall” of SVOD and Rise of AVOD & TVOD (Video Tech Trends)
• MPEG-DASH (Dynamic Adaptive Streaming over HTTP)
• Container Formats: The 4 most common container formats and why they matter to you.
• Quality of Experience (QoE) in Video Technology [2022 Guide]

The post Introducing Bitmovin’s Simple Encoding API appeared first on Bitmovin.

Here you will find everything a developer needs to know about this cutting-edge topic, in a a guide created by some of the most advanced video technology experts and engineers.

Take your time to read through the whole resource, or just skip to the chapter that interests you using the table of contents below.

What Are Compression Algorithms? 

When it comes to content distribution, especially in the form of video, the size of the content can make or break your business.

Even standard quality content files (video, audio, and text) end up taking up a lot of space, especially as applied to the transportation and/or distribution of the file.

To alleviate the potentially extremely high cost of storage and delivery everyone uses some form of compression algorithms to reduce file size.

The use of compression is of utmost importance to your success because it reduces the file size while maintaining the same user-perceived quality. At the time of this blog post, there are two variations of compression algorithms – lossy and lossless.

The focus of this post is lossy compression.

Introduction to Lossy Compression

Lossy compression means that compressed data is not exactly the same as it was originally, but a close approximation to it.

In most cases, the human eye wouldn’t even notice the difference between an original file and one compressed in a lossy way, but it yields a much higher compression ratio than the lossless compression, where an exact copy of the content is created. 

Lossy compression is one of the most important factors necessary in modern content distribution methods.

Without (lossy) compression the content we view every day wouldn’t be nearly as high quality as it actually is, and that’s just one of the pitfalls society might face without any kind of compression.

Other challenges viewers and distributors would face without (lossy) compression: slow load/buffer times, high delivery and storage costs, and limited distribution capabilities.

This blog acts as complementary material to our Video Developer Network – if you would like to learn about lossy compression algorithms in a classroom-style video format watch the video here.

What the Math?! Lossy Compression Ratios & Metrics in Digital Video

Lossy compression algorithms deliver compression ratios that are high and represent most multimedia compression algorithms in image, video, and audio content.

The goal of video and audio compression is to maximize the compression ratio and to minimize the distortion; a common trade-off in all compression technologies.

The standard formula for lossy compression algorithms is defined as “close-approximation”, measured by establishing various distortion metrics that specify how close the compressed content is to the original – the most common measures are defined below:

Perceptual Distortion

Perceptual distortion is a famous metric that has been used historically for assessing video quality. Distortion theory provides the framework to study the trade-offs between the data rate and the Distortion itself. 

In the graph above: Y-axis is the data rate and X-axis the distortion level.  If you have a high data rate and a zero distortion, it is a lossless compression scheme.

As soon as cost/spend limitations are considered (in the form of bandwidth and/or storage), data reduction rates will increase and image distortion will appear. 

Mean Square Error

Another measure of distortion is mean square error, where is X the input data sequence, Y is output data sequence and N is the count of elements: 

Peak-Signal-To-Noise Ratio (PSNR)

Then there is the Peak-Signal-To-Noise ratio (PSNR) which is calculated by comparing the size of an error relative to the peak value of a signal.

The higher the PSNR, the better the video quality. Signal-to-noise ratios are typically expressed in decibel units (dB).

A good ratio will register values of around 80db.

Having explained the metrics used to evaluate the accuracy and quality of lossy compression, it’s time to discuss how the compression process works.

Lossy Compression: the “two” step process

Step 1: Quantization

The step that adds the most distortion is quantization.

Quantization is the process of mapping input from a large set (like an analog signal) to numerical output values in a smaller (usually finite) set.

There are 3 different forms of quantization: uniform, non-uniform, vector.  

1. Uniform scalar quantizer – subdivides the domain of the input into output values at regular intervals, with the exceptions at the two outer extremes. 
2. Non-uniform quantizer – output values are not at equally spaced intervals. The output of the reconstructed value that corresponds to each interval is taken during quantization, the midpoint of this interval and the length of each interval is then referred to as the step size which can be denoted by a symbol.
3. Vector quantizer – high decoding complexity, output values can be distributed irregularly, not in a grid fashion – such as in the scalar quantizer case – because an output value represents a vector and not a scalar value.

Step 2: Transform coding

Transform coding is the second step in Lossy Compression.

Transform coding is the process of creating a quantized group of blocks (containing all pixels in a frame) of consecutive samples from a source input and converting it into vectors.

The goal of transform coding is to decompose or transform the input signal into something easier to handle.

There is a good chance that there will be substantial correlations among neighboring samples; to put it in other words, adjacent pixels are usually similar, therefore, a compressor will remove some samples to reduce file size.

The range of pixels that can be removed without degrading quality irreparably is calculated by considering the most salient ones in a block.

For example: If Y is the result of a linear transform T of the input vector X in such a way that the components of Y are much less correlated, then Y can be coded more efficiently than X.

If most information is accurately described by the first few components of a transformed vector Y, then the remaining components can be coarsely quantized, or even set to zero, with little signal distortion. 

As correlation decreases between blocks and subsequent samples, the efficiency of the data signal encode increases.

Spatial frequency is one of the most important factors of transform coding because it defines how an image (and the pixels within it) change throughout playback in relation to previous and future pixel blocks.

This graphs here depicts two variations:

Spatial frequency indicates how many times pixel values change across an image block. It’s key to note – the human eye is less sensitive to higher spatial frequency components associated with an image than lower spatial frequency components.

If amplitude (learn more about frequency components metrics here) falls below a predefined threshold, it will not be detected by the average human eye.

A signal with high spatial frequency can be quantized more coarsely and therefore maintain quality at lower data rates than a signal with low spatial frequency, which will need more data to provide the user with high perceived quality.

One of the other factors is – Discrete Cosine Transform (DCT) implements the measure of motion by tracking how much image content changes corresponding to the numbers of cycles of the cosine in a block.

The DCT is part of the encoding algorithm and converts pixel values in an image block to frequency values, which can be transmitted with lower amounts of data.

DCT is lossless – apart from rounding errors – and spatial frequency components are called coefficients. The DCT splits the signal into a DC – direct current component and an AC, alternating current component.

With the IDCT or Inverse Discrete Cosine Transform, the original signal is reconstructed and can be decoded and played back. 

Step 2.5: Other Transformation Formats

Wavelet
An alternative method of lossy compression is wavelet transformation; which represents a sigma with good resolution in both time & frequency and utilizes a set of functions, called wavelets to transform to decompose an input signal.

Wavelet-coding works by repeatedly taking averages and differences by keeping results from every step of different image parts, this is (almost) a multi-resolution analysis.

A wavelet transform creates progressively smaller summary images from the original, decreasing by a quarter of the size for each step. A great way to visualize wavelet coding is to consider a pyramid – stacking a full-size image, quarter-size image, sixteenth-size image, and so on, on top of each other.

The image has gone through a process of subsampling (through the wavelet transformation algorithm) decreasing the size but aiming at maintaining the quality in smaller iterations.

The image on the right in the top left quadrant has a compressed representation of the full-scale image on the left, which can be reconstructed from the smaller one by applying the wavelet coding transformation inversely.

Another example of lossy compressing a white and black image is:

2D Haar Transform

2D Haar Transform is the representation of a signal with a discrete non-differentiable (step) function – consider a function that represents on/off states of a device.

In the context of image decomposition for a  simple image applying the 2D Haar Transform would look like:

The image on the left represents the pixel values of the image on the right, an 8 x 8 image.

Applying a 2D Haar Transform for the second level, yields a linear decrease of the image size:

The calculated differences and image decrease allow for the image to be compressed with less data while keeping an eye on quality.

More compression means lower quality and higher quality means lower compression.

In the case of color images, the same applies:

In short, the goal of all compression algorithms is to achieve the highest possible compression ratio. For any video distributor compression ratios come to down to cost and quality considerations.

Which trade-off will yield the highest ROI? High compression and high quality at higher costs? The opposite? Or somewhere in the middle?

That’s for you to decide!

Did you enjoy this post? Check out our Video Developer Network for the full university quality videos.  (including a lesson on Lossless Compression)

More video technology guides and articles:

• Encoding Definition and Adaptive Bitrate: Video Compression Guide
• Back to Basics: Guide to the HTML5 Video Tag 
• What is a VoD Platform? A comprehensive guide to Video on Demand (VOD)
• Video Technology [2022]: Top 5 video technology trends
• HEVC vs VP9: Modern codecs comparison
• What is the AV1 Codec?

Did you know?

Bitmovin has a range of VOD services that can help you deliver content to your customers effectively.

Its variety of features allows you to create content tailored to your specific audience, without the stress of setting everything up yourself. Built-in analytics also help you make technical decisions to deliver the optimal user experience.

Why not try Bitmovin for Free and see what it can do for you.

This is a comprehensive guide to lossy compression algorithms. 

In this detailed post, we’ll cover the following topics:

• An Introduction to Lossy Compression
• Lossy Compression Ratios & Metrics in Digital Video
• Perceptual Distortion
• Peak-Signal-To-Noise Ratio (PSNR)
• Lossy Compression: the “two” step process
  • Step 1: Quantization
  • Step 2: Transform coding
• Wavelet Transformation
• 2D Haar Transform

So, if you are looking to learn about lossy compression algorithms then you are in the right place.

What Are Compression Algorithms? 

When it comes to content distribution, especially in the form of video, the size of the content can make or break your business.

Even standard quality content files (video, audio, and text) end up taking up a lot of space, especially as applied to the transportation and/or distribution of the file.

To alleviate the potentially extremely high cost of storage and delivery everyone uses some form of compression algorithms to reduce file size.

The use of compression is of utmost importance to your success because it reduces the file size while maintaining the same user-perceived quality. At the time of this blog post, there are two variations of compression algorithms – lossy and lossless.

The focus of this post is lossy compression.

Introduction to Lossy Compression

Lossy compression means that compressed data is not exactly the same as it was originally, but a close approximation to it.

In most cases, the human eye wouldn’t even notice the difference between an original file and one compressed in a lossy way, but it yields a much higher compression ratio than the lossless compression, where an exact copy of the content is created. 

Lossy compression is one of the most important factors necessary in modern content distribution methods.

Without (lossy) compression the content we view every day wouldn’t be nearly as high quality as it actually is, and that’s just one of the pitfalls society might face without any kind of compression.

Other challenges viewers and distributors would face without (lossy) compression: slow load/buffer times, high delivery and storage costs, and limited distribution capabilities.

This blog acts as complementary material to our Video Developer Network – if you would like to learn about lossy compression algorithms in a classroom-style video format watch the video here.

What the Math?! Lossy Compression Ratios & Metrics in Digital Video

Lossy compression algorithms deliver compression ratios that are high and represent most multimedia compression algorithms in image, video, and audio content.

The goal of video and audio compression is to maximize the compression ratio and to minimize the distortion; a common trade-off in all compression technologies.

The standard formula for lossy compression algorithms is defined as “close-approximation”, measured by establishing various distortion metrics that specify how close the compressed content is to the original – the most common measures are defined below:

Perceptual Distortion

Perceptual distortion is a famous metric that has been used historically for assessing video quality. Distortion theory provides the framework to study the trade-offs between the data rate and the Distortion itself. 

In the graph above: Y-axis is the data rate and X-axis the distortion level.  If you have a high data rate and a zero distortion, it is a lossless compression scheme.

As soon as cost/spend limitations are considered (in the form of bandwidth and/or storage), data reduction rates will increase and image distortion will appear. 

Mean Square Error

Another measure of distortion is mean square error, where is X the input data sequence, Y is output data sequence and N is the count of elements: 

Peak-Signal-To-Noise Ratio (PSNR)

Then there is the Peak-Signal-To-Noise ratio (PSNR) which is calculated by comparing the size of an error relative to the peak value of a signal.

The higher the PSNR, the better the video quality. Signal-to-noise ratios are typically expressed in decibel units (dB).

A good ratio will register values of around 80db.

Having explained the metrics used to evaluate the accuracy and quality of lossy compression, it’s time to discuss how the compression process works.

Lossy Compression: the “two” step process

Step 1: Quantization

The step that adds the most distortion is quantization.

Quantization is the process of mapping input from a large set (like an analog signal) to numerical output values in a smaller (usually finite) set.

There are 3 different forms of quantization: uniform, non-uniform, vector.  

1. Uniform scalar quantizer – subdivides the domain of the input into output values at regular intervals, with the exceptions at the two outer extremes. 
2. Non-uniform quantizer – output values are not at equally spaced intervals. The output of the reconstructed value that corresponds to each interval is taken during quantization, the midpoint of this interval and the length of each interval is then referred to as the step size which can be denoted by a symbol.
3. Vector quantizer – high decoding complexity, output values can be distributed irregularly, not in a grid fashion – such as in the scalar quantizer case – because an output value represents a vector and not a scalar value.

Step 2: Transform coding

Transform coding is the second step in Lossy Compression.

Transform coding is the process of creating a quantized group of blocks (containing all pixels in a frame) of consecutive samples from a source input and converting it into vectors.

The goal of transform coding is to decompose or transform the input signal into something easier to handle.

There is a good chance that there will be substantial correlations among neighboring samples; to put it in other words, adjacent pixels are usually similar, therefore, a compressor will remove some samples to reduce file size.

The range of pixels that can be removed without degrading quality irreparably is calculated by considering the most salient ones in a block.

For example: If Y is the result of a linear transform T of the input vector X in such a way that the components of Y are much less correlated, then Y can be coded more efficiently than X.

If most information is accurately described by the first few components of a transformed vector Y, then the remaining components can be coarsely quantized, or even set to zero, with little signal distortion. 

As correlation decreases between blocks and subsequent samples, the efficiency of the data signal encode increases.

Spatial frequency is one of the most important factors of transform coding because it defines how an image (and the pixels within it) change throughout playback in relation to previous and future pixel blocks.

This graphs here depicts two variations:

Spatial frequency indicates how many times pixel values change across an image block. It’s key to note – the human eye is less sensitive to higher spatial frequency components associated with an image than lower spatial frequency components.

If amplitude (learn more about frequency components metrics here) falls below a predefined threshold, it will not be detected by the average human eye.

A signal with high spatial frequency can be quantized more coarsely and therefore maintain quality at lower data rates than a signal with low spatial frequency, which will need more data to provide the user with high perceived quality.

One of the other factors is – Discrete Cosine Transform (DCT) implements the measure of motion by tracking how much image content changes corresponding to the numbers of cycles of the cosine in a block.

The DCT is part of the encoding algorithm and converts pixel values in an image block to frequency values, which can be transmitted with lower amounts of data.

DCT is lossless – apart from rounding errors – and spatial frequency components are called coefficients. The DCT splits the signal into a DC – direct current component and an AC, alternating current component.

With the IDCT or Inverse Discrete Cosine Transform, the original signal is reconstructed and can be decoded and played back. 

Step 2.5: Other Transformation Formats

Wavelet
An alternative method of lossy compression is wavelet transformation; which represents a sigma with good resolution in both time & frequency and utilizes a set of functions, called wavelets to transform to decompose an input signal.

Wavelet-coding works by repeatedly taking averages and differences by keeping results from every step of different image parts, this is (almost) a multi-resolution analysis.

A wavelet transform creates progressively smaller summary images from the original, decreasing by a quarter of the size for each step. A great way to visualize wavelet coding is to consider a pyramid – stacking a full-size image, quarter-size image, sixteenth-size image, and so on, on top of each other.

The image has gone through a process of subsampling (through the wavelet transformation algorithm) decreasing the size but aiming at maintaining the quality in smaller iterations.

The image on the right in the top left quadrant has a compressed representation of the full-scale image on the left, which can be reconstructed from the smaller one by applying the wavelet coding transformation inversely.

Another example of lossy compressing a white and black image is:

2D Haar Transform

2D Haar Transform is the representation of a signal with a discrete non-differentiable (step) function – consider a function that represents on/off states of a device.

In the context of image decomposition for a  simple image applying the 2D Haar Transform would look like:

The image on the left represents the pixel values of the image on the right, an 8 x 8 image.

Applying a 2D Haar Transform for the second level, yields a linear decrease of the image size:

The calculated differences and image decrease allow for the image to be compressed with less data while keeping an eye on quality.

More compression means lower quality and higher quality means lower compression.

In the case of color images, the same applies:

In short, the goal of all compression algorithms is to achieve the highest possible compression ratio. For any video distributor compression ratios come to down to cost and quality considerations.

Which trade-off will yield the highest ROI? High compression and high quality at higher costs? The opposite? Or somewhere in the middle?

That’s for you to decide!

Did you enjoy this post? Check out our Video Developer Network for the full university quality videos.  (including a lesson on Lossless Compression)

The post Lossy Compression Algorithms: Everything a Developer Needs to Know appeared first on Bitmovin.

In the context of video – Transcoding refers to the process of compressing video files as much as possible at minimal quality loss to represent (and transfer) information by using less data.

Essentially, video transcoding online is the conversion of a video file from one format to a better-compressed version to ensure consumers can stream content without buffering and at the highest possible qualities.

It’s easy to get Transcoding mixed up with Encoding. A good encoding definition is ‘the process of converting a raw file (codec) in to a compatible, compressed and efficient digital format’.

So, with that said, how does transcoding work? And how might it affect your everyday life?

Keep reading to find out!

Your camera (or device), your content, and video transcoding in your home

Picture this scenario: You’ve recently returned home to your comfortable sofa from your latest adventure or hobby. For the past few hours, your eyes have been locked on your computer, uploading and editing the HD videos that you captured earlier. You’ve finally finished the editing process on your laptop from your GoPro, high-quality Kodak cam, and/or Apple, Android, Windows, or other devices (Bitmovin is OS-agnostic after all!) and you’re ready to download and share your latest creation. The raw video files that you recorded on your device are significantly larger than your standard cloud storage, file sharing service, or social media platform can (or will) handle (pro-tip: check the raw size of a video file saved on the actual device – chances are, it’s huge!). Most raw HD video files amount to 18 GB of storage for every 60 seconds; based on an average of 1920 x 1080 pixels of a standard RGB 3x16bit uncompressed TIFF file.
To most, clicking export and then “share to social” are the final step to collecting those sweet sweet “likes.” But that’s not enough for you, you want to understand how and why to move the video content from one device to another. The first critical step is to hit “export and save”, most editing software (like the GoPro Quik for GoPro, Capture NX-D for Nikon, Capture Cam Express for Sony devices) will ask you to specify an output folder or Network Access Storage (NAS) location (ex: a hard disk connected to the wifi), a video & audio codec configuration, and a container file format, like MP4. Congratulations! You’ve now completed the first step and unlocked all the elements required to complete the video transcode.  
After you’ve confirmed the export, your computer might heat up – given the size of your newly created content, it’s not unexpected, your computer will require a lot of temporary storage (in terms of gigabytes) per second of exported video. Depending on your computer’s specifications the video transcode might take more or less time based on your RAM – this works inversely, lower RAM = slower transcodes / higher RAM = faster transcodes. From the consumer perspective – transcoding is as easy as the few clicks it takes to save and export to a new device. In short, transcoding is the process of converting one compressed (almost) losslessly video to a better-compressed video format. This is how video compression works as it moves from your computer to another local device. 
To learn more about Lossless compression (as opposed to lossy compression) and how it works, check out (the completely free) Video Developer Network: Lesson 1.2 What are Lossless Compression Algorithms. 
—-Keep reading to learn how transcoding works on Smart and Over-the-top (OTT) media devices—-

From your device to your computer, now transcoded wirelessly to your SmartTV

So, you’ve exported the video file from your editor and you know you have some friends or family coming over, and you want to show off your final product on your brand new high definition SmartTV. The short answer to this conundrum is to connect your device to the TV with an HDMI cable, but let’s be realistic, do you want to leave your comfortable couch for such an archaic action? Definitely not, we are in 2020 after all! You plan on streaming your video using some of the smart features on your device, but how does that work?
The editing software (like Quick for GoPro) has already created a semi-compressed file on your computer, but chances are that regardless of the server that your software used (GPU-based with plex or an outright transcoding solution), the file needs to be compressed and optimized further for streaming capabilities. This process of compressing multiple times across transitions is imperative to lowering buffering times and improving the user experiences as fewer resources (from the back-end) are required for your TV to decode and stream the content.
If your laptop/NAS has to load and send a very large file across your local network it might not yield stable playback on the client/TV device, because it would have to transmit a lot of data (4K quality for 1 minute is ~400 MB). So the transcoding server can employ a conversion process to HLS or DASH (like DASH for Plex client-server communication or HLS & Smoothstreaming for Google Chromecast) to achieve better quality and more stable table video streaming to your SmartTV (Try it yourself with a hosted Bitmovin player). The software on the TV must support what it receives from the transcoding server (like a Chromecast extension in your browser and the stick in the TV).

Using these extensions or Plex communications, a client device will support a video player that can playback the data received over Wifi. According to our 2020 Video Developer Report (illustrated below) the best video codec by device coverage is the H.264 codec, so that’s always a safe bet for which standard will operate across the most possible devices.

The ultimate transcode – sharing your content with the world!

To stream a video in a living room environment, you’ll need – on top of what we described before –  a transcoding server, which encodes the transcoded video and sends it to a player that has the playback requirements of the device you want to consume the content on. For example, the player must be able to handle different types of streams (like ts or fmp4 for HLS), packaged in different protocols (like DASH or HLS). We are talking about a device that is capable through an app, of playing back the encoded video. When transcoding for streaming it is important to create a video file that meets device client requirements for playback; unfortunately, most client devices do not support every codec or container format as they aren’t supported natively  – ex: any 2011 SmartTV. Converting and compressing (video) files at the local level is a small sample of what transcoding for the highest perceived quality. Achieving stable playback on your Local Area Network (LAN) is much easier, conceptually at least, than it is to transcode a video online – or even live! Maintaining quality and limiting size is even more important for web-based environments as giving way in either category will affect your viewer’s experience; be it lower quality content or long buffer times. Unlike the (semi)straightforward process of tra0nscoding your  content locally, publishing your video online takes a lot more steps on the backend:
Laptop Transcode => Living Room (OTT) Transcode => Internet Transcode (+ Content Delivery Network) => Online!
In short, once you’re device completed the steps to transcode to your Smart TV, it has two (and a half) additional steps:

1. Compressing and converting for internet transport – Uploading to a Content Delivery Network (CDN) – servers at “edge” locations (server locations at the EDGE of different networks)  near users with transcoded copies of your video that will further distribute your video

1. Delivered and played back on some stranger’s device (Yay internet!)

The edge locations allow faster transference and higher quality transference of data so that viewers of your video can see the great content that you are making without buffering and possibly as close to real event as possible (especially if you’re live-streaming).

Author’s Practical Example: End-to-End (E2E) video transcoding for Bitmovin

Now that you have a clear picture in mind of a daily transcoding process in your own home, I’ll shift to an ongoing real-life experience of transcoding video content for our recently published Bitmovin Video Developer Network, the go-to resource for people wanting to learn (more) about video development. The network contains multiple videos with university grade content.
I’m personally tasked with publishing all produced videos online. Our first video, Lesson 1.0 introduction & motivation (5 min runtime), was transcoded using two different methodologies  (to test for efficiency):

1. Standard H.264 profile encode at 1980×1080 pixels, resulting in  5GB(s) of video files 
2. Per-title encode, resulting in 200MB of video files (smaller by a factor 20x)

To ensure that our educational resource is truly universally streamable to users around the world it was important to transcode the videos at qualities and bitrates that all users can consume without buffering, therefore we used Per-Title methodology for all videos moving forward.

Bitmovin’s Video Developer Network

Introduced to the general public as of December 2019, the Bitmovin Developer Network was established with the sole purpose of introducing developers (and other interested parties) to the world of video development. Between our university-grade video lessons (all courses are completely free), global vid tech meet-ups, and learning labs, the developer network is the true home to grow the industry knowledge and capabilities of modern and future video technologies.
Are you interested in joining the Developer Network? All you need to do is sign-up with your email address at the bottom of the Developer Network Homepage. We’ll keep you up-to-date with the latest courses, meetups, and events! 

More video technology guides and articles:

Did you know?

Bitmovin has a range of VOD services that can help you deliver content to your customers effectively.
Its variety of features allows you to create content tailored to your specific audience, without the stress of setting everything up yourself. Built-in analytics also help you make technical decisions to deliver the optimal user experience.
Why not try Bitmovin for Free and see what it can do for you.

The post What is Video Transcoding? The Video Compression Basics appeared first on Bitmovin.

Growth of C++ 

As C++ evolves and improves, our Bitmovers are keen to upgrade our code alongside the powerhouse language. To do so, we look forward to implementing C++20 once compilers reach maturity with related features. As we prepare for the coming year, this blog acts as an opportunity to review all the features that have been added to C++ in 2014 and 2017 (as well as the improvements and uses that Bitmovin has applied)

About C++14

With this evolutionary iteration of C++, lambda functions have been improved considerably.
We can use keyword auto to let the compiler infer the type of input parameters making lambda functions working as templates:

Capturing variables on lambda function also gained a lot of flexibility thanks to generalized lambda capture:

C++ contexpr

It is possible to move more computation to compile time thanks to the fact that it is possible to write constexpr functions made of multiple statements and with conditional blocks:

Finally, it’s worth mentioning that a standard way to deprecate features has been introduced:

Coding with C++17

Variadic templates are much easier to implement with this evolutionary iteration of C++ thanks to folded expressions:

It is possible to have conditional blocks to be evaluated only at compile time as an alternative to function overloads:

Nested namespaces are less verbose:

We can tell the compiler that a returned value should not be ignored by the caller, thereby preventing bugs:

But the feature that we like most at Bitmovin is structured binding:

That makes iterating maps much more expressive:

Very useful classes have been added to the standard library:
Template class std::optional:

A non-mutable reference to a piece of text can be expressed with std::string_view:

Another template class std::variant has been added to implement type-safe unions:

With this evolutionary iteration C++ also gained standard support for file systems:

Conclusion

C++ is a powerful, efficient and wide-spread language, but to reach maximum effectiveness – it’s application requires developers to invest significant time in mastering the language and its diverse set of functions.
Us “Bitmovers” are naturally curious people and are constantly driven to try or learn new things. So it’s a pleasure to discover new hidden features or more elegant coding patterns every week. 
Did you enjoy this post? Then you may find the following blog posts interesting as well!

• Bitmovin Developers in Action: Epic Teams
• Custom Adaptation Logic in your HTML5 Video Player
• Bitmovin C# API Client for .NET – The New Kid on the Block
• Our OpenAPI journey with Standardizing SDKs

The post Celebrating C++: State of the Art Standards for All appeared first on Bitmovin.

Co- Founder Feature: Happy World Standards Day 2019 – Video Standards Create a Global Stage

Today, October 14, 2019, we celebrate the World Standards Day, “the day honors the efforts of the thousands of experts who develop voluntary standards within standards development organizations” (SDOs). Many SDOs such as W3C, IETF, ITU, ISO (incl. JPEG and MPEG) celebrate this with individual statements, highlighting the importance of standards and interoperability in today’s information and communication technology landscape. Interestingly, this year’s topic for the World Standards Day within ISO is about video standards creating a global stage. Similarly, national bodies of ISO provide such statements within their own country, e.g., the A.S.I. statement can be found here (note: in German). I have also blogged about the World Standards Day in 2017.
The numbers for video content created, distributed (incl. delivery, streaming, …), processed, consumed, etc. increases tremendously and, actually, more than 60 percent of today’s world-wide internet traffic is attributed to video streaming. For example, almost 700,000 hours of video are watched on Netflix and 4.5 million videos are viewed on YouTube within a single internet minute in 2019. Videos are typically compressed (or encoded) prior to distribution and are decompressed (or decoded) before rendering on potentially a plethora of heterogeneous devices. Such codecs (portmanteau of coder-decoder) are subject to standardization and with AVC and HEVC (jointly developed by ISO/IEC MPEG and ITU-T VCEG) we have two successful standards which even have been honored with Primetime Engineering Emmy Awards.

Img Source: Sandvine, Sep 10, 2019.

With this vast industry (and growing) looking to adopt standards to drive the future, we celebrate the creators, the innovators, and those looking for global digital solutions! Within Austria, we at Bitmovin drive to help this innovation – most recently earning the award for Living Standards in 2017 for our contribution to the MPEG-DASH standard, which enables dynamic adaptive streaming over HTTP. This standard (4th iteration to launch any day!) is now heavily deployed and has been adopted within massive products and services such as Netflix, Amazon Prime Video, YouTube, etc.
Standardization can be both source for and sink of research activities, i.e., development of efficient algorithms conforming to existing standards or research efforts leading to new standards. One example of such research efforts just recently started at the Institute of Information Technology (ITEC) at Alpen-Adria-Universität Klagenfurt (AAU) as part of the ATHENA (AdapTive Streaming over HTTP and Emerging Networked MultimediA Services) project. The aim of this project is to research and develop novel paradigms, approaches, (prototype) tools and evaluation results for the phases (i) multimedia content provisioning (video coding), (ii) content delivery (video networking), (iii) content consumption (player) in the media delivery chain, and (iv) end-to-end aspects, with a focus on, but not being limited to, HTTP Adaptive Streaming (HAS).
The SDO behind these standards is MPEG (officially ISO/IEC JTC 1/SC 29/WG 11), which has a proven track record of producing very successful standards (not only those mentioned as examples above) and Bitmovin is happy to be part of these efforts resulting into innovative products and services
View original post here
For other great reads about the state of the industry and video standards, check out the following links:

• 2019 Video Developer Report [E-Book]
• Multi-Codec DASH Dataset: An Evaluation of AV1, AVC, HEVC and VP9 [Blog Post]
• 2019 Bitmovin Magazine – August Edition [Magazine]

Don’t forget to sign-up for our weekly Newsletter to stay up-to-date on our latest blogs, events, or other downloadable content!

The post Happy World Standards Day 2019 from Everyone at Bitmovin! appeared first on Bitmovin.

At WWDC17 Apple presented updates and new features that will be available to their products soon including HEVC in HLS

This year Apple announced support for HEVC/H.265 for macOS High Sierra and iOS11. The release of iOS 11 is expected to happen soon after Apple’s iPhone 8 launch event on September 12th. Considering the fact that iOS users usually upgrade their system pretty fast, HEVC playback will soon be available on hundreds of million devices. This is evident, as only five months after iOS 10 was released to the public, it was installed on about 80% of active iOS devices. In January 2016 Apple announced it had over 1 billion active iOS devices. There was no updated number after that, however, when considering Apple is selling tens of millions of iOS devices each quarter, it’s reasonable to assume the total is above 1 billion by now.
That are numbers too large to be ignored considering the fact that HEVC/H.265 can reach up to 50% bitrate savings compared to AVC/H.264 according to Netflix’s experiments. This will allow to stream better quality to customers and saves storage and CDN costs for content providers.
The following image shows a quality comparison between HEVC/H.265 and AVC/H.264 encoded at 1000kbps. The quality improvement is clearly visible as there are multiple encoding artifacts visible in the AVC/H.264 version. Also if you look at the details of the coat or the lines in the background one can clearly see how HEVC/H.265 manages to preserve more details.

Multi Codec Streaming

With Bitmovin you can encode content with different codecs like AVC/H.264, HEVC/H.265, VP9, and recently also AV1. This allows to use the best codec for the platform when streaming content to your users. HEVC/H.265 and VP9 are more efficient than AVC/H.264 allowing to deliver higher quality with the same bitrate, or save costs by delivering similar quality to less bandwidth. VP9 is supported on Google Chrome, Firefox and Android devices which allows you to stream VP9 to about 70% of your users. For Safari there was still the need to use AVC/H.264 until now. Having the fast adoption rates for new iOS versions in mind, it is expected for iOS 11 to be installed on the vast majority of iOS devices soon, allowing to save bandwidth, storage and CDN costs by utilizing its HEVC/H.265 support.
Generating HEVC/H.265 content with fMP4 segments works out-of-the-box for HLS with Bitmovin as we do that today for HEVC MPEG-DASH content already. The trick to make it available as an HLS asset is just to reference the segments in the HLS playlist files in the same way as we do it today with fMP4 in HLS.

Playback of HLS with HEVC

With the Bitmovin Player you can deliver HEVC based HLS streams to a wide range of supported devices, enabling you to provide a better video quality to your customers, or to save storage and CDN costs by delivering the same video quality at significantly lower bitrates.
No matter if you build native iOS applications using our iOS SDK, or if you want to enable HLS HEVC playback in the browser, the Bitmovin Player provides you broad device and platform coverage for HLS content encoded with HEVC. Besides playback of HEVC based HLS streams in Safari on macOS High Sierra and iOS, the web-based Bitmovin Player is also able to play those streams on any browser which comes with built in HEVC support like the Microsoft Edge Browser.
If you have iOS 11 already installed on your device, you can test any HLS HEVC stream in our iOS SDK demo application available in the App Store. With macOS High Sierra or iOS 11 installed, you can also test your streams in Safari using our online demo section.

Test Vectors

There are a few ways to deliver HEVC content to users:

1. HEVC in HLS using MPEG-2 Transport Stream chunks, which Apple doesn’t support,
2. HEVC in HLS using fMP4 segments, which is what Apple announced on WWDC17 and our player supports,
3. HEVC in MPEG-DASH using fMP4 segments.

All of these options can already be created using the Bitmovin Encoder. For playback, it depends on the HEVC support in the browser. Obviously, Apple added this for Safari on macOS 11 and High Sierra, but also Edge on Windows 10 already supports it. HEVC can be streamed using HLS or MPEG-DASH to Edge with the Bitmovin Player.
We provide test vectors for the three above mentioned types for public testing:

• HEVC HLS with fMP4
• HEVC MPEG-DASH

Beside HEVC/H.265 we can also encode the same asset to AVC/H.264 and VP9 which we introduced earlier this year. With this configuration you can deliver the best codec to every device, improving quality and save costs. VP9 is supported on multiple platforms including Google Chrome, Firefox and Android devices and thus by about 70% of your users. Take in consideration that with VP9 you can save up to 50% on bitrate delivering similar quality you could save about 35% on CDN costs.
Here are some more test vectors including the VP9 codec:

• Multi-Codec (AVC/H.264, HEVC/H.265, VP9)
• VP9

Demo for HEVC HLS:

The post How the iOS 11 Release with HLS/HEVC Support will Help to Save Storage & CDN Costs appeared first on Bitmovin.

By using a single package format you can reduce your encoding, packaging and storage costs by halve and decrease your CDN costs by up to 10% as fMP4 has less overhead than MPEG-TS

At this years Worldwide Developer Conference (WWDC 2016), Apple introduced fragmented MP4 (fMP4) for HLS. Although this announcement was not such a big deal for Apple, the impact on the rest of the media industry is huge. In this blog post I will try to explain why, and also address some of the frequently asked questions around the topic of fMP4 for HLS.

Why is this such a big change? After all, it’s just a container format. It’s still HLS.

Very true, but this new container format halves encoding/packaging and storage costs. In the past you were required to multiplex each rendition/bitrate/resolution into two containers, MPEG2 Transport Stream (TS) for HLS and fMP4 for DASH, or maintain “just in time” packagers that do not scale well and cost you money on every request. Now that HLS supports fMP4 it can share the same encoded segments as the DASH manifest. On top of that, there is now also the potential for a major reduction in CDN costs for some businesses. TS is less efficient than DASH, with up to 10% more overhead. This means that CDN costs for HLS content can be reduced by up to 10% in certain cases.

Why do we need two formats, HLS and DASH, isn’t one enough?

Technically speaking it should be, and everybody would prefer that. Only having one format would make all of our lives much easier. Unfortunately, due to the proprietary nature of the Apple infrastructure, HLS is required on Safari, iOS and tvOS while on the other hand you need DASH to get native HTML5 playback on all other browsers.

Isn’t it possible to playback HLS in HTML5 on all browsers?

It is – and we can do this too – but for high resolutions and bitrates (just think about 360° videos in 4k and higher resolutions) it’s obviously not as effective (you need to remultiplex every chunk in javascript) and this costs performance, battery and latency, which makes such videos unplayable on non state of the art devices.

Which devices support HLS with fMP4?

It’s supported on iOS10, macOS and tvOS. Considering that Apple users are adopting new versions of iOS quite quickly – trends show that 80% of all Apple iOS users are already using iOS10 (https://mixpanel.com/trends/#report/ios_10), it’s already a large user base that could potentially benefit from HLS with fMP4. Same applies for OSX where HLS with fMP4 is available on Safari 10 which has by far the largest market share compared with other Safari desktop versions (https://www.stetic.com/market-share/browser/).

What does this mean for SVOD, DRM use cases?

For SVOD and DRM use cases the situation is a little bit more complicated. HLS with fMP4 as well as DASH support MPEG Common Encryption (MPEG-CENC). MPEG-CENC supports two major encryption modes, AES-CTR (Counter Mode) and AES-CBC (Cipher-Block Chaining) which are incompatible. Fairplay with fMP4 HLS uses AES-CBC, while PlayReady and Widevine with fMP4 DASH are using AES-CTR. This makes it currently impossible to use a single content encoding for all DRM systems – for the moment. I think that this will change in the future and Widevine has already added AES-CBC support on Chromecast and Android N devices (https://www.widevine.com/product_news.html). When Widevine continues to broaden their AES-CBC support and Playready follows it will be possible to use single encrypted encoding for all platforms.

• Streaming – DASH and HLS manifests are compatible since the beginning because HLS could be seen as a subset of the DASH standard.
• Container Format – This was a problem in the past as HLS just supported MPEG-TS segments and HLS is required on iOS. DASH on the other hand is container format agnostic and due to the fact that all recent browsers only support fMP4 natively, DASH was used with fMP4 mainly and therefore everybody was required to generate and store both formats. With the recent changes it’s possible to use HLS and DASH with the same segments which reduces your encoding/packaging efforts and storage footprint by half.
• Encryption Mode – This is only needed for SVOD/DRM use cases but here we have still an incompatibility that needs to be resolved. As FairPlay uses CBC and Widevine and PlayReady mainly use AES-CTR this is a problem and you are still required to generate two versions of your segments – one for Fairplay and another one for Widevine and PlayReady. Nevertheless, as described Widevine has just recently announced that they now also support AES-CBC for Chromecast and Android N devices (https://www.widevine.com/product_news.html). If Widevine and PlayReady continue to broaden that support we will have single format.
• Codec – On the codec layer, both, H264 as well as H265 can be multiplexed in fMP4 and if the browser or device supports it the player can playback the content.

So I think we all agree now that HLS with fMP4 is something that is pretty useful. Content encoding, packaging and playback solutions are still very rare on the market for this workflow but Bitmovin already offers a complete end-to-end workflow through our Bitmovin API that allows you to encode and playback HLS with fMP4.

HLS fMP4 from Encoding to Playback

Let’s start with the actual encoding of your content. Our new API can be utilized easily, with the PHP, Python, and Go API clients, which are already available. We will use the PHP API Client and its example to show you how to create fMP4 HLS content.
1. Get the Bitmovin PHP API Client
You can either download it from Github or install it using composer. Please see the API client’s repository for more information about the setup.
2. Initialize the Bitmovin API Client
In order to use the API client, we have to initialize it first.

$client = new \Bitmovin\BitmovinClient('INSERT YOUR API KEY HERE');

That’s it^^. The client is now ready to use. Now, we can start preparing the configurations for your input source, output destination, and encoding, containing all the renditions, you want to create for the fMP4 HLS content.
3. Create a input configuration
For the sake of simplicity we are using an HTTP(S) input, although many other input sources such as AWS S3, Google Cloud Storage, Microsoft Azure, Aspera, and (S)FTP are also supported.

$videoUrl = 'http://example.com/path/to/your/movie.mp4';
$input = new HttpInput($videoUrl);

4. Create an output configuration
Here you define either to directly transfer your encoding results to your preferred storage type (
AWS S3, Google Cloud Storage, Microsoft Azure, (S)FTP), or you can store the encoding on your own Bitmovin storage. Direct transfer as well as storage are features of our new Bitmovin API. The first allows you to keep the turnaround time of your encoding very low, so your encoded content becomes available as quickly as possible on your own storage. The latter enables you to keep a backup of your encoding, transfer them later on to another storage and so on. If you want to do both, you can do that as well now :). As AWS S3 is a very common cloud storage, we will use it for this example.

$s3AccessKey = 'INSERT YOUR S3 ACCESS KEY HERE';
$s3SecretKey = 'INSERT YOUR S3 SECRET KEY HERE';
$s3BucketName = 'INSERT YOUR S3 BUCKET NAME HERE';
$s3Prefix = 'path/to/your/output/destination/';
$s3Output = new S3Output($s3AccessKey, $s3SecretKey, $s3BucketName, $s3Prefix);

5. Create an encoding profile configuration
An encoding profile configuration contains all the encoding related configurations for video/audio renditions as well as the encoding environment itself. It’s now possible to select which region and cloud provider should be used to encode your content. This enables you, to locate the encoding infrastructure, where your input and/or output bucket is located. This could improve your download and upload speeds and keeps your costs for egress traffic low.
Create an encoding profile configuration

$encodingProfileConfig = new EncodingProfileConfig();
$encodingProfileConfig->name = 'Test Encoding FMP4';
$encodingProfileConfig->cloudRegion = CloudRegion::AWS_EU_WEST_1;

Create an video quality configuration

$videoConfig = new H264VideoStreamConfig();
$videoConfig->input = $input;
$videoConfig->width = 1920;
$videoConfig->height = 1080;
$videoConfig->bitrate = 4800000;
$encodingProfileConfig->videoStreamConfigs[] = $videoConfig;

Create an audio quality configuration

$audioConfig = new AudioStreamConfig();
$audioConfig->input = $input;
$audioConfig->position = 1;
$audioConfig->bitrate = 128000;
$audioConfig->name = 'English';
$audioConfig->lang = 'en';
$encodingProfileConfig->audioStreamConfigs[] = $audioConfig;

You might have noticed, that you need to provide an input for each Audio/Video stream configuration. This is another feature of our new API. Now, you can provide several input files for an encoding, create all the renditions you need, and use them in your manifest afterwards. One typical use-case would be, if you have separate files for your video and audio tracks.
6. Select Output Formats
Besides HLS and DASH, we do support Smooth Streaming, MP4, TS and of course HLS fMP4 by now as well. We will only use “HLS fMP4” in this example, but you could create all of them at once with one encoding attempt too.

$outputFormats = array();
$outputFormats[] = new HlsFmp4OutputFormat();

7. Create Encoding configuration
This configuration object acts as a container for all the previous configurations from above and will be passed to the BitmovinClient in order to start the encoding.

$jobConfig = new JobConfig();
$jobConfig->output = $s3Output;
$jobConfig->encodingProfile = $encodingProfileConfig;
$jobConfig->outputFormat = $outputFormats;

8. Start the encoding
Finally, we can start the encoding. runJobAndWaitForCompletion() will return as soon as the encoding is finished and transferred/stored successfully.

$client->runJobAndWaitForCompletion($jobConfig);

By reading this line, the encoding might be finished already 🙂 If not, we can use the spare time to quickly setup a Bitmovin player example to playback the created content once it is finished.

Playback

Now that we successfully created our HLS fMP4 content, we want to play it as well. This is as simple as it is for HLS TS content, because it works exactly in the same way. The minimum player configuration would look like the following:

{
   key: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
   source: {
       hls: "https://example.com/path/to/your/fmp4-hls-content-master-playlist.m3u8"
   }
}

A full example page would like the following:

<!DOCTYPE html>
<html>
<head>
   <meta charset="utf-8">
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
   <title>V6 fMP4 HLS</title>
   <script src="https://bitmovin-a.akamaihd.net/bitmovin-player/stable/6/bitmovinplayer.min.js"></script>
</head>
<body>
<div id="unique-player-id"></div>
<script type="text/javascript">
   var player = bitmovin.player("unique-player-id");
   var conf = {
      key: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      source: {
          hls: "https://example.com/path/to/your/fmp4-hls-content-master-playlist.m3u8"
      }
   };
   player.setup(conf).then(function (value) {
       console.log("Successfully created bitmovin player instance");
   }, function (reason) {
       console.log("Error while creating bitmovin player instance");
   });
</script>
</body>
</html>

With this configuration for our player, and the encoded content from our encoding-service, you can now play your HLS fMP4 content everywhere, with one SINGLE format.

HLS fMP4 Demo

Now, we have HLS fMP4 content as well as a player example, which is ready to be used. Before we give that a try, let’s have a look what actually changed compared to a conventional HLS playlist.
As fragmented MP4 requires an initialization segment, which you might already know from MPEG-DASH, it has to be referenced using the EXT-X-MAP tag (see picture below). Then the player is able to play the fragmented MP4 segments properly. Therefore, we had to add the EXT-X-MAP tag in every variant playlist and use EXT-X-VERSION with value “6” or higher.

 

Does it play everywhere?

Below you can see the Bitmovin HTML5 Adaptive Streaming Player in action, playing Fragmented MP4 through an HLS manifest on every browser. This demo will work on iOS 10, macOS, tvOS and all recent browser, including Edge, FireFox, Chrome and Safari, etc.

How much storage space can be saved?

Based on the video of that blog post we created a quick storage and CDN savings case. In the past it was necessary to encode DASH and HLS TS to playback this video on all browsers and platforms. Now it’s possible to use just HLS fMP4 for everything. The table below shows the output in Bytes for each format and the combination of DASH and HLS TS. In terms of encoding output, packaging and storage you would save here 50.89% as you just need the HLS fMP4 output which is more than half of the actual DASH and HLS TS output as the HLS TS output is bigger than the DASH output due to the fact that TS is less efficient than fMP4.
HLS fMP4HLS TSDASH + HLS TSSavings

This advantage helps us also on the CDN as shown in the second row. HLS fMP4 is more efficient than HLS TS and for every user that you serve with content you would need less bits to deliver the same quality. In that example you would immediately save 3.49% on your CDN costs. This could be much more on other services but our TS output is already optimized for low overhead.

Conclusion

All in all, HLS with fMP4 is already very useful, as long as you don’t have to deal with DRM protected content. It enables you to greatly improve your storage consumption, which reduces your overall storage and CDN costs, and it enables you to use a more efficient output format across all devices. If you want to give it a try you could use our new Bitmovin API, just a request an API key for free and try it out.

The post Halve your Encoding, Packaging and Storage Costs – HLS with fragmented MP4 appeared first on Bitmovin.

Apple HLS File Formats

Apple HTTP Live Streaming (Apple HLS) uses M3U8 playlists as a manifest file for their content files, which is an extension of the M3U format used for MP3 playlists. HLS is the only system that uses MPEG-2 Transport Stream (MP2TS) which adds a significant overhead in relation to the audio/video data, instead of MP4 or another ISO Base Media File Format based container. MPEG-2 TS consists of packets with 188 bytes in size, where each packet has headers with a varying size of 4 to 12 bytes. Therefore, the overhead caused by these headers increases proportionally with the segment size, which means that relative overhead does not tend to zero with increasing bitrates. Moreover, each MPEG-2 TS stream consists of several other data beside audio and video, such as Program Association Tables that introduce additional overhead. Additionally, audio and video streams are encapsulated in Packetized Elementary Streams (PES) which introduces an extra overhead per audio sample or video frame.

Manifest – M3U8

Apple HTTP Live Streaming (HLS) uses an M3U8 playlist as its manifest, which is a little bit different to the other approaches, e.g., MPEG-DASH or Microsoft Smooth Streaming that use an XML based manifest. VOD content with one quality can be described through the basic playlist feature, which means that the individual segments URLs are available in the main M3U8. If you want to offer multiple qualities as intended with adaptive multimedia streaming, you will have to use the variant playlist. Typically, a variant of a stream is the quality of the stream in a specific bitrate and/or resolution. Variant playlists are structured in the following in that there is one root M3U8 that references other M3U8s that describe the individual variants (qualities). The root M3U8 will then contain several EXT-X-STREAM-INF tags that indicate that the following URL is referencing another M3U8 playlist. Additionally, the EXT-X-STREAM-INF tag could contain several attributes such as bandwidth, codec, etc. Advertisements are supported in HLS through the EXT-X-DISCONTINUITY tag. This tag indicates a discontinuity in the media stream between the actual media segment and the one that follows. Furthermore, a basic encryption method is also available that allows AES-128 encryption of the media segments. The key server can be signaled in the M3U8 with the EXT-X-KEY tag and the URI attribute.

Generate your own Apple HTTP Live Streaming (Apple HLS) Content

The get started link below offers you a free trial with Bitmovin. With our Cloud Encoding Service and HTML5 Adaptive Streaming Player, you can get up and running with your first Apple HLS in a matter of minutes. Try it now!
Further Readings:

• Video Developer Report 2019
• Cloud Encoding Datasheet
• Video Player Datasheet
• Video Analytics Datasheet

The post Apple HTTP Live Streaming appeared first on Bitmovin.