Real Words or Buzzwords?: H.264 and I-frames, P-frames and B‑frames – Part 3 - RBCS

This is the 51st article in the award-winning “Real Words or Buzzwords?” series about how real words become empty words and stifle technology progress, also published on SecurityInfoWatch.com.

By Ray Bernard, PSP, CHS-III

Once seen as just a marketing term, ‘smart codecs’ have revolutionized video compression.

All-in-one RWOB

#1 Proof of the buzzword that killed tech advances in the security industry—but not other industries.

#2 Next Generation (NextGen): A sure way to tell hype from reality.

#3 Customer Centric: Why all security industry companies aren't customer centric.

#4 Best of Breed: What it should mean to companies and their customers.

#5 Open: An openness scale to rate platforms and systems

#6 Network-friendly: It's much more than network connectivity.

#7 Mobile first: Not what it sounds like.

#8 Enterprise Class (Part One): To qualify as Enterprise Class system today is world's beyond what it was yesterday.

#9 Enterprise Class (Part Two): Enterprise Class must be more than just a top-level label.

#10 Enterprise Class (Part Three): Enterprise Class must be 21st century technology.

#11 Intuitive: It’s about time that we had a real-world testable definition for “intuitive”.

#12 State of the Art: A perspective for right-setting our own thinking about technologies.

#13 True Cloud (Part One): Fully evaluating cloud product offerings.

#14 True Cloud (Part Two): Examining the characteristics of 'native-cloud' applications.

#15 True Cloud (Part Three): Due diligence in testing cloud systems.

#16 IP-based, IP-enabled, IP-capable, or IP-connectable?: A perspective for right-setting our own thinking about technologies.

#17 Five Nines: Many people equate high availability with good user experience, yet many more factors are critically important.

#18 Robust: Words like “robust” must be followed by design specifics to be meaningful.

#19 Serverless Computing – Part 1: Why "serverless computing" is critical for some cloud offerings.

#20 Serverless Computing – Part 2: Why full virtualization is the future of cloud computing.

#21 Situational Awareness – Part 1: What products provide situational awareness?

#22 Situational Awareness – Part 2: Why system designs are incomplete without situational awareness?

#23 Situational Awareness – Part 3: How mobile devices change the situational awareness landscape?

#24 Situational Awareness – Part 4: Why situational awareness is a must for security system maintenance and acceptable uptime.

#25 Situational Awareness – Part 5: We are now entering the era of smart buildings and facilities. We must design integrated security systems that are much smarter than those we have designed in the past.

#26 Situational Awareness – Part 6: Developing modern day situational awareness solutions requires moving beyond 20th century thinking.

#27 Situational Awareness – Part 7: Modern day incident response deserves the help that modern technology can provide but doesn’t yet. Filling this void is one of the great security industry opportunities of our time.

#28 Unicity: Security solutions providers can spur innovation by envisioning how the Unicity concept can extend and strengthen physical access into real-time presence management.

#29 The API Economy: Why The API Economy will have a significant impact on the physical security industry moving forward.

#31 The Built Environment: In the 21st century, “the built environment” means so much more than it did just two decades ago.

#32 Hyper-Converged Infrastructure: Hyper-Converged Infrastructure has been a hot phrase in IT for several years, but do its promises hold true for the physical security industry?

#33 Software-Defined: Cloud-computing technology, with its many software-defined elements, is bringing self-scaling real-time performance capabilities to physical security system technology.

#34 High-Performance: How the right use of "high-performance" can accelerate the adoption of truly high-performing emerging technologies.

#35 Erasure Coding: Why RAID drive arrays don’t work anymore for video storage, and why Erasure Coding does.

#36 Presence Control: Anyone responsible for access control management or smart building experience must understand and apply presence control.

#37 Internet+: The Internet has evolved into much more than the information superhighway it was originally conceived to be.

#38 Digital Twin: Though few in physical security are familiar with the concept, it holds enormous potential for the industry.

#39 Fog Computing: Though commonly misunderstood, the concept of fog computing has become critically important to physical security systems.

#40 Scale - Part 1: Although many security-industry thought leaders have advocated that we should be “learning from IT,” there is still insufficient emphasis on learning about IT practices, especially for large-scale deployments.

#41 Scale - Part 2: Why the industry has yet to fully grasp what the ‘Internet of Things’ means for scaling physical security devices and systems.

#42 Cyberspace - Part 1: Thought to be an outdated term by some, understanding ‘Cyberspace’ and how it differs from ‘Cyber’ is paramount for security practitioners.

#43 Cyber-Physical Systems - Part 1: We must understand what it means that electronic physical security systems are cyber-physical systems.

#44 Cyberspace - Part 2: Thought to be an outdated term by some, understanding ‘Cyberspace’ and how it differs from ‘Cyber’ is paramount for security practitioners.

#45 Artificial Intelligence, Machine Learning and Deep Learning: Examining the differences in these technologies and their respective benefits for the security industry.

#46 VDI – Virtual Desktop Infrastructure: At first glance, VDI doesn’t seem to have much application to a SOC deployment. But a closer look reveals why it is actually of critical importance.

#47 Hybrid Cloud: The definition of hybrid cloud has evolved, and it’s important to understand the implications for physical security system deployments.

#48 Legacy: How you define ‘legacy technology’ may determine whether you get to update or replace critical systems.

#49 H.264 - Part 1: Examining the terms involved in camera stream configuration settings and why they are important.

#50 H.264 - Part 2: A look at the different H.264 video frame types and how they relate to intended uses of video.

#51 H.264 - Part 3: Once seen as just a marketing term, ‘smart codecs’ have revolutionized video compression.

#52 Presence Technologies: The proliferation of IoT sensors and devices, plus the current impacts of the COVID-19 pandemic, have elevated the capabilities and the importance of presence technologies.

#53 Anonymization, Encryption and Governance: The exponential advance of information technologies requires an exponential advance in the application of data protection.

#54 Computer Vision: Why a good understanding of the computer vision concept is important for evaluating today’s security video analytics products.

#55 Exponential Technology Advancement: The next 10 years of security technology will bring more change than in the entire history of the industry to now.

#56 IoT and IoT Native: The next 10 years of security technology will bring more change than in the entire history of the industry to now.

#57 Cloud Native IoT: The next 10 years of security technology will bring more change than in the entire history of the industry to now.

#58 Bluetooth vs. Bluetooth LE: The next 10 years of security technology will bring more change than in the entire history of the industry to now.

More to come about every other week.

This article continues the discussion about the H.264/H.265 MPEG video standards, including the manufacturer innovations in smart codecs (once a marketing buzzword but now a very meaningful term). The reasons for discussing this complex video topic are presented in the Part 1 and Part 2 articles.

Those reasons include that video compression impacts video system cost and performance. Video compression reduces the burden on video networks and lowers data storage requirements. At the same time, it increases the data processing requirements – computer memory size and processing power.

Additionally, techniques such as motion-based recording can create problems of their own, such as spiking the processing and storage requirements by five times or more when rains, snow or lots of activity cause recording to be continuous on all outdoor cameras. On the other hand, the right AI-based video analytics can detect the difference between meaningless background changes like snow or rain and can eliminate the recording-spike phenomenon. Of course, running those kinds of analytics also increases the computer video processing power requirements.

There is some good financial news when you consider the exponential advancement of computing hardware and software technology and the two technology megatrends that result from it: decreasing technology costs and increasing technology capabilities. This means that shorter computing hardware lifecycles are appropriate for premises-based computing and storage so that you can stay ahead of the general need to increase the use of video and video analytics, and not run out of computing and storage capacity – potentially compromising security operations and investigations effectiveness.

Furthermore, utilizing a true cloud-based video VMS service and cloud-based analytics means having instantly-expanding compute and storage resources can give you a video technology infrastructure that expands and contracts with your actual usage on a minute-to-minute basis.

Evidence of the cost and capabilities megatrends having an impact is the move that Eagle Eye Networks made in early 2019 to cut its subscription prices by up to 45%. Smart cities are finding that a well-designed cloud-based approach to video can reduce the costs and efforts of video deployments for security and city operations by more than half, as Mexico City recently learned.

Now, back to our topic: smart codecs.

Today’s Codecs

As explained in the previous article, the first generation of video codec technology (codec being short for coding/decoding) dealt with improvements in compressing each video image. But they still sent the visually-same image over and over again when the scene the camera was viewing didn’t change. The next generation of codecs began sending partial images – only the parts that changed – in between the full images. This approach can reduce video storage dramatically, sometimes more than 80%.

Today’s video compression standards are very flexible due to the wide range of devices and software that video streams to across a wide variety of devices and networks, including the Internet. Thus, today’s security video technology benefits from the many advances made in video processing for movies, television and Internet video streaming. For example, the H. 264 video standard is at version 26 as of June 2019.

The result is that security camera makers can develop smart codecs that optimize video compression to minimize bandwidth and storage requirements while optimizing video quality based on the content and purpose of the video. This is done using the approaches specified in the H.264 and H.265 standards.

Smart Codecs

Originally, codecs set one compression level, one frame rate, and one I-frame interval (defining a group of pictures [GOP] that start with an initial full-image frame followed by a number of image-change frames, as explained in the Part 2 article) that all remained constant for a stream of video. Today’s smart codecs can change any or all of these values depending upon what’s happening in the video scene. Thus, smart codecs now can have:

Dynamic Compression. Continuously identify the regions of interest (ROI), such as image areas with people and vehicles, and maintain high quality/low compression for those image regions in each frame, applying low quality/high compression for other regions.
Dynamic I-frames. Increase the number of full-image frames (I-frames) in a video stream when there is activity, which reduces the compression artifacts (image distortion) for the frames in between the I-frames .
Dynamic FPS. Increase the number of frames sent per second, increasing the smoothness of moving people and objects, and assuring that important actions (such as a shoplifter hiding an item) aren’t missed.

A smart codec with dynamic FPS adjustment allows you to have fewer frames per second when there is no activity, providing the greatest savings in video storage without compromising the video quality of scene activity.

It is important to get the specifics about what any particular smart codec does, and to make sure the codec’s performance is appropriate for the type of scene and the activity in it. Dynamic FPS is important, for example, for street intersections, retail stores, airport checkpoints, transportation terminals, stairwells, doorways and hallways.

For example, Axis Communications created Zipstream and Avigilon created HDSM SmartCodec (HDSM stands for High Definition Stream Management). The Axis and Avigilon smart codecs adjust all three smart codec capabilities listed above. Most camera codecs address only the first two.

It’s also important to discover what specific smart codes settings you should use for a particular camera’s scene, given that the manufacturer’s default settings won’t be suitable for all camera scenes and activities.

Video Processing Requirements

Keep in mind that the better the video compression, the greater the computing power required to decompress the video. While smart codecs help reduce network bandwidth utilization and video storage requirements, they increase the video processing and memory requirements of video servers and sometimes of the devices for video viewing. Add video analytics into the mix, and it is likely time to give serious thought to what your future roadmap should look like for security video technology.

Ray Bernard, PSP CHS-III, is the principal consultant for Ray Bernard Consulting Services (RBCS), a firm that provides security consulting services for public and private facilities (www.go-rbcs.com). He is the author of the Elsevier book Security Technology Convergence Insights available on Amazon. Mr. Bernard is a Subject Matter Expert Faculty of the Security Executive Council (SEC) and an active member of ASIS International.