Trends in R&D on Optical and IP Transmission Technology for Program Production

2.1 Broadcast media progress and changes in device interfaces

Japanese TV broadcasting began in 1953, with black-and-white programming followed by color broadcasting, Hi-Vision (high-definition television; HDTV)^*3, and then the launch of the 4K/8K satellite broadcasting service in December 2018. The evolution of broadcast media has been accompanied by the development of new electrical interfaces for connecting the video devices used in program production. The changes in the main broadcasting media and device interface since the introduction of Hi-Vision are shown in Fig. 1.

In the 1990s, individual coaxial cables were generally used to carry high-definition analog component signals (Y, Pb, Pr)^*4 between devices. From the mid-1990s, however, there was steady progress in the standardization of digital transmission systems. In 1998, a serial transmission method known as High-Definition Serial Digital Interface (HD-SDI) for uncompressed high-definition digital signals (about 1.5 Gbps) compatible with 1080/60I^*5 was standardized by the Society of Motion Picture and Television Engineers (SMPTE) ^*6 as 292M^{2) *7}. That brought the digital interface into the mainstream.

Since 2000, R&D on 4K/8K Ultra-High-Definition Television (UHDTV) has been vigorous, and HD-SDI was initially adopted for the device interface. In 2006, the use of multiple 3G-SDI, which was standardized as SMPTE ST 424³⁾ for 1080/60P^*8, was adopted for 4K/8K. Currently, 12G-SDI, standardized as SMPTE ST 2082-1⁴⁾ in 2015, is mainstream. The standard methods are used as an interface for connecting devices with a transmission distance of about 100 m. Long-distance video transmission thus now requires optical fiber transmission technology.

Because the device interface described above (SDI) is not compatible with the 120 Hz frame frequency standard of 8K, a new interface was developed and standardized⁵⁾.

2.2 Transitions in optical fiber communication technology

(1) Backbone network^{6) 7)}

The interest in fiber optic communications increased significantly in 1970, when the potential for practical use was demonstrated by the announcement of an optical fiber with a transmission loss of 20 dB/km by Corning and the development of a semiconductor laser that operates continuously at room temperature by Bell Laboratories. In the 10 years that followed, there was a transition from the level of basic research to the possibility of practical application through achievements such as reduction of the transmission loss from 20 to 0.2 dB/km. In Japan, the Nippon Telegraph and Telephone Public Corporation (currently, NTT) began the commercial use of optical fiber transmission in 1981.

Subsequent progress in optical fiber communication technology for telecom carrier backbone networks is shown in Fig. 2. Telecom signals are mainly for voice, which is low in bandwidth and easily digitized, and digital data from computers, etc. Digital transmission has been used from the early days of optical fiber transmission. Even when the terminal equipment at both ends of the transmission line is expensive, as typified by submarine cables, fibers are sufficiently investment-effective when the capacity per fiber can be increased or the repeater distance can be extended. For that reason, vigorous R&D continued and progress has been rapid Fig. 2.

The initial increase in transmission capacity per fiber came from improvements in time division multiplexing (TDM) based on advances in optical devices. In the 1990s, fiber amplifiers came into practical use, and capacity was subsequently increased with wavelength division multiplexing (WDM) technology. In the 2010s, a further increase in capacity was achieved with multilevel modulation technology^*9 using digital signal processing and other such means. Various optical devices such as semiconductor lasers and optical fiber connectors^*10 that were developed at that time are also being used in LAN systems.

Concerning the type of signals handled by telecom backbone networks, the demand for voice communication dominated the demand for data communication until 2000 or so, but since then, data communication demand has increased and voice traffic has decreased. As a result, packet transmission has become the more efficient approach for voice as well, and a transition to a closed IP network^*11 referred to as Next-Generation Network (NGN), which began in 2008, continues to move forward. The same transition is also occurring in the access network^*12 as well as the backbone network. For example, the Nippon Telegraph and Telephone Corporation (NTT) plans to complete the transition to full-IP NGN by 2025.

(2) LANs

Local area networks (LANs) are used to connect devices within small areas such as ordinary homes, offices, laboratories, and factories. In the beginning, they were implemented with various types of technology, but Ethernet⁸⁾ became the mainstream after standardization of Fast Ethernet in 1995. Fast Ethernet provided a speed of 100 Mbps with a simple cost-effective mechanism^*13. The optical fiber tranmission for 100 Mbps Ethernet has also been standardized, and was commonly used in backbone networks for large buildings^*14. Since the advent of gigabit-per-second Ethernet, the optical fiber cable has taken the lead over the unshielded twisted pair (UTP) Ethernet cable in terms of standardization and widespread use. Ethernet speeds have continued to increase and the standard is now up to 400 Gbps. The data rates for the SDI used in video signal transmission and Ethernet are compared in Fig. 3. SDI includes coaxial cable transmission, and the achievement of higher speeds is considered difficult, so LAN technology is also being investigated as an interface for 4K and 8K video signals. Ethernet switches are increasing in capacity (the number of ports) and decreasing in price^*15; thus, interest in using them for audio and video signal switching (routing) is increasing.

3.1 Optical technology

Optical transmission technology was introduced to program production in response to the demand for long-distance transmission, which is problematic with a coaxial cable, and the need to increase capacity by transmitting multiple signals over a single cable. The use of optical fiber transmission for a live golf event in Japan in 1978 is an early example⁹⁾. At that time, digital transmission of video signals was difficult because video signals have a much larger bandwidth than audio signals, so video continued to be transmitted as standard TV composite signals^*16 or HDTV component signals (Y, Pb, and Pr) until the mid-1980s. The respective signals were generated by modulating the emission intensity of a light-emitting diode (LED) or semiconductor laser diode (LD) and transmitted over individual multimode fibers up to a distance of several kilometers^*17.

Our research on optical fiber transmission of program material began in earnest in the 1980s, as practical high-definition broadcasting had gained traction, and FM multiplexed optical transmission technology was developed with the objective of long-distance multichannel transmission, which was not previously achieved¹⁰⁾. In that technique, the carrier waves are frequency-modulated (FM) by the Y, Pb, and Pr component signals of the Hi-Vision material, respectively. The frequency-multiplexed signal is then used to modulate the light intensity. The technique was applied in a live transmission device for experimental Hi-Vision broadcasting from the 1988 Seoul Olympics. In the same year, in-station transmission equipment for standard TV implemented with the technique was introduced in the network operation center of the NHK Broadcasting Center.

In response to progress in the digitization of Hi-Vision program production in the early 1990s, technology was developed for transmitting uncompressed digital Hi-Vision signals at high speed (about 1.5 Gbps at that time) over long distances using optical fiber cables. For the efficient use of frequencies and clock recovery, a paired selected ternary (PST) code was adopted^*18 and a PST codec chip was developed to implement a practical digital transmitter for Hi-Vision¹¹⁾. That device was used for broadcasting the 1992 Barcelona Olympics and on many other occasions.

In the mid-1990s, practical technology for transmitting an optical signal modulated with serial data from an uncompressed digital Hi-Vision signal (about 1.5 Gbps) became available. We began research on high-density wavelength division multiplexing for optical transmission that is capable of distributing various types of high-data-rate signals such as Hi-Vision and standard television, and in 1995, feasible technology for multiplexing 32 optical signals at wavelength intervals of 1 nm was achieved. In cooperation with the NHK Engineering Administration Department and Broadcast Engineering Department, we developed a distribution system for Hi-Vision signals between the network operation center of the NHK Broadcasting Center and the studio and introduced the system in 1998¹²⁾.

In the 2000s, we began full-fledged R&D on 8K Super Hi-Vision (referred to simply as 8K below). At the same time, we also started R&D on optical fiber transmission technology for uncompressed 8K signals and IP transmission technology for compressed 8K signals for use in program production^{13) 14) 15) 16)}. The work on 8K long-distance transmission is described in chapters 4 and 5.

3.2 LAN and IP technology

Beginning in the mid-2000s, the incorporation of high-speed, general-purpose Ethernet and IP systems in live production systems was investigated as means of reducing transmission and routing costs with widely used LAN technology. One result of that work is the SMPTE ST 2022 standard ¹⁷⁾, which was issued in 2012 and is still in wide use for data transport in broadcasting stations.

Subsequently, interest in features such as bidirectionality and expandability afforded by IP technology increased, and work began on using the technology to build production systems that were more flexible^*19. Also, when transmitting high-resolution 4K and 8K video as SDI signals, multiple SDI signals must be connected and transmitted, which leads to the problem of system complexity. The use of IP technology was also studied as a solution to that problem, and various manufacturers developed and commercialized their own superior IP transmission systems. In particular, systems such as those installed in outside broadcasting vans can be highly independent, and their weight can be reduced by using an optical fiber rather than a coaxial cable. For those reasons, there are multiple cases of such IP transmission systems being introduced in Japan.

On the other hand, however, standardizing the IP transmission method to ensure the interoperability of devices would enable many manufacturers to enter the market and thus reduce the cost of introduction. For that reason, international standards organizations desired an early formulation of such a standard, and an IP transmission method for program production was standardized as SMPTE ST 2110 in 2017¹⁸⁾. The main differences between ST 2022 described earlier and ST 2110 are listed in Table 2. Typical usage is shown in Fig. 4¹⁹⁾.

With ST 2022, the SDI signal that is used at the broadcasting site serves as the interface, and only the part of the signal that is to be transmitted is sent over the IP network ( Fig. 4(a)). This is because the serial component signals (video, audio, and ancillary data, and so forth) that are superimposed in the SDI format are sequentially converted to IP format, and the resulting signal is suited to point-to-point transmission.

ST 2110, on the other hand, was intended to achieve greater flexibility in program production systems with IP technology. The IP signal serves as the interface, so each essential signal can be translated as a separate stream and point-to-multipoint transmission is possible. As shown in Fig. 4(b), for example, the video signal from a venue or studio can be transmitted to the studio control room of broadcasting station A, and after switching, the video can be sent to the venue and to the audio mixing room of broadcasting station B. The audio can be sent to the mixing room of broadcasting studio B.

Although the standardization of IP transport is proceeding in this way, IP production systems differ from SDI systems in that the IP addresses of program production devices and controls, such as switch routing, must be set rather than the simple connection of cables.

A prerequisite for remote production, which is expected to be implemented with IP technology, is that remote devices can be controlled via an IP network. In a production system that uses SDI, the device control functions use vendor-specific methods. The initial production systems that applied IP technology also used proprietary methods, but progress in IP transport standardization created the opportunity for an open standard for control methods as well. Discussions of control methods at the Advanced Media Workflow Association (AMWA)^*20 moved forward, and a control method standard for device discovery and addition to the network was issued in 2016. The AMWA control method^{20) 21)} was standardized with the cooperation of program production equipment vendors and broadcasting stations and was adopted as the recommended control method by the Alliance for IP Media Solutions (AIMS)^*21. Also, the European Broadcasting Union (EBU) announced that it will require vendors to support the AMWA control system²²⁾.

SMPTE, AMWA, EBU, and the Video Services Forum (VSF)^*22 created the Joint Task Force on Networked Media (JT-NM)²³⁾ to discuss a roadmap and standard implementation methods for the development of IP and cloud production facilities.

As described above, the standardization of the components of an IP control system has been progressing, and examples of use around the world have appeared. Work on using these technologies to achieve greater efficiency in program production systems is described in chapters 4 and 6.

4.1 Use of optical transmission technology

The transmission rates for 8K video are very high: about 144 Gbps for uncompressed full-featured 8K (frame frequency of 120 Hz, chroma sampling^*23 of 4:4:4, and bit depth of 12) and about 40 Gbps for uncompressed full-resolution 8K (frame frequency of 60 Hz, pixel sampling structure of 4:2:2, and bit depth of 10), the specifications currently used for program production. Thus, the increasing relative technical difficulty is a major issue in the development of 8K video transmission equipment and long-distance transmission.

The lines that are available for use in live 8K program production, etc., include dark fibers^*24, IP lines provided by telecom carriers, and Ethernet lines. When dark fibers are used, a system that is capable of economically extending the relay distance is needed. For the use of IP lines provided by a telecom carrier or Ethernet lines, the line cost must be reduced with low-delay, high-quality image compression, and error measures must be applied to mitigate the degradation of link quality such as packet loss and fluctuations in packet arrival time (jitter).

4.2 Issues related to using IP technology to improve efficiency in live program production

As described in chapter 3, standardization of the basic part of the IP production system has been moving forward and its application to routine program production (repeated tasks in the network configuration and production equipment configuration) has begun mainly in overseas countries^*25. The main issues for the future are listed below.

• Construction of a large-scale system
  How and to what extent system redundancy, stability, and ease of updating should be secured and how systems that can handle 2K, 4K, and 8K can be constructed, etc.
• Application to nonroutine program production tasks
  How to achieve efficiency in the construction and updating of the network system for program production.
• Software for the future
  How to further enhance functions with software and construct program production systems using general-purpose IT equipment such as servers and the cloud.
• Ensuring security
  What level of security is needed for a closed network that is dedicated to program production, what risks should be assumed, and what measures should be adopted to ensure security?

From the viewpoint of the increase in scale and efficiency described above, one issue is the development of network monitoring techniques for the IP production system. The IP network enables easy multiplexing of multiple signals, such as for video, audio, and control, but also introduces the problem of possible packet loss from network congestion, which does not occur in SDI systems. It is therefore important to monitor IP flow (the flow of the series of IP packets) through the IP lines in real time. Also, signals in an IP network are transmitted through a number of network switches, so the transmission route of each IP flow must be known to determine points of failure and deal with failures. While IP network monitoring technology is required, conventional network monitoring technology is not capable of real-time monitoring with the accuracy required by a program production system.

Although security basically requires the application of IT technology as needed, safety in the sharing of devices also requires a mechanism for preventing contention in device control.