As the Society of Motion Picture and Television Engineers celebrated its centenary, informitv delivered a presentation on ‘The Resolution Revolution’ at the SMPTE Annual Technical Conference in Hollywood. The proposition was essentially that we still have some way to go in terms of spatial resolution, bit depth and frame rate. Perhaps the time has come to think beyond fixed resolutions and frame rates. Rather than compromising quality, digital compression can enable dynamic resolution and an optimised experience.

William Cooper of informitv observed that the moving image has yet to match the highest resolution of still photography. A ‘high-definition’ video frame is only about two million pixels and even a 4K Ultra HD image is only 8 megapixels. An 8K image is only 33 megapixels, which is about the resolution of a professional digital still camera, although there are already cameras that capture 50 or 100 megapixels. Beyond this the limit becomes the resolving power of the lens rather than the limitations of the sensor.

William Cooper, the founder and chief executive of informitv.

It is often argued that it is difficult to discern the individual pixels beyond a certain distance. This is generally based on an assumption about visual acuity at around a minute of arc. However, it now understood that our vernier acuity or the ability to discriminate offset lines is five or ten times better than that. In any case, supersampling can reduce apparent aliasing effects, particularly on text and graphics, even if we cannot see the individual pixels.

According to one industry standard the ‘optimum viewing distance’ for an 8K screen is just 0.8 times picture height. That is just the point at which it may not be possible to see individual pixels. The reality is that we do not need to sit nearer a screen simply because it has higher resolution. What is the ideal viewing distance for a painting? It’s the wrong question.

A comfortable screen size and viewing distance will be determined by many factors, from the size of the room and number of viewers, to personal preferences. Besides, if we want to examine the quality of something we generally inspect it more closely. The closest viewing distance is really our minimum focussing distance, which varies according to our eyesight.

These days we may hold the image in our hand. Many tablet displays have a resolution greater than 300 pixels per inch. That happens to be the resolution generally accepted as a standard for commercial printing.

A high definition image at 300 pixels per inch would only be about the size of 6×4 inch photographic print. A 4K image, with four times the resolution, would only be about the size of 12×8 inch print. Even 8K is still only around the size of a 24×16 inch picture. This is clearly still smaller than a screen with a 55 inch diagonal, which is about 48 inches wide. So if we want large screens with photographic definition we may need 8K or higher resolution.

Relative screen sizes at 300 pixels per inch. Source: informitv.

Resolution is not just a question of height and width in pixels. The precision of colour and brightness information for each pixel depends on the bit-depth used. The number of bits determines how many values can be represented over the total range. 8 bits offer a maximum of 256 discrete levels or code values. With 10 bits and 12 bits there are 4 and 16 times as many levels available. However, there are already image sensors that can produce 14 or even 16 bits per sample. Computers tend to work in words of multiples of 8 bits. 16 bits offer a maximum of over 65,000 discrete levels, which is 256 times the resolution of 8 bits for double the storage.

Possible values increase with bit depth from 8 bits to 16 bits per channel. Source: informitv.

What about frame rate? We are no longer tied to the need for a constant frame rate by either the mechanics of the celluloid medium or the synchronisation of analogue television and video. Cinema movies have remained at 24 frames per second for around a century. This ‘film look’ has become so conventionally associated with cinema that many people object that higher frame rates produce a ‘video look’ that somehow seems too ‘real’. Ultra HD will deliver 50 or 60 frames per second, or even 100 or 120. With enough temporal resolution it is possible to synthesise lower frame rates and shutter speeds. Perhaps we don’t need a fixed frame rate, or even the concept of discrete frames. Perhaps we should be thinking in terms of sampling frequency and variable refresh rates.

If we simply keep doubling resolution in every dimension we face impractical data rates. Completely uncompressed 8K video at 120 frames per second would require almost 200 Gigabits per second, or 1.4 Terabytes per minute.

Uncompressed data rates for different resolutions. Source: informitv.

The solution is perceptually lossless digital compression. Rather than indiscriminately discarding data by using relatively low resolutions, we can sample at higher resolution and precision then use mathematical techniques to take advantage of spatial and temporal correlation to reduce redundancy. An image with four times the spatial resolution does not necessarily require four times the bandwidth when compressed. Similarly, encoding at twice the sample rate need not imply twice the data rate.

Perhaps we should consider spatial and temporal resolution as different dimensions of the same volume of information. It is possible that they could be flexible and variable, according to context. They could be established on a programme-by-programme basis, if not scene-by-scene or shot-by-shot. After all, we do not see the world in terms of pixels or frames. Sometimes we may benefit from more spatial detail, at other times we may be tracking rapid motion.

In the digital domain, data can be allocated dynamically according to the creative requirements of the particular content and the economic constraints of delivery. Digital compression allows bits to be deployed more flexibly and more intelligently.

It follows that rather than a single standard format, or a fixed number of bits, we need standards that can support dynamic systems that can describe their own requirements in terms of spatial and temporal resolution, which can compete for available resources to deliver the optimum experience for a particular viewing context.

The meeting paper “The Resolution Revolution: How many bits do we really need?” by Dr William Cooper and Dr Sue Farrell of informitv was presented at the SMPTE Annual Technical Conference 2016 in Hollywood.