Mobile Blog: Video, Media, Augmented Reality, Rich Media, IVVR

photo credit: qthomasbower

Applications cannot enlarge a small screen visually, but they can implement techniques that virtually increase the size of the display. One way is providing horizontal or vertical scrolling of the user interface to make new information visible while hiding other content. Another idea is a Peephole display that shows a different portion of a bigger picture when the phone is moved to the left, right, up or down. Unfortunately, neither approach works well with IVVR (Interactive Voice & Video Response). There are no positional sensors usable with 3G-324M, and scrolling requires fast screen updates with the ability to hold a key as long as the user wants to scroll. High delays in the current 3G-324M deployment and lack of transmitting the information that a key is hold for a time prevent the implementation of such features. However, applications can have multiple layers, such as a deck of cards that can be shown or hidden depending on the user’s selection. Furthermore, designs can take advantage of the media- streaming capabilities and multimodal information channels by providing some information using speech output, some with pictures or text, and others by using video sequences.

Mobile users demand visually attractive user interfaces that are clearly readable and intuitive to use. Application flow design is beyond the scope of this thesis, and every application and game will have its own characteristics to model and challenges to overcome. Nevertheless, some basic guidelines for slide-based IVVR applications can be given.

Slide-based applications such as IVR (Interactive Voice Response) Supplements shown in Figure 6 are best visually designed using pixel-based image editors such as Adobe Photoshop. The video codecs used in 3G-324M work in the YUV420 colour space, and the target image size is 176 x 144 pixels (QCIF (Quarter Common Intermediate Format)). With basic understanding of chroma-subsampling and how spatial and temporal compression in video codecs works, designers can create slides that will compress well while maintaining sharpness where essential. A precondition is to align the slide’s layout to a raster of 16×16 pixels with one subdivision (8×8) as seen in the figure below:

Template with 16x16 grid

To ensure best readability despite video compression, designers should use sans-serif fonts. Moreover, the font colour and the background colour should have a high difference in luminance. The human eye can distinguish difference in luminance easier than in colour; this fact is used by video compressors and is the foundation of chroma subsampling. For instance, a white font on a light yellow background is already hardly readable without compression. After compression, however, with only half of information available for colour differences when using YUV420, the font will not be distinguishable from the background. The author’s experiments showed that especially Microsoft’s Calibri font creates a nice typeface even after compression. Calibri’s subtly rounded stems and corners are perfect for H.263 DCT-based compressors that create smooth edges. Note that the minimum font size is 18px when lossy compressed in order to be readable for mobile users. We can only hope that next-generation IVVR applications can use T.140 or similar ways to transmit ASCII-text directly, making readability considerations obsolete.

Video call set-up times are generally between 1 to 5 seconds (1-second call set-up when using MONA specified in H.324 Annex K.) independently of the IVVR (Interactive Voice & Video Response) application one is going to use, which is sometimes faster than the initialisation process of complex J2ME (Java 2 Micro Edition) applications. This makes quick on-the-go lookup or entry of information pleasant. However, what happens when the caller needs to interrupt a gaming session or the call is interrupted because of missing network coverage or exceeded battery life? Games should enable users to start and stop with breaks in between, since the time they have to spend may be brief. Mobile games are used especially to pass time for just a couple of minutes or even seconds when waiting for the bus, riding the subway, or to relieve boredom during TV commercials. Therefore, all mobile applications need to apply ways to interrupt a session and quickly resume to the last state as the user desires. This requirement also applies to IVVR applications. As application and user data of IVVR programs can be completely stored on the server-side, there are no limitations to auto-save program states or record the user’s actions. With the caller’s unique phone number as an identifier, it is easy to develop resumable applications.

photo credit: phauly

In desktop video conferencing, the video conferencing application is normally bundled to an Instant Messaging software that includes text chat capabilities. Users can appoint or prearrange a video conference using textual chat. In contrast, the current evolution of video telephony in UMTS (Universal Mobile Telecommunications System) networks based on the circuit-switched 3G-324M service does not seamlessly combine video conferencing with other communication channels. The notion of video telephony in the mobile environment is nearer to standard voice calling than in the stationary world. Therefore, it is more likely that somebody will place a video call without prior announcement. This leads to privacy and inconvenience concerns. The callee might not want to be seen during a conversation for a variety reasons: A video call “turns you ugly” (Harlow) because the build-in cameras are usually not placed just above the user’s line of sight but in the suboptimal position below the nose. Further, the video quality is meagre, and lightning conditions are poor. People might feel that exposing their face over a video call invades their privacy and, most times, callees do not want callers to see how he or she looks. Furthermore, the use of video telephony can depend on social factors. Societies in South East Asian countries—for example, Malaysia—are considered non-confrontational. This can be seen when people make decisions on which channel they use for communication. The author’s experiences in South East Asia revealed that most people prefer non-confrontational communication such as SMS (Short Message Service), instant messaging or e-mail, even in the business environment or with good friends. Voice calling is avoided as much as possible for a first or unexpected contact. It is obvious that P2P (Peer-to-peer) video calling is considered even more intrusive—and therefore unlikely to succeed in these societies.

According to an informal research of Sachendra Yadav (Yadav), opinion leaders and technology experts feel that video calling does not add much to a conversation compared to voice calling. In comparison to desktop video conferencing, which is mostly free nowadays, the cost-benefit analysis leads to resistance for using mobile video telephony.

For many reasons, 3G video telephony as a person-to-person conversational service is not as successful as projected. The existing technical foundation for video calling can be used to deliver IVVR (Interactive Voice & Video Response) services. A wide range of IVVR applications is imaginable, and some service providers and network operators already deploy them. Furthermore, special IVVR applications such as P2P Video Avatar can even compensate the drawbacks of classic P2P video telephony, making P2P-alike video telephony successful after all.

photo credit: Iain Farrell

Even with a great deal of marketing, early attempts to convert users to the video telephony technology flopped (Jones and Marsden). In contrast, desktop video conferencing is incredibly popular for private person-to-person conversations and widely used for video conferencing in business environments such as telepresence for computer-supported cooperative work (CSCW).

In desktop video conferencing scenarios, typically a stationary computer is used. Camera and microphone are fixed and usually maintain the same distance from the person participating during the conversation. Moreover, lighting conditions are generally better than “on-the-go”, as a desktop is easier to illuminate correctly than a scene in the mobile environment. When performing mobile video telephony, lighting conditions change over time when the caller moves or the environment changes; moreover, the camera is usually not fixed. During mobile video telephony, the caller is likely to hold the handset in front of his face by extending his arm, making the video wiggly. In combination with the meagre bandwidth and low-resolution video, this can considerably degrade the video quality shown on the callee’s side. These considerations about the video quality problems in the mobile environment also play a major role in IVVR (Interactive Voice & Video Response) applications that take advantage of the instant video streaming capabilities that 3G-324M video telephony offers. Bad video quality negatively influences camera-based games, gesture recognition, or P2P (Peer-to-peer) services that intentionally change the video for dynamic video overlays such as for the P2P Avatar, because motion analysis algorithms perform better with a sharp and clear video signal.

IVVR (Interactive Voice & Video Response) Application: IVR (Interactive Voice Response) Supplement Example

IVVR applications use the IVVR mobile technology to create services based on 3G video calling. In my opinion, the easiest conceivable IVVR application is an IVR supplement that adds the video dimension to typical IVR services:

IVR Supplements are dialog-based IVVR applications founding on the notion of typical IVR applications for call dispatching or information services. They add the visual dimension to IVR applications by showing slides as a graphical representation of an IVR voice menu on the phone screen. This increases accessibility for the dumb, making it possible to interact with IVR where listening might not be desired—such as in business meetings or during a lecture—and accelerating perception of options and information.

Humans receive 80% of information by seeing, but only 15% by hearing—and 5% by feeling (Dahm). Additionally, seeing is a non-linear process by which people can perceive information simultaneously when skimming a text, for instance. In contrast, when the focus is on the transmission of facts without emotions, hearing is a linear process where the listener must wait until the speaker or recording has finished. The same information might be faster to transmit using written text or visualisations. Because humans can perceive information visually faster than auditively, IVVR takes advantage of this fact by using the handset’s display to present information.