Mobile Blog: Video, Media, Augmented Reality, Rich Media, IVVR

Especially in multiplayer scenarios, game developers need to discover ways to provide players with the freedom to interrupt a game without displeasing other players. Further, game concepts should cope with the interaction delays of IVVR (Interactive Voice & Video Response) technology. In single-player games, the game should automatically pause and save the current game state when a user hangs-up the video call, resuming the last game state when the user chooses to play again. Implementing a similar functionality in multiplayer games is more challenging. As real-time shooters require rapid interaction, they are inappropriate for the W-CDMA (Wideband Code Division Multiple Access) network. One way to cope with the high 3G network latency is turn-based multiplayer games in which fast reactions to other players’ decisions are not required. Moreover, some actions in a game can be performed asynchronously, such as character development or adding new items for sale in one’s in-game shop.

Event notifications combine well with asynchronous gameplay since it allows the game to contact the player when a certain kind of change in the game state has occurredor when other players are ready to play. In the sense of push-communication, gamers could be alerted by receiving a video call. When these alerts are received on a regular basis and only from a limited number of friends, they can increase pervasiveness without annoying the user. In such a case, users can decide whether they would like to accept the call when they are ready to play. Combining the concepts of asynchronous gameplay, event notifications, and a turn-based strategy helps developers to create entertaining multiplayer games suitable for IVVR.

photo credit: papalars

Did you already sign up for GSMA world congress in Barcelona 15-18 February 2010? Let’s meet: http://www.mobileworldcongress.com

photo credit: qthomasbower

Applications cannot enlarge a small screen visually, but they can implement techniques that virtually increase the size of the display. One way is providing horizontal or vertical scrolling of the user interface to make new information visible while hiding other content. Another idea is a Peephole display that shows a different portion of a bigger picture when the phone is moved to the left, right, up or down. Unfortunately, neither approach works well with IVVR (Interactive Voice & Video Response). There are no positional sensors usable with 3G-324M, and scrolling requires fast screen updates with the ability to hold a key as long as the user wants to scroll. High delays in the current 3G-324M deployment and lack of transmitting the information that a key is hold for a time prevent the implementation of such features. However, applications can have multiple layers, such as a deck of cards that can be shown or hidden depending on the user’s selection. Furthermore, designs can take advantage of the media- streaming capabilities and multimodal information channels by providing some information using speech output, some with pictures or text, and others by using video sequences.

Mobile users demand visually attractive user interfaces that are clearly readable and intuitive to use. Application flow design is beyond the scope of this thesis, and every application and game will have its own characteristics to model and challenges to overcome. Nevertheless, some basic guidelines for slide-based IVVR applications can be given.

Slide-based applications such as IVR (Interactive Voice Response) Supplements shown in Figure 6 are best visually designed using pixel-based image editors such as Adobe Photoshop. The video codecs used in 3G-324M work in the YUV420 colour space, and the target image size is 176 x 144 pixels (QCIF (Quarter Common Intermediate Format)). With basic understanding of chroma-subsampling and how spatial and temporal compression in video codecs works, designers can create slides that will compress well while maintaining sharpness where essential. A precondition is to align the slide’s layout to a raster of 16×16 pixels with one subdivision (8×8) as seen in the figure below:

Template with 16x16 grid

To ensure best readability despite video compression, designers should use sans-serif fonts. Moreover, the font colour and the background colour should have a high difference in luminance. The human eye can distinguish difference in luminance easier than in colour; this fact is used by video compressors and is the foundation of chroma subsampling. For instance, a white font on a light yellow background is already hardly readable without compression. After compression, however, with only half of information available for colour differences when using YUV420, the font will not be distinguishable from the background. The author’s experiments showed that especially Microsoft’s Calibri font creates a nice typeface even after compression. Calibri’s subtly rounded stems and corners are perfect for H.263 DCT-based compressors that create smooth edges. Note that the minimum font size is 18px when lossy compressed in order to be readable for mobile users. We can only hope that next-generation IVVR applications can use T.140 or similar ways to transmit ASCII-text directly, making readability considerations obsolete.

photo credit: phauly

In desktop video conferencing, the video conferencing application is normally bundled to an Instant Messaging software that includes text chat capabilities. Users can appoint or prearrange a video conference using textual chat. In contrast, the current evolution of video telephony in UMTS (Universal Mobile Telecommunications System) networks based on the circuit-switched 3G-324M service does not seamlessly combine video conferencing with other communication channels. The notion of video telephony in the mobile environment is nearer to standard voice calling than in the stationary world. Therefore, it is more likely that somebody will place a video call without prior announcement. This leads to privacy and inconvenience concerns. The callee might not want to be seen during a conversation for a variety reasons: A video call “turns you ugly” (Harlow) because the build-in cameras are usually not placed just above the user’s line of sight but in the suboptimal position below the nose. Further, the video quality is meagre, and lightning conditions are poor. People might feel that exposing their face over a video call invades their privacy and, most times, callees do not want callers to see how he or she looks. Furthermore, the use of video telephony can depend on social factors. Societies in South East Asian countries—for example, Malaysia—are considered non-confrontational. This can be seen when people make decisions on which channel they use for communication. The author’s experiences in South East Asia revealed that most people prefer non-confrontational communication such as SMS (Short Message Service), instant messaging or e-mail, even in the business environment or with good friends. Voice calling is avoided as much as possible for a first or unexpected contact. It is obvious that P2P (Peer-to-peer) video calling is considered even more intrusive—and therefore unlikely to succeed in these societies.

According to an informal research of Sachendra Yadav (Yadav), opinion leaders and technology experts feel that video calling does not add much to a conversation compared to voice calling. In comparison to desktop video conferencing, which is mostly free nowadays, the cost-benefit analysis leads to resistance for using mobile video telephony.

For many reasons, 3G video telephony as a person-to-person conversational service is not as successful as projected. The existing technical foundation for video calling can be used to deliver IVVR (Interactive Voice & Video Response) services. A wide range of IVVR applications is imaginable, and some service providers and network operators already deploy them. Furthermore, special IVVR applications such as P2P Video Avatar can even compensate the drawbacks of classic P2P video telephony, making P2P-alike video telephony successful after all.

photo credit: Iain Farrell

Even with a great deal of marketing, early attempts to convert users to the video telephony technology flopped (Jones and Marsden). In contrast, desktop video conferencing is incredibly popular for private person-to-person conversations and widely used for video conferencing in business environments such as telepresence for computer-supported cooperative work (CSCW).

In desktop video conferencing scenarios, typically a stationary computer is used. Camera and microphone are fixed and usually maintain the same distance from the person participating during the conversation. Moreover, lighting conditions are generally better than “on-the-go”, as a desktop is easier to illuminate correctly than a scene in the mobile environment. When performing mobile video telephony, lighting conditions change over time when the caller moves or the environment changes; moreover, the camera is usually not fixed. During mobile video telephony, the caller is likely to hold the handset in front of his face by extending his arm, making the video wiggly. In combination with the meagre bandwidth and low-resolution video, this can considerably degrade the video quality shown on the callee’s side. These considerations about the video quality problems in the mobile environment also play a major role in IVVR (Interactive Voice & Video Response) applications that take advantage of the instant video streaming capabilities that 3G-324M video telephony offers. Bad video quality negatively influences camera-based games, gesture recognition, or P2P (Peer-to-peer) services that intentionally change the video for dynamic video overlays such as for the P2P Avatar, because motion analysis algorithms perform better with a sharp and clear video signal.