Mobile Blog: Video, Media, Augmented Reality, Rich Media, IVVR

photo credit: *kaetzchen

Recently, I bought the HTC Touch Pro 2. As advertised by HTC, it is a professional communication tool. By upgrading it with Windows Mobile 6.5 its performance and functionality seemed boosted. A couple of days ago a friend of mine (he is not really into the mobile market, though) just bought the HTC HD2. Realizing the HD2 has double of processing power and double of RAM compared of my TP2, I quietly regret my decision. The HD2 is extremely fast, especially when browsing web pages. The only drawback of the HD2, and all touchphones, is the missing physical keyboard by nature. As I type a lot of emails on the go, sometimes blindly, I need a physical keyboard. But maybe Samsung solves this general problem with its Touchwiz UI? I think, HTC is about the turn the mobile phones market upside down. They are already very successful in the business sector. As they will start to market their products more to consumers for private and leisure use, they might rule out RIM and their blackberry; while wresting market share from Nokia and Samsung.

photo credit: phauly

In desktop video conferencing, the video conferencing application is normally bundled to an Instant Messaging software that includes text chat capabilities. Users can appoint or prearrange a video conference using textual chat. In contrast, the current evolution of video telephony in UMTS (Universal Mobile Telecommunications System) networks based on the circuit-switched 3G-324M service does not seamlessly combine video conferencing with other communication channels. The notion of video telephony in the mobile environment is nearer to standard voice calling than in the stationary world. Therefore, it is more likely that somebody will place a video call without prior announcement. This leads to privacy and inconvenience concerns. The callee might not want to be seen during a conversation for a variety reasons: A video call “turns you ugly” (Harlow) because the build-in cameras are usually not placed just above the user’s line of sight but in the suboptimal position below the nose. Further, the video quality is meagre, and lightning conditions are poor. People might feel that exposing their face over a video call invades their privacy and, most times, callees do not want callers to see how he or she looks. Furthermore, the use of video telephony can depend on social factors. Societies in South East Asian countries—for example, Malaysia—are considered non-confrontational. This can be seen when people make decisions on which channel they use for communication. The author’s experiences in South East Asia revealed that most people prefer non-confrontational communication such as SMS (Short Message Service), instant messaging or e-mail, even in the business environment or with good friends. Voice calling is avoided as much as possible for a first or unexpected contact. It is obvious that P2P (Peer-to-peer) video calling is considered even more intrusive—and therefore unlikely to succeed in these societies.

According to an informal research of Sachendra Yadav (Yadav), opinion leaders and technology experts feel that video calling does not add much to a conversation compared to voice calling. In comparison to desktop video conferencing, which is mostly free nowadays, the cost-benefit analysis leads to resistance for using mobile video telephony.

For many reasons, 3G video telephony as a person-to-person conversational service is not as successful as projected. The existing technical foundation for video calling can be used to deliver IVVR (Interactive Voice & Video Response) services. A wide range of IVVR applications is imaginable, and some service providers and network operators already deploy them. Furthermore, special IVVR applications such as P2P Video Avatar can even compensate the drawbacks of classic P2P video telephony, making P2P-alike video telephony successful after all.

photo credit: Iain Farrell

Even with a great deal of marketing, early attempts to convert users to the video telephony technology flopped (Jones and Marsden). In contrast, desktop video conferencing is incredibly popular for private person-to-person conversations and widely used for video conferencing in business environments such as telepresence for computer-supported cooperative work (CSCW).

In desktop video conferencing scenarios, typically a stationary computer is used. Camera and microphone are fixed and usually maintain the same distance from the person participating during the conversation. Moreover, lighting conditions are generally better than “on-the-go”, as a desktop is easier to illuminate correctly than a scene in the mobile environment. When performing mobile video telephony, lighting conditions change over time when the caller moves or the environment changes; moreover, the camera is usually not fixed. During mobile video telephony, the caller is likely to hold the handset in front of his face by extending his arm, making the video wiggly. In combination with the meagre bandwidth and low-resolution video, this can considerably degrade the video quality shown on the callee’s side. These considerations about the video quality problems in the mobile environment also play a major role in IVVR (Interactive Voice & Video Response) applications that take advantage of the instant video streaming capabilities that 3G-324M video telephony offers. Bad video quality negatively influences camera-based games, gesture recognition, or P2P (Peer-to-peer) services that intentionally change the video for dynamic video overlays such as for the P2P Avatar, because motion analysis algorithms perform better with a sharp and clear video signal.

Ganz einfach: Im Vergleich zur Festnetztelefonie müssen viele Hürden genommen werden! Anfang des vergangenen Jahrhunderts war Festnetztelefonie der große Hit. Und die Teilnehmer waren sprichwörtich “fest” verdrahtet. Zuerst manuell, dann mechanisch, später elektronisch, heute elektrisch wurden die Teilnehmerleitungen 1:1 miteinander mit Leitungen verbunden. Das nennt man auch leitungsvermittelt. Die Leitungen werden also einfach verbunden.

Mobiltelefonie ist um Potenzen komplizierter: Verbindungen sind nun kabellos und flüchtig. Unter optimalen Bedingungen funktioniert Mobiltelfonie so: Die beiden Gesprächspartner stehen fix in einer sichtlinie zueinander und haben Richtantennen auf einander gerichtet. Darüber wird unverschlüsselt ein Sprachsignal ausgetauscht. Nur das ist einfach keine Mobiltelefonie, vielmehr immobile Drahtlostelefonie. Typischerweise bewegen sich beide Teilnehmer, haben keine Sichtverbindung zueinander und befinden sich in unterschiedlichen Netzen, vielleicht sogar Ländern oder Kontinenten. Das wäre dann reale Mobiltelefonie.

Die Bewegung und Umgebung verursacht folgende Probleme: Das vom Mobilfunkmast abgestrahlte Signal wird in verschiedenen Luftschichten gebrochen, von Objekten reflektiert und Wänden gedämpft. Dabei ist es gut möglich, dass sich ein Signal dadurch so aufsplittet, dass Teile davon zu unterschiedlichen Zeiten, auf unterschiedlichen Wegen beim Empfänger eintreffen. Das Mobiltelfon muss diese also zusammensetzen.

Auch ist die Zahl der Mobilfunkteilnehmer in einer Funkzelle relevant: Je mehr Teilnehmer, desto mehr Signale stören sich gegenseitig. Dadurch sinkt die Sprachqualität. In extremen Szenarien, wie Großveranstaltungen oder Neujahr in Großstädten, “brechen” ganze Mobilfunknetze zusammen. In Wirklichkeit brechen diese natürlich nicht zusammen, sondern sobald die Anzahl der möglichen Verbindungen ausgeschöpft ist, werden keine neuen zugelassen.

photo credit: Steven Fernandez

Media compression, error concealment measures, and the characteristics of wireless networks have side effects on the quality of 3G video telephony and IVVR (Interactive Voice & Video Response) applications.

3G-324M requires only the use of speech codecs. In contrast to audio codecs, speech codecs are designed for speech transmission within a narrow frequency range, making them inappropriate for transmission of music or a range of artificial sounds. This fact needs to be considered when designing IVVR applications—especially games, as most games utilize music and sound effects to create an immersive atmosphere.

H.263 and MPEG-4 Part II baseline were designed for images of natural scenes with predominately low-frequency components, meaning that the colour values of spatial and temporally adjacent pixels vary smoothly except in regions with sharp edges. In addition, human eyes can tolerate more distortion of high-frequency components than of the low-frequency components (Kwon and Driessen). In reference to the explanation of Kwon and Driessen, video codecs used for 3G-324M video telephony are great for natural scenes and talking-head scenarios. Depending on the type of IVVR application, these characteristics work against a good user experience.

Typical desktop or web applications have a monochromatic user-interface with boxes, buttons, and fonts that are clearly readable. Based on user interaction, the user interface can change its appearance frequently, perhaps only for some parts of the user interface or perhaps the whole screen. It is obvious that codecs used for 3G-324M video telephony are unsuitable for this kind of video transmission. Compressing such user interfaces with H.263 creates blurred fonts and tattered buttons and lines, leading to a user interfaces too distorted for a good user experience. The comparable high round-trip delays can make interaction tedious, with interfaces that require a high rate of user interaction and screen changes.

Depending on the type of game, the compression characteristics of video codecs used in 3G-324M can be advantageous. Contemporary 3D games such as first-person shooters or simulation games try to model the game environment as realistic as possible, creating natural-looking scenes and making them appropriate for compression using video codecs defined for 3G-324M.

However, more problematic are the delay requirements for mobile games that are essential for a good gameplay experience. 3GPP (3rd Generation Partnership Project) defines a delay variation of below 75ms for real-time games  and considers first-person shooters the most demanding ones with respect to delay requirements (3GPP). Other types of games, such as turn-based strategy games or visual novels, may tolerate a higher end-to-end delay and may require lower data rates.