# Augmented Reality

## 01 - Introduction

### Mixed Reality
1. **What is Mixed Reality?**
- Presenting both, real world and virtual world objects in a single display
- Umbrella term for:
- Augmented Reality
- Augmented Virtuality
- Virtual Reality

2. **What is Augmented Reality?**
- virtual enhancement of the real world
- user sees real world
- with virtual objects placed in real world

3. **What is Augmented Virtuality?**
- real enhancement of a virtual world
- real data is mapped on virtual objects (textures etc.)
- >50% of the world is computer generated

4. **What is Virtual Reality?**
- fully virtual world
- using physical laws/world in virtual environment
- intent is to immerse the user

5. **Describe Milgrams Reality-Virtuality Continuum**
- Range from real environment to a purely virtual environment
- everything in between is mixed reality

6. **What are the AR Characteristics?**
- AR enhances users perception and interaction of the real world
- uses:
- head mounted displays (HMD)
- head-up displays (HUD)
- smartphones
- not limited to display technologies
- integrates additional info in real world
- tangible interfaces
- intuitive interaction with real life objects
- mobile or portable

7. **Tell differences between AR and VR**
- Tracking: AR high accuracy/VR low okay
- Display Device: AR small view/VR wide field of view
- Scene Generation: AR minimal rendering ok/VR realistic images required

### Classification

8. **Describe the term EWK**
- Extent of World Knowledge
- Knowledge of the displayed world
- where, what, where and what

9. **Describe the difference between an unmodelled and modeled world**
- unmodelled world could contain images or objects
- modeled world contains info about object positions, viewpoint observation, user interaction

9. **What does the term RF mean?**
- Reproduction Fidelity
- describes the quality in which the display can reproduce the content
- simple wireframes to hi-fidelity 3D animations
- video to high res 3D

10. **What does the term EPM mean?**
- Extent of Presence Metaphor
- how much the user is intended to feel present
- different ways to display the environment
- Monitor to HMD

11. **Explain the Milgram-Weiser Continuum**
- Combination of Milgram and Weiser Continuum
- Ubicomp vs Monolithic
- Real vs Virtual Environment
- Mobile Gaming more virtual and ubicomp

### Structure of MR Systems

12. **Name the main issues of AR**
- Calibration
- Registration
- Synchronisation
- Tracking
- Visual coherence
- Real-time
- mobile requirements
- Processor power limited

13. **Name the main issues of VR**
- Latency
- Lag
- Degree and metaphor of interaction
- Navigation
- Cybersickness

13. **Name the components of the sensor feedback loop**
- **User** gives
- **Input** to the
- **Reality Engine**. The Engine makes changes and
- displays the results on a **display**

### Categorising Mixed Reality

14. **Name the different Categories of MR**
- Telepresence
- AR
- Desktop AR/VR
- Fishtank VR
- Immersive VR

15. **Explain Telepresence**
- display of the real world
- audio and video data
- places user in another real existing place
- eg. robot control

16. **Explain AR**
- real world enhancement with additional content
- HMD, smartphones,...
- position tracking required (and often an issue!)

17. **Explain Desktop AR/VR**
- PC based
- 3d-glasses in front of display
- cheap

18. **Explain Fishtank VR**
- Projection screens
- display on small display area and for more users

19. **Explain Immersive VR**
- Goal: Immersion
- CAVE/HMD/curved screen
- user as far as possible isolated from the real world
- multiuser possible
- = VR
- full virtual world
- Cybersickness!

### Different types of AR
20. **Name the different types of AR**
- Head-up displays (automotive)
- screen based (smartphone)
- SAR = Spatial Augmented Reality (Textile, Automotive)
- stationary (marketing, lego)
- HMD-based (training, maintenance, assembly lines)
- smart glasses (logistic, military)
- smart lenses (replace HMDs)

21. **Name examples for every field of AR in real life**
- archaeology: documentation, cultural heritage
- Medicine: training, visualisation
- Entertainment: Smartphones, consoles
- tourism: info visualisation
- marketing: ads, catalogues
- logistics: guidance systems
- manufacturing: manuals, instructions

### Challenges and Development
22. **Name most common problems and challenges**
- graphics rendering HW and SW
- Tracking techniques (changes of viewer position properly reflected by rendered graphics)
- Tracker calibration and registration tools (precision)
- display hardware (virtual to real world)
- computer processing hardware for AR
- interaction techniques

23. **Name current focuses in development**
- input devices and displays
- improving optical tracking technology
- research and evaluation of displays
- software

---
## 02 - Computer Graphics
### Introduction

24. **What are computer graphics?**
- graphics created using computers, the representation and manipulation of pictorial data
- opposite of computer vision

### Color Perception
25. **What is responsible for Color Perception?**
- different cones (RGB)

### Color Models
26. **Name the different Color Models**
- RGB
- CIE
- HSV
- HLS
- CMYK

### Graphics Pipeline
27. **What is the goal of the Graphics Pipeline?**
- displaying 2D projection in 3D scene
- subsequential stages transforming data frame stage to stage
- geometric and rendering processes involved

27. **What are the steps of the Graphics Pipeline?**
- Vertices to 3D geometric primitives
- Modelling and transformation
- Lightning
- Clipping
- Rasterization
- Texturing and shading

28. **Name the basic primitive objects**
- points
- lines
- triangles

29. **Out of what are all 3D objects constructed?**
- mesh of triangles

29. **What do triangles typically have?**
- 3 Vertices interconnected by edges
- 1 normal vector indicating the front side
- Additionally normal vectors for vertices exist

28. **Name the different 3D primitives**
- Sphere
- box
- cylinders
- torus (Donut shaped form)

29. **Explain what happens in the modeling and transformation stage**
- primitives are transformed from the local coordinate system to the world coordinate system

30. **What is a transformation?**
- Scaling
- rotation
- translation
- as a 4x4 matrix

31. **What coordinate systems exist for 2d and 3d?**
- 2D
- Screen Coordinate System
- Viewport Coordinate System
- 3D
- World Coordinate System
- Object Coordinate System
- Left vs Right handed system

31. **What is a Frustum and what is Frustum Culling?**
- Is a volume where its contents are projected for rendering
- Has a near and a far clipping plane
- Culling means everything outside the area of the frustum is not rendered

31. **How do changes of FOV and aspect ratio affect an image?**
- High vertical/horizontal FOV make things smaller
- Different aspect ratios enlargen things on the corresponding side

32. **Which different type of lightning models exist?**
- Global lightning
- Local lightning (indirect)

33. **What different type of light sources exist?**
- directional light
- spot light
- point light

### Shading
34. **Name the different types of shading and explain it in short terms**
- Flat shading: norm-vector + color value (once per frame)
- Gouraud shading (intensity interpolation shading): Vertex + color interpolation
- Phong shading: Vertex normal vectors + color interpolation per pixel basis

### Texturing
35. **How can we change the surface of a 3D model?**
- Applying images on the 3D models/objects
- coordinates of texture are mapped onto the object

### Z-Buffer
36. **What is a Z-Buffer?**
- = stores distance from image plane on a per pixel basis
- expands between near and far clipping plane
- nonlinear
- higher precision for pixels closer to near plane

37. **Another name for Z-Buffer**
- Depth buffer

37. **What is Z-Buffer fighting?**
- Z-Buffer fighting is caused by close by polygons
- Usually results in flickering of the two objects where the intersection is

### Rasterization
38. **What happens in the rasterization step?**
- = scan conversion
- 3D model of the scene is rasterized and a 2D image is generated
- Colors from shading are evaluated
- Textures applied
- surface in front based on Z-Buffer are drawn

### High Dynamic Range
39. **What is HDR?**
- mapping the full color range
- for displaying on a screen not capable of showing the full color range
- = Tone mapping

### Basic Optimization
42. **Name different types of optimization operations**
- Anti-aliasing (removing sampling artifacts
- level of detail (less detail, better performance), can be done beforehand as dynamical processing is intense and error prone
- MIP Mapping (level of detail applied on textures)
- Billboards (simple representations of complex geometric objects)

### Scene Graphs
42. **What is a scene graph?**
- Collection of nodes in a graph or tree structure
- Node may have many children but only a single parent
- effect/changes of a parent is applied to all its child nodes
- Leaf nodes contain geometry

### Culling Techniques

42. **What are the most common culling techniques?**
- Frustum culling
- Backface culling (typ. biggest performance increase)
- Occlusion culling
- Contribution culling

43. **How can Frustum Culling be improved?**
- By using bounding volumes instead of testing each primitive type

44. **How does Backface Culling work?**
- faces where the face normal points away from the camera are not rendered

---
## 03 - Application Areas and Application Development

45. **Name industries where AR can become a use case or change the daily work**
- Medicine
- Military
- Product Presentation
- Manufacturing and Maintenance
- Logistics
- Manuals
- Automotive
- Annotatins
- Architecture
- Art and Entertainment

45. **Name examples for medicine use cases**
- Vein Visualisation
- Diagnosis and treatment
- Virtual Liver Surgery Planning System
- Training of staff

45. **Name examples for military use cases**
- simulations
- training
- Helmet mounted sights
- X-Ray vision of buildings

46. **Name examples for product presentation use cases**
- lego
- cars
- future products
- magazines
- games
- catalogues

47. **Name examples for manufacturing and maintenance use cases**
- error reduction
- time reduction
- remote support
- manuals
- assembly step tutorials
- training

48. **Name examples for logistics use cases**
- delivery
- communication
- picking
- quality assurance

49. **Name examples for automotive use cases**
- HUDs
- motorcycles

50. **Name examples for annotation use cases**
- Info of real life objects
- museums
- geotagging

51. **Name examples for architecture use cases**
- collaborative design platform
- collaborative design platform protocol
- virtual reality
- augmented reality

52. **Name examples for art and entertainment**
- virtual studios
- special effects
- facade AR
- westworld experience
- reality fighters
- pokemon go

### Application Development
53. **Name the steps in the app dev process**
- planning
- development
- testing
- evaluation

54. **On what should you focus on especially from the user point of view?**
- Navigation
- Interaction
- user feedback
- beware of the target audience

---
## 04 - Tracking Foundations and Optical Tracking

55. **What is tracking?**
- dynamic determination of spatial properties at runtime
- measure position and orientation of a physical object in real life
- for consistency between real world and virtual world

56. **What is tracking in AR used for?**
- display of augmentations
- interaction

57. **What can be tracked in AR applications?**
- objects
- features
- camera position

58. **What can be tracked in VR applications?**
- head
- hand and fingers
- eyes
- torso

59. **What must an AR app address/fulfill?**
- tracking
- calibration
- registration

### Characteristics of Tracking Technology

60. **Name the characteristics of tracking technology**
- Physical phenomena
- measure light, sound, gravity, ...
- Measurement principle
- signal strength, direction, time of flight
- Measured geometric property
- distances, angles
- Sensor arrangement
- sensor fusion (multisensors), sensor sync
- Signal resources
- Passive: magnetic field, light
- active: acoustic, optic, radio wave
- Measurement coordinates
- global vs. local, absolute vs. relative
- Degrees of freedom
- DOF; 3 translations (XYZ), 3 rotations (head, pitch, roll)
- workspace coverage
- working volume of tracking system 1m^2 to global eg. GPS
- spatial scan
- outside-In (Sensors outside the tracked device) vs Inside-Out (Tracked device contains sensors)
- static accuracy
- position and orientation tracking of a fixed object
- dynamic accuracy
- measurement of a moving object
- tracking latencies
- length of tracking, fixed and variable latencies, prediction

61. **How can accuracy be increased?**
- Kalman filter
- Combination of prediction and correction

61. **What is the Kalman Filter?**
- Sensor fusion / learning algorithm
- Tries to get the exact state from a set of faulty set of measurements (often noise)
- Finds the most optimum averaging factor for each consequent state

62. **Difference between fixed and variable latencies?**
- fixed: sampling sensors, pose estimation
- variable: network communication, buffering, synchronization

### Registration

63. **What are the common AR registration issues?**
- static errors (viewpoint and environment remain still)
- Causes are Optical distortion, errors in tracking system and incorrect viewing parameters
- dynamic errors (viewpoint and/or environment move)
- Causes are Delays and lags
- jitter (even with markers, objects could move w/o movement of them happening)

63. **Solution for jitter?**
- If 3rd measurement is in range of 1st measurement, interpolation of 2nd could be performed by using 1st and 3rd measurement as base

### Tracking Systems
64. **Name different types of tracking systems**
- GPS
- Inertial tracking
- Inertial sensing
- Inertialsensing (accelerometer)
- accelerometer and gyro
- optical tracking
- infrared light
- beam scanning
- videometric
- inside-out
- outside-in
- pattern recognition
- optical tracking w matrix code
- spatial scan
- SLAM (Simultaneous Localisation and Mapping)

65. **Explain Inertial tracking**
- 2 components (accelerometer, gyro)
- advantages: no stationary tracking, large indoor areas
- disadvantages: positioning errors, orientation problems

66. **Explain Inertial sensing**
- Information about relative transformation of a target
- conserves axis of rotation
- mechanical gyro

67. **Explain Inertial sensing - accelerometer**
- measuring linear acceleration of an object
- measures force on a mass (example: Spring with mass)

68. **Explain Accelerometer and gyro in combination**
- acceleration and orientation measurement
- measurements always relative to last position
- Beware of accumulative error

69. **Explain GPS**
- Global Positioning System
- signals from three to four satellites
- triangulation of three spheres

70. **Explain Optical tracking**
- different types of markers (active, passive (reflect light))
- image processing is used
- landmark recognition
- inside out vs outside in
- Infrared light possible as well

71. **Explain Infrared use of tracking**
- IR camera to track markers
- errors can occur like light source collision or bright surroundings

72. **Explain Inside Out tracking**
- The tracked device itself contains the cameras or the sensory equipment required for tracking
### Spatial Scan
73. **Explain Videometric**
- tracking of a users helmet
- with mounted cameras at the ceiling

74. **Explain Beam Scanning**
- Scanning optical beams on a reference
- sensors on target detect time of sweep of the beams

75. **Explain Outside-In**
- Cameras or sensors are placed outside the tracked device
- High speed cameras

76. **Explain multiscopy**
- two or more cameras
- triangulation to get spatial position
- multi-object measurement = orientation detection

77. **Explain Pattern Recognition**
- one camera
- shape and size of object detected (known from the system)

78. **Explain Optical Tracking - Matrix Code**
- barcode
- matrix markers attached on RL object
- = object and pattern recognition
- measurement and position calculation via the markers and a camera
- Steps:
1. Binarisation
2. Connected component analysis
3. Coordinate System estimator

79. **What is SLAM?**
- Simultaneous Localisation and Mapping
- Idea: place a robot in a room and let the robot build a map of the room
- using map to calculate the robot's location in the room

79. **What is odometry?**
- The use of data from the movement of actuators to estimate change in position over time

80. **What parts are included in SLAM?**
- Landmark extraction (object recognition)
- Data association (matching observed landmarks from different scans)
- State estimation (stochastic map building (Kalman Filter used))
- State update (position estimation + displacement calculation)
- Landmark update
- Application areas (AR tracking, robots [iRobot])

---
## 05 - Mobile Augmented Reality and Head-Mounted Displays

81. **What are the components of a mobile AR system?**
- HW computational platform
- Display
- Tracking
- WIFI
- wearable input and interaction
- software

82. **Name devices used with AR**
- backpack setups
- smartphones
- tablets
- consoles

83. **What is used from a mobile device to track the device's location?**
- Communication technology
- WPAN (Bluetooth)
- WLAN
- WWAN (GSM and UMTS, LTE)

84. **Name the main challenges of mobile AR**
- limited computational resources
- size, weight
- battery size
- tracking and registration
- 3d graphics
- real time performance
- networking

85. **Name software for mobile AR**
- OpenGL
- ARKit
- ARCore
- mixare
- AndAR

86. **Name products of Head Mounted displays commonly known**
- HoloLens
- Google Glass
- Google CardBoard
- Samsung GearVR
- HTC Vive
- PlayStation VR
- Oculus Rift

86. **Name binocular cues**
- Accommodation, change of lens to bring object at certain depth in focus
- Vergence, movement of eyes to map an object on the corresponding side of the retina
- Binocular disparity, the difference in image location of an object seen by the left and right eye

86. **What is the Screen Door effect?**
- It looks like you’re viewing the world through a mesh screen, and is a result of the black, empty spaces between pixels when seen up close.

87. **How do optical see-through HMDs work?**
- mostly used in AR
- similar approach as Head up displays in cars
- real world light reduction
- blending virtual objects in
- optical blending requires no additional computing power

88. **How do video see-through HMDs work?**
- recording real world
- combining real with virtual world
- presenting on a "monitor"
- no depth info
- separate video streams for RL and virtual world
- resolution is limited

89. **Compare video and optical see-through hdms**
- Advantages of optical blending
- simple
- computational cheaper than video blending
- Field of view not a major issue
- distortion needs to be compensated
- display of real world is not degraded
- direct view of real world still visible in case of power outage
- no eye offset like on video blending
- Advantages of video blending
- fully obscured real world
- composition on pixel-by-pixel basis
- no semi-transparent systems
- easier to match brightness of RL and virtual objects
- RL and virtual view delays can be matched
- problem with brightness (washed out real world or virtual world)

90. **Compare smart devices, smart glasses and hdms**
- smart devices
- intuitive interaction
- have to be held
- mainly monoscopic
- smart glasses
- low weight
- reduced interaction possibilities
- very low FoV
- HDMs
- hands-free interaction
- encumbrance of the user
- low FoV

91. **What is a Field of View?**
- Field of View is the area the user can see through a HDM, AR Smartphone/Tablet or smart glass

91. **What different form of display technology exist?**
- Prism lenses
- WaveGuide Arrays
- Half silvered mirrors

---
## 06 - Spatial Augmented Reality and Tangible User Interfaces

92. **What is Spatial Augmented Reality?**
- Actual objects or real world surfaces are used as display area

92. **What is part of spatial AR?**
- Augmented Surfaces
- projection mapping
- industry applications
- virtual sandbox

93. **What do tangible user interfaces allow?**
- more users
- co-located interaction
- encumbering one user

94. **What can be used as user interface?**
- actual object surface
- RL world surface

95. **What are the advantages of tangible user interfaces and the real world displays?**
- display surfaces may not be just walls
- flexible projection config
- camera for calibration and tracking
- rendering using texture mapping hardware
- projector = fixed location

96. **Main problems with projector based tangible user interfaces**
- gemoetric correction
- color correction
- intensity correction
- display on arbitrary surfaces

### Augmented Surfaces

97. **What are Augmented surfaces?**
- using real world objects as extension for displaying information
- like table or wall
- interaction must be possible

98. **What does the term hyperdragging mean?**
- spatial dragging of virtual objects
- like drag'n' drop
- dragging from one display to another (other display is RL object)
- establishing a link between RL object and virtual object

99. **What components do we need for augmented surfaces?**
- Anchor cursor (link between RL object and virtual object)
- Object aura (area around physical object)

10. **What do we need and need to do to add virtual objects to a physical object?**
- A link between physical and virtual object is needed
- area around physical object is needed
- data attaching to physical object
- binding virtual object to aura of physical object

10. **What do we need for Augmented surface setups?**
- communication via WIFI
- projection on info table or wall
- identification of physical objects via markers and cam
- additional camera (tracking)

1. **What is projection mapping?**
- projection on surface or real life object like a house
- calibration is needed
- Complex calibrations corrections with help of projector camera (Procam)
- for advertisement or art projects

1. **Explain the term AR sandbox**
- kinect
- projector
- sandbox
- Depth images are captured
- heat map generation

1. **Name other augmented surface projects**
- Augmented climbing wall
- pico projectors
- mono-on-mono
- SixthSense and Wear Ur World

### Tangible User Interfaces

1. **What is a tangible user interface?**
- physical object representing an object in the virtual world
- may have physical controllers attached
- TUI provides access to virtual information
- how: intuitive physical manipulation
- try to integrate UI in surrounding world

1. **On what objects can a TUI be attached to?**
- solid
- liquid
- gases

1. **Name projects with TUI**
- TUISTER
- Toolstone
- marker based input
- iOrb
- magnifing lens
- Keyers
- ChairIO
- ShoeSoleSense

1. **How to build a TUI?**
- Potentiometer
- Arduino board

---
## 07 - Interaction

1. **What is interaction in general?**
- influence of two entities on each other
- interaction with environment occurs if user input is responded by the system
- action --> reaction

1. **What two types of interaction exist?**
- Abstract interaction (e.g. gesture using data gloves)
- natural interaction (direct connection; virtual object pick up)

1. **What other ways of interaction exist?**
- egocentric interaction
- exocentric interaction

1. **What are the differences between AR and traditional desktops?**
- no mouse or keyboard
- 2D input not ideal for 3D object manipulation
- multi-touch
- speech/gesture recognition

1. **What is marker based interaction?**
- use markers as pointing devices
- e.g. state change mechanisms
- visual representation by icons
- buttons/sliders
- Problems:
- The action which can be performed is not always obvious to the user
- Too many markers can be confusing

1. **How can marker based interaction be used?**
- shake
- rotation
- hiding/showing

1. **What can be visualized with marker based interaction?**
- Menus
- Arrows
- Symbols
- Texts
- Icons

1. **Tell the difference between manipulative and semaphoric gestures**
- manipulative: couple target with gesture
- semaphoric: symbolic/representation of static or dynamic gestures

### Motion Capturing

1. **What different types of motion capturing technologies exist?**
- markerless (Kinect, leap motion)
- simple markers (commercial systems)
- bodysuit (VR domain)

1. **What is Motion Capturing used for?**
- used for skeletal animation of virtual characters

1. **Explain hand tracking**
- Use of LEDs, reflectors or color markers (FTIP)
- data gloves
- collision detected by distance between finger position and virtual object
- multi-finger-detection: grabbing usable
- problem: detection of multiple hands

1. **Tell me something about gesture tracking**
- good for SAR and HMDs
- HoloLens, Meta, Leap Motion
- no defined starting point (problematic)
- Can lead to fatigue (weight of hand etc.)

1. **Explain head tracking**
- detection of nodding and head shaking
- knowledge about head position and orientation

1. **Explain body tracking**
- used for recognizing bowing down, sitting
- e.g. Kinect

### Gaze based interaction

1. **What is gaze based interaction?**
- common in VR
- Eye tracking
- gaze direction can be calculated by head transformation and eye tracking

1. **What is used if eye tracking not available?**
- approximation of gaze vector

### Speech based interaction

1. **Explain Speech based interaction**
- Good because hands-free
- complex sentences distract user from detailed scene
- avoid speech based interaction on heavy visual load

1. **Name the different types of speech based interaction**
- Commandos
- numerical input
- free recognition

1. **What approaches exist?**
- Commands + numerical input
- interpreting free recognition
- e.g. sphinx, julius

### Interaction Techniques

1. **Explain the Canonical 3D manipulation tasks**
- Selection –> the task of manually identifying an object
- Position –> the task of positioning an object from an initial to a final, terminal, position
- Rotation –> the task of rotating an object from an initial to a final orientation

1. **Explain interaction processes according to bowman**
- split up into three main tasks, Selection, Manipulation and Release
- main tasks consist of sub-tasks
- manipulation requires selection
- selection does not always lead to manipulation
- combine above mentioned techniques to create new ones

1. **Name the manipulation methods**
- Direct user control (interface gestures mimicking real world interaction
- physical user control (objects user can physically touch)
- virtual control (devices a user can virtually control)

1. **Which variables affect the manipulation?**
- distance to the object
- object size
- translation distance
- amount of rotation
- object density

1. **On what do manipulation techniques depend?**
- interaction tasks
- variables

1. **Explain the interaction technique Ray-casting**
- = selection technique
- ray cast from observer position in scene
- collision between ray and interaction object is calculated
- collision detection only in specific areas
- after selection give feedback to user

1. **Explain the interaction technique Aperture**
- = selection technique using hed and hand sensor
- selection cone is cast into scene
- depends on distance between hand and head size

1. **Explain the image plane techniques (an interaction technique)**
- use of hand, finger, head tracking
- Headcrusher (ray between head and center of an area)
- sticky finger (ray between head sensor and finger sensor)
- lifting palm (open palm below object)
- framing hands (palm and thumbs in corner of frame)

1. **Explain the hand tracking technique jog dial**
- fingertip recognition
- fingertip orientation tracking

1. **Explain the hand tracking technique jog dial + slider**
- fingertip recognition
- rotation
- slider recognition (sliding the fingertips)

1. **Explain the interaction technique "Studierstubes PIP (Personal Interaction Panel)"**
- interaction via tablet and pen tracked
- tablet/pen = tangible interface
- asymmetric interaction (dominant hand)

1. **Explain the interaction technique Pick-and-Drop**
- natural extension of drag and drop
- stylus used
- network communication
- switch objects from one device to another

### Visual Hints

1. **What components are used for visual hints?**
- tangible tooltips (on object recognition tooltip appears)
- tangible bubble (like tooltip but a speech bubble)
- ghosted hints (what action you can do with the object)

1. **What types of hints exist?**
- textual hints
- diagrammatic hints
- ghosted hints
- animated hints
- placing annotations in the environment
- Hedgehog labelling

### Menus

1. **How can a menu be represented?**
- textual
- graphically
- 3D Objects
- WIMP interfaces

1. **What are menus used for?**
- changing states
- adjusting scalar values
- choosing from objects/options

1. **How can a menu be placed?**
- world referenced
- object referenced
- head referenced
- body referenced
- device referenced

---
## 08 - Advanced Rendering

1. **What does advanced rendering do?**
- issue to map real world data in virtual world data together
- ideally in real time
- mapping virtual content correctly

1. **What needs to be taken into account for real world to virtual world mapping?**
- consistent lighting
- use of shadows
- realness factor of virtual experiences
- virtual people

1. **What two aspects of the above question can be taken care of in AR?**
- consistent lighting
- use of shadows

1. **Why can they be an issue?**
- just partial known environment
- occlusion between real and virtual environment
- lighting conditions need to match

### Stereoscopy

1. **What is stereoscopy vision?**
- 120° FoV
- binocular cues

1. **Name the binocular cues**
- Accommodation (change of lens to bring object in focus)
- vergence (movement of eyes to map object on the correspinding side of the retina)
- binocular disparity (objects with different distance from eye focus)

1. **Name the issues for binocular cues**
- accomodation-vergence conflict
- Focal plane is always on the display surface
- Accommodation is always set to get the focal plane sharp
- Vergence has to change due to disparity
- In real life accommodation and vergence always match
- When using a display they can’t match, unless the display provides different depth layers
- binocular rivalry ( perception alternates between different images presented to each eye)
- diplopia
- suppression

1. **Name the types of perspective projection**
- parallel (two cameras with off-set)
- toed-in (for each eye a camera, pointed towards single focal point)
- off-axis

1. **Name the techniques of rendering stereoscopic images**
- Temporal (Shutter glasses)
- spatial (HMD)
- color (Anaglyph)
- autostereoscopic displays (barrier / lenticular displays)

1. **Name the relevant topics for stereoscopy in AR**
- HMDs
- depth perception
- autosterescopic tablets

### Depth Perception

1. **What depth perception can occur?**
- oculomotor cues (binocular display)
- monocular (induced by motion)

1. **Name additional depth cues**
- Depth of field
- Motion parallax
- Occlusion
- Shadows
- Shading

1. **Name examples for monocular cues**
- relative size
- relative height
- familiar size
- texture gradient

1. **Name examples of motion induced monocular cues**
- deletion and accretion
- motion parallax

### Distance Perception
1. **Why are distances underestimated by users in a virtual environment?**
- measurement methods
- technical factors
- human factors
- environmental variables
- personal variables

### Visual Coherence

1. **What is a visual coherence?**
- consistent environment between virtual and real content
- seamless blending of virtual objects in real environment
- easier to handle with video based approach

2. **What are problems of Visual Coherence?**
- Occlusion
- Lighting and Shading
- Shadows

### Occlusion

1. **When does occlusion happen?**
- if virtual objects are in front of real objects
- if virtual objects are behind real objects
- one of the strongest depth cues and should be resolved

1. **What is one of the main challenge of AR?**
- correct occlusion between real and virtual objects
- because no info about real world coordinates available from the beginning

1. **Name approaches which solve this issue**
- modeling real world beforehand
- generating depth maps
- use of optical flow
- real world object detection and extraction

1. **What is phantom rendering?**
- phantom = virtual object represented as real object
- phantom rendered invisibly only the Z-Buffer is modified

1. **How is phantom rendering done?**
- draw image on color buffer
- disable writing to color buffer
- render phantoms on real scene in z-buffer
- enable writing to the color buffer
- draw virtual objects

1. **Explain the model free occlusion**
- use of depth sensors to generate depth map
- virtual content and real content merged in z-buffer
- real environment is not modeled beforehand
- can be freely modeled based on sensor input
- Approaches: special HW or assumptions

1. **What are the sources of inaccuracy?**
- phantom models not 100% accurate
- errors in static registration
- errors in dynamic registration
- correction can be done in image space (post processing)

1. **How can occlusion refinement be done?**
- edge detection
- Camera has edge detection
- AR application searches for edges
- Edge smoothing
- Polygon renderer displays rendered results
- Others compute optical flow between camera image and model

1. **What is probabilistic occlusion?**
- assumptions about the real world model
- tracking required for rough representation
- transparency increases from inner to outer models

### Lighting

1. **What is photometric registration?**
- consistent illumination between real and virtual objects
- knowledge of illumination of real environment is required
- remote light sources and local illumination are easier to compute
- consider reflection, refraction and shadows for rendering

1. **Explain image based lighting**
- HDR maps used
- representing illumination in physical units
- radiance & irradiance map

1. **What is a radiance map?**
- environment map representing light from the point of the observer

1. **What is an irradiance map?**
- environment map representing outgoing light after reflection

1. **Explain light probes**
- way to physically record radiance maps

1. **Explain the differences between active and passive light probes**
- Passive:
- reflective sphere placed to capture real world (300° FoV)
- determination of light sources
- Active:
- placing a camera in the scene
- record light conditions
- fisheye, 360° camera

1. **Explain the term offline light capturing**
- pre-processing light sources
- multiple images can be used to generate a radiance map

1. **When do we need photometric registration?**
- light probes too complicated
- offline capturing too complex
- instead using regular camera frames

1. **From where can we use data for photometric registration?**
- static images
- Help: user input
- specular reflections
- Reflection in human eye
- diffuse reflections
- Spherical harmonics
- Face detection with machine learning
- shadows
- Knowledge of casting geometry has to be known
- Has to be classified correctly
- outdoor
- Position and time for sun position

### Shadows

1. **What do shadows add to the scene?**
- level of realism to rendered image
- visual clues to determine spatial relationships

1. **Name the requirements for shadows**
- real objects have corresponding 3D geometry (phantom model)
- known light source
- 4 different configurations available

1. **How are shadows displayed?**
- shadow volumes (project a ray from the light source through each vertex of a shadow casting object into infinity)
- shadow maps (Scene is rendered from the view of the light source)

### Diminished Reality

1. **What does diminished reality mean?**
- taking objects out of the real world
- by superimposing graphics

1. **Which three problems address diminished reality?**
- determination of region of interest (dynamic set of pixels)
- observation of hidden areas (view of background behind the ROI has to be integrated)
- new content for removed content

1. **Tell me something about the region of intereset**
- Region of interest is a dynamic set pixels on the screen to be replaced
- It can be freely visible or partially or fully occluded
- User may outline ROI, use strokes, selection rectangle
- Object behind ROI can be used to define ROI
- ROI could also be an articulated object which has to be tracked to be removed

### Stylized Rendering

1. **What is stylized rendering?**
- emphasizing regions / aspects of the scene
- - can be used for visualisation of art projects

1. **Which types of emphasizing exist?**
- cartoon shading
- pencil sketch shading
- painterly rendering

---
## 09 - Collaborative Augmented Reality

1. **What is CSCW?**
- Computer supported cooperative work

1. **Name problems of CSCW**
- Disparity between people who work and the ones who get the benefit
- Breakdown of intuitive decision making
- seams (spatial, temporal functional constraints)
- major categories of seams:
- functional
- Discontinuities between different functional workspaces
- Seams between shared and personal space
- Force the user to change modes of operation
- cognitive
- disruption between existing and new work practices
- Abandoning acquired skills and learning new skills
- Forcing the user to learn new ways of working
- Often rejected if users have to change the way they work

1. **To minimize seams CSCW should support what?**
- support of existing tools
- support of existing working techniques
- visual/audio communication between participants
- Collaborators must be able to maintain eye contact and gaze awareness
- Users must be able to bring real world objects into the interface

1. **Which types of collaboration spaces exist?**
- interpersonal space (video conferences)
- shared workspace

1. **For collaborative AR - which setup possibilities do you know?**
- co-located (same physical space)
- geographically dislocated

1. **What are the issues with VR HW concerning CSCW?**
- images rendered for one viewpoint
- human has to be rendered as avatar
- low portability

1. **Advantages of AR in CSCW?**
- users can see each other's facial expression
- reference of real objects
- no full rendering of environment needed
- real users and virtual world on one display

1. **Name the five key advantages of collaborative AR environments compared to traditional VR setups**
- Virtuality
- augmentation
- cooperation
- independence
- individuality

1. **Name aspects of collaborative AR users?**
- typically co-located
- no issues with viewpoint control
- private and public space sharing has to be considered

1. **Name aspects of collaborative AV and VR users?**
- typically dis-located
- users immersed in VE
- viewpoint problem
- VE should replace real world

1. **Advantages of AR collaboration?**
- shared virtual content
- perception and sharing of stereoscopic 3D data
- seperation between public and private space
- inclusion of tangible, real-world objects

1. **Issues of AR collaboration?**
- small FOV

1. **Name different coll. AR Applications**
- TeamWorkStation
- Translucent overlay of live video images of computer screens, two cameras for faces, private + public space
- ClearBoard
- Projection of video stream of remote user directly on the display
- AR^2Hockey
- Mixed Reality Stage
- Projection of virtual characters and models in a real down-scaled environment
- Transvision
- Visualisation of CAD data in a shared environment
- cAR/PE!
- WearCom
- Collaborative Web Space
- 3D web browser, commands via voice
- blue-c
- Empathy Glasses
- VITA
- offsite visualisation of an archaeological dig, combination of various displays
- Display in WIM and life size
- AR Tennis
- Co-located communication via Bluetooth, multi modal
- CheckMate
- HoloPortation

1. **What are awareness cues of AR Collaboration?**
- users have to perceive remote users action
- view frustum visualisation
- gaze vector visualisation
- potential issue: visual clutter

1. **Name awareness cues of AR Collaboration?**
- Empathy Glass

1. **What are hybrid solutions?**
- simular issues to Networked Virtual Environments
- real objects hard to transfer into virtual space
- AR users represented in VR/ VR awareness parameter visible in AR
- General problem of awareness parameters is visual clutter
- prototypes make us of hand presentation and eye tracking

1. **Networked Virtual Environments?**
- Researches in:
- Scalability
- Responsiveness
- Collaboration
- Architectures
- Communication protocols
- Human factors
- Provide useful and interesting asset to coll. AR

1. **Design Guidelines for Collaboration in MR?**
- as much information of the remote environment
- 3D Mesh of environment
- updated 3D Mesh in realtime
- provide independent PoV
- as much Awareness cues as possible
- transmitting speech
- information of posture
- poiting of hand should be modelled
- cues for events outside due FOV
- possibility to turn awareness cues on and off
- usability and comfort
- comfortable interface

---
## 10 - Evaluation Basics

1. **Evaluation in general**
- important for app usability
- performed by experts
- three groups preferred (HCI experts, developers, users)

1. **What are typical problems for evaluation?**
- Evaluation and user studies are one of the main topics in the humanities community
- developers typically don't come from humanities community
- demand for guidelines is high

1. **What are typical approaches of evaluation?**
- expert studies
- empirical studies
- questionnaires during and after use
- observation of the test subjects during interaction

1. **When and what data should be collected for evaluation?**
- beforehand:
- goal of a study
- setup of tests
- number and nature of tasks to complete
- during evaluation
- completion time of tasks
- afterwards:
- statistical evaluation
- graphical evaluation

1. **What possible uses of psycho-physiological and neurophysiolocig measurements?**
- estimation of mental workload
- stress
- strain
- level of cognitive performance
- alertness
- arousal

1. **How can these measurements be performed?**
- Electroencephalography (EEG)
- Event-related potentials (ERP)
- Electrocardiography (ECG)
- Skin conductance response (SCR)

1. **Which statistical analysis methods/tools are used?**
- Arithmetic mean
- Median
- Frequency distribution
- T-test
- Bonferroni
- ANalysis Of VAriance
- Variance
- Standard Deviation
- Quantiles, quartiles and percentiles
- Displaying data: charts

1. **What data should be gathered and prepared for evaluation?**
- amount of participants
- gender
- age
- profession
- degree of experience
- forms
- schedule
- test plan

2. **How to evaluate MR apps?**
- investigate human activity
- participant observation
- in situ interviews
- user recordings (video)
- collaborative viewing of recordings of a group
- transcription
- use of the three design loops:
- Loop1:
- User requirements
- Technology opportunities
- Selection and evaluation of solutions
- Loop2:
- usability studies (not always needed)
- proof of concept
- use of virtual prototyping
- Loop3:
- ergonomic research
- general evaluation

1. **Examples of MR App evaluations?**
- Mixed Reality stage
- Lighthouse Trial

1. **What are the development domains of MR App evaluations?**
- Behavioural domain - user centered approach
- Interaction with the application
- Usability of the user interface
- Application content
- constructional domain
- Api development
- every domain has experts

1. **Whats the 4 step sequential analysis?**
- User task analysis
- Expert guidelines-based evaluation
- Formative user-centred evaluation
- Summative comparative evaluations

1. **Standards for Questionnaires?**
- NAS Task Load Index
- Subjective assessment tool to rate the perceived workload of the user
- System Usability Scale

1. **Aspects of evaluating with wearable computing applications using a HMD?**
- for outdoor applications
- main issue: supervisor is not able to see what the subject sees
- interconnection of HMD via WIFI to supervisor's laptop

1. **Problems with AR applications?**
- Latency
- Depth perception
- Adaptation
- Fatigue and eye strain