An interdisciplinary course covering emerging immersive media, HCI, computer vision, graphics, and computer networks. In addition to lectures, the course includes experiential sessions with a variety of state-of-the-art spatial computers like AR/VR headsets and smart glasses.
- Fundamentals: computational and human spatial intelligence
- Fundamentals: visual media, spatial computers, legacy computers
- Information, digital world representations, and Internet
- 2D, flat/stereo 360, 3D and volumetric videos: RGB-D, point cloud, mesh, NeRF
- Monocular, stereoscopic, and multiview videos, and world models
- Acquiring immersive content for network delivery
- Compression for RGB, depth, point clouds, and mesh sequences
- Streaming fundamentals: stored, live, and interactive protocols
- Streaming immersive content: videos, point clouds, meshes, holograms, spaces
- Visual and wireless sensing for human motion tracking
- Building immersive systems such as 3D telepresence and spatial web applications
- Tracking eyes, hands, face, head, body; outside-in and inside-out systems
- ARKit, Unity, Open3D, Open4D, Pytorch3D, and networked AR/VR platforms