EECE 5698: Networked XR Systems, Spring 2024

Logistics

Instructor: Mallesham Dasari
Office: 650D EXP Building
   
Class Times: TuFr 9:50AM-11:30AM
Class Room: Room 129, Forsyth Building
Office Hours: WedTh 4:00PM-5:00PM; also on Appointment or open doors
Contact: m.dasari@northeastern.edu

News

Class Description

This is an interdisciplinary course covering the following topics from emerging multimedia, computer networks, vision and graphics. In addition to the regular lectures, the class will also have experiential sessions with a vareity of state-of-the-art XR headsets in the market.

Grading Criteria

Tentative Class Schedule

Subject to minor tweaks throughout the semester.
Date Topics Lecture slides & Readings Notes
01/09 Introduction, networked applications, properties, basics of XR systems. Lecture Slides
01/12 XR headsets, internals, hardware, software, and tools. Lecture Slides , Open3D basics. Homework1 out. Due 01/22.
01/16 Sensors, cameras, depth sensors, lidars, sensing, algorithms. Lecture Slides
01/19 3D data structures, point clouds, depth maps, geometric meshes, neural representations, mono, stereo, and multiview. Lecture Slides
01/23 Capturing 3D data for network transmission, outside-in and inside-out capture, latency and bandwidth trade-offs. Lecture Slides
01/26 Compression fundamentals, 2D video compression. Lecture Slides
01/30 Depth map compression, adopting 2D video codecs, standalone depth compression. Lecture Slides , Depth compression methods based on standard video codecs. Homework2 out. Due 02/12.
02/02 Point cloud compression, MPEG VPCC, GPCC. Lecture Slides
02/06 Geometric mesh compression, Draco, Inter-frame mesh compression. Lecture Slides
02/09 Machine learning advances in compression. Lecture Slides
02/13 Class canceld due winter storm.
02/16 Streaming fundamentals, on-demand, conference calls, live broadcasting. Lecture Slides
02/20 2D video streaming, adaptive bitrate algorithms. Lecture Slides Homework3 out. Due 03/01.
02/23 360-degree video streaming. Lecture Slides Project idea due.
02/27 Point cloud streaming, point cloud quality metrics, adaptive algorithms. Lecture Slides
03/01 Mesh streaming, decimation, mesh quality metrics, adaptive algorithms. Lecture Slides
03/05 Spring break, No class
03/08 Spring break, No class
03/12 Progressive transmission of XR content. Lecture Slides Homework4 out. Due 03/21.
03/15 XR Experiences Session
03/19 Class in and about Metaverse.
03/22 Instructor out of town, No class
03/26 Rendering basics, Real time rendering, Performance, Rendering offloading. Lecture Slides
03/29 Edge rendering, local streaming, WiFi, mmWave, THz, optical links, challenges and opportunities. Lecture Slides
04/02 Hybrid rendering, optimal scheduling, WebRTC. Lecture Slides Project midterm evaluation.
04/05 Tracking Fundamentals: Eye tracking, head tracking, outside-in, inside-out tracking Lecture Slides
04/09 Tracking Fundamentals: Hand tracking, full body tracking, face tracking Lecture Slides
04/12 RF Tracking, sensor fusion for tracking, RF, Visual and IMU sensor fusion. Lecture Slides
04/16 Instructor out of town, No class
04/19 Advances in neural rendering, implicit representations for XR systems. Lecture Slides
04/26 Final project submission.

University Statement

Academic Integrity

A commitment to the principles of academic integrity is essential to the mission of Northeastern Uni-versity. The promotion of independent and original scholarship ensures that students derive the most from their educational experience and their pursuit of knowledge. Academic dishonesty violates the most fundamental values of an intellectual community and undermines the achievements of the entire University.

As members of the academic community, students must become familiar with their rights and respon-sibilities. In each course, they are responsible for knowing the requirements and restrictions regarding research and writing, examinations of whatever kind, collaborative work, the use of study aids, the appropriateness of assistance, and other issues. Students are responsible for learning the conventions of documentation and acknowledgment of sources in their fields. Northeastern Uni-versity expects students to complete all examinations, tests, papers, creative projects, and assign-ments of any kind according to the highest ethical standards, as set forth either explicitly or implicitly in this Code or by the direction of instructors.

Go to http://www.northeastern.edu/osccr/academic-integrity-policy/ to access the full academic integri-ty policy.

Student Accommodations

Northeastern University and the Disability Resource Center (DRC) are committed to providing disability services that enable students who qualify under Section 504 of the Rehabilitation Act and the Ameri-cans with Disabilities Act Amendments Act (ADAAA) to participate fully in the activities of the universi-ty. To receive accommodations through the DRC, students must provide appropriate documentation that demonstrates a current substantially limiting disability.

For more information, visit http://www.northeastern.edu/drc/getting-started-with-the-drc

Diversity and Inclusion

Northeastern University is committed to equal opportunity, affirmative action, diversity and social jus-tice while building a climate of inclusion on and beyond campus. In the classroom, members of the University community work to cultivate an inclusive environment that denounces discrimination through innovation, collaboration and an awareness of global perspectives on social justice.

Please visit http://www.northeastern.edu/oidi/ for complete information on Diversity and Inclusion.

TITLE IX

Title IX of the Education Amendments of 1972 protects individuals from sex or gender-based discrim-ination, including discrimination based on gender-identity, in educational programs and activities that receive federal financial assistance. Northeastern’s Title IX Policy prohibits Prohibited Offenses, which are defined as sexual harassment, sexual assault, relationship or domestic violence, and stalking. The Title IX Policy applies to the entire community, including male, female, transgender students, faculty and staff. In case of an emergency, please call 911.

Please visit www.northeastern.edu/titleix for a complete list of reporting options and resources both on- and off-campus