Putting the ‘A’ in STEAM: Professor Aditi Majumder uses Computer Science to Amplify Art

UCI Professor of Computer Science Aditi Majumder’s, Ph.D., journey into the virtual/augmented reality world stems back to a young girl growing up in Kolkata, India around her mother, a professional musician and her father, a civil engineer. Both parents unknowingly planted the hybrid seed of art and science, which led Majumder to pursue a blossoming career in virtual/augmented reality that intertwines with her deep appreciation for art and music.

Through her research interest in novel display technologies and computational cameras and projectors, Majumder would later be instrumental in developing a very successful digitization platform for a complimentary domain of augmented reality that does not involve VR glasses, and instead incorporates projectors,–or projection-based augmented reality. The platform’s development involved about $1.5 million of research funding over 12 years, close to 70 publications in premier venues, four “best paper” awards, eight patents and finally, Summit Technology Laboratories–a company to commercialize the platform she co-founded in 2016.

In 1996 she began her path to a Ph.D. at University of North Carolina, Chapel Hill—a top school for VR. Her advisor, Henry Fuchs, Ph.D.,  often known as the “father of virtual reality” in the computer graphics community, and Greg Welch, Ph.D., turned their focus from wearable VR gear (“wearables”), to VR sans wearables. She was one of the first student members of the “Office of the Future” project, which, during a decade dedicated to flannel and dial-up internet, was devoted to building the technological concepts needed to make projection-based AR.

“The dream was to be able to track the user and augment digital content from multiple projectors on top of any real life object and then create a feedback loop between these two so that users can trigger change in the augmented digital content,” said Majumder. “My Ph.D.’s thesis focused on putting these different projectors together to make the color appear as if it were coming from a single device.”

Majumder joined several trailblazers at UNC to work on a then niche technology that involved projectors and cameras with long pre-calibrations, which still requires a trained person to set up the projection–a process that is costly and very time consuming.

As part of her thesis, she worked in the Argonne National Laboratory, operated by the University of Chicago, for two years and, after graduating, Majumder joined UCI’s Computer Science Department in 2003.

“My goal was to get this same technology done at a low cost on very unconstrained devices,” said Majumder. “As I started as junior faculty in 2003, my research direction at UCI was to develop a comprehensive platform that would enable cinematic quality projection-based AR using low cost off-the-shelf devices and automated deployment by relatively everyday users.”

As her work progressed at UCI, Majumder’s expert reputation provided exceptional consulting opportunities for several technologically-focused companies. She worked with startup Ostendo Technologies to build the first projection-based curved screen desktop display, which was later marketed by NEC/Alienware and won the “Best Display” award at the Consumer Electronics Show in 2009. Majumder also consulted with Disney Imagineering, Cubic Defense, an integrated technology solutions provider for transportation and defense, and UC Santa Barbara’s AlloSphere. The AlloSphere is a three-story research facility that uses multiple modalities to represent complex data.

In 2009, she received a National Science Foundation CAREER Award and $700,000 for her research “Ubiquitous Displays via a Distributed Framework,” which allotted funds until 2015 to further develop the technology. These funds allowed her and her Ph.D. students to dig deeper into VR research and, from 2009 until about 2015, determine the possibilities for the VR immersive technologies, such as simulation training, hospitality, retail, education, and entertainment. Following these funds, in December 2016, her company, Summit Technology Laboratories, received a SBIR Phase 1 award of $225,000, which served as the company’s main funding source in 2017.

Summit Technology Laboratories

Summit Technology Laboratories (STL) was developed in 2015 and in 2017, operations gained more traction from additional SBIR Phase 1 funding.

STL focuses on multiple interactive projectors using projection mapping, which is a projection technology used to turn irregularly shaped objects into a display surface.

Today, projection-based experiences are carefully curated and set up as prohibitively expensive installations—think: Disneyland’s Cinderella Castle during nighttime fireworks show, which can only be “seen” en masse rather than as an aesthetically interactive experience. STL aspires to bring this technology to the masses by enabling automated quick deployment and multiple interaction modalities (e.g., laser pointers, hand gestures, mobile devices) in addition to the use of multiple inexpensive projectors to create a similar high quality interactive experience with the additional capability of multi-user interaction.

Niche projectors would significantly bring down the cost of these experiences.

Through STL, Majumder brings these technologies to the mainstream, showcasing colorful artistic projections at public events, such as her team’s premier feature at the Segerstrom Centers for the Arts “Brilliance: A Night of Music and Light” event in April. Attendees were prompted to use a laser pointer to pop colorful balloons on a concave projection screen, download photos of the event to a kiosk display and interact with the multi-projector display, which featured several colorful art pieces projected onto a vase.

Projection-Based AR

Projection-based AR provides the ability to view a large amount of data at a high resolution. The image is formed by several projectors, each of which covers a part of the projection surface. By observing specialized patterns from the projectors using one or more cameras, STL’s patented projection aggregation and integration software (PRAIS) provides seamless imagery that can cover any complex surface, creating a unique visual experience for the user.

“The most innovative aspect of STL’s technology lies in using the same cameras to track user interactions (via hand gestures or laser pointers) to now enable multiple users to interact simultaneously with the projected 3D surface,” said Majumder.

Therefore, STL’s technology transforms 3D objects or spaces of any shape and size, from cups and vases to cylinders, domes and rooms, into magical aesthetic spaces for shareable and immersive experience for multiple users. When using projectors, the technology turns any 3D shapes into a display.

It can be used on unlikely shapes like surfaces of buildings and large and small abstract art forms. These projectors can augment any object, like a relief map to show the effect of flooding on a landscape model. Mini projectors, PCs, or foldable or inflatable screens can be used to create ultra-high resolution portable displays for varied applications. STL envisions people creating their own extremely light-weight 4K displays for mobile devices. Their displays may be used in immersive environments, such as flight simulation, museums, commodity planetariums, offices, and public venues.

“Flat is yesterday,” said Majumder, which is her company’s motto. “The goal is to put interactive displays anywhere at any time.”

STEAMin’

Majumder is very passionate about and a big proponent of incorporating “art” into the acronym “STEM,” or science –technology-engineering-math. Majumder attributes her natural knack for music and dance to the environment in her childhood home, which was constantly filled with music, dance and innumerable performing arts students due to her mother’s longtime career in Indian music and music training.

“Growing up, I never realized how much I picked up these skills,” said Majumder. “It was just osmosis.”

Majumder and her husband, M. Gopi, associate dean of student affairs and UCI professor of computer science, Ph.D., and whom she refers to as her “greatest partner in crime,” are connoisseurs of good music, dance and movies that form the primary hobby and source of entertainment for their family. Gopi is co-founder of STL along with Majumder. They met as Ph.D. students at UNC Chapel Hill and continue to work together even today, both in research and STL.

“He is the man behind the woman,” said Majumder with a twinkle in her bespectacled eyes.

Majumder has two daughters, four and 13 years. Her 13-year old loves math, music and dance alike – piano, choir and Indian classical dance being her primary non-academic activities.

“Whether it is research or STL, my artistic self helps me incorporate aesthetics into any prototype or system,” says Majumder. “Engineering and systems design need a tremendous amount of focused and detailed scientific endeavor. Amidst that, engineers tend to forget their important role in creating an aesthetically radical expression of human creativity. With the combination of engineering and art, we get a magical technology.”

As an educator and an entrepreneur, Majumder has always strived to make scientists and engineers who work with her aware of this reality and she believes that an engineer with an artistic bent is much better off than without.

In her work, Majumder is always aware of the aesthetic benefits of user-friendly design when performing interactive systems for ground-breaking user experiences, namely for clients who work with graphic visual displays. The hope is that these clients should view STL’s products as tools that help them convey complex concepts and data to the “everyday” user with the most appealing interfaces – both from a usability and aesthetically pleasing standpoint.

“The backbone of the technology is central – it has to be solid and general,” said Majumder.

“Yet, without a good artistic eye for design of the user interfaces, the technology will be locked up in a research lab and not see the light of the day.”

Read the full article at: http://innovation.uci.edu/2018/05/putting-the-a-in-steam-professor-aditi-majumder-uses-computer-science-to-amplify-art/