The Sum of the Parts

By Andrew Myers

From the Apple Watch to Fitbit, the rage in technology these days is all about wearable, touchable devices. Tech gurus and cutting-edge engineers talk of a day when all the many electronic devices that surround us will be networked together in a magical electrical collaboration that will change how we live our daily lives. Many of these devices, if not all, will be controlled by touch.

Some have taken to calling this vision the “Internet of Things,” but others have begun to believe even that grandiose thinking may be too narrow. The “Internet of Everything” is more accurate, they say.

Unfortunately, for millions of people who are paralyzed or challenged by neuromuscular diseases, that vision of an “everything” Internet may be largely (and literally) out of reach.

The myriad touch-based interfaces that are filling, and will continue to fill, our world will be lost on them. And so, too, will the ability to control their lives and, perhaps, the dream of an independent life.

Buz Chmielewski is one of those people. At seventeen, a surfing accident confined him to a wheelchair. His legs no longer worked, but Chmielewski retained some movement in his arms and hands.

Putting that “Internet of Everything” into play for Chmielewski and others like him is what motivates a team of UMBC engineers working together as the ECLIPSE Cluster.

“Ironically, these are exactly the people who need these devices most of all, but they won’t be able to access or control the electronics that surround them if they can’t use a touch device,” says Nilanjan Banerjee, an assistant professor of computer science and electrical engineering at UMBC. “We’re trying to give them that control of their lives.”

Indeed, the numbers of those who need such help navigating new technologies are bigger than most would imagine. In the United States alone, some 1.5 million people are hospitalized yearly just from spinal injuries, strokes, and traumatic brain injuries.

So the researchers who make up the ECLIPSE Cluster are designing and building devices that will provide greater facility of use of electronics and other technology.   They are collaborating with willing partners like Chmielewski and other researchers – including those at the University of Maryland, Baltimore – to find out if they work and how they can be improved.

With his fellow professors of engineering Ryan Robucci and Chintan Patel, and a group of eager students, Banerjee is imagining ways that the highly connected world of tomorrow and designing products today that could vastly improve the lives of people like Chmielewski.

MANUFACTURING TOUCH

The touchscreens and touch-based wearables at the heart of technological innovation don’t work very well for people with motor control issues.

Touchscreens are too rigid and too fragile for many of these users on a day-to-day basis and they are too sensitive to inadvertent motions to be practical for people with motor control problems.

“We’re working on touchless gestural devices that use sensors similar to what is used in your smartphone to control household electronics,” says Banerjee. “But in our case, the hand or arm only needs to be in the vicinity of the pad, not actually on it, for the sensor to detect the motion.”

So why not use flexible fabric-based touch interfaces? The ECLIPSE researchers observe that while this technology solves the first two issues, it raises a third one. These interfaces require direct and repeated contact with the fabric, which can cause abrasions from the constant rubbing, often without notice to users who may lack sensation in the extremities.

“These abrasions can become severe, so we really wanted a device that did not require the user to make physical contact,” Robucci says.

Working closely with Sandy McComb Waller, an associate professor of physical therapy and rehabilitation science at the University of Maryland School of Medicine, the ECLIPSE Cluster has designed and developed a fabric integrated with sensors that can read a person’s slightest gesture.

The ECLIPSE device is unassuming in appearance. It looks like a simple patch of fabric about the size of an electric heating pad. Woven into the fabric, however, is a grid of electrically conductive threads that can sense the changing electrical charge when a human hand or finger passes close by.

In engineering terms, these are known as capacitive sensors. They work in a way similar to the touchscreen technologies in smartphone and tablet computers, but they don’t require the user to physically touch the screen.

“In this case, close is more than good enough,” Banerjee says.

Such controllers might one day be integrated into bed sheets, pillows, wheelchair pads, or clothing. Their potential uses? Manipulate light switches, televisions, thermostats, or even make a 911 call. Virtually any electronic device, even a computer, could be controlled from this one device.

TRAINING THE DEVICE

The flexible fabrics hold many advantages over their glass-screened cousins. First off, they conform to practically any surface without any loss of sensitivity. A user could wear it over a knee one day, on a forearm the next, and the device would work the same. No rigid glass touchscreen could ever be so versatile.

Second, the ECLIPSE device is trainable. “We started out with predefined gestures, but the people we are designing for have limited range of motion and most have limited motor control, so they didn’t like it. It was too difficult for everyone to learn the same motions,” Robucci says.

Instead, the ECLIPSE team recalibrated. They began designing algorithms that help the users record their own gestures and to tell the computer what the gestures mean.

“Sometimes, a user is capable of only a finger twitch, but we can even map those motions to specific instructions. People will be able to control the devices around them with the slightest of movements,” Banerjee says.

Best of all, perhaps, the ECLIPSE sensors are ultra-low power and extremely affordable, costing just 5 to ten cents per device to produce.

Down the road, Banerjee talks of forward-looking applications that will help not only those with physical challenges, but just about everyone on the planet.

“You could imagine putting these sensors in car seats, where a driver might be falling asleep at the wheel or distracted looking at a smartphone. Our capacitors can be embedded to detect whole body motion: Is the driver facing the road? Is he is on a phone? Is she falling asleep?” Banerjee explains. “Instead of an Internet of Everything, we’ll have an Internet of Everyone.”

Banerjee adds that these or similar systems based on similar technologies also could be deployed in life-saving situations, such as in cribs to detect breathing patterns that could prevent Sudden Infant Death Syndrome (SIDS), a mysterious killer in which infants simply stop breathing in their sleep.

GETTING END RESULTS

Things are beginning to blossom for ECLIPSE, especially as researchers and funders outside the close-knit UMBC community to begin take note of their work and invest in it.

Last August, the National Science Foundation and the National Institutes of Health placed a big bet on their work, seeding the group with a $650,000, three-year grant to fuel their research. TEDCO, a program sponsored by the State of Maryland to facilitate transfer and commercialization of technology flowing out from the state’s research labs, soon followed with an additional $150,000. Microsoft Research and its “Lab of Things” rapid prototyping program are also supporters of the ECLIPSE work.

Robucci says that one key to the success of the ECLIPSE Cluster and its researchers are their focus – and capacity – to see the big picture and make it happen within their own lab. Banerjee is the software expert. Robucci focuses on analog to digital signal processing that turns the real world into data a computer can understand. And Patel is a chipmaker – an expert in very large-scale integrated circuits. The goal of ECLIPSE is to build end-to-end cyber-physical systems.

“We’re working on complete systems here, not on individual components,” observes Robucci, “so we have to be able to imagine the end result – the product – and then work at all levels…to design and optimize the components and algorithms to make it a reality.”

Though ECLIPSE began as a loose affiliation of like-minded engineers, it is quickly growing into something more – an incubator of ideas and solutions to some of the most challenging questions of the day.

“If you’re working alone you often think, ‘Can I do this? Is this possible?’ and then you have to find the answers on your own,” Robucci says, “but at ECLIPSE we’re sort of all-inclusive. If you ask the same question, someone else has the answer, ‘Yeah, we could do that.’”

Banerjee agrees: “I couldn’t do it alone, and neither could Ryan or Chintan. Each of us brings unique and important expertise to the equation and the sum is greater than the parts. It’s as if you added three individuals together and you end up as four.”