Human Machine Symbiosis | Pattie Maes | TEDxBrussels

Human Machine Symbiosis | Pattie Maes | TEDxBrussels Experts

Translator: Kim Paulissen Reviewer: Denise RQ A major milestone took place in the history of humanity about ten or 15 years ago. We became cyborgs. At the age of 12 or 14, kids get a smartphone and forever after, they have that phone or the successor with them day and night, often consulting the device more than 100 times a day.

When we used to talk about the two halves of our brain, we used to have a picture in mind of the left and the right hemisphere, but increasingly, this is the picture that we should be thinking about: the digital and the biological brain. Increasingly, we rely on our digital brain to answer questions, when we are asked something, rather than using our biological brain to come up with a solution. Unfortunately, the two halves of our brain don’t really talk to each other very well.

The communication or the interaction is very limited and very inefficient. One problem is the bandwidth between the two. We mostly just use our two fingers, our two thumbs, to input into the device, and we rely on the tiny, little screen to receive output from the device.

Second, these devices require our complete attention to be operated, which means that we can no longer pay attention to whatever is around us in the physical world: the people that we are with, or maybe the tree that we are about to walk into. Last, these devices and the software really haven’t been designed with the goals of people in mind. The different apps on your smartphone are all competing for your attention, trying to distract you as much as possible and basically, trying to make you as inefficient and ineffective at accomplishing your goals as possible.

And this really results in a lot of multitasking and very short attention spans. I’m not the first to notice that this is a problem, and people have generally responded in two different ways. One type of people will say that we should unplug. We should take breaks from our technology and ween us from our dependence on these technologies before it’s too late.

While another camp of people have argued that we should embrace this development and that really, it is people’s true destiny and natural destiny, really, to augment themselves with evermore powerful tools and technologies and to ultimately merge with technology itself. You can’t me in the embrace camp. I believe that we will never go back, and that we will continue on this trajectory of merging with the machine, becoming one with the machine.

However, I believe that what is needed is that we radically and urgently redesign the experience of how a person interacts with their digital devices. I’ve for a while now argued for a new approach, a new type of experience for how we work with our smartphones, our laptops et cetera. And there’s really three ways in which this new approach is different from what we are used to today. The first difference is that the system is very aware of the user’s context and also, the user’s internal state. It uses sensors to know what is around the person, what the person is paying attention to, et cetera, as well as sensors to notice maybe what the brain activity of the user is, what their heart rate is, their breathing rate, et cetera.

Second, the device should be always on, always augmenting, really, the experience that the user has of their surroundings by for example, using augmented reality glasses, but basically, in very seamless, non disruptive ways. And the third difference with today’s devices that we all use is these new types of symbiotic devices have to be more proactive. They have to basically make suggestions, recommendations, and offer relevant information and interventions, given whatever it is that the user is currently trying to do, given the state of the user, and given the goals of the user.

So in my previous TEDx talk, seven years ago, I gave an example of a system that illustrates this approach, the sixth sense device. Basically, the device consists of a camera and a projector that you wear around your neck, and the camera constantly notices what the user is doing, and what they are trying to accomplish. And the projector finds relevant information and interfaces and makes those available by projecting them onto the objects you have in your hand or onto your hands themselves. It can give you audio augmentations, and so on, as well.

So notice that the two experiences here are completely synergistic and integrated. The person experiences their physical surroundings at the same time as benefiting from all the digital information and services. I believe that this new approach, this symbiotic relationship between a human and a machine will help people with self-actualization. It will help them with growing and developing into the people they want to become.

All of us want to change and grow in some way. Some of us may want to become more efficient, better at communicating, better at relationships. Others may want to be thinner, healthier, happier. Yet, others may want to learn some new skills, some new knowledge, some new languages.

And today, we rely on books, or posts, or maybe we consult a therapist or a coach to help us change and develop in the direction we want to develop. But I think that increasingly, we will actually rely on these more symbiotic forms of our personal devices to help us change. I’ll give you some examples in the rest of the talk of some experiments that we have been doing at my laboratory at MIT that show how these more symbiotic interfaces can help us with making decisions, or changing the way we make decisions, learning, remembering, and regulating our mood.

First, take decision making. Behavioral economists have shown that up to 90% of the decisions we make and the actions we perform are the result of a very intuitive, fast system that often results in actions and decisions that are suboptimal, that are not really in line with our own goals. We believe that these symbiotic devices can help us keep our goals and maybe our New Year’s resolutions in mind, when we are just out in the real world performing, making some decisions. Here’s an example of a system that uses the Microsoft HoloLens augmented reality headset to constantly look at what decisions the person is making. When you look at, say, the Coca-Cola bottles for more than three seconds, it assumes you may want to take one of these Coca-Cola bottles off the shelf and can give you some real time feedback, trying to encourage you to keep your goals in mind of maybe consuming less sugar.

Of course, for every person, the goals may be different, and you would tell the device what your specific goals are that you want it to make you more mindful of. These devices can also help us with learning new skills, new languages, new knowledge. We’ve been building again using augmented reality devices, we’ve been building systems that let you learn in the real world. So, instead of reading a book, or having to sit and watch a post, or taking a class, you actually, if you want to learn a language, go and explore the world, and everything around you can be labeled in the language that you are interested in learning.

The system also knows what words you already know, you have already learned, can give you new words for the same object, or more complex words, sentences, et cetera. Not all of these devices have to rely on a smart AI system to make these real-time augmentations. In this system, we use a Google Glass headset, and the person on the left constantly streams whatever is in front of him – in this case, in the supermarket – to another person, who may well be on the other side of the globe, who is labeling all the objects in the language the person wants to learn.

For example telling them, “This is called a grapefruit, whatever you’re holding right now. And that’s called an apple,” et cetera. You can even talk to that person, ask them some questions about how to interact with a clerk, for example, if you need some help. Similarly here, this augmented reality system basically makes it possible for a user who needs help, in this case, operating a laser cutter, with calling up an expert, and the expert can make her hands show up in real time in his visual field, so she can actually instruct him what buttons to push, how far to slide, different sliders, what gestures to perform, et cetera, while also talking to him about what it is he is trying to accomplish. Symbiotic devices will also help us with our memory.

This is a system which constantly records the last couple of seconds, or minutes rather, of your experience, so that if you missed something that someone said, you can quickly hit rewind, and listen, and see the experience again. Or it uses a wristband that detects when you shake someone’s hand. And it automatically records the face of the person and the name of the person, every time you basically shake someone’s hand.

Translator: Kim Paulissen Reviewer: Denise RQ A major milestone took place in the history of humanity about ten or 15 years ago. We became cyborgs. At the age of 12 or 14, kids get a smartphone and forever after, they have that phone or the successor with them day and night, often consulting the device more than 100 times a day.

When we used to talk about the two halves of our brain, we used to have a picture in mind of the left and the right hemisphere, but increasingly, this is the picture that we should be thinking about: the digital and the biological brain. Increasingly, we rely on our digital brain to answer questions, when we are asked something, rather than using our biological brain to come up with a solution. Unfortunately, the two halves of our brain don’t really talk to each other very well.

Human Machine Symbiosis | Pattie Maes | TEDxBrussels Dating

The communication or the interaction is very limited and very inefficient. One problem is the bandwidth between the two. We mostly just use our two fingers, our two thumbs, to input into the device, and we rely on the tiny, little screen to receive output from the device.

Second, these devices require our complete attention to be operated, which means that we can no longer pay attention to whatever is around us in the physical world: the people that we are with, or maybe the tree that we are about to walk into. Last, these devices and the software really haven’t been designed with the goals of people in mind. The different apps on your smartphone are all competing for your attention, trying to distract you as much as possible and basically, trying to make you as inefficient and ineffective at accomplishing your goals as possible.

And this really results in a lot of multitasking and very short attention spans. I’m not the first to notice that this is a problem, and people have generally responded in two different ways. One type of people will say that we should unplug. We should take breaks from our technology and ween us from our dependence on these technologies before it’s too late.

While another camp of people have argued that we should embrace this development and that really, it is people’s true destiny and natural destiny, really, to augment themselves with evermore powerful tools and technologies and to ultimately merge with technology itself. You can’t me in the embrace camp. I believe that we will never go back, and that we will continue on this trajectory of merging with the machine, becoming one with the machine.

However, I believe that what is needed is that we radically and urgently redesign the experience of how a person interacts with their digital devices. I’ve for a while now argued for a new approach, a new type of experience for how we work with our smartphones, our laptops et cetera. And there’s really three ways in which this new approach is different from what we are used to today. The first difference is that the system is very aware of the user’s context and also, the user’s internal state. It uses sensors to know what is around the person, what the person is paying attention to, et cetera, as well as sensors to notice maybe what the brain activity of the user is, what their heart rate is, their breathing rate, et cetera.

Second, the device should be always on, always augmenting, really, the experience that the user has of their surroundings by for example, using augmented reality glasses, but basically, in very seamless, non disruptive ways. And the third difference with today’s devices that we all use is these new types of symbiotic devices have to be more proactive. They have to basically make suggestions, recommendations, and offer relevant information and interventions, given whatever it is that the user is currently trying to do, given the state of the user, and given the goals of the user.

So in my previous TEDx talk, seven years ago, I gave an example of a system that illustrates this approach, the sixth sense device. Basically, the device consists of a camera and a projector that you wear around your neck, and the camera constantly notices what the user is doing, and what they are trying to accomplish. And the projector finds relevant information and interfaces and makes those available by projecting them onto the objects you have in your hand or onto your hands themselves. It can give you audio augmentations, and so on, as well.

So notice that the two experiences here are completely synergistic and integrated. The person experiences their physical surroundings at the same time as benefiting from all the digital information and services. I believe that this new approach, this symbiotic relationship between a human and a machine will help people with self-actualization. It will help them with growing and developing into the people they want to become.

All of us want to change and grow in some way. Some of us may want to become more efficient, better at communicating, better at relationships. Others may want to be thinner, healthier, happier. Yet, others may want to learn some new skills, some new knowledge, some new languages.

And today, we rely on books, or posts, or maybe we consult a therapist or a coach to help us change and develop in the direction we want to develop. But I think that increasingly, we will actually rely on these more symbiotic forms of our personal devices to help us change. I’ll give you some examples in the rest of the talk of some experiments that we have been doing at my laboratory at MIT that show how these more symbiotic interfaces can help us with making decisions, or changing the way we make decisions, learning, remembering, and regulating our mood.

First, take decision making. Behavioral economists have shown that up to 90% of the decisions we make and the actions we perform are the result of a very intuitive, fast system that often results in actions and decisions that are suboptimal, that are not really in line with our own goals. We believe that these symbiotic devices can help us keep our goals and maybe our New Year’s resolutions in mind, when we are just out in the real world performing, making some decisions. Here’s an example of a system that uses the Microsoft HoloLens augmented reality headset to constantly look at what decisions the person is making. When you look at, say, the Coca-Cola bottles for more than three seconds, it assumes you may want to take one of these Coca-Cola bottles off the shelf and can give you some real time feedback, trying to encourage you to keep your goals in mind of maybe consuming less sugar.

Of course, for every person, the goals may be different, and you would tell the device what your specific goals are that you want it to make you more mindful of. These devices can also help us with learning new skills, new languages, new knowledge. We’ve been building again using augmented reality devices, we’ve been building systems that let you learn in the real world. So, instead of reading a book, or having to sit and watch a post, or taking a class, you actually, if you want to learn a language, go and explore the world, and everything around you can be labeled in the language that you are interested in learning.

The system also knows what words you already know, you have already learned, can give you new words for the same object, or more complex words, sentences, et cetera. Not all of these devices have to rely on a smart AI system to make these real-time augmentations. In this system, we use a Google Glass headset, and the person on the left constantly streams whatever is in front of him – in this case, in the supermarket – to another person, who may well be on the other side of the globe, who is labeling all the objects in the language the person wants to learn.

For example telling them, “This is called a grapefruit, whatever you’re holding right now. And that’s called an apple,” et cetera. You can even talk to that person, ask them some questions about how to interact with a clerk, for example, if you need some help. Similarly here, this augmented reality system basically makes it possible for a user who needs help, in this case, operating a laser cutter, with calling up an expert, and the expert can make her hands show up in real time in his visual field, so she can actually instruct him what buttons to push, how far to slide, different sliders, what gestures to perform, et cetera, while also talking to him about what it is he is trying to accomplish. Symbiotic devices will also help us with our memory.

This is a system which constantly records the last couple of seconds, or minutes rather, of your experience, so that if you missed something that someone said, you can quickly hit rewind, and listen, and see the experience again. Or it uses a wristband that detects when you shake someone’s hand. And it automatically records the face of the person and the name of the person, every time you basically shake someone’s hand.

This system here, again, uses augmented reality in combination with the ancient technique of the Memory Palace, which the Greeks and the Romans had invented. And it uses that to help you remember or memorize a random set of facts, for example, the list of winners of the Super Bowl, the big championship in the Unites States of football. And what the system does is as you follow a path or a route that you’re very familiar with, for example, from the subway to your office, it places images in different places or in sequence of all the teams that have won the Super Bowl over the last 20 years: the Jets, the Cowboys, the Packers, the Steelers, the Dolphins, et cetera. And because you basically, we’ve tested this, and you only have to experience this once, to be able to remember a totally random set of facts for up to a year.

Because this random set of facts becomes associated in your brain with these actual locations along the route from your home to your office. We can also help people with regulating their mood and state of mind. We’ve built systems such as this one, which uses an EEG sensor to note your brain waves, as well as a wristband to pick up your heart rate, galvanic skin response, et cetera.

And the system releases a pleasant smell whenever it senses, based on the data that you are anxious, or maybe that you are having a nightmare when you are sleeping, it may release vanilla, or rose scent, or something like that. While if you actually are a little bit too sleepy, for example, when you’re driving at night, or you’re studying, or something, it can sense that again, that you’re not being focused enough, and it releases a more sharp scent, like peppermint, to be able to help you sort of wake up and pay attention. This is a similar system, which again, uses an EEG headband to notice what your brain activity is. And it uses that in combination with either a virtual reality headset or an augmented reality headset and gives you special powers if your brain is very calm and focused. So you actually get this neural feedback in a visual form and you can do things like make things catch on fire or make things levitate as long as your brain is very concentrated and focused.

But as soon as you get distracted, the system actually, basically shows you that, in terms of the visual feedback. In summary, I think that we should embrace the fact that we are now cyborgs and will forever be cyborgs. But I strongly believe that we should redesign the devices that we carry with us all the time, and that we should make them become a more natural extension, a more seamless extension of ourselves that assists us, empowers us, and augments us. Thank you. (Applause)

Leave a Reply

Your email address will not be published. Required fields are marked *