Introduction to the class & expectations

Supporting Readings

  • Bush, Vannevar – As We May Think (link)
  • Clark, Andy – Natural Born Cyborgs (book)
  • Engelbart, Doug – Augmenting Human Intellect: A Conceptual Framework (link)
  • Engelbart, Doug – Mother of All Demos (video)
  • Lanier, Jaron – You Are Not a Gadget (link)
  • Licklider, J.C.R. – Man Computer Symbiosis (link)
  • Maes, Pattie – Agents that Reduce Work and Information Overload (link)
  • Mann, Steve – Cyborg: Digital Destiny and Human Possibility in the Age of the Wearable Computer (book)
  • Mann, Steve – Wearable Computing: Toward Humanistic Intelligence (link)

Augmenting Memory

Mandatory Readings

  • Brooker, Charlie – Black Mirror S1E3: The Entire History of You (link)
  • Carey, Benedict – How we learn: Introduction and Part 1 (book)
  • Sparrow, Betsy – Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips (link)


  • Bell, Gordon (feat.) – Microsoft Researcher Records His Life in Data. WIRED Magazine (link)
  • Bell, Gordon – Total Recall (book)
  • DeVaul, Richard – The Memory Glasses (link)
  • Foer, Joshua – Moonwalking with Einstein: The Art and Science of Remembering Everything (book)
  • Madan – Augmented Memory: A survey of the approaches to remembering more (link)
  • Maguire, Eleanor – The neuroscience of memory (lecture)
  • Pontius, Erin – Exploring Emergent Technology: Memory Augmentation Implants (link)
  • Rhodes, Bradley – Remembrance agent (link)
  • Wilson , Matt – Reading the Minds of Rats (link)


Motor augmentation and substitution/ Skin augmentation

    Mandatory Readings

  • Jaron Lanier – Homuncular Flexibility (link)
  • Hugh Herr – Exoskeletons and orthoses: classification, design challenges and future directions (link)
  • Katia Vega – Beauty Technology: Designing Seamless Interfaces for Wearable Computing, Chapter 1 and 2 (link)

  • Stelarc – Third Hand / Exoskeleton (link)
  • IKO Creative Prosthetic System (link)
  • Hyungkoo Lee (link)
  • Vega, Katia, Marcio Cunha, and Hugo Fuks. “Hairware: conductive hair extensions as a capacitive touch input device.” Proceedings of the 20th International Conference on Intelligent User Interfaces Companion. ACM, 2015. pdf
  • The illustrated Man, Prologue pdf
  • Guterstam, A., Petkova, V. I., Ehrsson, H. The Illusion of Owning a Third Arm. PLoS ONE. 6 (2). 2011. (link)
  • Liu, Xin, et al. “Wearability factors for skin interfaces.” Proceedings of the 7th Augmented Human International Conference 2016. ACM, 2016. pdf
  • Faye Wu, Harry Asada – “Hold-and-Manipulate” with a Single Hand Being Assisted by Wearable Extra Fingers. In Proc. ICRA ’15 (link)
  • Machine to be another (link)
  • Beetle wrestler (link)
  • Pedro Lopes, Alexandra Ion, Patrick Baudisch – Impacto: Simulating Physical Impact by Combining Tactile Stimulation with Electrical Muscle Stimulation. In Proc. UIST ’15 (link)
  • Pedro Lopes, Patrik Jonell, Patrick Baudisch. Affordance++: Allowing Objects to Communicate Dynamic Use. In Proc. CHI ’15 (link)
  • Emi Tamaki, Miyaki Takashi, Jun Rekimoto – PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proc. CHI ’11. (link)

Sensory substitution and augmentation

    Mandatory Readings

  • Nagel, S. K., Carl, C., Kringe, T., Märtin, R., & König, P. (2005). Beyond sensory substitution–learning the sixth sense. Journal of Neural Engineering, 2(4), R13–26. (link)
  • Auvray, M., Hanneton, S., Lenay, C., & O’Regan, K. (2005). There is something out there: distal attribution in sensory substitution, twenty years later. Journal of Integrative Neuroscience, 4(04), 505–521. (pdf)
  • Deroy, O., & Auvray, M. (2012). Reading the world through the skin and ears: a new perspective on sensory substitution. Frontiers in Psychology, 3. (link)
  • Rao, R. (2017) Scientists see a mind controlled virtual reality . Web Article. (link)
  • Scott Novich, David Eagleman – Using space and time to encode vibrotactile information (link)
  • Daniel Kish – How I use sonar to navigate the world (link)
  • Kolarik et al – A summary of research investigating echolocation abilities of blind and sighted humans (link)
  • Feelspace (link)
  • Maeda, Taro, et al. “Shaking the world: galvanic vestibular stimulation as a novel sensation interface.” ACM SIGGRAPH 2005 Emerging technologies. ACM, 2005. (link)
  • Hashimoto, Yuki, et al. “Straw-like user interface: virtual experience of the sensation of drinking using a straw.” Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology. ACM, 2006. (link)
  • Koizumi, Naoya, et al. “Chewing jockey: augmented food texture by using sound based on the cross-modal effect.” Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology. ACM, 2011. (link)
  • Several references from:
  • Schwartzman – See Yourself Sensing (link)
  • Ortiz et al – Recruitment of Occipital Cortex during Sensory Substitution Training Linked to Subjective Experience of Seeing in People with Blindness (link)
  • Thomson et al – Perceiving invisible light through a somatosensory cortical prosthesis (link)
  • Ward and Meijer – Visual experiences in the blind induced by an auditory sensory substitution device (link)
  • Ward and Wright – Sensory substitution as an artificially acquired synaesthesia (link)
    • Additional

    • Uexküll, J. von, A stroll through the worlds of animals and men: A picture book of invisible worlds. Semiotica (Vol. 89).
    • F. A. Geldard, “Some neglected possibilities of communication.,” Science, vol. 131, no. 3413, pp. 1583–1588, May 1960.
    • P. Bach-y-Rita, C. C. Collins, F. A. Saunders, B. White, and L. Scadden, “Vision Substitution by Tactile Image Projection,” Nature, vol. 221, no. 5184, pp. 963–964, Mar. 1969.
    • P. Bach-y-Rita, “Tactile Sensory Substitution Studies,” Annals of the New York Academy of Sciences, 1013(1), pp. 83–91, Jan. 2006.
    • Kaczmarek, K. A.. The tongue display unit (TDU) for electrotactile spatiotemporal pattern presentation. Scientia Iranica, 18(6), 1476–1485. 2011.
    • D. R. Chebat, F. C. Schneider, R. Kupers, and M. Ptito, “Navigation with a sensory substitution device in congenitally blind individuals,” Neuroreport, 2011.
    • J. K. O’Regan and A. Noë, “A sensorimotor account of vision and visual consciousness,” Behavioral and Brain Sciences, vol. 24, no. 5, pp. 939–973. 2001.
    • J. K. O’Regan and A. Noë, “What it is like to see: A sensorimotor theory of perceptual experience,” Synthese, vol. 129, no. 1, pp. 79–103, 2001.
    • S. K. Nagel, C. Carl, T. Kringe, R. Märtin, and P. König, “Beyond sensory substitution—learning the sixth sense,” Journal of Neural Engineering, vol. 2, p. R13, 2005.
    • S. K. Nagel, C. Carl, T. Kringe, R. Märtin, and P. König, “Beyond sensory substitution—learning the sixth sense,” Journal of Neural Engineering, vol. 2, p. R13, 2005.
    • S. E. Guttman, L. A. Gilroy, and R. Blake, “Hearing What the Eyes See,” Psychological Science, vol. 16, no. 3, pp. 228–235, Mar. 2005.
    • P. Meijer, “An experimental system for auditory image representations,” Biomedical Engineering, vol. 39, no. 2, pp. 112–121, Feb. 1992.
    • L. Merabet, L. Battelli, S. Obretenova, S. Maguire, P. Meijer and A. Pascual-Leone, “Functional Recruitment of Visual Cortex for sound encoded object identification in the Blind,” NeuroReport, Vol. 20, No. 2, pp. 132 – 138, January 2009
    • E. Striem-Amit and A. Amedi, “Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People ‘Seeing’ by Using Sounds,” Current Biology, vol. 24, no. 6, pp. 687–692, Mar. 2014.
    • E. E. Thomson, R. Carra, and M. A. L. Nicolelis, “Perceiving invisible light through a somatosensory cortical prosthesis,” Nature Communications, vol. 4, pp. 1482–, Feb. 2013.
    • M. Auvray and E. Myin, “Perception with compensatory devices: from sensory substitution to sensorimotor extension,” Cognitive Science, vol. 33, no. 6, pp. 1036–1058, 2009.
    • Classen, Constance. The Deepest Sense: Studies in Sensory History. University of Illinois Press. Kindle Edition, Locations 468-471. 2012.
    • M. Auvray, S. Hanneton, C. Lenay, and K. O’Regan, “There is something out there: distal attribution in sensory substitution, twenty years later,” Journal of Integrative Neuroscience, vol. 4, no. 4, pp. 505–521, 2005.
    • R. Held, Y. Ostrovsky, B. de Gelder, T. Gandhi, S. Ganesh, U. Mathur, and P. Sinha, “The newly sighted fail to match seen with felt,” Nature Neuroscience, vol. 14, no. 5, pp. 551–553, May 2011.
    • Davis, L. J., “The Disability Studies Reader, 2nd ed.” New York: Routledge, 2006.

Students present a design exercise related to the first 4 weeks

  • Buxton – sketching user experiences
  • Don Norman – The Design of Everyday Things

Embodiment in Virtual Reality

    Mandatory Readings

  • Peck, Tabitha C., et al. “Putting yourself in the skin of a black avatar reduces implicit racial bias.” Consciousness and cognition 22.3 (2013): 779-787. (link)
  • Meyerbröker, Katharina, and Paul MG Emmelkamp. “Virtual reality exposure therapy in anxiety disorders: a systematic review of process‐and‐outcome studies.” Depression and anxiety 27.10 (2010): 933-944. (link)
  • Yee, Nick, and Jeremy Bailenson. “The Proteus effect: The effect of transformed self‐representation on behavior.” Human communication research 33.3 (2007): 271-290. (link)

  • Fox, Jesse, and Jeremy N. Bailenson. “Virtual self-modeling: The effects of vicarious reinforcement and identification on exercise behaviors.” Media Psychology 12.1 (2009): 1-25. (link)
  • Guadagno, Rosanna E., et al. “Virtual humans and persuasion: The effects of agency and behavioral realism.” Media Psychology 10.1 (2007): 1-22. (link)
  • Hershfield, Hal E., et al. “Increasing saving behavior through age-progressed renderings of the future self.” Journal of Marketing Research 48.SPL (2011): S23-S37. (link)
  • Slater, Mel, et al. “Inducing illusory ownership of a virtual body.” Frontiers in neuroscience 3 (2009): 29. (link)
  • Madary, Michael, and Thomas K. Metzinger. “Real virtuality: a code of ethical conduct. recommendations for good scientific practice and the consumers of vr-technology.” Frontiers in Robotics and AI 3 (2016): 3. (link)
  • Bailenson, Jeremy N., et al. “Transformed social interaction: Decoupling representation from behavior and form in collaborative virtual environments.” PRESENCE: Teleoperators and Virtual Environments 13.4 (2004): 428-441. (link)
  • Won, Andrea Stevenson, et al. “Homuncular flexibility in virtual reality.” Journal of Computer‐Mediated Communication 20.3 (2015): 241-259. (link)

Brain Interfaces

    Mandatory Readings

  • Moran Cerf: Free Won’t, TEDx (link)
  • Abdulkader, Sarah N., Ayman Atia, and Mostafa-Sami M. Mostafa. “Brain computer interfacing: Applications and challenges.” Egyptian Informatics Journal 16.2 (2015): 213-230.  (link)

  • L. George, A. Lecuyer, An overview of research on “passive” brain-computer interfaces for implicit human-computer interaction, International Conference on Applied Bionics and Biomechanics, Venice, Italy, 2010. (link)
  • E.M. Peck, D. Afergan, B.F. Yuksel, F. Lalooses, and R.J.K.  Jacob, “Using fNIRS to Measure Mental Workload in the Real World,” pp. 117-139 in Advances in Physiological Computing, ed.  S.H. Fairclough and K. Gilleade, Springer (2014). (link)
  • A. Girouard, E. Solovey, L. Hirshfield, E. Peck, K. Chauncey, A.  Sassaroli, S. Fantini, and R. Jacob, “From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI,” pp. 221-237 in Brain- Computer Interfaces: Applying our Minds to Human-Computer Interaction, ed. D.S. Tan and A. Nijholt, Springer (2010). (link)
  • First brain-to-brain ‘telepathy’ communication via the Internet (link)
  • Brain-controlled robots (link)


The potential of subliminal interfaces: Exploiting the science of physical intelligence

Presentation: Judith (link) Xin (link)

    Mandatory Readings

  • Andreas Riener, Guido Kempter, Timo Saari, and Kenneth Revett – Subliminal Communication in Human-Computer Interaction (pdf)
  • Thalma Lobel – Sensation (link) (link to intro) (contact TAs for physical copy)

  • Mlodinow – Subliminal: how your unconscious mind rules your behavior (link)
  • Subliminal Persuasion and Its Potential for Driver Behavior Adaptation (link)
  • Subliminal priming and persuasion: Striking while the iron is hot (link)
  • Anat Arzi, Yael Holtzman, Perry Samnon, Neetai Eshel, Edo Harel, and Noam Sobel – Olfactory Aversive Conditioning during Sleep reduces cigarette-smoking behavior (link)
  • Jasper H. B. de Groot, Monique A. M. Smeets, Annemarie Kaldewaij, Maarten J. A. Duijndam and
    Gün R. Semin – Chemosignals Communicate Human Emotions (link)
  • Costa, Jean, et al. “EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions.” Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2016. (link)
  • Masashi Nakatani –
  • MLThesis: BrightBeat , Essence
  • Philippot, Pierre, Gaëtane Chapelle, and Sylvie Blairy. “Respiratory feedback in the generation of emotion.” Cognition & Emotion 16.5 (2002): 605-627.
  • White, Gregory L., Sanford Fishbein, and Jeffrey Rutsein. “Passionate love and the misattribution of arousal.” Journal of Personality and Social Psychology 41.1 (1981): 56.
  • Costa, Jean, et al. “EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions.” Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2016.
  • Valins, Stuart. “Cognitive effects of false heart-rate feedback.” Journal of personality and social psychology 4.4 (1966): 400.

Second design exercise

Augmented Creativity
Hybrid creativity: human-machine artists

Mandatory Readings

  • Margaret A. Boden – CREATIVITY IN A NUTSHELL (pdf)
  • Computers and Creativity (pdf)


FreeD: Human-Computer Collaboration in 3D Craft (pdf)
A manifest for digital imperfection (link)
Computational Creativity (link)
Creative AI: The robots that would be painters (link)
Harold Cohen’s AARON Paint System
Tandem (Page 214-216) (pdf)
HoloARt: Painting with holograms in Mixed Reality (link)

Symbioses in Science Fiction

  • Her (2013)
  • Black Mirror
  • HyperReality (link)
  • Nathan Shedroff, Christopher Noessel – Make it So (link)
  • HAL 9000 in 2001: A Space Odyssey –
  • The Veldt/The World that the Children Made
  • David Cronenberg – eXistenZ
  • Spike Jonze and Charlie Kaufman – Being John Malkovich
  • Ex Machina (2015)
  • Lord of the Rings
  • Gattaca (1997)
  • sight (link)
  • Joseph Ruben – Dreamscape
  • Tomorrowland (2015)
  • Memento (2000)
  • Lucy (2014)
  • The man who changed his mind (1936)
  • Lathe of heaven
  • Hitchhiker’s guide to the galaxy

Final project presentations