My contributions to date have been in each of these seven areas, though my greatest effort has been on the first four, leaving questions and analyses about the office of the future and information architecture for my future work.

1.        Methodology: Several years ago I constructed a computational environment for studying Tetris which allows non-intrusive tracking of a subject's interactions with a Tetris game at millisecond levels. We recorded the state of the game at each moment and the type of action undertaken. Massive amounts of data were gathered and analyzed on people playing Tetris for tens of hours. This gave us a virtually unique data set of how people move from novices in an interactive game to becoming experts 10 to 20 hours later. Several companies expressed interest in using this environment in their research labs and Interval may still be using it. The method of recording large amounts of data and then data mining on the basis of logical queries is now used in several research projects around the country. I have also grappled with the challenges of developing a reliable methodology for experimentally studying how people play and improve at jigsaw, counting coins, solving block problems interactively. I made solid advances in studying selected areas of activity and have published most of my results (On Distinguishing Epistemic From Pragmatic Action, Interactive Skill in Scrabble, Complementary strategies: why we use our hands when we think – the work on comparing strategy evolution between young and old is still in preparation). However, I regard the general problem of how experimentally to study the natural evolution of cognitive strategies as inadequately solved and an important topic for future research. A closely related problem arises in Human Computer Interaction where experimental approaches seem to leech out most of the real life aspects of our way of interacting and adapting to computational environments, and so requires new methodological principles.

One of these new methods which I am developing involves creating special digital capture tools for recording the activity of teams in what may be called semi-structured contexts. In this approach, subjects are put in a reasonably natural context then assigned specific tasks and asked to perform those tasks under varying conditions. A huge amount of data is collected this way: video via camcorders, video of desktop activity, separate audio recordings of each person. To make sense of all this data we have designed an activity review tool which takes 4 tracks of synchronized audio-video recordings and allows us to machine review the 4 tracks looking for onsets of conversation, and dozens of other machine identifiable events. The system has been designed further to allow investigators to review the tapes and add new lines of commentary. This allows us to lay down dozens of tracks of ethnographic coding. When these annotations are examined in conjunction with the machine annotations we are able to increase the complexity of our activity analysis. The design and implementation of this activity review tool has been undertaken in conjunction with Dave Nadeau at the San Diego SuperComputer Center and is supported by the ONR.

2.        The nature of cognitive coupling. The general problem of cognitive coupling, and coordination more particularly, is a field rich in possibilities. So far I have undertaken three sets of studies that have contributed to our understanding of the time course of adaptations we make to our environment, and the computational advantage of making such adaptations.

a.        The first set examines the ways humans use the space around them strategically to help them in their tasks (The Intelligent Use of Space, Complementary strategies: why we use our hands when we think, and in Adapting the World Instead of Ourselves). I showed how the idea of amortized complexity could be used as an explanation of why people prepare their environments and I classified the major ways both humans and, at times, animals have of adapting the world to improve their overall adaptive advantage.

b.       In the second set I have begun to distinguish forms of coordinative mechanisms and am now pursuing this in the context of command centers for disaster management. I reported on this in my conference opening lecture at the European Cognitive Science Society (Distributed Cognition, Coordination and Environment Design). This work builds on the concepts of epistemic and complementary actions I introduced a few years ago (On Distinguishing Epistemic from Pragmatic Actions, Complementary strategies: why we use our hands when we think, Adapting the World Instead of Ourselves, and Are All Actions Performative or Exploratory).

c.        In some new studies on the nature of interruption and mechanisms for recovery, (part of which is discussed in A Few Thoughts on Cognitive Overload) I am exploring how interruption is related to frames of activity. People can multitask if they are prepared, but stress and cognitive overload occur the more we have to rely on ourselves alone to re-establish a tight coupling with our environments.

d.       Finally, in examining the problem of metacognition in elearning environments I have considered how visual design can affect coordination. An initial report of this can be found in Metacognition, Distributed Cognition and Visual Design.

3.        How interactive strategies evolve. I am currently completing a paper with Larraine MacDonald on how old and young develop strategies for improving visual search and memory on a pick and place task. We conducted this experiment over a two year period and will be submitting the paper soon. It required administering fairly extensive neuropsychological pretests, a recall post test. Coordination with elderly subjects proved more time consuming than we expected. Another paper that is in progress is a write up of experiments on interruption, how people initially react and then adapt to an environment in which interruption is common.

4.        How a theory of interactivity fits with the principles of evolutionary biology. Although I have been interested in the nature of adaptation at different time scales in much of my experimental and analytical work and reported on it in many papers, the only paper in which I explored the implications of the theory for evolutionary biology was in the paper Adapting the World Instead of Ourselves. I believe this paper was well received in the zoological world as immediately after its publication I received letters from several groups of evolutionary biologists at Oxford, Harvard, Princeton and Stanford who, apparently, are now turning to this topic at a non-cognitive level.

5.        Context Aware venues/Office of the future. In Adaptive Rooms, Virtual Collaboration and Cognitive Workflow, I articulated an ontology of objects necessary for designing adaptive rooms. Virtual entities can be classified as passive, reactive, active and information tools and spaces. Self transforming objects are a special type of active object. Using this ontology it is possible to define an adaptive room as a collection of self-transforming objects situated in an architecturally active room containing information spaces and tools. Well designed adaptive rooms combine the insights of ubiquitous computing – that computerization should be everywhere, transparently incorporated – with the insights of augmented reality – that everyday objects can be digitally enhanced to carry more information about their use. The result is an integrated virtual environment in which people can cooperate and collaborate on projects in a way that leverages the simulation strengths of computers. This work was picked up by Ted Krueger, Director of Information Technology in the School of Architecture, University of Arkansas and as a result I was invited to contribute to the Architecture journal Convergence. The paper I wrote “Changing the Rules: Architecture in the Millennium” is an examination of how changes in technology are making irrelevant some of the architectural principles discussed by Alexander.

The theme of how context aware technology may change our conceptions of design is explored further in the article The Context of Work. In that article I introduce the idea of an ‘entry point’ to characterize the attractors in a ‘space’ where users can jump off into another ‘space’.

On a practical side, I have made a digital glass wall to join the two sections of my lab which are physically divided. I will soon be increasing the complexity of that design in a joint project with Calit2.

6.        Information architecture. I have yet to publish on this topic in journals. I have written several technical documents for the Worldbank on the design principles of their intranet but none of these are in the public domain. My contributions on this topic, therefore, should be seen as practical. I designed and oversaw the implementation of a large intranet (with more than 350,000 pages). This design is of interest, not just because of its scale, but because it focuses on the knowledge management needs of a major knowledge producer: the Worldbank. It is still the design which the WorldBank uses after 4 years. I also designed the information architecture for the Worldbank’s external website in 1998-9. (It has more than 100,000 pages), and most recently I designed the architecture of the Global Development Gateway with several hundreds of thousands of pages. I believe these products count as intellectual contributions, much as patents do.

7.        Elearning Environments. I spent a significant part of the last 5 years building large web based software systems for elearning. My systems have gone through several redesigns and will no doubt go through more. The biggest efforts though were spent on designing a large course and learning management system and a team collaboration environment. Both were written in asp and sql.

a.        To review the learning management system visit cogsci 187A – Cognitive Aspects of Multimedia Design, and Cogsci 25 – Intro to Web Programming To see how to create a course visit our test site. Try this out login as visitor (password = test).

This entire system, containing hundreds of asp files and tens of thousands of lines of asp code is dynamic, driven by an SQL database and completely customizable. By storing all data in a database we are able to create and manage an arbitrary number of courses, and to make it simple for teachers to build new courses quickly. This shell is currently being tested by several faculty throughout UCSD campus and will soon be in use for many courses in 6^th college, and some at UC Irvine. It has been designed with ease of use in mind, and the needs of professors as well as students. On the teacher’s side I have incorporated principles of instructional design to help professors design better lectures and structure their courses more powerfully. I studied the workflow of teachers and introduced features to make it easier to manage TA’s and simplify grading. Students have a secure portfolio to collect their work and to have private discussions with their teachers. To reduce the burden of grading, the system has been designed to support portfolio based assessment, student grading of other students, quiz taking and auto-grading, and team based work.

b.       To review the collaborative shell for team based projects look are some of the ways students have used the shell. Login as visitor (password = test)

This team collaborative system was designed to allow small to medium sized teams of students to work together remotely with each other and a client. I interviewed students taking my project based class that the hardest thing they found was getting together to work on their project together. This application grew from a simple shared document management system into a dynamic tool that allows team members to define their own navigation, to comment on gallery images they uploaded, and to comment on files and announcements. Task assignment and tracking is another major feature of the system. Students can assign each other tasks, dues dates, they can comment on their tasks, accept, reject and reassign them. Again the design of this tool was based on workflow analysis of small teams.

[ top ]