Will our 'digital social pulse' diagnose and predict our behaviour?

Was struck by the phrase 'Digital Social Pulse' from Deborah Estrin at TEDMED last week, chatted with her and the Open mHealth initiative, working to fix the data API problem with a clean standard. But mostly she reinforced this idea about the data of YOU having diagnostic and predictive ability. That coupled with a request also last week from a public spaces project asking for a super short public art proposal, I thought about how we can use both data collected about us, with data collected in a neighborhood and begin to build a narrative....a story that helps us understand ourselves and the places we live.

Our ‘digital social pulse’ will be the way we live and how we move through the world. Imagine when city streets will be both smart and how we discover and understand more about ourselves. It’ll be fun. It will be beautiful. It will support our need as humans to feel apart of our community.

Connect self-tracking data gathered from GPS and accelerometers on our phone, fitbit devices, fuelbands, watch sensors with crowd-sourced proximity data gathered in the downtown neighborhood to accumulate and build a narrative story about how people walk, shop, eat, drive and encounter random experiences in an urban neighborhood.

Connect the small-data of you, your digital social pulse with how an urban  downtown functions as a social space.

Draw people into repetitive experiences where their presence builds and adds to the narrative of the downtown street-scene. Lure them toward experiences with small curious installations to alter their normal path. Offer a visible method for pedestrians and riders to signal on their phone when they are contributing. Allow people to learn something new about their patterns and see something about themselves they wouldn't normally notice. Delight and surprise. Combine the digital with the physical.

We move through spaces and sense the scale of human proportions, distance and space. We move in close when something grabs our attention and unconsciously walk toward it. Chance conversations can shift our mood. Something bright and shiny in the distance can shift our path and pull us toward it.

My focus in the past year has been tracking movement and travel by walking and photo gathering through a city. I’m using a fitbit and Basis watch which tracks steps, temperature and pulse. Openpaths, which gathers time and GPS coordinates. I often supplement with time-lapse photography using a little camera I wear around my neck to document the visual experience. The art installation pieces use underlying algorithms – as I experiment with ways to use weather, color, sense and data to build human data portraits from patterns of behaviour.

Every time I look at future scenarios of data tracking – it takes on a hugely sinister theme. Intense fear, worry, anxiety, loss of privacy to government agencies or global brand advertisers is the typical response. At times I think it is inevitable. Every bit of data about us will be known. Should we worry, or can we personally begin to know more and self-experiment. Rather than just be fearful, we might grab even more data and play with it, test it out, experiment with how it feels and build something we desire and embrace.

So, in anticipation of a time where total surveillance is normal, sensors are seamlessly embedded in our clothes and in tiny bits we swallow with vitamins. Location coordinates are gathered from phones and recordings of what we see, say and hear are invisibly captured, I've been collecting data to experiment and test ways of converting data into abstract patterns which can communicate encoded meaning about us and how we live in the neighborhoods we love.

There is an opportunity to connect the data we gather about ourselves and marry it with the data collected about how we live and move thru a downtown neighborhood. There is a story that can be created and the combination can be illuminating.

5 questions

5 questions from Paul van den Bosch who writes for Second Sight blog in the Netherlands 1. Why are you so interested in measuring yourself? 

I was interested in the notion of little bits of time, and why we  jump from one thing to the next and began trying to capture a little diary of how I spent my time. And realized its unbelievably difficult. I started measuring my sleep cycles as an easier…almost automatic way to measure how I spent my sleep (and yes, it’s all little 5 and 10 minute bits of time), and simultaneously found Ben Lipkowitz online who measured how he spent his minute-by-minute time for the past 7 years. Was curious about the subliminal, underlying metrics of a person. Like a data portrait measured and captured of you.

2. In your TEDtalk you talked about data patterns helping to understand ourselves: have you already discovered things about yourself? And was that after you analyzed the data or after you created your artworks?

My idea is that innate patterns of behaviour and body function are like an individual fingerprint. Your heart rate, sleep patterns, walking and travel patterns, speech patterns, food you eat…literally everything you start to imagine that can be measured creates a portrait of you. You have an individual rhythm that’s particular to you and can also be a reflection of your personality. For example, my sleep patterns are pretty steady, I have a very low standard deviation of my sleep score, versus my husband whose sleep score swings wildly. I realized it was a reflection of both our personalities.

My process is to analyze and create, analyze and create, test something, throw it away, do it again, analyze and create….you get it.

3. How do you see the future of Quantified Self / measuring ourselves; for example do you think that measuring ourselves will become common? Will everybody then become a data analyst, peering over his spreadsheets and dashboards, trying to make sense of all these data?

I think self-tracking will become invisible, and embedded into apps that we use and don’t even think about. No, we won’t pour over spreadsheets, but self-tracking data will be pervasive… for example - our clothes will have sensors that know how much we moved our arms and legs, and thus will send accurate data about our caloric expenditure to our Fitbit or fuelband dashboard. Our watch will track our heart-rate-variability and give us little stress alerts. And my best fantasy is that all our data is turned into patterned wall texture that is spit out from a 3D printer in the walls of our home. Recyclable, updatable. Human data portraits as art on our walls….mindfulness made tangible.

4. What would you like to measure / investigate, that is currently not possible?

It is incredibly hard to accurately and easily track what you eat. The nutrition, the content and make-up of what you consume would be fantastic to track automatically. I want it to be as easy as how I capture my weight, fat and BMI by just stepping on my wifi scale.

5. Do you think science can learn from / be inspired by (your) art? 

Massive braggadocio to think an artist has an effect on scientific research….but here’s my thought. It seems like the work on the human Connectome project to map the pattern of connections in the brain might find there is an innate pattern of you. And it’s a rhythm that is inherently familiar, and repeats itself in many facets of you. And weirdly there is an resonance when you encounter those same patterns in the real world.

I do think big-data research teams can learn from artists on how to deliver data that is addictive, beautiful and something you WANT to look at. Much of the self-tracking data graphics is so literal and dull, they need to ARTIFY the visual presentation of data. Come talk to me ;-).

Artify...will we talk about artification - same as we talk about gamification?

We'll talk about ARTIFY the same way we talk about GAMIFY today...a way to make something addictive.

Art is a lens to think about, stop and examine an experience. It gets you to feel something and really stop and consider. Sometimes you think about it over and over and roll it around in your head for days or months afterward.

I predict we'll see phone apps (applications and usage on smart phones) begin to incorporate metaphor and artification of data and information to pull you into repetitive use. It's not about designing a more beautiful interface, but it's about presenting data and information in a way that gets you to see it anew. I think self-tracking data for weight, sleep, walking, running, heart-rate....and all the new body sensors will find ways to artify the data - so you can easily see patterns in your behaviour to anticipate and gain insight to you.

Don't fear your phone watching you (the art of self-surveillance)....my TED talk from TEDxAustin

Data becomes physical: data --> art --> 3D printing

Very soon, all the data about us will be easily and invisibly tracked via sensors in our clothes and little patches we stick on our skin. The data will be privately ours. But then WHAT? You’ve got to have a way to extrapolate, summarize, compare and deliver a quick portrait of how you’re doing and what it means. Big data is meaningless if it can’t give me a picture that’s compelling and something I want to spend time with. A quick glance at pixilated graphics and status measurements is not enough.

I think pattern is the answer. Physical pattern. A language of pattern that’s readable to convey a sense of rhythm and proportions that are the compilation you. Complex, yet understandable, definitely recognizable.

Human in feel, hand-built libraries will rely on physical materials produced with wall size laser cutters and 3D printers on invisible x/y axes mounted on the ceiling. Digital tools produce sculptural texture and visual pattern quickly, easily, disposably. Renewed, updated, and recycled. In the spaces we live and work. For ourselves, households and teams of people who work together.

Responding to your stress level, mood, travels, sleep rhythms, nutrition level, body movement, posture, body strength, inflammation, calorie expenditure, productivity, financial ups/downs, word patterns…more than is listable here.

Encoded patterned language – visual and beautiful, steps ahead of data visualization, with complex models and algorithms to capture and convey the experience surrounding the data. Ambient, sensing walls that feel like ART, but deliver a narrative, patterned language that is reinforcing, probably anticipating, but definitely responding to all the data collected about you.

Will it kill the mystery of being human, simply magnify our defects or use sensors and a mass of measurements to acknowledge and present patterns of data that lure you into something beautiful and strangely compelling?

Stop thinking like scientists and start thinking like artists. Data meets art…meets 3D printing.

Texas Biennial catalog from last year just came online