research – GRAVITRON [notes]

Autonomous Dynamic Visualisation

I am a strong proponent of speculative design. The simple argument is that unshackling design from the usual constraints and social expectations gives designers the headroom to create the unusual and unexpected, or to take a critical or even antagonistic position. For research, speculative experimentation can drive innovation and lead research in exciting new directions. When I create speculative work I do so without expectation of immediate practical or commercial application but I know that ideas and techniques developed during a speculative process inevitably find their way into more conventional works. A fine example of this is my collaboration with Crowd Convergence.

Crowd Convergence provide a social media moderation service which allows their customers to aggregate various streams and filter the results. It’s perfect for large-scale events like sporting matches where organisers want to broadcast social media updates on a big screen and need to curate the content (ie. block anything offensive). Having seen some of my Twitter experiments, Crowd Convergence contacted me about creating visualisers for their social media service. I was stoked to receive the invitation and somewhat surprised that my speculative work had found such a directly relevant application.

“AIR”, the first of the Crowd Convergence pieces, extended techniques from one of my earlier experiments to create a 3D motion graphics sequence. After displaying a status post, the view zooms through a cloud of status updates, twisting and turning to arrive at the next post in the stream. Below, AIR in situ at London’s “Clothes Show Live” and at the FINA World Junior Swimming Championships in Dubai.

AIR example — AIR in situ at London’s Clothes Show Live. From Stylonylon.

AIR Example 2 — AIR in situ at the FINA World Junior Swimming Championships in Dubai.

Photowall, as the name suggests, renders a social stream as a fullscreen tiled wall of photos, overlaying the screen name of the author and any associated status text.

Photowall example — Photowall in situ at the 2014 World Mobile Congress in Barcelona.

Both of these works make extensive use of CSS 3D transforms; AIR for its rendering of 3D space, and Photowall for the transitions of each image tile. As I’ve stated previously (here and here), the evolution of CSS is a brilliant example of the changing practice of graphic design. Concepts and techniques derived from print design are being redefined and extended to address the transient qualities of the computer screen. It’s an active area that is evolving quickly.

Another aspect of the project that is worth noting is the format of the works. While they behave like fullscreen desktop software, they are in fact standard web applications utilising html, css and javascript, and running in a browser window. As someone who has dabbled with all sorts of IDEs and programming languages, it’s marvellous what can today be achieved within the context of the humble browser.

Crowd Convergence case study video of their work at the 2014 World Mobile Congress in Barcelona. Features glimpses of my AIR and Photowall visualisers, and also shows some interaction with the moderation console.

Another novel aspect of the works is their autonomy – unlike typical web interfaces the Crowd Convergence pieces operate without typical user interaction. Instead of waiting for clicks or text input the pieces poll the Crowd Convergence server and retrieve data containing the content to display and the instructions for its playback. So, while not interactive in the typical sense (buttons, hyperlinks, text input etc.), these autonomous works can be controlled to some extent through Crowd Convergence’s moderation console; stop, start, duration of transition, duration of hold, ordering of status posts, as well as customising features such as colours, typefaces, etc.

Of course, an important consequence of their autonomy is that the works need enough smarts to cope with variation or disruption. It’s definitely the more time-consuming aspect of production – testing for “what-ifs” and edge cases.

I think there is massive potential for autonomously visualising networked data and information – you only have to note the sheer quantity and scale of the public screens you encounter on a daily basis. The norm for many of these large public screens is to serve as billboards, displaying a queue of static posters, but there is no reason they can’t be used in more dynamic and interactive ways.

The Crowd Convergence collaboration is just one example of my recent forays into autonomous forms of visualisation. It is an area that is ripe for innovation, and one that I am continuing to explore.

Driving Forces

In March 2014 I had the pleasure of presenting at the “Driving Forces” conference at the excellent ANU Art School. The conference bi-line: “The Role of Artists and Designers in Interdisciplinary Research” gives a pretty precise idea of what the conference was all about. Mitchell Whitelaw and I based our presentation on our experiences developing exploratory web interfaces for a beautiful collection of high res scans of “The Queenslander”, an early 20th century rural magazine. Our work was commissioned by the State Library of Queensland, a fantastic institution renowned for its progressive outlook regarding both physical and virtual manifestations of the modern library.

Discover the Queenslander preview — Work-in-progress

The conference, organised by Erica Seccombe, was really inspiring and we were fortunate to share our session with Nola Farman and Leah Heiss – I strongly recommend checking the work of both.

Our work, “Discover the Queenslander”, will be live on the SLQ site soon. Here is an abstract of our presentation…

An Interdisciplinary Machine: Reflections on Digital Practice-led Research
Mitchell Whitelaw and Geoff Hinchcliffe
Centre for Creative and Cultural Research
University of Canberra

Abstract

In this paper we reflect on our own practice as designers, artists and programmers to argue that computation is a fertile site of interdisciplinarity, and software production is an inherently creative field ideally suited to practice-led enquiry.

Computation has been interdisciplinary since its inception; the earliest computing machines were used by physicists, meteorologists, cryptographers and biologists. Computation is in a sense indifferent to disciplines, reducible ultimately to a simple set of formal operations. This is not to reduce or dissolve disciplinary differences, but to create a common ground, a machine that in its indifference fosters connections between disparate domains. And computation is intrinsically pragmatic – it makes things happen, a verb, not a noun. It does not simply link different domains, but engages them in action, in joint projects and creations.

While code and programming have been professionalised and “disciplined” through computer science and software engineering, the explosion of the Web and its accessible programming languages drew broad participation in new forms of software production. Twenty five years later, we find computing at the heart of creative practice as evidenced by the Art+Code movement.

Our recent work with the State Library of Queensland demonstrates this interdisciplinary pragmatism in action. Our work relies on the practical affordances of both code and code culture: powerful software toolkits ready to be appropriated and recombined for novel outcomes. Data becomes a creative material as well as a shared language; in this project heritage collection data links us with the concerns and conventions of historians, librarians, archivists and information managers. Computation enables our joint project: creating rich new forms of exploration and engagement with digital collections. Computation also priveleges a practice-led approach. With code as a medium, we work through rapid ideation and experimentation towards an outcome that is only apparent in retrospect. We produce software, but we are not software engineers – rather we sketch, play, copy and paste; coding is a hands-on practice with its own pleasures and pitfalls, rather than a rationalised process. As artists and designers we respond to the specificity of each collection and the qualities of our materials; the solutions are bespoke, customised to each context but also produce generalisable knowledge.

Virga

May 1 2013 saw the launch of “Virga”; a data lighting sculpture produced in collaboration with renowned Australian designer Robert Foster of F!nk Design. Pat Coppel, of Make Designed Objects, invited Rob and I to complete the installation as part of the renovation of the flagship store in Carlton. If you’ve not had opportunity to visit, Make Designed Objects is a wonderful slice of Melbourne retail, brimming with amazing designer wares. It also has a significant online presence and the concept for the data sculpture was to create some kind of bridge between the bricks & mortar and online manifestations of the Make retail business. Pat describes the work:

Data Aesthetics in Retail Space is a collaborative project between Make Designed Objects, Robert Foster of Fink & Co and Geoff Hinchcliffe of University of Canberra’s Faculty of Arts & Design.

Virga (an observable streak or shaft of precipitation that falls from a cloud but evaporates or sublimes before reaching the ground) is the product of that collaboration; an LED light and data sculpture formed by internationally acclaimed designer-maker Robert Foster that colourfully expresses itself based on the digital data fed into its environment.

What data?
Any data we choose!
Want to watch a colourful representation of the seasonal nature of Make’s sales data?
Feed in the data.
The change in inner Melbourne maximum average daily temperatures from 1913-2013?
Feed in the data.

With bricks and mortar retailers rapidly migrating to the World Wide Web why not bring a bit of the World Wide Web back into bricks and mortar retail? Why not feed the traffic data from the Make website into Virga and see what happens?

The real joy of Virga lies in its abstract representation of a digital world in a rapidly evolving bricks and mortar retail environment.

And it looks way cool…

The commissioning of Virga is clear evidence of Make’s ongoing commitment to building an exceptional retail experience – both on and offline. It’s refreshing to see when there is so much doom & gloom about the future of business for Australian retailers. As Pat Coppel explains:

In a market and world where increasingly we have a physical and virtual version of almost everything; people, businesses, streetscapes… it’s not surprising that online retail is booming. And while the news reports a great deal of fear for bricks and mortar retail, this ever changing market presents opportunities for innovation in brokering an extended relationship between the virtual and physical aspects of a retail business such as Make.

It is from within this landscape that Pat Coppel, Director of Make has initiated this collaboration between retailer, designer-maker and academic. The result is Virga; a beautiful sculptural installation that translates our technological data into a spectacular visual language of light and colour. As such, Virga creates a playful narrative around the virtual and physical instances of Make Designed Objects.

Now that the Carlton refurbishment is complete, Make are turning their attention to their online store and I’ll no doubt be posting about it soon. But for now, some details about the Virga development…

Virga is comprised of eleven individual lighting forms, each with two individually controllable RGBW (Red, Green, Blue, White) LED units. Inside each of the lighting forms is an Arduino mega with Wifly card for wireless network connection. The lighting array is orchestrated by a server-side application which parses the data and transmits instructions to each of the lighting units.

Each of the Arduinos acts as a simple slave and the heavy lifting is performed by the web server. The reasoning is that the Arduinos have a very limited amount of memory whereas the web server is powerful, fast and also much easier to edit and update than the physically concealed Arduinos.

Programming the Arduinos was new to me and coming from web scripting, the Arduino code seemed extremely brittle at first. However, it didn’t take too long to become acquainted and I was soon entranced by the wonder of working with something outside the screen. Fortunately I had the help of colleague Chris Hardy when it came to wiring the Arduinos to the LED lighting units. The trick was in finding the correct drivers for the LED units. Once the hardware was configured, I was able to experiment with the lights and get a feel for their tolerances and capabilities.

Ultimately the code I created for the Arduinos is akin to an animation class – its role is to understand lighting values as well as timing and transition values. One of the more difficult aspects of working with 11 independent asynchronous network devices is synchronising their timing. My solution was to create a sync process which is executed on start-up. It works like this: on start-up each unit connects to the network, checks-in with the server and awaits further instructions. The server logs the arrival of each lighting unit and stores an adjustment value for each. Once all units have checked-in, the server commences the choreography, sending each unit its time-adjusted lighting sequence.

Virga development at F!nk workshop. Rob Foster – Right.

In many respects, the delineation of roles and responsibilities on this project was determined by the particular expertise that Rob and I brought to the project. Rob crafted the lighting forms, including the custom mounts and housings for the LED units, while I took care of the data side of things; programming arduinos to control each lighting unit, and creating a server-side application to orchestrate the lighting array. But in this project the line between data and design was very blurry, and it required us both to be very hands-on at the meeting point of physical and digital. That meant lots of soldering of LEDs and testing in situ at the Make store.

I loved creating a work outside the LCD screen, working with hardware and collaborating with a designer whose knowledge of materials and material production is simply exceptional (I will have to do a post some time on Rob’s F!nk workshop and the astounding Jules Verne industrial contraptions he devises to realise his elegant design forms). It’s ironic that my “getting away from the LCD” project ultimately produces another screen analog; an array composition of pixel-like lighting forms. Mitchell Whitelaw’s eloquent article “AFTER THE SCREEN: ARRAY AESTHETICS AND TRANSMATERIALITY” highlights the interesting qualities and tensions in post-screen works such as Virga. Where the ubiquitous digital screen aims for generality (an ability to display any content at all) and self-effacing slightness (an attempt to disappear as a neutral substrate for content), Virga instead attempts to lower the resolution of the grid and emphasise the material presence of the array elements (Whitelaw). I think in the case of Virga, the tension between physical and digital is integral to its purpose: creating a dialogue between the physical and virtual manifestations of the Make business.

Pat Coppel, Rob Foster, Geoff Hinchcliffe