Future Design Software: The ExerciseUI

Recent research (see Men’s Health, NPR & NYTimes) has shown that sedentary office jobs, like those of designers, shorten lives, increase heart disease and increase weight. Since the research is conclusive and given the numbers of sedentary computer users about to enter their 50’s in the next decade the risk of Health and Safety lawsuits for employers increases. Designers are particularly at risk due to the addition of deadline hyper-focus and overstress. Sedentary jobs could become the next biggest wave of health lawsuits since the OOS/RSI lawsuits of the 1990s. If this happens then work place computer usage will need to change to be less sedentary.

At first employers will provide subsidized gym memberships, but these will be of limited use because gym attendance needs extra hours in the day. Employers will mandate exercise breaks but stressed users will not take these when under deadline pressure. The solution will be to include more movement in the actual everyday usage of computers.

There already is a growing awareness of the fitness potential of gesture based user interfaces. Nintendo already sell exercise related games that are entirely gesture based under the Wii Fit name. PlayStation Move and Xbox Kinect are Sony's and Microsoft’s answers to gesture based gaming. Since the exercise potential of gestural user interfaces is obvious then it will make sense to build exercise into the office computer user interfaces of the future. Imagine controlling your computer and getting a work-out at the same time. An ExerciseUI is the cross over between gestural user interfaces and physical fitness.

Since we already have widely available consumer gestural technology then the only barrier to adoption is social and cultural.

Widespread adoption of ExerciseUIs will cause gym memberships to fall. However, personal trainers can find new roles on user interface design teams where they will consult on safe and good mixtures of movement for users. Since exercise is a very individualized thing, offices will have consultant personal trainers who will work with users to customize their ExerciseUIs for their health and ability needs.

At first people will complain about the extra physical effort needed to drive the ExerciseUI but their fitness levels will gradually increase. Short-term productivity may decrease compared to current mouse GUIs but longer term productivity and morale should increase for many tasks. The reason is that more exercise will improve physical health, mental health and give happy endorphin boosts.

During periods of deadline stress the extra physical movements and speed to drive the computer faster will help burn off deadly overstress hormones. Stressful periods that overwork the body into anaerobic stiffness will physically force users to rest between tasks. This will help them mentally balance stressful work better as they learn to take breaks when their body forces them and rest their brains.

Workplaces will need to change dress codes to allow for more exercise friendly clothing. Showering facilities will also be needed. ExerciseUIs will also require a bit more space per user compared to current office cubicles.

Provided ExerciseUIs give good productivity for common office computing tasks then there are few downsides and many benefits to adopting them. Lawsuit pressure might force this to occur sooner rather than later.

(Read more articles in the Future Design Software series.)
(Read about the first ExerciseUI project: WorkOut Poker Episode 1

Future Design Software: Design Anywhere, Anytime

Mobile computing and cloud computing are big trends in computing at the moment. Mobile computing (and tablet computing) is the push for smaller devices that travel with us. Cloud computing is the push for computing to happen in large data centers connected to over the internet instead of on local machines. Graphic design has resisted these trends because it requires flexible and powerful workstations but it is undeniable that these two trends will have an impact on future design software.

There is an saying that “The best camera is the one that’s with you” (Chase Jarvis). This could also be applied to the idea that the best computing device is the one that’s with you. As mobile devices increase in capability it is inevitable that we will be doing more and more design work on them. Today, capable mobile devices can update websites, color correct photographs, edit videos and sketch concepts. While the productivity does not yet match desktop computers with more powerful processors and larger screens, it might be quicker to just do something on a mobile device in the moment than travel back to a more capable computer.

This is highlights an inherent weakness with desktop workstations – they are not easily portable and thus remain fixed in location. The trend is for people to spend less time at a personal non-portable computer and get out and about more. This means that personal computing becomes less important and mobile computing increases in importance.

Laptop computers are not really ideal. They are somewhat portable but battery charge life is still too short, the device too heavy for casual just-in-case carrying and the screen and trackpad combination not ideal for interaction. Laptops lack the always-on property of other mobile devices. Laptops will live on as slightly more portable versions of personal desktop computers but will generally lose out to other mobile devices like tablets.

At present most cloud based services work on data-storage with a web-application front end and maybe some custom software installed on local machines. In the future more applications with better interaction than current web-application silos will live in the cloud also. Computing will also be pushed into the cloud. This means that when the local device encounters a task that overwhelms it then that processing task can be seamlessly performed on the cloud. Seamless means with no user intervention required. Imagine if videos and complex 3D material rendered faster than realtime because of the economies of scale that large cloud computing datacenters can achieve.

The future will have more public kiosk computers and less laptops. When a mobile device is not enough then a designer can login to another other computer (such as a kiosk) and have their data, applications and processing power immediately available in a secure fashion. This can happen on a computer borrowed in a client company or a kiosk computer in a coffee shop/library. Desktops and laptops will become less numerous as more portable mobile devices become more prevalent and designers can rely on access to convenient access to computing kiosks.

Designers will then be freed to travel more – networking, visiting clients, getting in touch with their inspiration, discovering their target audiences. Journeys like this will enrich the design process rather than be seen as unwelcome distractions from productivity. Design might become more human again.

(Read more articles in the Future Design Software series.)

Future Design Software: Streamlining Critique

Critique is important for designers to judge the appropriateness of their designs and to progress in their craft. Unfortunately for the working designer the pressures of budget and time and the difficulty of finding people to critique work mean that critique rarely happens. Future design software could change that.

Ideally designers should get critique from designer peers, clients and the target audience. Each brings a different perspective to the work that is important for different reasons. Client critique is usually built into the design process because it is the easiest to do – there is often a single client representative who critiques before they signoff work.

Critique from peer designers is more difficult to come by. There are design communities like Behance and DeviantArt that will critique but those communities are too public for pre-release design and the level of critique is often not high – despite being enthusiastic. Design critique should come from experienced people whose opinions you trust. These people are probably already in your social network (like Facebook).

Research I co-supervised for Kerrin Meek’s Honours work looked at how social media could be used for designer critique. While in the early stages, many good ideas were explored. The central idea is that by extending critique notifications into the activity stream of others and making those critiques easy to accomplish, peer designer critique might become more successful.

Facebook already allows for posting images and collecting comments. This can functionality can be extended by adding semantic differentials, alternative image voting, and critiques where hotspots can be highlighted for comment. This can be done via a third-part website that integrates tightly into the Facebook experience using the existing Facebook API.

Target audience critique is a more difficult to organize – especially with the speed demanded by many design projects. Market research companies already have large databases of willing participants sorted by demographics that are used to working online. The missing piece of future design software is a website where designers can place their work for critique in standard forms (comments, votes, surveys, questionnaire, semantic differentials), assign a budget, and have that targeted towards a particular demographic. If the process was thus streamlined through future design software then costs could come down and turn-around times improved. As costs and time decrease then the feasibility of adding user critique to a project increases.

Future design software has the potential to revolutionize designer peer critique and make target audience critique more affordable both in terms of time and money. Would you consider expanding critique in your projects?

(This is another article in the Future Design Software series.)

Future Design Software: Ranged Imaging Cameras

Ranged image cameras are a recent technology that capture distance information as well as light. As software support develops these cameras may come to have a big impact on the way that photos are taken and manipulated for use in graphic design compositions.

A single 2D photograph that includes distance information becomes 2.5D. This is not full 3D because the camera has only captured light and distance information from a single perspective. However future photo manipulation tools can take advantage of depth information. Depth information will help software determine more naturally the objects contained within a scene so that pixel masking will become easier. Simply every adjacent pixel at a similar depth is probably part of the same object. Combining existing 2D edge detection with the distance information will make automatic object selection much more accurate and faster. So objects can be moved, moved or replaced much easier.

Depth information will allow for stretching in the Z dimension to create interesting perspective effects. By manipulating the depth information directly and recasting the perspective into 2D space rooms could appear longer, a car on a road accelerates and a scene can be flatten like an old cartoon.

The most interesting possibilities come when multiple ranged image cameras are used. A simple rig of two (or more) cameras starts to give 3D capabilities. The rotation of objects could be tweaked, the angle of the viewport could be tweaked or the position of the camera changed. Such data could be fed into 3D software or 3D painting programs with a little conversion.

If objects are moved in 3D space then the lighting will look wrong. Fortunately, in 3D with pixels and ranges, it could be possible to guess the light sources and perhaps even automatically modify the pixels as they are rotated to fix lighting effects. Or the designer could leave the lighting incorrect on purpose – relying on the near undetectable incongruity of “wrong” lighting to have an attracting effect on the viewer.

Technologies like QuickTimeVR would move in a far more life-like manner if ranged image information is taken into account. Presently zooming in QuickTimeVR tends to stretch objects that are near to the camera, but off centre from the zoom. Utilizing distance information would let QuickTimeVR handle these objects in a more natural way.

The possibilities for prosumer level ranged imaging are enormous and could have a big impact on the way we edit images in future.

(Read more articles in the Future Design Software series)

Future Design Software: Studio+ Project Workflow Tools

Our current software enables us to work at the level of documents but has very little to help with the wider issues of managing a project and the associated work flows. While there is software that solves parts of this problem the real power comes when the pieces come together as an integrated whole. What might such a piece of software look like? Let’s call this hypothetical software Studio+.

At the beginning of a project client details and project details could be input into Studio+. Projects can be based on project templates from a library (or an existing project). Once all deliverables, deadlines and other parameters are known then a full contract can be generated for the client and briefs for the design teams. Studio+ will track the status of such project and alert if something is taking too long. Studio+ will also create appropriate disk directories for projects and schedule appropriate backups.

Studio+ can also create briefs for outsourcing parts of the work to freelancers. Briefs can be sent to certain people or published to an online marketplace for bids. Studio+ has tools to help project managers select from freelance bids, send out briefs, send out source files and integrate completed files into the overall project. Freelancers can temporarily be given shared access to project files.

Once the project is underway Studio+ can automatically associate files and folders with a project. This means that time-based billing can occur automatically when a designer opens a file for work – of course this can be manually modified if the designer leaves the file open while they go for lunch. While time-based billing might not be in use, studio managers will want to know how long designers are spending on tasks to better plan their future human resource needs.

Studio+ has knowledge of the files necessary to create milestone outputs. Once the project reaches a milestone then the files can be assembled and converted for delivery to the client for sign off. Perhaps a studio manager will want to do a final approval before the milestone material is sent to the client. The signoff process can be fully automated via return email.

Studio+ can also communicate status updates to clients and report both the overall progress of a project and when the client is expected to do something (sign off approval or provide content). By integrating project management tools Studio+ is able to estimate the expected finish date of a project. Perhaps clients can engage better with the design process when their role is known.

Studio+ also knows the relationship between a rendered file and the source file that created it. When a source file is changed the designer is asked if they would automatically like to update the rendered file. This applies to JPEGs, Videos, Animations and PDFs.

Once a project is completed Studio+ can archive files for the future and change the file backup policy. Internal reflective reviews can be incorporated to help designers feedback process improvement ideas to management. The account managers will also be signalled to suggest when to do follow up calls to the client.

Better project management and workflow tools can help make the studio more efficient and provide valuable metrics for the proper allocation of resources.

(This is the third article in Future Design Software series.)

Future Design Software: File|Save is stupid. What should replace it?

Why must users make conscious decisions when to save a file? Surely a computer can work this out for itself? Imagine if saving files just happened automatically. Imagine being able to undo actions somebody performed on a file ten years ago? It is generally not possible with the current way that files are saved but a software change can make this possible. Welcome to a rant I’ve had for the last decade.

Current Save actions place a snapshot of the current state of a file on permanent storage. Forget to save and it’s only one power outage away from the loss of hours of work. Files are created by a stream of user actions which each change the state of the file. If a computer saved actions as they were being performed then the current state of a file can be recreated by just replaying the action stream. Work can resume where the user was just seconds before an interruption.

File | Save is replaced by a “Bookmark Progress Milestone” command that inserts a status bookmark into the action stream that indicates some significance to that state of the file. A user should be able to annotate these bookmarks – and even rewind the action stream to add a bookmark to a past state. If a user wants to create different variations of the same file (e.g. a designer wanting to make a blue version and a red version) then the action stream can be forked. Our current Undo actions should rewind the action stream then create a fork in the action stream when the user resumes performing actions. This means that even an Undo action can be undone.

It should be left to the computer to decide when to create snapshots of the file state. For speed more snapshots can be created. The computer could create snapshots after every computationally expensive action just to make opening the file quicker in future. Snapshots at bookmarks are probably reasonable too.

Snapshots can be deleted to save storage space because only the action stream is needed to recreate the file state at any point in time. Snapshot culling should be integrated into the operating system so that it can be triggered automatically when storage space runs low even if the original creating programme is not longer available. Some actions in the action stream could be merged and perhaps some useless forks culled – like the forks created by Undo actions.

File | “Save as ...” still has an important role. It takes any point in an action stream (usually the most current one) and allows the user to save a snapshot to a different file type (e.g. a Photoshop file to a JPEG or a document to a PDF). “Save as …” should also create a bookmark in the action stream.

Storage space is cheap and managing storage space should be a task we delegate to the operating system. Why waste precious human time? The only reasons to keep Snapshot based savings are based on historical constraints that no longer exist in modern computer systems. Time for File | Save to go!

(This is the second article in Future Design Software. Read the first article: Design at the level of thought, message and feeling.)

Future Design Software: Design at the level of thought, message and feeling.

Graphic design software has steadily increased in power. At first the focus was on improving the visual expression capability of the software but gradually, with the addition of work-flow tools, productivity has become important. However, today’s software is limited by a direct manipulation mindset.

Direct manipulation interfaces allow designers to tweak individual elements on screen by using a mouse and keyboard (and/or commonly a tablet). This means that the designer’s vision must be transformed into discrete steps of computer interactions - think of it like the visual equivalent of programming. Any adjustments to the “feel” of a design piece are non-trivial and will often involve many interactions.

What if there was software for designers that worked more at the level of thought, message and feel? There are two areas of research that combine to produce the design software of the future: Generative Design and Aesthetic Science.

Generative design systems take parameters then produce a set of candidate designs for a designer to select from. Sophisticated generative design systems can then use those selected items to further iterate and produce more (and hopefully better) candidate designs. Imagine feeding the text and image library for a poster to software and having it come up with thousands of concepts for you within seconds.

Aesthetic Science tests (usually following Psychology modes of testing) for aesthetics that are universal. Once an aesthetic principle has been established as universal then it can be algorithmically modelled. A computer can run the algorithms to judge design pieces on an aesthetic level. Yes, aesthetics is a complicated area but we do not need a complete understanding to produce computer tools that can still be useful.

Examples of existing work in aesthetic algorithms are Donald Knuth and the TEX typesetting system, auto-balance/contrast and colour correction in photographic software and the Text Colour Contrast algorithms used by the W3C. My own research is searching for algorithms for measuring the design principle of unity. Unity has already been established as an aesthetic universal by fMRI brain scanner research in Europe. My research is in the blue-sky stage.

A criticism of generative design is that the generated candidate designs tend to break too many aesthetic universals. The designer ends up weeding out useless chaff rather than thinking at the level of message and feel.

Future design software could use a generative system approach but incorporate aesthetic algorithms to pre-screen the concepts shown to the designer. The designer can then discard designs, keep designs and even use a semantic differential like system for rating the feel of designs. Then, the generative system can iterate more concepts – for as many rounds as the designer deems necessary. Changing the feel of a design is simply a matter of asking for something in emotive terms : e.g. A bit happier and a stronger sense of community. At the end the designer gets a file they can load into direct manipulation software (like InDesign) for refinement. This process could take less than five minutes.

UPDATE 20130401: Complexity Science (eturnerx blog article) could provide insights that help build a generative design system. Conventional science methods might be too limited.

Welcome to a more human and productive design future. There will be more articles in this series.

I invite your comments. Your reactions inspire me. I hope my ideas are worthy of sharing with your posse.

This is the first article in the Future Design Software series. Read the second article: File|Save is stupid. What should replace it?.)