The ideas behind Niseq

let created = `Date (2020, 9, 7) in

I recently gave a talk at Bornhack where I spoke about the ideas and the technology behind niseq - note that this post is not a direct copy/paste of the talk.

So what is niseq? In a sentence, it's an instrument for making live visual art - especially of the abstract kind. At the same time it's an explorative instrument, for finding interesting relations between (single or groups of) images - which enables one to work in an intuitive and immediate way with footage, in stark contrast with how one would work with ordinary video-editing applications (NLE's).

I developed niseq in my spare time throughout the past couple of years, and have been working on the idea of it for about a decade (where I in the meantime also worked on related projects, studied computer science and worked as a programmer). The idea initially stems from how I came up with manually moving the playback-cursor around in my video-footage, while making a video-installation at the Fatamorgana school of photography.

Since the beginning of july I quit my job as a programmer, to work on niseq instead, now trying to make a living on making art and art-technology.

Moving in time

Niseq supports effects and many other things - but the core technological feature of niseq is that one can access any image on disk at 60 frames per second. This enables live, free movement through time - which is hard for me to do justice in a sentence, so I'll list some examples for how I use it currently:

  • instantly switching between 'streams of images', controlled by different complex internal representations
  • running an image-sequence at any possible playback-speed
    • certain visual patterns emerge when you are e.g. close to the integer divisible speed-multiplier, relative to the length of the image-sequence
  • finegrained manual control of which image is shown when, so the images shown are coupled as closely as possible (in time and space) to my human input to niseq
  • sequencing any set of images + effects together, and recording this as a new sequence to be played upon later (recursion of play)
    • therefore breaking the original relations between images, stemming from e.g. a video, and creating new relations live

I have a lot of different ideas on the drawing board for how to extend niseq to exploit this feature even more.

Moving in space

Just before the recent Bornhack performance, I implemented an effects system in niseq, where each image can have effects applied and recorded for later. For now this e.g. includes zooming and stretching images - which is what I call 'moving in space', and also includes fading images together.

Live instrument

A major focus for niseq is being an instrument for live creation of visual art. I use the word instrument in the same sense as when one says 'musical instrument'. Here I mean there being a direct relation between what the human player does with the instrument, and what can be experienced from the output of the instrument. And this relation should be observed within a short timeframe. When I say output in this sense, I mean the final output - not a temporary output that is not intended for an audience.

This has consequences for the implementation of niseq, which I'll go into detail with in a later blogpost.

Seen from the outside, this means that niseq is designed to:

  • be directly manipulated by the player of the instrument, with minimal delay
  • be extensible with more tangible and bodily expressive interfaces for human interaction
  • be stable, for live performing

Of course, as niseq is software, I also have features that enable automation of certain things - for example one can record a rhythm of switching between different streams of images, and most things that one controls directly are 'settings' that control simply automated streams of events.

Concerning the importance of extensibility: keyboard and mouse is a good approximation for a good interface for interaction, but it is not expressive in communicating what the player actually does, towards the audience. In this scope of design there are many interesting potentials I want to explore.

In the context of history

The idea for niseq initially came to me through experimentation and intuition, and a feeling of there being something missing in the language of cinematic expression of most cinematic experiences I had - especially the mainstream cinema, which I think often has degraded to a fast-food alternative to books.

I've found that the ideas of the french Dada and Surrealist movements of the 1920's correspond closely with my idea of the unlived potentials of cinema, and I consider the art of niseq part of the related cinéma pur movement.

The non-narrative nature of experimental electronic music has inspired me a lot, as it leads to more direct forms of emotional, aesthetic and atmospheric experiences. This relation to music also existed for the surrealist/cinéma pur filmmaker Germaine Dulac, who wrote (1):

... The cinema can certainly tell a story, but one mustn't forget that the story is nothing. The story is a surface. The seventh art, the art of the screen, is the palpable rendering of the depth which extends beneath this surface: the musical ineffable... Visual impact is ephemeral, it's an impact you receive which suggests a thousand thoughts. An impact analogous to that provoked by musical harmonies.

Collaborations

I've been performing a couple of 'silent cinema' shows with niseq, which I want to do more of in the future - but right now I'm looking at making collaborations with musicians for making improvisational jam sessions, hopefully ending up in some interesting audiovisual live concerts. If you are interested in collaborating, write to me at rand at r7p5 dot earth.

Footnotes

  1. Dada and Surrealist Film / edited by Rudolf E. Kuenzli, First MIT Press edition, 1996,