The ideas behind Niseq

let created = `Date (2020, 9, 7) in

I recently gave a talk at Bornhack where I spoke about the ideas and the technology behind niseq. In this blogpost I'll give a less technical overview of what niseq is.

In a sentence, niseq is an instrument for making live visual art - especially of the abstract kind. At the same time it's an explorative instrument, for finding interesting relations between single images or groups of images - which enables one to work in an intuitive and immediate way with footage. This workflow is very different from how one would work with ordinary video-editing applications (NLE's). This non-immediacy of NLE's is one of my original reasons for coming up with the idea for niseq.

I developed niseq in my spare time throughout the past couple of years, and have been working on the idea of it for about a decade (where I in the meantime also worked on related projects, studied computer science and worked as a programmer). The idea initially stems from how I came up with manually moving the playback-cursor around in my video-footage, while making a video-installation at the Fatamorgana school of photography.

Since the beginning of july I quit my job as a programmer to work on niseq, hopefully ending up with new works of art and performances.

Moving in time

The core technological feature of niseq is that one can access and show any image loaded from disk at 60 frames per second. This enables live, free movement through time - which is hard for me to do justice in a sentence, so I'll list some examples for how I use it currently:

  • instantly switching between 'streams of images', controlled by different complex internal representations
  • running an image-sequence at any possible playback-speed
    • certain visual patterns emerge when you are e.g. close to the integer divisible speed-multiplier, relative to the length of the image-sequence
  • live finegrained control of which image is shown when, so the images shown are coupled as closely as possible to the human input
  • sequencing any set of images + effects together, and recording this as a new sequence to be played upon later, which enables recursion of play
    • this redefines the relations between images, so new image-sequences are constructed of images originally stemming from several stills and videos

Moving in space

Just before the recent Bornhack performance, I implemented an effects system in niseq, where each image can have effects applied and recorded for later. This e.g. includes zooming and stretching images - which is what I call moving in space. Effects also includes fading images together, brightness/contrast and bandpass-filtering (added later).

For now I'm mostly interested in exploring less effect-based workflows, as I see a great value of the resulting art in being relateable to the original material.

Live instrument

A major focus for niseq is being an instrument for live creation of visual art. I use the word instrument in the same sense as when one says 'musical instrument'. Here I mean there being a direct and instant relation between what the human player does with the instrument, and what can be experienced from the instrument.

As niseq is a software instrument, it also has automative features - for example, one can record a rhythm of switching between different streams of images. Though the current primary focus of niseq has been on direct human control, I'm planning to explore generative features at a later point - inspired by how modular synthesizers work. Generative input to niseq is already supported, as niseq is includes a client/server structure, which makes it easily extensible.

Another important reason for extensibility of niseq: keyboard and mouse, which is the current interface to niseq, is a good approximation for a powerful and intuitive interface - though this interface not expressive in communicating what the player actually does towards the audience. New hardware interfaces to control niseq is also a dimension I'm planning to explore.

In the context of history

The idea for niseq initially came to me through experimentation and intuition, and a feeling of there being something missing in the language of cinematic expression of most cinematic experiences I had - especially the mainstream cinema, which I think often has degraded to a fast-food alternative to books.

I've later found that the ideas of the french Dada and Surrealist movements of the 1920's correspond closely with my idea of the unlived potentials of cinema, and I consider the art of niseq part of the related cinéma pur movement.

The non-narrative nature of experimental electronic music has inspired me a lot, as it leads to more direct forms of emotional, aesthetic and atmospheric experiences. This relation to music also existed for the surrealist/cinéma pur filmmaker Germaine Dulac, who wrote (1):

... The cinema can certainly tell a story, but one mustn't forget that the story is nothing. The story is a surface. The seventh art, the art of the screen, is the palpable rendering of the depth which extends beneath this surface: the musical ineffable... Visual impact is ephemeral, it's an impact you receive which suggests a thousand thoughts. An impact analogous to that provoked by musical harmonies.

Collaborations

I've been performing a couple of 'silent cinema' shows with niseq, which I want to do more of in the future - but right now I'm looking at making collaborations with musicians for making improvisational jam sessions. If you are interested in collaborating, write to me at rand at r7p5 dot earth.

Footnotes

  1. Dada and Surrealist Film / edited by Rudolf E. Kuenzli, First MIT Press edition, 1996,