AI-Songlines for Corporate Data

Ensuring a single source of truth for Corporate Artificial Intelligence Data

Introduction.

Distributed organisations, at least the ones I have dealt with, have IT platforms made up of separate, loosely linked systems characterised as a collection of single-point solutions.

These platforms often include cloud-based data warehouses to meet reporting needs, but invariably, they are partitioned according to point data sources.

A collection of single-point applications does not provide a single source of truth for precisely describing and understanding corporate-wide data.

To remedy this situation, I discuss a design for a data system that would provide an organisation with an AI-supervised and validated single source of truth for all its applications.

AI and the Data Problem.

Data comes in two forms. Unstructured, where you know what the data object contains but not exactly how it is arranged, and structured data, in which you know exactly what the data object contains and exactly how it is arranged.

Knowledge of data is known as metadata. Metadata must include timelines because data, especially transactional data, loses its value over time.

Successful AI applications depend upon powerful AI algorithms processing validated and normalised data with accurate supporting metadata.

Common Data Issues I Have Encountered.

One organisation’s inventory of trucks was dispersed over three depots. Truck data was maintained at each depot. Managers wanted to analyse the entire fleet to identify cost savings using AI.

The existing practice was to combine data from the depots to form a table (matrix) with a row for each truck and a column for each of its attributes, such as load, colour, registration, and truck type (e.g., articulated or rigid).

Each depot, for historical reasons, had a unique set of codes for each of its truck’s colour schemes. One way of handling this would be for trucks in the amalgamated data to have a colour attribute for each depot.

Another way is to have a single attribute with the colour code normalised across all possible depot values. This situation creates issues with the AI algorithm, both in performance, predictions, and how managers would interpret the result.

I have been grappling with similar data and metadata issues for some time. In 2016, I came across the aural memory training technique called Songlines.

Songlines were developed, refined, and practiced by Australian Aboriginals over many thousands of years. I became convinced that the memory training techniques of Songlines held the solution to my data and metadata problems.

An Overview of Songlines

After completing my PhD in 2016, I studied Lynne Kelly’s book “The Memory Code” published by Allen & Unwin. Her book described how cultural memory was passed down through countless generations of indigenous Australians using a memory technique called Songlines.

The Songline embodies the soul of the Australian Aboriginal culture. When tribal elders perform a songline, the tribe as a whole experiences the rich cultural heritage that emphasizes an individual’s role in the tribe and the tribe’s connectedness to the land that sustains them.

Songlines have accurately preserved the knowledge of the Aboriginal culture over many thousands of years. In fact, this culture is the oldest continuous living culture on Earth.

The Songline is literally a path in the landscape linking sacred locations. Songlines are bound to the progression of seasons. Tribal elders carefully coordinate the travel timing between sites to ensure harmony between season and ritual performances. Elders perform Songlines through recitals, songs, and dance — they literally “sing the landscape.”

A site’s location dictates the order of performances in a Songline, as the tribe must walk between locations.

Distinctive landscape features and the presence of animals at each site act as memory cues for rituals. Australian Aboriginals also make use of rock art, bark paintings, and carvings to act as memory aids.

I believe everyone has their own Songline. I relive my songline whenever I walk in the forest near my apartment. Over two decades, I became familiar with the trees and often stopped beside my favourites.

I see and hear kookaburras, magpies, and lorikeets. The bark of each turpentine tree tells its own story. My walks, actual and remembered, merge to give me a sense of time and ancestry.

Unashamedly, I use Songlines’ sense of connectedness in my own thinking. I distinguish my practice from critical thinking by naming it “Songline Thinking.”

Songline Thinking and Data Abstraction.

Songlines are all about connectedness with the landscape. The diagram below shows the simplified abstraction of a Songline. An activity takes place at a location and involves participants from the tribe.

Resources such as bark paintings may be used in the activity. Participants travel to each location following a path in the landscape and then move on once the activity is completed.

The simplest songline is a single activity performed at a single location. This is shown in the diagram below, and I refer to this as a Songline Unit. The diagram emphasizes connectedness.

Taking the structure of the Songline Unit, I have designed a programmable IT object. All entities in the object representing a Songline Unit, namely location, activity, participant, and resource, have attributes that may be inspected, amended, or used by an appropriate AI tool.

In keeping with the Songline motif, the “world” is made up of all possible Songlines and is simply referred to as the “Landscape.”

The Songline Unit accounts for time, as each activity takes time to perform. Activities in Songlines have time dependencies. The “Path Link From” may represent a path back in time to an activity at an already visited location. The “Path Link To” could represent a path to a planned or possible future location. It’s conceivable that a “Path Link To” could point to a broad spectrum of possible future undiscovered Songline Units.

Connected Songline Units form a Songline used for a specific purpose, for example, building, maintaining, and housing a truck, undertaking a course of study, or walking through a forest.

My intention is to develop the ability to construct Songlines using a series of AI tools that parse all available data and metadata. In this way, constructed Songlines can be developed, adapted, and fitted to manage data or take into account data anomalies.

The Songline structure is a metaphor for the table of contents in a book. It provides a familiar and accepted way to explore Songline’s data.

AI-Songlines.

I am planning a “deliverable prototype” of an AI-Songline. A deliverable prototype provides a useful outcome but requires expert intervention to achieve a result. In “design thinking” terms, this would be called a heuristic.

The simplified process diagram below implies a waterfall approach to using AI tools. Here, an AI tool only executes if the previous tool completes its task successfully.

In my proposed prototype, all AI tools are active. Each tool, upon completing a process on a single data item, sets its processing status. Other tools check the item’s status to see if they are required to process the data item. Analysis and reporting is always based on the condition of data at a particular point in time.

Coming back to the truck colour problem, AI-Songline tools could map depot colours to unique colour codes stored within the AI-songline’s metadata. When the data is accessed for viewing, the AI-Songline colour code would be mapped back to the truck’s depot colour code for a display to users.

My development platform is based around Anaconda 3. I have TensorFlow and PyTorch as my prime libraries. I use scikit-learn’s tools for testing ideas and exploring data relationships.

Using multiple AI-Songline tools enables data triangulation to identify possible processing issues and biases introduced by individual AI libraries.

Aspirations

An AI-Songline acts as a “single source of truth” approach to recording, analysing and understanding business processes and interactions. All reports and predictions are based on the use of validated data in its correct context.

AI-Songlines could be applied to dynamic situations such as contact tracing and disaster relief.

Blessed be.