Skip to content

Upload data

Overview

There are three methods for uploading data into Conode:

  1. šŸ—„ļøĀ Uploading files directly > Jump there
  2. šŸ¤–Ā Uploading from a database > Jump there
  3. āž”ļøĀ Uploading via the Conode API > Jump there

But before we dive into these methods, let's first go through some pointers for accessing the upload function, the types of data encoding, and the preliminary views that will open once data is uploaded using any of the three methods above.

Accessing the Upload Menu

To upload files into Conode, navigate to theĀ Uploadbutton located in the top LHS of the homepage, or the top RHS of the menu when you have an existing graph open. You can either import data into an entirely new graph usingĀ New Graph, or to the graph you are currently working inĀ Add to Graph.

Data Encoding

For structured data sources, the tables will be converted according to Conodeā€™s graph structure.

For media and unstructured text, each file will be uploaded onto a node. The node label will replicate the file title, and the media or text will be contained in the node URL. Learn more about this in customising node properties.

Starting Views

After uploading your data from files, Conode will generate some preliminary views as a suggestion of where to begin your exploration.

The first view will always be calledĀ HomeĀ and contain a full picture of the taxonomy defining your data. If more than one file was uploaded at a time, the home view will populate with multiple taxonomies which you can thenĀ fuseĀ to create a single connected graph of all your data.

Depending on the type and content of file, other views which may open areĀ Histograms,Ā Bar Plots,Ā Embeddings, andĀ Spatial Views. You can see more examples of the types of views commonly opened during data upload in our Tutorials and Blog examples.

šŸ—„ļøĀ Uploading Files Directly

To upload files into Conode, navigate to theĀ UploadĀ button located in the top LHS of the homepage, or the top RHS of the menu when you have an existing graph open. You can either import data into an entirely new graph usingĀ New Graph, or to the graph you are currently working inĀ Add to Graph.

After opening the upload menu, hit UPLOAD FILEĀ to import any locally stored files.

The following file types and extensions are supported:

  • Structured and semi-structured ā†’ .csv,Ā .json,Ā .xml
  • Media ā†’Ā .jpg,Ā .jpeg,Ā .png,Ā .mp4,Ā .mov.
  • Unstructured text ā†’ .txt, .pdf
  • ASCII Point Cloud data ā†’ Ā .xyz,Ā .asc,Ā .obj,Ā .ptx
  • OpenScenario files with theĀ .xoscĀ are also accepted.

FYI

There is currently a 10MB size limit on each file uploaded to Conode. If you wish to work with larger files, pop an email over toĀ shona@drisk.ai.

šŸ¤–Ā Uploading from a Database

To enable fast, iterative, and crucially code-free graph building from large structured data sources that underpin every enterprise, we have built the Conode Upload Agent. This agent allows users to upload data using only natural language. You interact with the agent by asking for the data you would like to upload, which it will then generate a script in SQL, where you can tweak or expand, and once correct, run.

Our helpful AI agents are here to accelerate your work, so make use of them!

1. Setting Up a Connection Database

After opening the upload menu, hit UPLOAD FROM DATABASEĀ to pull in data it from an external source.

We currently support 3 types of databases:

  • Sqlite: Data Source=c:\mydb.db;Version=3;Password=myPassword;

  • MySQL mysql://user:password@host:port/database

  • PostgreSQL: postgresql://user:password@host:port/database

Tip

Use the Retool Utility tool for generating connection strings.

2. Navigating Your Schema

The view which opens -defined as the Home view, will display nodes for all the features which define your data, grouped by the table in which they exist. A key benefit of Conode is that your data is automatically fused during import, and therefore you will notice some overlap. Thanks to the knowledge graph structure, we now have only one node representing each idea, rather than duplicates for every table.

ā­ Did You Know?

Your database has been translated into a fused knowledge graph, from which you see the structure, key distributions and quickly can dive in to any corner of interest.

We now have complete visibility over the schema which defines our domain. This was shown in the data upload video above, where Conode has automatically translated and fused together three, high-dimensional, unstructured data sources into a single unified knowledge graph to interact with directly.

When importing the database schema, the Data Retrieval agent samples 100 rows from each table, providing a quick overview of the database content. This allows you to familiarise yourself with the data before selecting and importing the specific subsets that are most relevant.

Click through the features in the schema and open up the Inspect Agent to check out the possible values in each categorical feature.

You can read more about how to Inspect your data here

3. Building Graph from Schema

Given the size of enterprise database, loading every quest into Conodeā€™s in-memory graph would at some point overwhelm the browser. We have ambitious scalability goals, so while our engineers diligently work to meet these, we build graphs whilst selectively importing the most relevant portions of the database.

To load in the portion of our database we care about without having to write any code, we will use the Conode Upload Agent.

  1. Open the Conode Upload Agent.

  2. Access the Database Connection Node

    As with everything in Conode, your database connection is stored on a node. The connection string itself is stored in the Content field of the ā€œPostgres Connection Nodeā€ which lives in your Home view.

  3. Feed the Database Connection Node to the Agent

    To begin a session with the Upload AI Agent, add, or drag and drop the connection node into the Connection Node box. Youā€™re now set up!

  4. Prompt away!

    You can use natural language to question the upload agent about the contents of your database, or dive in with instructions on which data you would like to import to Conode.

Here are some common examples you can ask of the data:

  • Import only instances where the event happened on (a certain date).

  • Import only instances where the event took place at (a certain place).

You can combine as many conditions as you like to focus on a subset of the data, and get this all code-free!

āž”ļø Uploading via the Conode API

Import data from a Web API using a HTTP request with the URL, headers, and JSON body with either a 'GET' or 'POST' method.

The Conode API offers a python-based route to uploading data onto the platform. Check out our github for more information:

Last update: 2025-01-13