Tables and storage¶

Start from ordinary tables, create an AnnNet graph, and compare storage/interchange formats on the same small graph.

In [1]:

import annnet as an

import annnet as an

Create a graph from a table¶

In [2]:

import polars as pl

edges = pl.DataFrame(
    {
        'edge_id': ['e1', 'e1_parallel', 'e2', 'e3'],
        'source': ['A', 'A', 'B', 'C'],
        'target': ['B', 'B', 'C', 'D'],
        'weight': [0.9, 0.45, 0.7, 0.4],
        'directed': [True, True, True, False],
        'relation': ['activates', 'phosphorylates', 'inhibits', 'binds'],
        'slice': ['baseline', 'stimulated', 'stimulated', 'stimulated'],
    }
)

G = an.from_dataframe(edges, schema='edge_list')
print('shape:', G.shape)
print('slices:', G.slices.list())

import polars as pl edges = pl.DataFrame( { 'edge_id': ['e1', 'e1_parallel', 'e2', 'e3'], 'source': ['A', 'A', 'B', 'C'], 'target': ['B', 'B', 'C', 'D'], 'weight': [0.9, 0.45, 0.7, 0.4], 'directed': [True, True, True, False], 'relation': ['activates', 'phosphorylates', 'inhibits', 'binds'], 'slice': ['baseline', 'stimulated', 'stimulated', 'stimulated'], } ) G = an.from_dataframe(edges, schema='edge_list') print('shape:', G.shape) print('slices:', G.slices.list())

shape: (4, 4)
slices: ['default', 'baseline', 'stimulated']

Export dataframe tables¶

In [3]:

tables = an.to_dataframes(G)
print('tables:', sorted(tables))
tables['edges'].select(['edge_id', 'source', 'target', 'relation', 'weight'])

tables = an.to_dataframes(G) print('tables:', sorted(tables)) tables['edges'].select(['edge_id', 'source', 'target', 'relation', 'weight'])

tables: ['edges', 'hyperedges', 'nodes', 'slice_weights', 'slices']

Out[3]:

shape: (4, 5)

edge_id	source	target	relation	weight
str	str	str	str	f64
"edge_0"	"A"	"B"	"activates"	0.9
"edge_1"	"A"	"B"	"phosphorylates"	0.45
"edge_2"	"B"	"C"	"inhibits"	0.7
"edge_3"	"C"	"D"	"binds"	0.4

Visualize the imported edge table¶

This confirms that the table was interpreted as the intended directed and undirected interactions.

In [4]:

from annnet.utils import plotting

plotting.plot(
    G,
    backend='graphviz',
    show_edge_labels=True,
    edge_label_keys=['relation'],
)

from annnet.utils import plotting plotting.plot( G, backend='graphviz', show_edge_labels=True, edge_label_keys=['relation'], )

Out[4]:

CSV and Excel ingestion¶

In [5]:

import os
import tempfile
import pandas as pd

tmp = tempfile.mkdtemp(prefix='annnet_tables_')
csv_path = os.path.join(tmp, 'edges.csv')
xlsx_path = os.path.join(tmp, 'edges.xlsx')

edges.write_csv(csv_path)
pd.DataFrame(edges.to_dicts()).to_excel(xlsx_path, index=False)

from_csv = an.from_csv(csv_path)
from_excel = an.from_excel(xlsx_path)

print('csv shape:', from_csv.shape)
print('excel shape:', from_excel.shape)
print()
print(open(csv_path, encoding='utf-8').read())

import os import tempfile import pandas as pd tmp = tempfile.mkdtemp(prefix='annnet_tables_') csv_path = os.path.join(tmp, 'edges.csv') xlsx_path = os.path.join(tmp, 'edges.xlsx') edges.write_csv(csv_path) pd.DataFrame(edges.to_dicts()).to_excel(xlsx_path, index=False) from_csv = an.from_csv(csv_path) from_excel = an.from_excel(xlsx_path) print('csv shape:', from_csv.shape) print('excel shape:', from_excel.shape) print() print(open(csv_path, encoding='utf-8').read())

csv shape: (4, 4)
excel shape: (4, 4)

edge_id,source,target,weight,directed,relation,slice
e1,A,B,0.9,true,activates,baseline
e1_parallel,A,B,0.45,true,phosphorylates,stimulated
e2,B,C,0.7,true,inhibits,stimulated
e3,C,D,0.4,false,binds,stimulated

Native, Parquet, JSON, and NDJSON outputs¶

In [6]:

native_path = os.path.join(tmp, 'graph.annnet')
parquet_path = os.path.join(tmp, 'graph_parquet')
json_path = os.path.join(tmp, 'graph.json')
ndjson_path = os.path.join(tmp, 'graph_ndjson')

an.write(G, native_path)
an.to_parquet(G, parquet_path)
an.to_json(G, json_path, indent=2)
an.write_ndjson(G, ndjson_path)

round_trips = {
    'native': an.read(native_path).shape,
    'parquet': an.from_parquet(parquet_path).shape,
    'json': an.from_json(json_path).shape,
    'ndjson files': sorted(os.listdir(ndjson_path)),
}
round_trips

native_path = os.path.join(tmp, 'graph.annnet') parquet_path = os.path.join(tmp, 'graph_parquet') json_path = os.path.join(tmp, 'graph.json') ndjson_path = os.path.join(tmp, 'graph_ndjson') an.write(G, native_path) an.to_parquet(G, parquet_path) an.to_json(G, json_path, indent=2) an.write_ndjson(G, ndjson_path) round_trips = { 'native': an.read(native_path).shape, 'parquet': an.from_parquet(parquet_path).shape, 'json': an.from_json(json_path).shape, 'ndjson files': sorted(os.listdir(ndjson_path)), } round_trips

Out[6]:

{'native': (4, 4),
 'parquet': (4, 4),
 'json': (4, 4),
 'ndjson files': ['edge_slices.ndjson',
  'edges.ndjson',
  'hyperedges.ndjson',
  'nodes.ndjson',
  'slices.ndjson']}

Inspect a JSON and NDJSON payload¶

In [7]:

import json

with open(json_path, encoding='utf-8') as handle:
    json_doc = json.load(handle)
with open(os.path.join(ndjson_path, 'edges.ndjson'), encoding='utf-8') as handle:
    edge_lines = [line.strip() for line in handle if line.strip()]

print('JSON top-level keys:', sorted(json_doc))
print('JSON edges:')
print(json.dumps(json_doc['edges'], indent=2))
print('edges.ndjson:')
for line in edge_lines:
    print(json.dumps(json.loads(line), indent=2))

import json with open(json_path, encoding='utf-8') as handle: json_doc = json.load(handle) with open(os.path.join(ndjson_path, 'edges.ndjson'), encoding='utf-8') as handle: edge_lines = [line.strip() for line in handle if line.strip()] print('JSON top-level keys:', sorted(json_doc)) print('JSON edges:') print(json.dumps(json_doc['edges'], indent=2)) print('edges.ndjson:') for line in edge_lines: print(json.dumps(json.loads(line), indent=2))

JSON top-level keys: ['directed', 'edges', 'multigraph', 'nodes', 'x-extensions']
JSON edges:
[
  {
    "id": "edge_0",
    "source": "A",
    "target": "B",
    "directed": true,
    "weight": 0.9,
    "relation": "activates"
  },
  {
    "id": "edge_1",
    "source": "A",
    "target": "B",
    "directed": true,
    "weight": 0.45,
    "relation": "phosphorylates"
  },
  {
    "id": "edge_2",
    "source": "B",
    "target": "C",
    "directed": true,
    "weight": 0.7,
    "relation": "inhibits"
  },
  {
    "id": "edge_3",
    "source": "C",
    "target": "D",
    "directed": false,
    "weight": 0.4,
    "relation": "binds"
  }
]
edges.ndjson:
{
  "id": "edge_0",
  "source": "A",
  "target": "B",
  "directed": true,
  "weight": 0.9,
  "relation": "activates"
}
{
  "id": "edge_1",
  "source": "A",
  "target": "B",
  "directed": true,
  "weight": 0.45,
  "relation": "phosphorylates"
}
{
  "id": "edge_2",
  "source": "B",
  "target": "C",
  "directed": true,
  "weight": 0.7,
  "relation": "inhibits"
}
{
  "id": "edge_3",
  "source": "C",
  "target": "D",
  "directed": false,
  "weight": 0.4,
  "relation": "binds"
}

CSV and Excel are useful at human-facing boundaries. Native .annnet and Parquet are better when AnnNet remains the source of record. JSON and NDJSON are convenient when another system expects plain structured text.