donbr

Crisis Management - a personal perspective

The COVID-19 pandemic highlighted critical gaps in our crisis response capabilities, particularly in an industry where I worked directly. This paper examines the requirements for effective training and preparedness during acute phases of the outbreak.

A fundamental challenge lies in preparing both human operators and their supporting AI systems to respond to unprecedented scenarios. Crisis events, by their nature, often fall outside of the standard validation parameters used to test AI systems' generalization capabilities.

The ongoing refinement of the Graph Simulation Pipeline and associated open-source datasets represents a crucial element in advancing crisis preparedness. These tools enhance our ability to transform overwhelming data streams into actionable intelligence during evolving situations.

The framework incorporates two key components:

Replay Mode: leverages tools like NetworkX and TigerGraph to model temporal dependencies using historical crisis

🚀 Multi-Stakeholder Perspective: Air Cargo Disruptions During COVID-19 (One-Week Interval)

📌 Global Meta Event: COVID-19 Outbreak (March 2020 - Global Lockdowns, Border Closures, & Supply Chain Shocks)
📆 Time Window: March 16–22, 2020 (One-Week Interval of Major Cargo Disruptions)
🌍 Disruption: Rapid lockdowns, flight cancellations, PPE (Personal Protective Equipment) demand spikes, and re-routing bottlenecks.

Key Stakeholders & Their Perspectives:

📦 Shipper (Medical Equipment Manufacturer) → Focus: Single Critical PPE Shipment
✈ Airline Cargo Operations (Major Carrier) → Focus: Hundreds of disrupted flights & rerouted shipments
🌏 Freight Forwarder (Global Logistics Company) → Focus: Managing demand spikes, re-bookings, and rate volatility

Knowledge Graph Construction

Some fundamental concepts

Data Ingestion

NOTE: mermaid.js diagrams below are based on some inspiring content from the Connected Data London 2024: Entity Resolved Knowledge Graphs masterclass.

graph TD

╭──────────────────────────────────────────────────────────── New run ────────────────────────────────────────────────────────────╮ │ │ │ Correlate TP53BP1 protein interactions with mRNA co-expression patterns in TCGA breast cancer data. │ │ │ │ If you have the required libraries and functions to complete the task: │ │ 1. run the code │ │ 2. limit the response to the first 5 records for initial validation │ │

Code Examples from smolagents Documentation

Proposal: Enhanced Interactive Knowledge Graph (IKG) for Real-Time Situational Awareness

Objective

To design and develop a multi-agent, dynamic, and interactive knowledge graph (IKG) system that integrates real-time data fusion, ontology-based semantic layers, and advanced human-in-the-loop capabilities. This system will enhance situational awareness (SA) for Air Force operations by allowing autonomous agents to dynamically manage, analyze, and adapt knowledge graphs, ensuring timely and effective decision-making in rapidly evolving environments.

Title: Integrating Arize Phoenix and Apache Iceberg for Local Telemetry Data Management and Querying

Authors: Don Branson

Abstract

In modern data observability workflows, capturing and managing telemetry data is crucial for debugging and improving machine learning systems. This paper demonstrates the integration of Arize Phoenix, an open-source observability platform, with Apache Iceberg, a high-performance table format for data lakes, to create a scalable and efficient local telemetry data management and querying system. We present a step-by-step implementation for capturing telemetry data as Parquet files using Arize Phoenix on a local system and using Apache Iceberg to enable schema evolution, time travel, and efficient queries. This solution bridges the gap between data observability and data lake management for machine learning monitoring.

FAQ: Resolving Ollama Service "Permission Denied" Error

Issue

When starting the Ollama service, it fails with the following error message:

Error: could not create directory mkdir /usr/share/ollama: permission denied

A Hypothetical Medium Post...

I created a cool prototype for airline data last year but misplaced the code, so I recently created this "synthetic" prototype stringing together a few excerpts from old ChatGPT messages and simple prompt engineering.

I haven't found the datasource yet, but going through the process has been such a gift. I knew these concepts were becoming essential again and this helped to both remember key concepts AND generate excitement around new opportunities.

Reminds me of Hypothetical Document Embeddings... appparently HyDE can help me retrieve stuff from long term memory too.

Feel free to have a laugh at my expense!