Network Device Collection

5.1 Supported Vendors & OS Types

5.2 Inventory File Format

Devices are defined in a YAML inventory file. By convention this file is named inventory.yaml and stored in the project directory.

devices:
  # Cisco IOS core switch
  - hostname: sw-core-01
    host: 10.0.0.1
    device_type: cisco_ios
    username: netops
    password: "{{ env.SWITCH_PASSWORD }}"
    port: 22
    timeout: 30

  # Arista EOS distribution switch
  - hostname: sw-dist-01
    host: 10.0.0.2
    device_type: arista_eos
    username: netops
    password: "{{ env.SWITCH_PASSWORD }}"

  # PAN-OS firewall (includes firewall policy collection)
  - hostname: fw-edge-01
    host: 10.0.0.10
    device_type: paloalto_panos
    username: admin
    password: "{{ env.FW_PASSWORD }}"

  # Juniper SRX firewall
  - hostname: fw-srx-01
    host: 10.0.0.11
    device_type: juniper_junos
    username: netops
    password: "{{ env.FW_PASSWORD }}"

  # Fortinet FortiGate
  - hostname: fw-fg-01
    host: 10.0.0.12
    device_type: fortinet
    username: admin
    password: "{{ env.FW_PASSWORD }}"

  # Cisco ASA
  - hostname: fw-asa-01
    host: 10.0.0.13
    device_type: cisco_asa
    username: cisco
    password: "{{ env.FW_PASSWORD }}"

Supported device_type Values

Credential Security

Never store credentials in plain text in inventory files. Use environment variable references ({{ env.VAR_NAME }}) or a secrets manager. For Docker deployments, use Docker Secrets or pass credentials via environment variables.

5.3 Running Collection

Collect Raw Data

# Collect from all devices in the inventory file
meshq collect --source inventory.yaml

# Collect from a specific device (by hostname)
meshq collect --source inventory.yaml --device sw-core-01

# Collect with a higher timeout (seconds, default 30)
meshq collect --source inventory.yaml --timeout 60

# Run in parallel (default 4 workers; increase for large networks)
meshq collect --source inventory.yaml --workers 8

Raw CLI output is saved to ~/.meshoptixiq/cache/<hostname>/. Each collection run overwrites the previous cache for that device.

Check Collection Status

meshq status
# Shows last collection time, device count, and any collection errors

Scheduling Regular Collection

For continuous network intelligence, schedule collection with cron or a container orchestrator:

# Cron example — collect every 4 hours
0 */4 * * * /usr/local/bin/meshq collect --source /opt/meshoptixiq/inventory.yaml \
  && /usr/local/bin/meshq parse \
  && /usr/local/bin/meshq ingest

Parse and Ingest

# Parse all cached device data
meshq parse

# Parse a specific device only
meshq parse --device sw-core-01

# Ingest parsed data into the configured graph backend
meshq ingest

# Ingest and reinitialise schema constraints first
meshq ingest --init-schema

# One-command pipeline: collect, parse, and ingest in sequence
meshq collect --source inventory.yaml \
  && meshq parse \
  && meshq ingest

Parsing produces normalised Pydantic models and reports counts: devices, interfaces, IPs, MACs, VLANs, endpoints. Parsing errors are reported per-device and do not abort the run.

Verifying Ingested Data

# Check summary counts via the API
curl -H "X-API-Key: $API_KEY" \
  http://localhost:8000/queries/summary_stats/execute \
  -H "Content-Type: application/json" \
  -d '{"parameters": {}}'

# Expected response
{
  "total": 1,
  "rows": [{
    "device_count": 12,
    "interface_count": 247,
    "endpoint_count": 1843,
    "vlan_count": 18
  }]
}

5.4 Distributed Collection Pro+

For large networks or horizontally-scaled deployments, MeshOptixIQ supports a distributed collection model using a Redis work queue. Multiple collector worker processes pop device tasks from a shared queue, SSH into the devices in parallel, and write results back to the queue.

Prerequisites

Distributed collection requires Redis. Set the REDIS_URL environment variable on all API and worker instances:

export REDIS_URL=redis://redis:6379

Dispatching Devices to the Queue

# Push all devices from inventory.yaml to the Redis work queue
meshq collect --dispatch --source inventory.yaml

# The dispatcher is typically run as a one-shot job
# and does not block — it exits once all devices are enqueued.

Running Collection Workers

# Start a single worker (pop → SSH → ingest loop)
meshq collect --worker

# Customise the polling interval (seconds, default 5)
meshq collect --worker --poll-interval 5

Each worker instance continuously pops device tasks from the queue, collects raw CLI output, parses, and ingests the data into the configured graph backend. Workers exit cleanly when no tasks remain and the queue is empty.

Docker Compose Cluster Setup

The docker-compose.cluster.yml file in the repository provides a ready-to-use configuration with two collector workers and a one-shot dispatcher service:

# Start the full cluster stack (API + Redis + collectors + dispatcher)
docker compose -f docker-compose.cluster.yml up -d

# Scale workers independently
docker compose -f docker-compose.cluster.yml up -d --scale collector=4

# Check queue status
curl -H "X-API-Key: $API_KEY" http://localhost:8000/collect/status

API Endpoints for Queue Management

5.5 Ansible Dynamic Inventory Export Pro+

MeshOptixIQ can generate an Ansible-compatible dynamic inventory from the graph, grouping devices by vendor and automatically placing firewall devices into a dedicated firewalls group.

CLI Export

# Export to stdout (JSON format, compatible with ansible-inventory --list)
meshq export --format ansible

# Save to a file
meshq export --format ansible --output /etc/ansible/inventory.json

# INI format (legacy Ansible format)
meshq export --format ansible-ini --output /etc/ansible/inventory.ini

REST API

# JSON format (default)
GET /inventory/ansible
X-API-Key: your-api-key

# Legacy INI format
GET /inventory/ansible?format=ini
X-API-Key: your-api-key

Example JSON Output

{
  "all": {
    "hosts": {
      "sw-core-01": {
        "ansible_host": "10.0.0.1",
        "ansible_user": "netops",
        "vendor": "cisco",
        "os_version": "17.3.4"
      },
      "fw-edge-01": {
        "ansible_host": "10.0.0.10",
        "ansible_user": "admin",
        "vendor": "paloalto",
        "os_version": "10.2.3"
      }
    },
    "children": ["cisco", "paloalto", "firewalls"]
  },
  "cisco": {
    "hosts": ["sw-core-01", "sw-dist-01"]
  },
  "paloalto": {
    "hosts": ["fw-edge-01"]
  },
  "firewalls": {
    "hosts": ["fw-edge-01", "fw-srx-01", "fw-fg-01"]
  },
  "_meta": {
    "hostvars": { ... }
  }
}

Using with ansible-inventory

# Use as a dynamic inventory script
ansible-inventory -i "http://localhost:8000/inventory/ansible" --list

# Or reference the exported file
ansible-inventory -i /etc/ansible/inventory.json --list

# Run a playbook against firewall devices
ansible-playbook -i /etc/ansible/inventory.json \
  --limit firewalls \
  playbooks/audit_fw_config.yaml