Skip to content

Connector Architecture

Overview

Floh's connector system provides a unified interface for integrating with external services, whether SaaS platforms, on-premise applications, or custom APIs. Connectors abstract the specifics of each integration behind a standardized execution model, allowing workflows to interact with any system through a consistent set of commands.

Design Principles

  1. Multi-modal execution — Support built-in, scripted, OAS-derived, and external connectors through a single registry
  2. Secure by default — Sandboxed script execution, encrypted secrets, and scoped permissions
  3. Observable — Integrated logging with configurable levels, impact analysis, and audit trails
  4. Testable — Built-in mock mode at multiple granularity levels
  5. Evolvable — Schema versioning with breaking change detection

Execution Models

┌─────────────────────────────────────────────────────────────────┐
│                    Connector Registry                           │
│                                                                 │
│  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌───────────┐│
│  │  Built-in   │  │   Script   │  │    OAS     │  │  External ││
│  │  (Native)   │  │ (QuickJS)  │  │ (Generated)│  │  (HTTP)   ││
│  └──────┬─────┘  └──────┬─────┘  └──────┬─────┘  └─────┬─────┘│
│         │               │               │               │       │
│         ▼               ▼               ▼               ▼       │
│  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌───────────┐│
│  │ TypeScript  │  │ Worker     │  │ Worker     │  │  HTTP     ││
│  │ function    │  │ Thread +   │  │ Thread +   │  │  POST     ││
│  │ call        │  │ QuickJS VM │  │ QuickJS VM │  │  /execute ││
│  └────────────┘  └────────────┘  └────────────┘  └───────────┘│
└─────────────────────────────────────────────────────────────────┘

Built-in (built_in)

Native TypeScript connectors compiled into the server. These have direct access to Node.js APIs and are registered in the connector registry at startup. Examples include the http, delay, google-workspace, and outbound scim connectors. The outbound SCIM connector is reference-grade for IdP provisioning use cases — see Outbound SCIM Connector.

Built-in types are authored with the defineConnector() framework under modules/connectors/define/ — a typed DSL that captures a connector's schema, commands, and handlers in one module and auto-derives:

  • The runtime handler registered with registerConnector() (via auto-discover.ts at bootstrap).
  • The ConnectorSeed emitted to the connector_type table (via derive-seed.ts), preserving the legacy configSchema wire format.

Adding a new built-in type is a single module under handlers/ plus one entry in handlers/index.ts; routes.ts, registry.ts, and seed-connectors.ts require no edits. See the Creation Guide for worked examples.

Best for: Core integrations that ship with Floh, performance-critical connectors.

Script (script)

Custom JavaScript connectors executed in a sandboxed QuickJS environment via Node.js worker_threads. Scripts communicate with the host through a controlled floh.* API surface.

Best for: Custom integrations, user-authored connectors, connectors that need isolation.

OAS-Derived (oas)

Connectors auto-generated from OpenAPI 3.x specifications. The OAS parser extracts operations, parameters, and security schemes to produce commands and a generated script that runs in the same QuickJS sandbox as script connectors.

Best for: Rapid integration with APIs that provide OpenAPI specs.

External (external)

Remote connectors accessed over HTTP. Floh sends a POST request to the connector's /execute endpoint with the command, parameters, and metadata. External connectors implement a standardized protocol.

Best for: Connectors running in separate processes, microservices, or managed by third parties.

Data Flow

Workflow Step → StepExecutor → Mock Check → ConnectorRegistry
                    │                              │
                    ▼                              ▼
              ConnectorLogger            Execution Model Router
                    │                    ┌────┬────┬────┬────┐
                    ▼                    │BI  │Scr │OAS │Ext │
              LogService                 └────┴────┴────┴────┘
                    │                              │
                    ▼                              ▼
              system_log table            ConnectorResult
  1. The StepExecutor receives a connector step from the workflow engine
  2. It looks up the connector definition from the database
  3. A ConnectorLogger is created with the connector's configured log level
  4. If mock mode is active (step, connector, run, or system level), mock execution runs instead
  5. Otherwise, the ConnectorRegistry routes execution based on execution_model
  6. The result is returned to the workflow engine as output variables

Database Schema

connector_definition table

Column Type Description
id UUID Primary key
name VARCHAR Unique connector name
type VARCHAR Connector type (maps to registry handler)
description TEXT Human-readable description
version VARCHAR Semantic version
execution_model VARCHAR built_in, script, oas, external
script_source TEXT JavaScript source for script/OAS connectors
mock_data JSONB Static mock scenarios per command
mock_script TEXT JavaScript source for dynamic mocks
mock_enabled BOOLEAN Whether mock mode is active
log_level VARCHAR Minimum log level: trace/debug/info/warn/error/fatal
debug_logging BOOLEAN Override log level to debug when true
oas_spec TEXT Original OpenAPI spec (for reference)
endpoint_url VARCHAR External connector endpoint URL
endpoint_auth JSONB Authentication config for external endpoints
schema_version VARCHAR Schema version for breaking change tracking
category VARCHAR Organizational category (e.g., crm, hr, finance)
tags JSONB Array of tags for filtering
icon VARCHAR Icon identifier for UI display
deprecated_at TIMESTAMP When the connector was deprecated

system_log table additions

Column Type Description
connector_id VARCHAR Links log entries to a specific connector

Security Model

Secret Encryption

Connection configuration containing secrets (API keys, tokens, passwords) is encrypted at rest using AES-256-GCM. Secret fields are identified via the configSchema — any field with secret: true is encrypted before storage and decrypted only at execution time.

Script Sandboxing

Script connectors run inside quickjs-emscripten VMs in isolated worker_threads:

  • Memory limit: Configurable per connector (default 16 MB)
  • CPU limit: Execution timeout (default 30s, configurable)
  • No filesystem access: Scripts cannot read/write files
  • No network access: All HTTP calls go through floh.http.* proxied to the main thread
  • No module imports: Scripts must be self-contained

Built-in HTTP Connector URL Policy

The built-in http connector enforces outbound URL safety checks before sending requests:

  • Only http and https URLs are allowed
  • Loopback and local names such as localhost are blocked
  • Private/link-local IP ranges (RFC1918, 169.254.0.0/16, ::1, fe80::/10, fc00::/7) are blocked
  • Known metadata hosts such as metadata.google.internal are blocked

This policy prevents workflows from using interpolated variables to call internal network targets.

Permission Model

Permission Operations
connector:read List, view, get registry, impact analysis
connector:manage Create, update, delete, test, execute, parse OAS

Mock Connector Guardrails

Built-in test connectors (test-ldap, test-db, test-activedirectory) are kept available for workflow design and dry-run development in production tenants, but are explicitly labeled and constrained:

  • /api/connectors/registry now marks connector handlers with isMock.
  • Manual execution of mock connectors via /api/connectors/:id/execute is restricted to users with the admin role.
  • Manual executions are audit-tagged as connector.mock_executed (real integrations use connector.executed) so log and compliance queries can distinguish simulation traffic from real integration activity.

Resource Sync & User Matching

Connectors that declare sync-capable commands (e.g., listUsers) can synchronize external records into the connector_resource table. A post-sync matching step then links each resource to a Floh user account.

Match strategies

Strategy Key field(s) DB lookup
email resource.email v_user.email (case-insensitive)
email_and_issuer resource.email + issuer from dot-path v_user.email + v_user.upstream_issuer (compound key)
upstream_identity resource.externalId v_user.upstream_id
external_id resource.externalId v_user.sub

The email_and_issuer strategy resolves the issuer value from the synced resource using a configurable dot-path (issuerSourcePath, default attributes.identityIssuer). This is useful when matching against identity providers where the same email address may appear under different issuers.

When createUsers is enabled and users are auto-created during sync, the upstream_issuer and upstream_id fields are populated from the resource attributes so that future compound matching works correctly.

Versioning & Backwards Compatibility

Connector types follow semantic versioning (MAJOR.MINOR.PATCH). The system enforces version discipline at multiple levels.

Semver Enforcement

  • Create: The POST /api/connector-types endpoint validates that version is a valid semver string.
  • Update: When configSchema changes on PUT /api/connector-types/:id, the system runs compareSchemas() to detect breaking/minor/patch changes and requires the caller to supply a version that satisfies the bump level — or set autoBump: true to let the system compute the next version.
  • Seed time: Built-in connector seeds run compareSchemas() at startup against the existing DB row. Breaking changes log a warning so operators notice schema drift between releases.

Schema Change Detection

The compareSchemas() function in schema-versioning.ts classifies changes as:

Change Classification
Command removed Breaking (major)
Parameter removed Breaking (major)
New required parameter (no default) Breaking (major)
Required connectionConfig field added Breaking (major)
Output removed Breaking (major)
connectionConfig field removed Breaking (major)
New command added Minor
New parameter with default Minor
Optional connectionConfig field added Minor
New output added Minor

Use POST /api/connector-types/:id/compare-schema or POST /api/connectors/:id/compare-schema (instance-level, compares against the instance's type schema) to preview changes before applying them.

Command Deprecation

Commands can be marked deprecated in the configSchema:

{
  "commands": {
    "listUsers": {
      "params": ["limit"],
      "description": "List users",
      "deprecated": true,
      "deprecatedMessage": "Use listUsersV2 instead",
      "addedInVersion": "1.0.0",
      "removedInVersion": "3.0.0"
    }
  }
}

When a deprecated command is invoked at runtime, the system emits a structured warning log ("Deprecated command invoked") without failing the execution. This allows operators to track deprecated command usage and plan migrations.

Capabilities Endpoint

GET /api/connector-types/:id/capabilities and GET /api/connectors/:id/capabilities return the connector's command map with deprecation metadata and current version, enabling UI and automation to discover what a connector supports.

External Protocol Versioning

External connector /execute requests include a protocolVersion field (currently "1.0") so remote connectors can detect which request shape Floh is sending:

{
  "protocolVersion": "1.0",
  "command": "listUsers",
  "config": { ... },
  "params": { ... },
  "variables": { ... },
  "metadata": { "stepId": "...", "runId": "...", "timeout": 30000 }
}

Future protocol versions will be additive within a major version — fields may be added but never removed.

Migration Types

The shared ConnectorMigration type supports declarative upgrade metadata:

interface ConnectorMigration {
  fromVersion: string;
  toVersion: string;
  description: string;
  commandRenames?: Record<string, string>;
  paramRenames?: Record<string, Record<string, string>>;
}

This enables future tooling to automate connector instance config and workflow step migrations when connector types are upgraded.

Module Structure

packages/server/src/modules/connectors/
├── registry.ts            # Public execution dispatch + `executeConnector`
├── registry-internal.ts   # Leaf module: in-memory built-in registry map
├── repository.ts          # Database operations
├── routes.ts              # API endpoints
├── connector-logger.ts    # Structured logging via LogService
├── connector-debug.ts     # Console debug logging (CONNECTOR_DEBUG env)
├── mock-engine.ts         # Mock execution engine
├── schema-versioning.ts   # Breaking change detection
├── impact-analysis.ts     # Workflow impact scanner
├── oas-parser.ts          # OpenAPI spec parser
├── script-runtime.ts      # Worker thread management
├── script-worker.ts       # QuickJS sandbox (runs in worker)
├── agent-protocol.ts      # On-premise agent protocol
├── agent-routes.ts        # WebSocket agent endpoints
├── rotate-keys.ts         # Secret key rotation
├── http-url-policy.ts     # Outbound URL safety checks (SSRF guard)
├── authifi-connector.ts   # Built-in Authifi connector (legacy style)
├── authifi-client.ts      # Authifi API client
├── test-connectors.ts     # Built-in test connectors (legacy style)
├── define/                # `defineConnector()` framework
│   ├── define-connector.ts   # `defineConnector()` / `defineCommand()` / `t.*`
│   ├── schema-builders.ts    # `t.*` builders (string, secret, select, raw, …)
│   ├── types.ts              # `ConnectorDefinition`, `InferShape<>`, etc.
│   ├── dispatcher.ts         # `executeDefinition()` + `runInCodeMock()`
│   ├── derive-seed.ts        # `ConnectorDefinition` → legacy `ConnectorSeed`
│   ├── register-definition.ts# Bridge to `registerConnector()`
│   └── auto-discover.ts      # `loadBuiltInConnectors()`
├── clients/               # Shared client helpers for built-in handlers
│   └── http-connector-client.ts # SSRF-safe `connectorHttpRequest()`
└── handlers/              # Built-in connector modules
    ├── index.ts              # BUILT_IN_DEFINITIONS + auto-registration
    ├── http.ts               # http connector (defineConnector-based)
    └── delay.ts              # delay connector (defineConnector-based)

Authoring → registration → seeding flow

handlers/widget.ts            (defineConnector({ name, commands }))
      │
      ▼
handlers/index.ts             (BUILT_IN_DEFINITIONS.push(widget))
      │
      ├─▶ loadBuiltInConnectors() → registerDefinition() → registerConnector()
      │   (available to `executeConnector("widget", …)` at runtime)
      │
      └─▶ seed-connectors.ts → toConnectorSeed(widget) → upsertConnectorType()
          (creates `connector_type` row so UI can list + instantiate it)

Tests and the server boot path both import registry.ts, which side-effect-imports handlers/index.js — the registration invariant ("importing the registry gives you all built-ins") is preserved by the registry-internal.ts leaf module, which breaks what would otherwise be a cycle between registry.ts and handlers/*.