Skip to main content

Architecture

Perpetual's architecture pairs Perpetual Cloud (a hosted SaaS application) with a data plane installed into a dedicated cloud account inside your AWS billing organization. The data plane stores security events in an Iceberg-compatible, S3-based lakehouse and analyzes them with Perpetual's serverless query engine, a design which scales storage separate from compute, and which keeps broad event retention economically practical.

Deployment boundaries for PerpetualCloudTrail and custom applications write directly to event data in S3. CloudWatch Logs and CloudWatch metric streams deliver through Firehose. Perpetual ingestion reads the event data and sends operational telemetry to an S3 bucket hosted by Perpetual. Agent harnesses use MCP to invoke application tools. The API can invoke Simba, Perpetual's agent loop. Simba uses the tools and either an LLM on Bedrock or an external model provider. The API, Simba, and tools access stored results through a result proxy. The data plane uses outbound connections for control.Your AWS organizationExisting AccountsCustomAppsCloudWatchLogsCloudWatchMetricsFirehoseCloudTrailEvent Data(S3)Data Plane AccountLLM(Bedrock)Results(S3)IngestionQuery Engine(Lambda)Iceberg(EKS)Lakehouse(S3)Your GCPCloud Storage(coming soon)Pub/SubYour AzureBlob Storage(coming soon)Event HubsYour model providersAnthropicOpenAIOpenRouter…Your SaaS servicesCrowdStrikeO365SlackDuoGWS Audit1PasswordBitwarden…Perpetual CloudToolsSimbaResultProxyMCPAPIControlTelemetry(S3)Your corporate networkUser machineAgent harnessMCP-compatibleharnessesBrowser(Perpetual web app)3567891011412

Numeric labels represent the following interaction boundaries:

  • 1. Public MCP: Streamable HTTP
  • 2. Public API: HTTPS
  • 3. Bedrock: Scoped role with temporary STS credentials
  • 4. Model providers: Outbound HTTPS to model provider APIs
  • 5. Query results: Vended S3 credentials
  • 6. Control: Outbound command and status channel
  • 7. S3 ingestion: Scoped IAM role
  • 8. GCP ingestion: Outbound connections
  • 9. Azure ingestion: Outbound connections
  • 10. SaaS ingestion: Outbound API polling
  • 11. Telemetry: Metrics and logs to Perpetual-hosted S3

The data plane runs in a dedicated AWS account​

The Perpetual data plane runs in a dedicated AWS account. You install it through CloudFormation by following a versioned link out of the data plane configuration UI in Perpetual Cloud. The resulting stack creates and manages data-plane infrastructure as a single unit.

You can think of this dedicated account as a virtual appliance. The account, including the data and compute resources inside it, is yours. Perpetual configures and updates it through its managed deployment and update process. You operate it by using Perpetual Cloud, which communicates commands and other instructions through its control channel. Perpetual data planes expose no management endpoint to the internet.

Connectors bring events into the data plane​

A connector brings raw events from one external source into the data plane's lakehouse. Perpetual supports four source shapes:

  • S3 buckets such as those populated by cloud services, Amazon Data Firehose, or custom applications. Perpetual integration accesses these by assuming a scoped IAM role configured with each connector resource. Perpetual stores no AWS credential for the source
  • SaaS APIs that Perpetual knows how to poll via outbound connections
  • Streaming brokers such as Google Cloud Pub/Sub and Azure Event Hubs
  • Endpoints that the data plane exposes for receiving inbound HTTP requests that carry events

Perpetual natively ingests event streams from Google Cloud Pub/Sub and Azure Event Hubs. The SaaS products in the diagram are examples from a rapidly growing list of supported APIs.

Storage and compute scale independently​

Inside the data plane, Perpetual stores events and materialized datasets using a lakehouse architecture. A lakehouse combines structured, indexed metadata with columnar files in object storage to deliver an open, warehouse-style storage layer.

Perpetual’s query engine uses lakehouse metadata to plan reads and distributes query work across serverless workers that share access to object storage. The same engine executes interactive queries for hunts and investigations and materializes deployed derived datasets, including signal datasets produced by detections. Query compute scales with the work these queries perform rather than retained event volume. Cost and latency still depend on query complexity and how much data the engine must read.

Storage efficiency makes broad ingestion practical​

Perpetual does not meter ingestion for billing purposes, and the data plane design keeps the marginal cost of ingestion and retention low enough for broad collection. Events land in S3 in the dedicated AWS account and are stored in a compressed, columnar format. That cost structure lets teams retain useful events instead of discarding them merely to control ingestion cost. A broader history preserves evidence for threats and questions that nobody knew to ask when the events arrived.

Tables and materialized views can set a retention period or inherit the Space default.

Compatible engines can read lakehouse tables​

The lakehouse stores ingest tables and materialized derived datasets in an Iceberg-compatible table format. Compatible engines can read the table data without Perpetual's query engine when they can reach the Iceberg catalog and have read access to the underlying S3 objects.

Perpetual assumes exclusive access for ingestion, materialization, and maintenance. It does not allow external systems to write to managed Iceberg tables. This restriction protects the catalog and table state that Perpetual coordinates as it maintains those datasets.

Interoperability applies to stored tables. Hamelin queries and resource definitions remain specific to Perpetual, as do virtual views, detections, and application workflows. A compatible engine can read the rows without inheriting the logic that produced or operates them.

Perpetual Cloud provides three interaction paths​

Perpetual Cloud exposes three ways to work with Perpetual: the web application and public API, Simba, and remote MCP access for external agent harnesses.

When using Perpetual Cloud, your browser calls Perpetual's public API directly. You can also script against it, given that it is published as OpenAPI.

Perpetual Cloud also runs Simba, Perpetual's agent, which provides a native chat experience for working with every aspect Perpetual. Simba's model provider is configurable: it can use Bedrock in the dedicated AWS account or either OpenRouter or Anthropic, if you configure it with an API key. For Bedrock, Simba assumes a scoped IAM role in the data-plane account with temporary STS credentials.

External harnesses connect through MCP access, Perpetual's public remote MCP endpoint over Streamable HTTP. MCP makes Perpetual's full suite of agent tools available to any harness, but it cannot quite match the native feel of Simba, which can directly interact with UI components and link to specific application pages and visualizations.

Perpetual Cloud is where you manage and monitor the data plane​

Perpetual Cloud gives you one place to direct data-plane work, monitor its operation, and retrieve selected query results. It does so without exposing a management endpoint in the data-plane account or moving event history or materialized datasets out of the lakehouse.

The data plane fetches query requests and deployed resource definitions from Perpetual's control service through its outbound control channel. It reports deployment status and component versions through the same channel. Data-plane components use the fetched definitions to ingest events, maintain materialized datasets, and run detections.

Data-plane components send metrics and logs to a telemetry-focused S3 bucket that Perpetual hosts in each region. This telemetry powers monitoring in Perpetual Cloud and gives your dedicated Perpetual support team visibility into data-plane operation. The data plane can also ingest its own telemetry as event data.

Interactive query results follow a separate return path. The query engine writes them to S3 in the dedicated AWS account. Through the control channel, the data plane vends temporary S3 credentials to Perpetual Cloud as needed. The result proxy uses those credentials to read selected results for the web application.