Skip to main content

Deploy

Deploy is the mechanic that turns authored content into a live dataset. A deployable resource carries a deploy boolean. While deploy is false the resource is saved content and produces nothing. Setting it to true stands up the dataset the resource defines.

Eight resource kinds are deployable. The four dataset resources (connectors, tables, views, and unions) deploy themselves. Four deployable content resources (detections, parsers, mappers, and lookups) deploy by standing up a dataset resource they own, and each reports that dataset's state as its own deploymentStatus. Only the four dataset resources reach the data plane, which stands the dataset up and reports back on it. Setting deploy is an edit to the resource, so it takes write access on the resource's Space, and a deploy change staged in a changeset takes effect when the changeset commits.

The deploy field​

deploy is required on a connector, table, view, union, detection, and lookup. It is optional on a parser and a mapper, where it defaults to false.

An integration has no deploy field. An integration goes live when you deploy the parsers and mappers under it, and its status reports the worst state across the parser tables and the composite view. That field uses a narrower enum than deploymentStatus: UNDEPLOYED, DEPLOYED, UPDATE_PENDING, or FAILED.

What each kind stands up​

Resourcedeploy: true stands up
ConnectorIngestion from the source, landing raw events as the connectors.<name> dataset.
TableThe dataset at the schema.name you chose, materialized to storage.
ViewThe dataset at the schema.name you chose. A materialized view writes its rows to storage. A virtual one resolves at query time.
UnionA virtual view over the union's member datasets, resolved at query time. The data plane writes no rows to storage.
DetectionA continuous table of the detection's signals, at signals.<detection>.
ParserAn ingest table of the parsed rows, at events.<integration>_<parser>.
MapperOne branch of the integration's single composite view, at events_ecs.<integration>.
LookupThe lookup's backing dataset at lookups.<lookup>. A computed lookup gets a materialized view, a static lookup a static table.

The dataset lands in the same Space as the resource that stood it up.

Perpetual derives a content resource's dataset name from the resource's own name, lowercased, with runs of spaces, dots, and hyphens collapsed to underscores and other punctuation dropped. When the derived name for a detection's signal table, an integration's composite view, or a lookup's backing dataset is already taken by another dataset in the Space, Perpetual appends _2, _3, and so on, so a second detection named "Failed logins" lands at signals.failed_logins_2. A table, view, or union takes the dataset name you give it, which may be top-level or schema-qualified, and a connector's name must start with connectors..

Undeploying and archiving​

Setting deploy back to false on a deployed resource tears the dataset down. deploymentStatus runs CLEANUP_PENDING, then CLEANUP_IN_PROGRESS, then UNDEPLOYED. The data plane drops the dataset from the catalog, and for a materialized dataset that drop purges the files behind it, so the rows do not survive an undeploy. The resource itself stays, so you can deploy it again from the same definition.

Archiving stops the work without dropping the dataset. The archived field leaves a deployed dataset in place and reports ARCHIVED. Its rows stay queryable, and the data plane stops refreshing them and stops the retention and maintenance it would otherwise run against them. A resource must already be deployed before you can archive it, and a view must also be materialized.

Deployment status​

deploymentStatus is a returned field on every deployable resource. It reports what the data plane has done with the resource's committed version, so it can disagree with what deploy asks for.

ValueMeaning
UNDEPLOYEDdeploy is false and no dataset stands. Also the state a resource lands in once cleanup finishes.
DEPLOY_PENDINGdeploy is true and the data plane has not reported on this version yet.
DEPLOY_IN_PROGRESSThe data plane is standing the dataset up for the first time.
DEPLOYEDThe dataset stands and matches the resource's committed version.
UPDATE_PENDINGThe committed version is ahead of the version the data plane last reported on. Also the state while an upstream dataset this one reads is unavailable.
UPDATE_IN_PROGRESSThe data plane is applying the new version.
REFRESH_IN_PROGRESSThe data plane is refreshing a materialized dataset's rows.
FAILEDThe last deploy, update, refresh, or cleanup failed. The value does not say which of the four.
ARCHIVEDThe resource is deployed and archived. See Undeploying and archiving.
CLEANUP_PENDINGdeploy went to false on a resource that had deployed, and teardown has not started.
CLEANUP_IN_PROGRESSThe data plane is dropping the dataset.

When a deploy fails, the data plane's error lands in statusDetails on the resources that return that field, such as a table.

A dataset becomes referenceable in a query only once its deploy succeeds. The catalog a query compiles against holds the latest version of each dataset that reached DEPLOYED, so committing a query edit to a deployed table leaves the previously deployed version in the catalog until the new version deploys.