Overview
MAPS provides a data processing layer that allows you to filter, transform, aggregate, and analyze event streams in real time.
This layer sits between event ingestion and delivery and is responsible for:
- Filtering events with selectors (including ML-enabled filters).
- Transforming payloads using schemas (field mapping, enrichment, format conversion).
- Aggregating multiple input topics (N→1) into a single output topic using windowed fan-in aggregation.
- Applying statistics to a single topic stream (1→1) to produce summary events.
- Enforcing schemas for validation and routing.
Without a schema, event payloads are treated as opaque — no type information is available beyond basic inference.
With a schema, MAPS can perform type-aware filtering, transformations, aggregation, and statistical analysis.
Components
- Schemas: Define the structure and types of event payloads. Critical for validation, transformation, and analytics.
- Filtering: Use selectors or ML functions to route or drop events.
- Transformations: Change event payloads, map fields, convert formats (JSON, Avro, Protobuf, CSV), or enrich with new data.
- Aggregation (N→1): Windowed fan-in across multiple topics. Supports per-input selectors, per-input transformer chains, contribution policies (FIRST/LAST), and optional output transformation. Produces an aggregated envelope event on a single output topic.
- Statistics (1→1): Stream analytics over a single topic feed. Produces JSON summary events (Base, Advanced, Moment, etc.) from one input stream.
Processing Flow (Conceptual)
Key Notes
- Processing is stream-oriented and happens inline with event flow.
- Every step (filter, transform, aggregation, stats) produces valid events that can be bridged, routed, or archived like any other message.
- Schemas are the foundation: without them, MAPS treats payloads as opaque, which limits processing capabilities.
Where Each Feature Applies
| Capability | Client subscriptions (e.g., MQTT 5, STOMP, AMQP) | Inter-server inbound | Inter-server outbound |
|---|---|---|---|
| Filtering | Yes — protocol-native subscription filters where supported | Yes | Yes |
| Transformations | Yes — inbound/outbound for clients | Yes | Yes |
| Aggregation | Yes — server-side, topic-based fan-in | Yes | Yes |
| Statistics | Yes — single-topic stream analytics | Yes | Yes |
Notes
- Filtering (clients): supported on protocols with subscription filtering (MQTT 5, STOMP, AMQP, others that expose selectors).
- Filtering (inter-server): available on both ingress and egress links.
- Aggregation: combines multiple topics into one output topic; useful before downstream filtering, routing, or statistics.
- Statistics: summarizes a single topic feed; client-side vs inter-server availability depends on your current deployment wiring and namespace considerations.