Schemas
Schemas define the structure and types of event payloads. They are the foundation of the MAPS data processing pipeline.
Why Schemas Matter
- Validation: Ensures every message conforms to the expected format.
- Transformation: Enables field-level mapping, renaming, and enrichment.
- Filtering: Provides type-aware evaluation of conditions.
- Statistics: Allows numeric fields to be aggregated correctly and string fields to be counted.
Without a schema, payloads are treated as opaque and only minimal inference is possible.
Supported Formats
| Name | Description |
|---|---|
| JSON | Javascript Notation |
| CBOR | Concise binary object representation |
| XML | extensible markup language |
| ProtoBuf | Google Protobuf |
| AVRO | Apache AVRO |
| MessagePack | Like JSON but smaller |
| CSV | Comma seperated Values |
| QPID-JMS | Apache QPID JMS Messages |
| RAW | Opaque data load |
| Native | Single Native Java types |
Schema Registration
Schemas are registered with the server and can be bound to topics.
When an event is published to a topic, the schema (if present) is applied automatically.
Schema in Processing Flow
⚠️ No schema → opaque payload. Stats and transformations may be limited.