Skip to main content

Machine Learning in Maps — Overview & Configuration

The Maps ML Model Manager provides a unified way to load, cache, refresh, and serve models to Maps components. It supports file, s3, nexus, and maps backends with consistent loading and refresh behaviour.

Where ML runs in Maps (Scope)​

  • JMS Selectors are the only execution point for ML models in Maps.
  • Models are invoked from selector expressions to accept, reject, or route events.
  • Analyzers and other parts of the pipeline may compute stats, but model execution is through JMS selectors only.

Example (illustrative only):

selector: random_forest.classify_prob('engine_rf', temperature, pressure, rpm) > 0.9

(Exact function names depend on your configured operations.)

Post‑install requirement (Smile)​

Due to licensing, Smile artifacts are not bundled. After installation, run once per node:

  • Linux/macOS: {MAPS_HOME}/bin/download-smile.sh
  • Windows: {MAPS_HOME}\bin\download-smile.bat

This fetches the Smile 4.3.0 set required by your build.

Packaging expectations (no Maps‑enforced standard)​

Maps consumes whatever format the framework uses:

  • TensorFlow: SavedModel directory or GraphDef.
  • ONNX: a single .onnx file (+ optional sidecars).
  • Smile: whatever the algorithm expects; downloader provides libraries.

You may wrap assets in a ZIP for transport, but Maps does not require a manifest or ZIP structure.

Configuration — top level​

MLModelManager:
  enableCaching: true
  cacheSize: 100
  cacheExpiryMinutes: 60
  autoRefresh:
    enabled: true
    intervalMinutes: 10

  modelStore:
    type: file   # file | s3 | nexus | maps
    config:
      file:
        path: "{{MAPS_DATA}}/models"

  # Optionally keep critical models warm
  preloadModels: modelA.onnx,modelB

Notes:

  • Caching: LRU with idle expiry.
  • Auto‑refresh: Hot‑swaps updated models without downtime.
  • Failure handling: Keeps last healthy model if refresh fails.

Backend configuration reference​

1) File​

modelStore:
  type: file
  config:
    file:
      path: "{{MAPS_DATA}}/models"

2) S3 (incl. MinIO)​

modelStore:
  type: s3
  config:
    s3:
      region: ap-southeast-2
      accessKey: <your-access-key>        # or use instance role/ENV
      secretKey: <your-secret-key>
      bucket: maps-models
      prefix: ml                          # optional path prefix
      endpoint: https://minio.local:9000  # optional for S3-compatible vendors

3) Nexus (Raw Repository)​

modelStore:
  type: nexus
  config:
    nexus:
      url: https://repository.mapsmessaging.io/maps_ml_store
      # Auth via standard HTTP client settings (basic/token)

4) Maps (remote Maps server as model source)​

modelStore:
  type: maps
  config:
    maps:
      url: https://maps-central.example.com/api/v1/modelstore
      apiKey: <maps_api_key>
      namespace: production

The Maps Model Store REST API is documented separately.

Learning & anomaly detection (via JMS selectors)​

  • Learning: Maps can train (e.g., Random Forest) from event streams; you can trigger training and retraining based on counts/time/drift.
  • Anomaly detection: Use selector functions to emit a boolean/score and route outliers (e.g., to outliers/...). The algorithm is your choice (no single method enforced).
  • General inference: Classification, enrichment, or routing decisions can be based on model outputs—again, only via JMS selector evaluation.

Security​

  • Always use TLS to remote stores.
  • Prefer instance roles or tokens over inline secrets.
  • For maps, scope API keys by namespace on the server side.