# infinimesh > infinimesh is an Apache-2.0, self-hosted IoT backend written in Go. It is the device layer for your IoT and AI stack: device identity, access as a permission graph (per customer and per partner), reported and desired shadow state, MQTT 3.1.1/5 and HTTP ingest. Addons stream device data to Kafka, KafScale, Postgres/TimescaleDB, InfluxDB, webhooks, an MQTT broker, Elasticsearch, Snowflake, SAP HANA, S3, GCS and Azure Blob. The open source core is complete and free. No feature is held back for a paid tier. infinimesh has no dashboards. In 3.4: rules and alarms on device state with ack and clear, and an OTA contract in core (target version per device, reported status, an OTA addon must register before updates switch on), an audit log, namespace quotas, CoAP ingest on UDP 5683, and MQTT bridge sessions in Redis. The OTA addons are hawkBit, Mender, and a signed webhook. They are released in addons v3.4.2 as source. You build them. They are not published images. The webhook addon needs core 3.4.2 or newer, which serves it the namespace webhook URL. The current release is v3.4.3 for core, inf-cli, platform-operator and the Helm chart. The Go module path is github.com/infinimesh/core/v3. It is not an HA MQTT cluster yet. It helps EU device makers give customers access to device data, but it does not make a product Data Act compliant by itself. AI ready (coming soon, not built yet): device state for AI agents over an MCP server and for any LLM through an addon for OpenAI-compatible APIs (vLLM, Ollama, Mistral, OpenAI, Anthropic, Gemini), scoped by the same permission graph. Models can run locally so data stays on your servers. ## Docs - [Installation](https://github.com/infinimesh/core/blob/main/docs/installation.md): Docker Compose on one machine - [Device auth](https://github.com/infinimesh/core/blob/main/docs/device-auth.md): certificates, tokens, MQTT login, HTTP ingest - [Public TLS](https://github.com/infinimesh/core/blob/main/docs/public-tls.md): running on a public domain - [API reference](https://infinimesh.org/api/): REST view of the stable v1 API, OpenAPI spec at https://infinimesh.org/api/openapi.json. Devices send their own JSON with POST /v1/ingest and a device token (16 KB max, 60 requests a minute), and read desired state with GET /v1/ingest/desired. - [API versioning](https://github.com/infinimesh/core/blob/main/docs/api-versioning.md): the v1 API is stable, REST under /v1, breaking changes checked against proto v1.0.0 - [Device shadow streams](https://github.com/infinimesh/core/blob/main/docs/shadow.md): account and namespace streams, event ids, read-only streaming token for addons - [OTA contract](https://github.com/infinimesh/core/blob/main/docs/ota.md): target version per device, and the hawkBit, Mender, and signed webhook addons - [Docs and examples](https://github.com/infinimesh/docs): ESP32 MQTT, ESP32 HTTP, Pico W, curl, small-device TLS guide ## Code - [core](https://github.com/infinimesh/core): API, registry, shadow, MQTT bridge, console - [addons](https://github.com/infinimesh/addons): data sinks, OTA addons, and the write-an-addon guide - [inf-cli](https://github.com/infinimesh/inf-cli): the inf command - [proto](https://github.com/infinimesh/proto): API schemas ## Optional - [Kubernetes operator](https://github.com/infinimesh/platform-operator) - [Cluster recipes](https://github.com/infinimesh/k8s) --- Source: https://github.com/infinimesh/core # infinimesh [![Go Reference](https://pkg.go.dev/badge/github.com/infinimesh/core/v3.svg)](https://pkg.go.dev/github.com/infinimesh/core/v3) [![Coverage](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/infinimesh/core/badges/coverage.json)](https://github.com/infinimesh/core/actions/workflows/go.yml) [![Release](https://img.shields.io/github/v/release/infinimesh/core)](https://github.com/infinimesh/core/releases) [![License](https://img.shields.io/github/license/infinimesh/core)](https://github.com/infinimesh/core/blob/main/LICENSE) ![The inf CLI creates a namespace, a device and a rule, reports a hot reading, lists the raised alarm and acks it](https://raw.githubusercontent.com/infinimesh/core/main/docs/media/inf-cli-demo.gif) infinimesh is an open source IoT platform you run yourself. It gives you a device registry, device auth, a device shadow (digital twin), an MQTT bridge, and REST, gRPC and Connect APIs. It runs on Docker Compose or Kubernetes. What you get: - MQTT 3.1.1 and 5, with TLS 1.2 and 1.3 - Device shadow with reported and desired state - Messages from device to cloud and from cloud to device - Multi-tenant namespaces with graph-based access control - Rules on device state that raise alarms you ack and clear ([docs/rules.md](https://github.com/infinimesh/core/blob/main/docs/rules.md)) - An OTA contract: target version per device and reported status, switched on once an OTA addon registers ([docs/ota.md](https://github.com/infinimesh/core/blob/main/docs/ota.md)) - CoAP ingest on UDP 5683 for private networks ([docs/coap.md](https://github.com/infinimesh/core/blob/main/docs/coap.md)) - Fleet provisioning with bulk create and factory claims ([docs/fleet.md](https://github.com/infinimesh/core/blob/main/docs/fleet.md)) - An audit log, namespace quotas, Prometheus metrics, and MQTT bridge sessions in Redis - A web console - Docker Compose and Kubernetes setups ## Quickstart You need Docker with Docker Compose. ```bash git clone https://github.com/infinimesh/core.git cd core cp .env.example .env docker compose up -d ``` Open https://console.127.0.0.1.nip.io It redirects to HTTPS with a local self-signed certificate. Accept that certificate separately for `https://console.$BASE_DOMAIN`, `https://api.$BASE_DOMAIN`, and `https://media.$BASE_DOMAIN`. The browser keeps one exception per host. With `BASE_DOMAIN=localhost` those hosts are `console.localhost`, `api.localhost`, and `media.localhost`. Until `api` is accepted, login fails with no message. Until `media` is accepted, the media page shows `Offline :(`. Log in as `infinimesh` / `infinimesh`. Change the password right away. Images come from GHCR. Set `INFINIMESH_VERSION` in `.env` to pin a release. `v3.4.3` pulls that exact tag. A release candidate does not move `latest`. `BASE_DOMAIN` in `.env` is the root host for all services. The default, `127.0.0.1.nip.io`, resolves to your own machine, so it works locally with no `/etc/hosts` edits. For real use, set `BASE_DOMAIN` to your own domain (see [docs/public-tls.md](https://github.com/infinimesh/core/blob/main/docs/public-tls.md)). Other services: - API (gRPC, Connect, REST): `http://api.${BASE_DOMAIN}` - Media files: `https://media.${BASE_DOMAIN}` - Traefik dashboard: `http://traefik.${BASE_DOMAIN}` ArangoDB and RabbitMQ stay on the Compose network. Their admin UIs are not routed through Traefik or Caddy. Redis `6379` and AMQP `5672` are not published on the host. The public overlay can bind AMQP to `127.0.0.1`. Default logins. Change them before you expose anything. - Platform admin: user `infinimesh`, password from `INF_DEFAULT_ROOT_PASS` (`infinimesh` by default) - ArangoDB, inside the Compose network: user `root`, password `openSesame` - RabbitMQ, inside the Compose network: `infinimesh` / `infinimesh` (from `.env`) MQTT listens on 8883 (TLS with `hack/server.crt` and `hack/server.key`). Plaintext 1883 listens only when `MQTT_PLAINTEXT=1`. The local Compose file sets that. `hack/server-ec.crt` is a P-256 certificate with a subject alternative name for localhost. Small-device TLS is in [docs/guides/small-devices-tls.md](https://github.com/infinimesh/core/blob/main/docs/guides/small-devices-tls.md). ## When not to use it - You want a fully managed cloud service and do not want to run servers. - You only have a few devices and one user. A plain MQTT broker is simpler. ## CLI (inf) The CLI lives in [infinimesh/inf-cli](https://github.com/infinimesh/inf-cli). The easiest way to get it on macOS or Linux is Homebrew: ```bash brew tap infinimesh/inf-cli https://github.com/infinimesh/inf-cli brew install inf ``` New Homebrew versions ask you to trust a tap before they load it. If `brew install` says the tap is untrusted, run `brew trust infinimesh/inf-cli` and try again. Prebuilt archives for Linux, macOS and Windows are on the [releases page](https://github.com/infinimesh/inf-cli/releases). You can also install it with Go 1.27 or newer: ```bash go install github.com/infinimesh/inf-cli@latest ``` `go install` names the binary `inf-cli`. The docs call it `inf`, so add an alias or rename it: ```bash mv "$(go env GOPATH)/bin/inf-cli" "$(go env GOPATH)/bin/inf" ``` Or build from source: ```bash git clone https://github.com/infinimesh/inf-cli.git cd inf-cli go build -o inf . ``` A short session against the local stack: ```bash API=api.localhost:443 # log in as the admin account. The default Traefik certificate does not cover api.localhost. inf login --api "$API" --username infinimesh --password infinimesh --insecure-skip-verify # list namespaces and devices inf namespaces list inf devices list # issue a device token that may post state inf devices token --allow-post > device.token # report state for the device inf devices state --token "$(cat device.token)" --report '{"online":true,"temp":21.5}' ``` Create devices in the console, or with `inf devices create --crt `. Run `inf help` and `inf --help` for all flags. The device token is for the API. MQTT needs a client certificate registered on the device. On 8883 the device sends the certificate. On 1883 the username is the device id and the password is the base64 SHA-256 fingerprint of the certificate, and basic auth must be turned on for the device. Topic rules, QoS, and session behaviour are in [docs/device-auth.md](https://github.com/infinimesh/core/blob/main/docs/device-auth.md). ```bash mosquitto_pub -h api.${BASE_DOMAIN} -p 1883 \ -u -P "$(base64 < fingerprint.bin)" \ -t devices//state/reported \ -m '{"temp":21.5}' ``` ## Documentation - [docs/installation.md](https://github.com/infinimesh/core/blob/main/docs/installation.md): prerequisites and Docker Compose setup - [docs/public-tls.md](https://github.com/infinimesh/core/blob/main/docs/public-tls.md): Traefik or Caddy with Let's Encrypt on a public domain - [docs/device-auth.md](https://github.com/infinimesh/core/blob/main/docs/device-auth.md): device tokens, certificates and MQTT - [docs/rules.md](https://github.com/infinimesh/core/blob/main/docs/rules.md), [docs/ota.md](https://github.com/infinimesh/core/blob/main/docs/ota.md), [docs/coap.md](https://github.com/infinimesh/core/blob/main/docs/coap.md), [docs/fleet.md](https://github.com/infinimesh/core/blob/main/docs/fleet.md): the 3.4 features - [CLI guide](https://github.com/infinimesh/docs/blob/main/guides/cli-getting-started.md): login, quotas, rules, OTA addon registration and audit with `inf` - [api.swagger.json](https://github.com/infinimesh/core/blob/main/api.swagger.json): API schema, copied from the pinned [proto](https://github.com/infinimesh/proto) module. `sh scripts/swagger_check.sh` fails if the copy drifts. `sh scripts/buf_lint_pin.sh` lints that module. ## Checks `make commit-check` runs `fmt`, `race`, `test-fuzz`, and `code-ql-gate`. `fmt` writes Go files that `gofmt` would change. `race` is `go test -race`. `test-fuzz` runs each `Fuzz` target for `FUZZ_TIME` (default `30s`). `code-ql` runs the CodeQL CLI (`brew install --cask codeql`) and writes SARIF under `.tmp/codeql/`. The gate fails when a result is level `error`, ignoring paths under `vendor`. `jq` is required for the gate. Lower the Go database limit with `CODEQL_GO_RAM_MB` if the machine is tight. GitHub Actions runs the same race and fuzz steps, and CodeQL on pull requests with `security-extended` and `security-and-quality`. ## Releases Current version: `v3.4.3`. inf-cli, platform-operator and the Helm chart in k8s are released with the same version from the core tag. See [CHANGELOG.md](https://github.com/infinimesh/core/blob/main/CHANGELOG.md) and [Releases](https://github.com/infinimesh/core/releases). A tag `vX.Y.Z` or `vX.Y.Z-rN` publishes `ghcr.io/infinimesh/core/` and `:latest`. A revision tag also publishes `:vX.Y.Z`. A tag `vX.Y.Z-rc.N` publishes only that tag, and the GitHub release is a pre-release whose notes start with "for testing, not for production". The release text is the matching `## [tag]` section in `CHANGELOG.md`. The same tags on [platform-operator](https://github.com/infinimesh/platform-operator) publish `ghcr.io/infinimesh/platform-operator`. The Helm chart stays on its stable `image.tag` unless you set an RC tag yourself. RC images older than 30 days are deleted. ## Repositories - [core](https://github.com/infinimesh/core): this platform - [proto](https://github.com/infinimesh/proto): gRPC, Connect and OpenAPI schemas. Regenerate with `buf generate` there. - [inf-cli](https://github.com/infinimesh/inf-cli): the `inf` command - [platform-operator](https://github.com/infinimesh/platform-operator): Kubernetes operator - [k8s](https://github.com/infinimesh/k8s): cluster install recipes - [addons](https://github.com/infinimesh/addons): data sinks and OTA addons for hawkBit, Mender, and a signed webhook. The latest addons release is v3.4.2. It ships as source, with no published images and no Helm chart entries. - [website](https://github.com/infinimesh/website): the site at [infinimesh.org](https://infinimesh.org/) ## Contributing See [CONTRIBUTING.md](https://github.com/infinimesh/core/blob/main/CONTRIBUTING.md). Questions and bugs go to [GitHub issues](https://github.com/infinimesh/core/issues). ## License Apache 2.0. See [LICENSE](https://github.com/infinimesh/core/blob/main/LICENSE). --- Source: https://github.com/infinimesh/core # API versioning The schema lives in [github.com/infinimesh/proto](https://github.com/infinimesh/proto). `api.swagger.json` in this repo is a copy of that module. `sh scripts/swagger_check.sh` fails when the copy drifts. `sh scripts/buf_lint_pin.sh` runs `buf lint` on that module. `buf breaking` runs in the proto repository against the latest tag. Until that tag exists, the check is skipped. Protobuf packages are `infinimesh.*.v1`. REST methods are under `/v1`, and the previous path stays as an alias for one minor release. Create is `POST` on the `/v1` path. `exp`, `inf`, and `GetRequest.pool` remain, marked deprecated, beside `expires_at`, `infinite`, `devices`, and `device`. API errors are Connect errors. Each one carries a `google.rpc.ErrorInfo` reason on domain `infinimesh.org`. The shadow process returns that same reason on its gRPC status. The API keeps the reason when it converts the status. `POST /v1/ingest` posts reported JSON for a device token that names exactly one device and may post. The body is at most 16 KB, and that device is limited to 60 requests a minute. `GET /v1/ingest/desired?wait=N` returns desired JSON. `wait` is 0 to 60 seconds. A matching `If-None-Match` returns `304`. --- Source: https://github.com/infinimesh/core # Device Authentication & MQTT infinimesh has no device SDK. A device brings its own MQTT client. Devices authenticate with a TLS client certificate. Port 8883 requires the certificate. Port 1883 accepts the same certificate's SHA-256 fingerprint, base64-encoded, as the MQTT password. The username is the device id. The device must be enabled, and basic auth must be enabled for that device. Turn basic auth on in the console (the lock on the device card) or with `POST /devices//toggle/basic`. A device token (JWT) is for the HTTP, gRPC and Connect API. The bridge does not accept that token as the MQTT password. A token from device `Get` is read-only and expires in 24 hours. A token from `GetByFingerprint` may post reported state and expires in 24 hours. `POST /devices/token` with `post` true may write reported state without raising the token to the caller's level. Desired state still requires manage level. When `expires_at` and `exp` are both unset, that token expires in 24 hours. Each device token has a `jti`. `POST /v1/devices/token/revoke` denies that id until the token would have expired. `POST /v1/ingest` accepts a JSON object as reported state. The bearer token must name exactly one device and be allowed to post. The body is at most 16 KB. `GET /v1/ingest/desired` returns the desired JSON. Port 1883 is not encrypted. Anyone who knows the fingerprint can log in. Use it for local tests. ## Certificate Generate a key/cert pair (self-signed example): ```bash openssl genrsa -out device.key 4096 openssl req -new -x509 -sha256 -key device.key -out device.crt -days 365 \ -subj "/CN=demo-sensor" ``` Upload the public cert when creating the device (UI or CLI `inf devices create` with `--crt device.crt`). TLS: ```bash mosquitto_pub --cafile hack/server.crt \ --cert device.crt --key device.key \ -t devices/demo-sensor/state/reported \ -m '{"temp":21.5}' \ -h api.${BASE_DOMAIN} -p 8883 --tls-version tlsv1.2 ``` Plain MQTT, same device. The password is the base64 SHA-256 fingerprint of `device.crt`: ```bash FP=$(openssl x509 -in device.crt -outform DER | openssl dgst -sha256 -binary | base64) mosquitto_pub -h api.${BASE_DOMAIN} -p 1883 \ -u demo-sensor -P "$FP" \ -t devices/demo-sensor/state/reported \ -q 1 \ -m '{"temp":21.5}' ``` On a public domain the broker certificate is the Let's Encrypt certificate for `mqtt.$BASE_DOMAIN`, so the system CA bundle validates it. See [public-tls.md](https://github.com/infinimesh/core/blob/main/docs/public-tls.md). ## API token The CLI uses a device token against the API, not against MQTT: ```bash inf devices token --allow-post > demo-sensor.token inf devices state --token "$(cat demo-sensor.token)" --report '{"online":true,"temp":21.5}' inf devices state inf devices state --patch '{"threshold":50}' ``` `inf devices state mqtt` can publish too. `--basic` is `device-id:password`. The CLI default topic is still `shadow/devices/%s/desired`, which this bridge rejects: ```bash inf devices state mqtt --basic "demo-sensor:$FP" \ --topic 'devices/%s/state/reported' --report '{"temp":21.5}' inf devices state mqtt --device-id demo-sensor --basic "demo-sensor:$FP" \ --topic 'devices/%s/state/desired' --desired ``` ## Topics A device may publish only `devices//state/reported` and subscribe only to `devices//state/desired`. `` is that device's id. Wildcards and any other topic, including another device's, are rejected. A publish to `desired` is not stored as reported. The retain flag is accepted on `reported`. The stored reported state is that payload. Payloads must be JSON objects. Desired state is pushed when the device subscribes, at the QoS granted for that subscription. `shadow/devices/...` is not a bridge topic. ## QoS, session, and will QoS 0 is delivered once, with no acknowledgement. QoS 1 publishes receive `PUBACK`. QoS 2 uses `PUBREC`, then `PUBREL` and `PUBCOMP`. The reported payload is stored when `PUBREL` arrives. A repeated QoS 2 packet id completes that handshake and is stored once. Desired state is sent at the subscribed QoS. An MQTT 5 desired publish carries content type `application/json` when the sender did not set one, plus any response topic, correlation data, and user properties supplied with the send. Those properties stay on the message through retries and a session resume. QoS 1 and 2 stay inflight until `PUBACK` or `PUBCOMP`, and are retried with the DUP flag. MQTT 3.1.1 keeps that inflight state for one hour when the clean session flag is 0. MQTT 5 keeps it for the session expiry interval in the CONNECT packet. Expiry 0 drops the session when the connection closes. The state lives in the bridge process. A bridge restart clears it. MQTT 5 `CONNACK` advertises a receive maximum of 32, a maximum packet size of 1 MiB, and a topic alias maximum of 16. Inbound topic aliases use that server maximum for the connection. The topic alias maximum in the client's `CONNECT` does not change it. An alias is forgotten when the connection closes. The client's maximum packet size limits publishes the bridge sends. An inbound packet larger than 1 MiB is rejected. The bridge reads the client keepalive and closes the connection after one and a half times that interval with no packet. It also checks that the device is still enabled at least every 30 seconds. A disabled device is disconnected with not authorized, and that disconnect does not store the will. MQTT 5 `CONNACK`, `PUBACK`, `PUBREC`, `SUBACK`, and `DISCONNECT` carry reason codes. The bridge accepts session expiry, the client's receive maximum, the client's maximum packet size, topic aliases, message expiry, content type, response topic, correlation data, and user properties. Message expiry of zero is acknowledged and not stored. A later expiry is applied again while the message waits to be sent or for `PUBREL`; once that time has passed the message is dropped. The remaining interval is written on the outbound publish. The bridge holds at most 32 inbound QoS 2 packet ids. Another MQTT 5 QoS 2 publish receives `PUBREC` reason `0x93` and is not stored. A held id is dropped after two minutes, or sooner when its message expiry elapses. A `PUBREL` for a missing id receives `PUBCOMP` reason `0x92`. A will message is stored as reported state when its topic is `devices//state/reported` and the connection drops without a `DISCONNECT`. The bridge does not publish a reply to the response topic. ## Troubleshooting - MQTT connection closed right after connect: the certificate is not registered, the device is disabled, or on 1883 the username is not the device id, the password is not the fingerprint, or basic auth is off. - 401/permission errors: ensure token includes the device with at least read; reissue with `--allow-post` for writes. - Connection refused: check broker host/port and that `docker compose ps` shows `mqtt-bridge` healthy. - TLS failures: locally, `--cafile hack/server.crt`. On a public domain the broker certificate is the Let's Encrypt certificate for `mqtt.$BASE_DOMAIN`, so the system CA bundle validates it. See [public-tls.md](https://github.com/infinimesh/core/blob/main/docs/public-tls.md). --- Source: https://github.com/infinimesh/core # Device shadow `GET /devices/states/stream` follows reported state, desired state, and connection. Connection state published on `mqtt.incoming` is delivered to the device stream. A sync snapshot includes the stored connection. A device token streams the devices in that token. An account token streams the devices that account can read. `namespace` limits an account token to one readable namespace. Leaving it empty streams every namespace the account can read. An account stream emits `DEVICE_ADDED` for the devices present at subscribe, and later when a device is created, moved in, renamed, retagged, or shared directly with that account. It emits `DEVICE_REMOVED` when a device is deleted or moved out. A later `DEVICE_ADDED` for a device already on the stream refreshes its title, namespace, and tags. Each message has `eventId` and `sequence`. `resume_after` replays later events from that stream. Reported and desired state carry `version`, which increments when that side is patched. State messages include `title`, `namespace`, and `tags` when the registry has them. The same methods are also served under `/v1/devices/{device}/state`. The previous `/devices/states` paths remain. `only_delta` defaults to false. A state message is then the stored shadow with the published change merged in. `only_delta` true sends that published change. `include` limits a state message to `REPORTED`, `DESIRED`, or `CONNECTION`. An empty `include` sends all three. Membership events are sent whole. `GET /devices/states`, `PATCH /devices/states`, and key removal still require a device token. `POST /devices/token` with `stream` true and an empty device map mints a read-only token for the caller account. `namespace` limits that token to one namespace the account can read. The token opens `GET /devices/states/stream` and the sync stream. It is rejected by every other API. --- Source: https://github.com/infinimesh/addons # infinimesh addons Module `github.com/infinimesh/addons`. Connectors that sit beside [core](https://github.com/infinimesh/core). They are not part of the platform process. Apache-2.0. ## Layout | Addon | What it does | |---|---| | [`timeseries`](https://github.com/infinimesh/addons/blob/main/timeseries/README.md) | API to RedisTimeSeries, with Grafana | | [`kafka`](https://github.com/infinimesh/addons/blob/main/kafka/README.md) | API to a Kafka or KafScale topic, JSON keyed by device id | | [`ota-hawkbit`](https://github.com/infinimesh/addons/blob/main/ota-hawkbit/README.md) | hawkBit Management API for namespace OTA | | [`ota-mender`](https://github.com/infinimesh/addons/blob/main/ota-mender/README.md) | Mender open source Management API for namespace OTA | | [`ota-webhook`](https://github.com/infinimesh/addons/blob/main/ota-webhook/README.md) | Signed HTTP posts for namespace OTA | | [`webhook`](https://github.com/infinimesh/addons/blob/main/webhook/README.md) | API to an HTTP endpoint, one signed JSON POST per shadow update | | [`postgres`](https://github.com/infinimesh/addons/blob/main/postgres/README.md) | API to a Postgres table, optionally a TimescaleDB hypertable | | [`influxdb`](https://github.com/infinimesh/addons/blob/main/influxdb/README.md) | API to InfluxDB 2, InfluxDB Cloud or InfluxDB 3, one point per shadow update | | [`mqttbridge`](https://github.com/infinimesh/addons/blob/main/mqttbridge/README.md) | API to an existing MQTT broker, JSON per shadow update on a topic template | | [`cloudconnect`](https://github.com/infinimesh/addons/blob/main/cloudconnect/README.md) | API to CSV files, then S3, Google Cloud Storage or Azure Blob | | [`redisstream`](https://github.com/infinimesh/addons/blob/main/redisstream/README.md) | API to the Redis stream `objects`, read by the sinks below | | [`elasticsearch`](https://github.com/infinimesh/addons/blob/main/elasticsearch/README.md) | Redis stream to the Elasticsearch index `infinimesh` | | [`snowflake`](https://github.com/infinimesh/addons/blob/main/snowflake/README.md) | Redis stream to the Snowflake table `infinimesh.device_states` | | [`saphana`](https://github.com/infinimesh/addons/blob/main/saphana/README.md) | Redis stream to the SAP HANA table `devices` | [`sdk`](https://github.com/infinimesh/addons/blob/main/sdk/README.md) holds the shared API client, the device worker manager, the shadow runner, the JSON message and the batcher used by the shadow addons, and the Redis stream consumer. To write your own addon, see [docs/write-an-addon.md](https://github.com/infinimesh/addons/blob/main/docs/write-an-addon.md). ## Run From this repository, with `API_URL` pointing at a running platform: ```bash docker compose -f timeseries/docker-compose.yml up --build ``` The sink compose files (`elasticsearch`, `snowflake`, `saphana`) already start `redisstream`, so you do not run it on its own as well. Each compose file's build context is the repository root. Set `USERNAME`, `PASSWORD`, and `API_URL` in that file. Placeholders are `CHANGEME` and `https://api.example.com`. ## Talking to core The SDK uses the core v3 gRPC API through the generated clients in [infinimesh/proto](https://github.com/infinimesh/proto), the same way `inf` does. - `USERNAME` and `PASSWORD` are a standard infinimesh account. A fresh core install has the root account `infinimesh` with the password from `INF_DEFAULT_ROOT_PASS`. A separate read only account is better for addons. Creating a plugin, including an OTA plugin, needs a root token from `POST /v1/token`. See [Write an addon](https://github.com/infinimesh/addons/blob/main/docs/write-an-addon.md#register-an-ota-plugin). - `ota-hawkbit`, `ota-mender`, and `ota-webhook` do not use `USERNAME` or `PASSWORD`. They send `OTA_ADDON_TOKEN`, the bearer from `inf plugins create`. None of them is published as an image. The webhook addon reads the namespace webhook URL with that token. It needs core 3.4.2 or newer. The core v3.4.1 image does not serve that read. - `API_URL` is the gRPC address. `https://api.example.com` uses TLS on port 443, which the core compose setup routes to the api service. `http://api.example.com:8000` uses plain gRPC on the `grpc` entry point, or `http://api:8000` from inside the core compose network. With no scheme, TLS is used. - Addons see every device the account can see. For each device they ask core for a short device token with read access and stream its shadow. `webhook`, `postgres`, `kafka`, and `mqttbridge` also send the desired and connection state unless their `*_ONLY_REPORTED` setting is `true`. `influxdb` adds the desired state with `INFLUX_INCLUDE_DESIRED=true`. `timeseries`, `redisstream`, and Cloud Connect use only the reported state. The v3 shadow has no version, so the `version` field the sinks store is empty. `SNOWFLAKE_INITDB=true` creates the Snowflake database and table. Cloud Connect commands are `cloudconnect/csvwriter`, `s3`, `cloudstorage`, and `blob`. ## Checks `make commit-check` runs `fmt`, `race`, `test-fuzz`, and `code-ql-gate`. `fmt` writes Go files that `gofmt` would change. `code-ql` runs the CodeQL CLI and writes SARIF under `.tmp/codeql/`. The gate fails when a result is level `error`, ignoring paths under `vendor`. `jq` is required. GitHub Actions runs the same race and fuzz steps, and CodeQL on pull requests. ## Build ```bash go test ./... go build -o redisstream ./redisstream ``` --- Source: https://github.com/infinimesh/docs # infinimesh docs infinimesh is an open source (Apache-2.0) device layer: devices connect over MQTT or HTTP, and infinimesh keeps their identity, permissions and state, then hands the data to the tools you already run. The server lives in [infinimesh/core](https://github.com/infinimesh/core). This repo holds small device examples and practical guides. ## Examples Each example is one sketch or script with a short README. | Example | Board | Protocol | | --- | --- | --- | | [examples/esp32-mqtt](https://github.com/infinimesh/docs/tree/main/examples/esp32-mqtt) | ESP32 (Arduino) | MQTT over TLS on 8883 with a device certificate | | [examples/esp32-http](https://github.com/infinimesh/docs/tree/main/examples/esp32-http) | ESP32 (Arduino) | HTTPS with a device token | | [examples/picow-mqtt](https://github.com/infinimesh/docs/tree/main/examples/picow-mqtt) | Raspberry Pi Pico W (MicroPython) | MQTT over TLS on 8883 with a device certificate | | [examples/curl](https://github.com/infinimesh/docs/tree/main/examples/curl) | Any computer | HTTPS with a device token | ## Guides - [guides/small-devices-tls.md](https://github.com/infinimesh/docs/blob/main/guides/small-devices-tls.md): why TLS fails on small chips like the ESP8266, and how to fix it. - [guides/node-red.md](https://github.com/infinimesh/docs/blob/main/guides/node-red.md): send device data to Node-RED with the MQTT bridge or webhook addon. ## Good to know - Devices bring their own MQTT or HTTP client. infinimesh has no device SDK. - MQTT topics: a device publishes to `devices//state/reported` and subscribes to `devices//state/desired`. `` is the device uuid. - Boards without TLS (for example ATmega Arduinos) should talk to a local gateway, such as a Raspberry Pi or an ESP32, which then connects to infinimesh over TLS. See the guide above. - HTTP devices send their own JSON with `POST /v1/ingest` and a device token, and read desired state with `GET /v1/ingest/desired`. The v1 API is stable, see [API versioning](https://github.com/infinimesh/core/blob/main/docs/api-versioning.md). - The boards themselves have not been tested yet. What was tested is listed in each README. ## Check before you commit ```bash make commit-check ``` This runs `make lint` and then compiles the examples. `make lint` runs shellcheck, markdownlint, a lychee link check, clang-format on the sketches, actionlint, yamllint and gitleaks. The compile step runs a MicroPython compile check and `arduino-cli compile` for both ESP32 sketches. On Linux, `sh scripts/install-lint-tools.sh` installs the lint tools; on macOS use `brew install shellcheck markdownlint-cli2 lychee clang-format actionlint yamllint gitleaks`. ## License Apache-2.0. See [LICENSE](https://github.com/infinimesh/docs/blob/main/LICENSE). # Related projects - [KafScale](https://kafscale.io/): Kafka-compatible streaming that keeps the log in S3. The infinimesh Kafka addon works with it - [KafClaw](https://github.com/KafClaw/KafClaw): agent runtime in Go that connects AI agents through Kafka topics - [Scalytics open source](https://www.scalytics.io/open-source): all open source projects from the same team - [Lascaris](https://www.scalytics.io/lascaris): the governed and supported platform from Scalytics that infinimesh is part of ## Author of infinimesh - [Alexander Alten](https://www.novatechflow.com/2026/10/infinimesh-34-device-layer.html): initiator and lead engineer of infinimesh. CTO at Scalytics. Writes at novatechflow.com