Skip to main content


This tutorial walks through how an application can leverage common streaming data patterns to ingest Stellar network transaction data using a few select packages from the Stellar Go Repo collectively known as the 'Ingestion' SDK:

The Ingestion SDK packages

  • utility package to convert prices from network transaction operations to string
  • utility package with convenient wrappers for accessing history archives, and avoid low-level http aspects
  • provides parsing functionality over the network ledger data, converts to more developer-centric LedgerTransaction model
  • provides the captive core backend implementation
  • provides convenient pre-configured settings for Testnet and Mainnet networks
  • a complete Golang binding to the Stellar network data model

Ingestion project setup

Project requirements

To build an example streaming network ingestion pipeline from live Stellar network transaction data, you'll need:

  • A developer workstation with Go programming language runtime installed
  • An IDE to edit Go code, VSCode is good if one is needed
  • A newly initialized, empty Go project folder. mkdir pipeline; cd pipeline; go mod init example/pipeline
  • stellar-core must be installed on your workstation and available on your o/s PATH

The Stellar network data model is defined in an IDL format expressed in XDR encoding. Our example application is only interested in a small subset of the overall transaction data model related to buying and selling of assets, i.e. a payment, and defines its own data model internally:

Timestamp: uint
BuyerAccountId: string
SellerAccountId: string
AssetCode: string
Amount: string

The example application will run a network ingestion pipeline to derive a smaller ApplicationPayment model from the Stellar network transaction data model as 'source of origin' and thus enable the application to avoid large compute resources that would have been required for maintaining storage of the full Stellar network data model.

The ingestion pipeline will perform three distinct stream processor roles:

Inbound Adapter

Acts as the 'source of origin' for the pipeline. Retrieves LedgerCloseMeta generated from a Stellar network using captive core. LedgerCloseMeta is the top-level aggregate in the Stellar data model of which all Stellar network transaction data is nested within. Publishes the LedgerCloseMeta onto the pipeline.


Subscribes to receive LedgerCloseMeta from the pipeline. Uses the Go SDK package to parse the nested network data model for payment operations and convert those into a new instance of application data model ApplicationPayment instances. Publishes ApplicationPayment to the pipeline.

Outbound Adapter

Acts as the termination of the pipeline, it subscribes to receive ApplicationPayment and publishes the data off the pipeline and to an external data store, a ZeroMQ Publisher Socket, which is essentially a message broker.


Refer to Ingestion Pipeline Sample Application for complete code demonstrating usage of the 'ingestion' SDK packages to create these adapters and transformers and run a live pipeline against the Stellar network.