Back Home

Intent Driven Development

Data Collector and Data Collector Edge

https://streamsets.com/documentation/datacollector/3.11.x/help/datacollector/UserGuide/Getting_Started/GettingStarted_Title.html

  • Data Collector - design and execution engine that streams data in real time. Use Data Collector to route and process data in your data streams.
  • Data Collector Edge - StreamSets Data Collector EdgeTM (SDC Edge) is a lightweight execution agent without a UI that runs pipelines on edge devices. Use SDC Edge to read data from an edge device or to receive data from another pipeline and then act on that data to control an edge device.
  • StreamSets Control Hub is a central point of control for all of your dataflow pipelines. Use Control Hub to allow your teams to build and execute large numbers of complex dataflows at scale.

Teams of data engineers use the shared repository provided with Control Hub to collaboratively build pipelines. Control Hub provides full lifecycle management of the pipelines, allowing you to track the version history and giving you full control of the evolving development process.

Basic Components

Code Development

Testing - StreamTest Test Framework

Error Handling

 
streamsets.txt · Last modified: 2019/10/18 06:51 by root
 
RSS - 200 © CrosswireDigitialMedia Ltd