go-pipeline

Configurable data processing in golang.

Install

go get github.com/Focinfi/go-pipeline

Processing Flow

1. Handler-*: References a existing Handler
2. Builder-*: Builds a Handler with config
3. Independent Handlers can process parallelly

Handler

type HandleRes struct {
	Status  HandleStatus           `json:"status"`
	Message string                 `json:"message"`
	Meta    map[string]interface{} `json:"meta"`
	Data    interface{}            `json:"data"`
}

type Handler interface {
	Handle(ctx context.Context, reqRes *HandleRes) (respRes *HandleRes, err error)
}

Pipe / Parallel / Line

1. Pipe

   1. It is a Handler
   2. Instanced by config, contains a internal handler
   3. The internal handler can be built by a builder or refrenced by anther Handler
   4. Run the internal hanlder with timeout

2. Parallel

   1. It is a Handler
   2. Contains a list pipes of Pipe
   3. Parallelly run the every Pipe.Handle

3. Line

   1. It is a Handler
   2. Contains a list of Pipe
   3. Sequently run the every Pipe.Handle
   4. Create a Line with JSON