wiki:FlowLib

Flow

back

note: this is not (yet) finished - the grammar compiles the afd file, the vm and overall main are not the trunk


Rationale

The idea is not very new: define a (work) flow of operations on a stream. In unix it is handled by piping(text) streams through executables. A modern day version would be  Yahoo! pipes ...

But you need to pass all data over stdin/out. What if you want a chain of commands where the medium is something else - bad design is one comment here ... But still, what if you processes are cpu bound (video encoding etc) and you need a string of these to complete you workflow.

Flowlib is an attempt to define eventhandlers and a routing mechanism. The events are filesystemevents, the routing moves the files around over various directories (and creates them as needed).


Design

Boost.Spirit is used to define a grammar and executions. The grammar sets up an AST (abstract syntax tree) after which the whole thing is 'dumped' in a VM (virtual machine) for processing.

This way you write many flows and it will run ... & stderr/out go to file.

In a later stage, one could add a gui, say in Python ...


Dependencies


Example

Example grammar:

# abonja flow definition

# executables live a directory defined by 'static.bin'
# next, search the PATH environmen variable

setup
.bin(./bin)
.env(PATH)

# variables start with $
# .name should be findable in 'static' othewise an error is issued

$copy
.executable("c.bat")
.command_line(--copy-from ./a --copy-to ./b --input-file _1)
.triggers(1)
.trigger_name(.end)

$echo
.executable(e.bat)
.command_line(--input-file _1)
.triggers(1)
.trigger_name(.begin)

queue
.location(./a)
.log(./logs/xqueue.log)
.remove_log(no)
.dry_run(no)

# predefined events are '.begin' and '.end'
# convention: all tools write a .end file on finish
 
directory
.name(a) 
.on_begin(queue, echo)
.on_end(queue, copy)
.on_trigger(.move)(queue, move)

# the idea is: after this file is parsed, then all directories
# are created and queue+process is created too, so the
# 'flow' is ready for action

directories
.sequence(a, b, c)

Use