Hey fellas, is there a package that creates DAGs in order to visualize a data processing pipeline? I mean, suppose I have a database, and I extract some data and apply a series of transformations and joins and so on, and produce a final dataframe. I wish to visualize this pipe in a diagram...
@Andrew Baas, do we have something like this in AlgebraicJulia?
Does Semagrams.jl fit here?
Semagrams wouldn't really work here, as that's more of a graphical editor for these.
Catlab.jl does this kind of visualization of wiring diagrams through Graphviz and includes an @program
macro for defining a wiring diagram (see this page of the documentation https://algebraicjulia.github.io/Catlab.jl/dev/generated/wiring_diagrams/wd_cset/). This requires you to define the generators (component boxes with input/output signatures) as a presentation (using @present
) and then describe the data pipeline as a series of functions in the @program
macro.
There is also an imperative API for constructing these programmatically, which could be more useful in this case (https://algebraicjulia.github.io/Catlab.jl/dev/generated/wiring_diagrams/wiring_diagram_basics/)
Thanks for the answers. I actually am looking for something more in the graphical... I guess I'll write some functions in Luxor to do what I want.
If one of the diagrams in Kroki is suitable (Pikchr for example), it might be easier to use Kroki.jl instead.
Last updated: Nov 06 2024 at 04:40 UTC