I have asked this before but got no response, so now I'm trying again with a slightly different formulation:
Is there some goto package for concatenating arrays without creating a new array (or modifying one of the arrays to concatenate)?
Use case is to avoid that things like FillArrays.jl and Arrow columns end up as an Array
when concatenated. I know this is not exactly what one might call lazy concatenation, but I couldn't come up with a short name for it.
ComponentArrays.jl seems to do this, but it also seems to do other things (e.g. index components by name) so maybe there is something more tailor made.
Do you need the full array interface or just iteration? Base.Iterators.flatten
gets you the iteration interface but not the array interface.
If you do need the array interface, I'm pretty sure that Lazy.jl's lists satisfy at least most of it, so you're looking for Lazy.flatten
.
Nevermind. I forgot that Lazy's functions all assume that you're working with lists rather than letting you bring your own type.
Found it. You're looking for RecursiveArrayTools.ArrayPartition.
While not strictly what you are looking for, transducers might help for transformation pipelines where you don't want to allocate intermediates.
Thanks!
Unfortunately I forgot to state that I want it to be an indexable AbstractArray
or whatever the requirements are for having it as columns in a DataFrame
.
RecursiveArrayTools looks like it could work though at first glance.
Last updated: Nov 06 2024 at 04:40 UTC