Stream: helpdesk (published)

Topic: package for "bit packing"?

Expanding Man (Feb 14 2021 at 19:00):

Does anyone know if a package exists for efficiently "packing" bits? That is, combining sequences of bits of lengths not necessarily divisible by 8 and returning a Vector{UInt8}, so for example, the 5-bit sequence (represented as a UInt8) 0b00011111 combined with the 3-bit sequence 0b00000111 gets combined into the 1-element array [0xff]

Mason Protter (Feb 14 2021 at 19:16):

I think @Jakob Nybo Nissen has a package for this, I'll see if I can find it.

Jakob Nybo Nissen (Feb 14 2021 at 19:17):

https://github.com/jakobnissen/BitBuffer.jl - no tests, and I wrote it as one of my first Julia "packages", so dragons be there

Mason Protter (Feb 14 2021 at 19:17):

I think I was thinking of this one: https://github.com/jakobnissen/StackCollections.jl, so not exactly what you wanted.

Mason Protter (Feb 14 2021 at 19:23):

Wait, isn't this what BitVector already does?

Jakob Nybo Nissen (Feb 14 2021 at 19:25):

BitVector is only for single-bit entries, it can't have e.g. an array of 3-bit integers

Mason Protter (Feb 14 2021 at 19:27):

Right, but @Expanding Man wants it to all get packed together right?

julia> BitVector([[1,1,1,1,1]; [1,1,1]]).chunks
1-element Vector{UInt64}:
 0x00000000000000ff

Expanding Man (Feb 14 2021 at 19:28):

Yeah, I was thinking about using BitVector, but I think it might wind up being extremely inefficient. My concern is that if I start out with a bunch of bytes, converting them to bools to stick in the BitVector owuld be expensive

Expanding Man (Feb 14 2021 at 19:29):

but maybe not... I should test it. I was going to try doing that with BitOperations.jl

Mason Protter (Feb 14 2021 at 19:29):

Yeah, that's a fair concern.

Jakob Nybo Nissen (Feb 14 2021 at 19:31):

I would probably implement an AbstractVector with an index saying how many bits are filled, and which offset it's at. A little manual, perhaps

Expanding Man (Feb 14 2021 at 19:33):

Yeah, but writing an implementation that shifts bits around to pack them together seems like a big pain in the ass and not something I particularly want to do

Last updated: Oct 02 2023 at 04:34 UTC