The first major decision to make is whether fields are packed MSB-first or LSB-first

Unfortunately in many cases like image and video codecs, that decision has been made for you already --- and not in the direction that's most efficient for a CPU (but of course is pretty much immaterial for hardware.) I suspect "It’s easy to fool yourself into thinking that one variant is better than another because it looks prettier" was what drove those decisions.

The big-endian/MSB-first order requires extra "bit movement" if you are maintaining a bit buffer in a register, since the bits of interest are always going to reside in the topmost bits, but you often need them in the lowest bits to use them. With little-endian/LSB-first, the bits of interest are already in the lowest position, and naturally "fall off the end" as you shift right and replenish into the upper bits. IMHO another reason to prefer little-endian: it's very logically consistent.

This article is awesome, I don't work in this kind of thing at all, and I feel like it's a good exersize for my brain.

Part 2: https://news.ycombinator.com/item?id=16419774

Fun small educational project: If you want to learn how to work with units smaller than a byte, implement Base64 from scratch. It’s really fun!

Almost everything on this guy's blog is amazing if you are interested in hardware, optimization and that kind of thing.