Work in progress PR for chunked item support. As of writing this is largely functional, but is incomplete and may need a refactor.

• What is large item support?

Items have always required a single chunk large enough to hold its header, key, and data. This is done via a slab allocator, which is split into many sized slabs. The "largest available slab", became the largest item you could store into memcached. This is where the 1 megabyte limit came in.

The larger the item size limit, the less efficient the slab allocator is, as there is more space between slab classes.

With this change, the max slab chunk size is decoupled from the max item size, and items larger than the max chunk size now consist of multiple chunks chained together.

This has a number of benefits:

• Performance is nearly identical, as larger items will fill bandwidth much earlier than small items.
• The max slab class size can be much smaller (I've been settling toward 16k after testing), down from 1mb. This means the rest of the slab classes can be closer together.
• Since larger items use data at a much smaller granularity (ie 16k), they have less memory overhead than before.
• The item size max can be hundreds of megabytes without any major impact.

There are some downsides:

• It complicates the slab allocator a bit and adds a few extra code paths in reading/writing items.
• Freeing a single item no longer guarantees enough memory to store a new single item. Pulling chunked items to free will need some careful tuning. This may not be awful considering very small items will continue to be 1:1, and the number of loops necessary to free memory for large items is reasonable given the bandwidth required to store them.

• The current status

This largely works. It's been bench tested but not torture tested. Some tests have been written and I've fiddled with it manually for a while. There's a decent amount of code left to write, and I feel like it might need a little rewriting or simplification. This implementation is making the deep slab allocator more aware of how items are structured which feels like a layer violation: items.c should instead ask the slab allocator for a specific number of chunks, but then we might have to loop through them twice, or pass a callback function into the slab allocator to initialize the chunks. Moving more of the implementation up to the item layer would allow it to more intelligently garbage collect when freeing up chunks for a new chunked item as well.

Could use some feedback on any obvious bugs I've left in.

Work remaining:

• Write more tests (+ fix more existing tests + tests for 00-startup.t)
• Protect binary sasl auth from getting a chunked item
• Implement chunked-aware append/prepend commands
• Implement chunked-aware slab page mover. This is also complicated.
• Run burn-in test
• Clean out debug code

Punting or doing later:

• Test using readv to reduce number of read() syscalls on store.

Run burn-in tests on very large items (hundreds of megabytes)
Any useful stats that could be added? Something to indicate how many large items have been seen, at least.