Stream block_dump, block_add and block_set (#216)
* Stream the block_dump data while reading the contents
* Thread state
* Also stream the block_set and block_add commands
* tls endpoint: do the streaming for block_add/block_set
* stream_to_block: at the end, truncate to desired size
* various fixes, use bounded streams
* revise block_dump with errors
* safety for receiving streams and appropriate error handling
* Add streaming compression
* Add streaming uncompress, and add back compression
* albatross-client: also use streaming compression
* the stream compression used to emit 65536 zero bytes, we skip the flush until we read something from the stream
* Provide output buffer before Zl.Def.encode
* stream_to_fd: always consume the stream until the end
* Fix the way we compress/uncompress blocks
* albatross-client: don't lwt.async, but wait for the task (Lwt.both) to finish
* simplify block_set/block_add protocol, just write length-prefixed chunks
* tls-endpoint: stop relay loop on `Block_data None
* Fix cancellation in `Send_stream
Before this patch a malicious client could ask for a block dump and then
disconnect. For a 40 GB block device this would result on my machine
albatrossd spending more than 1m30s of CPU time for no good.
With this patch the streaming is cancelled as soon as we can't send to
the client anymore.
* Don't report the error twice
If we cancel a dump stream the canceller is responsible for reporting
the error.
* separate old and new block_device_image (new one uses a stream)
* add new commands for block_add and block_set, to phase out the earlier ones
* tls endpoint (read_stream_write): write empty chunk to signal eof
* client: block_create is really an add followed by block_set if a file is provided
* client: ensure that the certificate is not longer than 2 ^ 24 (otherwise it can't be transmitted via TLS)
* Allow for draining compressed stream if malformed
---------
Co-authored-by: Reynir Björnsson <reynir@reynir.dk>
Co-authored-by: Calascibetta Romain <romain.calascibetta@gmail.com>
eabbe4
-
Aug 14 10:36 +00:00