lz4_flex

What's Changed

• add minimal security policy by @Marcono1234 in https://github.com/PSeitz/lz4_flex/pull/203
• Fix get_maximum_output_size overflow on 32-bit targets by @dglittle in https://github.com/PSeitz/lz4_flex/pull/205
• lz4_block exposes option to reuse compression dict by @matthewfollegot in https://github.com/PSeitz/lz4_flex/pull/207

New Contributors

• @Marcono1234 made their first contribution in https://github.com/PSeitz/lz4_flex/pull/203
• @dglittle made their first contribution in https://github.com/PSeitz/lz4_flex/pull/205
• @matthewfollegot made their first contribution in https://github.com/PSeitz/lz4_flex/pull/207

Full Changelog: https://github.com/PSeitz/lz4_flex/compare/0.12.0...0.13.0

What's Changed

• Fix integer overflows when decoding large payloads by @teh-cmc in https://github.com/PSeitz/lz4_flex/pull/192
• chore(readme): add python binding impl by @LVivona in https://github.com/PSeitz/lz4_flex/pull/190

New Contributors

• @teh-cmc made their first contribution in https://github.com/PSeitz/lz4_flex/pull/192
• @LVivona made their first contribution in https://github.com/PSeitz/lz4_flex/pull/190

Full Changelog: https://github.com/PSeitz/lz4_flex/compare/0.11.5...0.12.0

Documentation

• Docs: add decompress block example

Fixes

• Handle empty input in Frame Format #120

Empty input was ignored previously and didn't write anything. Now an empty Frame is written. This improves compatibility with the reference implementation and some corner cases.

• Fix: Small dict leads to panic #133

compress_into_with_dict panicked when the dict passed was smaller than 4 bytes. A match has the minimum length of 4 bytes, smaller dicts will be ignored now.

Features

• [breaking] invert checked-decode to unchecked-decode #134

invert `checked-decode` feature flag to `unchecked-decode`
Previously setting `default-features=false` removed the bounds checks from the
`checked-decode` feature flag. `unchecked-decode` inverts this, so it will needs to be
deliberately deactivated.

To migrate, just remove the `checked-decode` feature flag.

• Allow to pass buffer larger than size #78

This removes an unnecessary check in the decompression, when the passed buffer is too big.

• Add auto_finish to FrameEncoder #95 #100

Empty input was ignored previously and didn't write anything. Now an empty Frame is written. This improves compatibility with the reference implementation and some corner cases.

• Autodetect frame blocksize #81

The default blocksize of FrameInfo is now auto instead of 64kb, it will detect the blocksize
depending of the size of the first write call. This increases
compression ratio and speed for use cases where the data is larger than
64kb.

• Add fluent API style contruction for FrameInfo #99 (thanks @CosmicHorrorDev)

This adds in fluent API style construction for FrameInfo. Now you can do

let info = FrameInfo::new()
    .block_size(BlockSize::Max1MB)
    .content_checksum(true);

Performance

• Perf: Faster Decompression #113 #112

Replace calls to memcpy with custom function

• Perf: optimize wildcopy #109

The initial check in the the 16 byte wild copy is unnecessary, since it is already done before calling the method.

• Perf: faster duplicate_overlapping #114

Replace the aggressive compiler unrolling after the
failed attempt #69 (wrote out of bounds in some cases)

The unrolling is avoided by manually unrolling less aggressive.
Decompression performance is slightly improved by ca 4%, except the
smallest test case.

• Perf: simplify extend_from_within_overlapping #72

extend_from_within_overlapping is used in safe decompression when
overlapping data has been detected. The prev version had unnecessary
assertions/safe guard, since this method is only used in safe code.
Removing the temporary &mut slice also simplified assembly output.

uiCA Code Analyzer

Prev
Tool 	    Skylake	IceLake 	Tiger Lake 	Rocket Lake
uiCA Cycles 28.71 	30.67 		28.71 		27.57

Simplified
Tool 	    Skylake	IceLake 	TigerLake 	Rocket Lake
uiCA Cycles 13.00 	15.00 		13.00 		11.00

• Perf: remove unnecessary assertions

those assertions are only used in safe code and therefore unnecessary

• Perf: improve safe decompression performance 8-18% #73

Improve safe decompression speed by 8-18%

Reduce multiple slice fetches. every slice access, also nested ones
, carries some overhead. In the hot loo

What's Changed

• Add support of decoding legacy frames by @yestyle in https://github.com/PSeitz/lz4_flex/pull/66

New Contributors

• @yestyle made their first contribution in https://github.com/PSeitz/lz4_flex/pull/66