If a flate memory limit is set, reject decode parameters that would cause
the limit to be exceeded and treat stream as unfilterable.

Fixes oss-fuzz issue 394463491