superset-frontend proxy is not working when superset backend decides to use zstd encoding about superset HOT 9 OPEN

The issue you're encountering with Apache Superset's frontend proxy not supporting zstd encoding likely stems from an indirect dependency update within Superset's Python environment. While the webpack.proxy-config.js is configured to handle gzip, br, and deflate encodings, it does not include support for zstd ¹. This absence means that when the backend serves content with zstd encoding, the proxy cannot decode it, leading to the display issues you've observed.

To identify the package that might have introduced zstd as a default encoding, you would need to review recent changes to the dependencies listed in Superset's pyproject.toml. Although zstd is not listed as a direct dependency, it could be introduced through other libraries, especially those related to data processing or compression. For instance, the pandas package, with its [performance] extra, includes optional dependencies that could utilize zstd compression, such as pyarrow ².

Adding support for zstd in the Node.js proxy would require integrating a decompression mechanism into webpack.proxy-config.js, similar to how other encodings are handled. This integration would depend on the availability of a Node.js module capable of decompressing zstd encoded data. As of the last update, adding such support might involve using third-party modules like node-zstd, though direct support in Node.js for zstd is not yet standard.

For a more immediate solution, ensuring that all developers are using the same version of Python packages, as you've done, can prevent the issue from occurring by avoiding the introduction of zstd encoding. Additionally, investigating the specific versions of indirect dependencies related to compression could help pinpoint the source of the zstd encoding.

To continue the conversation, mention @dosu.

from superset.

For me the culprit package was Flask-Compress 1.15.
Rolling back Flask-Compress to 1.14 and removing zstandard solved the problem.

You can pip install pipdeptree and then run pipdeptree -r -p zstandard to see what depends on zstandard.

from superset.

Hi @jakubjosef thanks for reporting this, and for the research into the cause. I ran into the same issue and found that adding COMPRESS_REGISTER = False into my superset_config.py file fixes it for me when developing locally. We'll look into either documenting this somewhere or fixing it more longer-term.

from superset.

@jakubjosef so that would be on a fresh install from pypi as in pip install apache-superset ?

Doing a bit of guessing here, since all of our CI installs the pin packages in requirements/development.txt, maybe the unpinned version of Superset has this issue? If that's the case it points towards:

• as a workaround, you leveraging docker images build on top of pinned deps, or using the pinned deps in your installation scripts
• for us, we could run some CI tests against a vanilla pip install apache-superset, so at least we know when an open dependency breaks a build - which would inform pin-pointing the issue and changing the version range for that dep

from superset.

Yes, in our install script we are using pip install apache-superset==4.0.0rc1. I see 4.0.0 is now officially published, so we can try to use it, but I think we'll face the same issue eventually.

from superset.

but I think we'll face the same issue eventually

My suggestion would be either:

• use the base docker, which installs the pinned deps ahead of installing the apache-superset package
• fetch the requirements/base.txt from the repo (with the associated version tag you're trying to build as in git checkout 4.0.0) and pip install those pin deps ahead of installing the superset package (that has loose dependency ranges). At that point might as well pip install -e . to install from the repo as opposed to pypi, but should be same as pip install apache-superset==4.0.0

On our side, there's a question as to whether we should publish a more strict or "with depenencies pinned" version of the package to work around all this....

from superset.

I also encountered the same problem. After modifying config.py =>COMPRESS_REGISTER = False, the page no longer had strange symbols, but there were many resources on the page that failed to load; May I ask how I can solve this problem?
superset:3.1.3
python: 3.9.7
node: 16.20.2
npm: 8.19.4

from superset.

@Amylee520 We basically took a list of exact versions from another working machine (IIRC the command was pip freeze) and use it as source of truth for pip on new device. Not an ideal solution but we were in bit time rush.

from superset.

@tyrius02 Thank you for identifying what exact package version is causing this problem, it can help others. We were not able to identify exact cause when we faced this problem.

from superset.