Since we download JS SDKs in a for loop which invokes a separate docker container for each `curl` run, we seem to be triggering some sort of a DoS protection. And rightfully so as the old method causes TCP and TLS churn although we advertise we support HTTP/1.1 and HTTP/2.
This patch does a few things:
1. Uses `curl`s globbing support to download all files in one go, maxing TCP and TLS reuse. This should fix the DoS protection
2. Uses `curl`'s `--compress` option to make things even more efficient
3. Uses `curl`'s `--create-dirs` to save 1 docker container run per version for creating the directory
4. Removes the `-I` `HEAD` checks in favor of a `-f` fail option combined with `|| true` which makes curl fail and not write the output on a non-200 response while still allowing the script to succeed
5. To make sure the above approach works, it adds a file size test, requiring all downloaded files to be larger than 1kB
Hopefully fixes https://github.com/getsentry/sentry/issues/22715#issuecomment-2458066842
### Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
* Update minimum docker compose requirement
docker compose down <service> is now required.
* Update docker compose version to new minimum in CI config.
Previous situation: we do most of our "check minimum requirements" work in the aptly named check-minimum-requirements.sh. This would be a natural home for the "check if docker even exists on this system" verification, but we actually run call into docker info to set some environment variables before this script gets run, in detect-platforms.sh. So I've adde the "does docker exist" check to that file, directly before we call docker info.
* Reference paths relative to the current script or project root
Before this PR:
- some scripts change the current working directory and use relative paths
- different approaches are taken to know which directory a script is running in
- paths are sometimes relative, sometimes absolute, sometimes traversing directories
After this PR:
- scripts do neither change nor care much about the current working directory
- a unified approach determines the directory of the current script
- paths are always relative to the project root
This should resolve an issue I already tried to fix with https://github.com/getsentry/self-hosted/pull/1798,
where the contents of `./sentry` were not copied
into the built container image,
thus `enhance-image.sh` did not apply.
Co-authored-by: Amin Vakil <info@aminvakil.com>
Not all environments are KVM, and not all virtualized CPUs show SSE4.2 support even if they do support it. This adds an escape hatch if you know what you are doing.
In this PR we fix our usage of jq:
1. Use the container. Before we were accidentally using jq installed in the environment
1. Rewrite generate_breadcrumb_json to read log file all at once
1. Build docker image for jq in unit tests
