archive-gh-me/self-hosted
1
0
Fork 0
mirror of https://github.com/getsentry/self-hosted.git synced 2025-12-08 19:46:14 +00:00
Code Issues Projects Releases Wiki Activity

fix: Use js.sentry-cdn.com for JS SDK downloads (#3417)

Browse Source 
Since we download JS SDKs in a for loop which invokes a separate docker container for each `curl` run, we seem to be triggering some sort of a DoS protection. And rightfully so as the old method causes TCP and TLS churn although we advertise we support HTTP/1.1 and HTTP/2.

This patch does a few things:

1. Uses `curl`s globbing support to download all files in one go, maxing TCP and TLS reuse. This should fix the DoS protection
2. Uses `curl`'s `--compress` option to make things even more efficient
3. Uses `curl`'s `--create-dirs` to save 1 docker container run per version for creating the directory
4. Removes the `-I` `HEAD` checks in favor of a `-f` fail option combined with `|| true` which makes curl fail and not write the output on a non-200 response while still allowing the script to succeed
5. To make sure the above approach works, it adds a file size test, requiring all downloaded files to be larger than 1kB
This commit is contained in:
Burak Yigit Kaya 2024-11-07 12:25:53 +00:00 committed by GitHub
parent 2a7abf215e
commit 49e30a7356
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 9 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
_unit-test/js-sdk-assets-test.sh
View File

@ -10,6 +10,7 @@ source install/setup-js-sdk-assets.sh
sdk_files=$(docker compose run --no-deps --rm -v "sentry-nginx-www:/var/www" nginx ls -lah /var/www/js-sdk/)
sdk_tree=$(docker compose run --no-deps --rm -v "sentry-nginx-www:/var/www" nginx tree /var/www/js-sdk/ | tail -n 1)
non_empty_file_count=$(docker compose run --no-deps --rm -v "sentry-nginx-www:/var/www" nginx find /var/www/js-sdk/ -type f -size +1k | wc -l)
# `sdk_files` should contains 5 lines, '4.*', '5.*', '6.*', `7.*` and `8.*`
echo $sdk_files
@ -23,4 +24,9 @@ echo "$sdk_tree"
test "5 directories, 17 files" == "$(echo "$sdk_tree")"
echo "Pass"
# Files should all be >1k (ensure they are not empty)
echo "Testing file sizes"
test "17" == "$non_empty_file_count"
echo "Pass"
report_success

20
install/setup-js-sdk-assets.sh
View File

@ -30,25 +30,11 @@ if [[ "${SETUP_JS_SDK_ASSETS:-}" == "1" ]]; then
echo "Found JS SDKs: v${latest_js_v4}, v${latest_js_v5}, v${latest_js_v6}, v${latest_js_v7}, v${latest_js_v8}"
versions=("$latest_js_v4" "$latest_js_v5" "$latest_js_v6" "$latest_js_v7" "$latest_js_v8")
variants=("bundle" "bundle.tracing" "bundle.tracing.replay" "bundle.replay" "bundle.tracing.replay.feedback" "bundle.feedback")
versions="{$latest_js_v4,$latest_js_v5,$latest_js_v6,$latest_js_v7,$latest_js_v8}"
variants="{bundle,bundle.tracing,bundle.tracing.replay,bundle.replay,bundle.tracing.replay.feedback,bundle.feedback}"
# Download those versions & variants using curl
for version in "${versions[@]}"; do
$dcr --no-deps --rm -v "sentry-nginx-www:/var/www" nginx mkdir -p /var/www/js-sdk/${version}
for variant in "${variants[@]}"; do
# We want to have a HEAD lookup. If the response status code is not 200, we will skip the variant.
# Taken from https://superuser.com/questions/272265/getting-curl-to-output-http-status-code#comment1025992_272273
status_code=$($dcr --no-deps --rm nginx curl --retry 5 -sLI "https://browser.sentry-cdn.com/${version}/${variant}.min.js" 2>/dev/null | head -n 1 | cut -d$' ' -f2)
if [[ "$status_code" != "200" ]]; then
echo "Skipping download of JS SDK v${version} for ${variant}.min.js, because the status code was ${status_code} (non 200)"
continue
fi
echo "Downloading JS SDK v${version} for ${variant}.min.js..."
$dcr --no-deps --rm -v "sentry-nginx-www:/var/www" nginx curl --retry 10 -sLo /var/www/js-sdk/${version}/${variant}.min.js "https://browser.sentry-cdn.com/${version}/${variant}.min.js"
done
done
$dcr --no-deps --rm -v "sentry-nginx-www:/var/www" nginx curl -w '%{response_code} %{url}\n' --no-progress-meter --compressed --retry 3 --create-dirs -fLo "/var/www/js-sdk/#1/#2.min.js" "https://browser.sentry-cdn.com/${versions}/${variants}.min.js" || true
echo "${_endgroup}"
fi