It's hard to debug docker compose failure on other repositories since they can't see the `docker compose ps` and `docker compose logs`. One problem occurred here: https://github.com/getsentry/relay/pull/4940
This PR aims to provide both commands if failure happens.
* ci: run tests on arm64
* ci: runner name should be arm, not arm64
* ci: retain old job name to not mess with CI protection rules
* ci: integration test should not use plural form
As discussed, removing the upgrade test since it doesn't provide that much utility as we are already testing upgrades from restoring docker volumes from cache
In this version, there's a new `--pull` argument for `docker compose run` which we will start leveraging, especially with `sentry-admin` command.
Should come with a slight speed boost.
Just starting up services for Snuba or Sentry migrations takes up to a minute sometimes and we do this even when there are no migrations, just because one of the Sentry or Snuba migrations change. This patch splits the caches up so only the necessary one runs, saving further time. It also uses the `LATEST_TAG` as the cache key for upgrade tests as the image versions or data will never change for a certain tag once it is release.
Instead of using direct image hashes, only use hashes from migrations folders for each respective image for cache key generation. Should increase cache hit rate significantly as we don't have migrations much.
Also swaps the key order from `sentry-snuba` to `snuba-senry` assuming Snuba has less frequent migration additions.
This is an initial transitionary patch before landing #3516. Once we land this, we will update users of the old action to use this one and remove that repo. Then land #3516 safely.
Great thing is, with this patch and the subsequent update to getsentry/action-self-hosted-e2e-tests to use this one, all the repos would be using the Docker Volume caching we introduced in #3488.
Docker Compose is much more robust nowadays compared to the past where we had to maintain tests for both v1 and v2 and then a specific version of v2. Hence, we are removing tests for the older versions of Docker Compose with this patch. We also remove the separate tests for customizations and w/o customizations as the one with customizations should cover the one without them anyway. This reduces the CI workload to 25% of what it was
This patch caches all DB volumes based on the sentry and snuba images to avoid doing the same migrations over and over for every test run.
This shaved off a whole minute from "Install self-hosted" jobs (so ~20% speed increase).
Left side: cached re-run -- Right side: no-cache initial run
`geoipupdate` is not used by any other service nor it is needed for any
service to run. Moreover, it is a one-shot command, causing
`docker compose up --wait` to fail when it exits with a non-zero status.
This happens when one has not yet set up their credentials and they may
choose to never do this.
This PR removes `geoipupdate` from the `docker-compose.yml` file and
moves the command directly into the geoip related script. One may run
this whenever they want to update their GeoIP database.
This PR needs an accompanying docs change.
* Update minimum docker compose requirement
docker compose down <service> is now required.
* Update docker compose version to new minimum in CI config.
Fixes#1888
This integration test:
uploads a symbol file, which indicates where to find the symbols for the minidump
uploads a sample minidump, which creates an event we can check for
This test is based on the test here: 28f4f36b4a/tests/symbolicator/test_minidump_full.py (L72)
* fix failing unit tests
* fix unit test pipeline
* add newline to unit-test file
* preserve semantics from docker volume test
* add error-handling tests
* reorganize file structure of unit tests
* add comment for using _file format in ensure relay credentials test
* add error handling unit tests
* use unit-test directory
* stack trace should be from the unit test directory
* Slightly better fail signal
* Clearer start as well
Co-authored-by: Chad Whitacre <chadwhitacre@sentry.io>
