Saw this error on the self-hosted e2e tests:
```
snuba-uptime-results-consumer-1 | Error: Invalid value for '--storage': 'uptime_monitor_checks' is not one of 'spans', 'metrics_raw', 'errors', 'search_issues', 'generic_metrics_distributions_raw', 'generic_metrics_counters_raw', 'generic_metrics_sets_raw', 'generic_metrics_gauges_raw', 'eap_items', 'groupassignees', 'outcomes_raw', 'querylog', 'functions_raw', 'transactions', 'replays', 'profile_chunks', 'profiles', 'groupedmessages', 'group_attributes'.
```
Seems like we don't need that consumer anymore, since it's moved to EAP.
* Remove symbolicator volume creation command
* Remove symbolicator volume
* Create sentry-symbolicator on docker compose up
* Pass volume name to remove_command
* Remove sentry-symbolicator from unit test
* feat: Use S3 node store with garage
* lol, fix bash
* moar bash
* lol
* hate bash
* fix moar bash
* Add healthcheck to garage service
Co-authored-by: Reinaldy Rafli <aldy505@proton.me>
* revert +x
* fix healthcheck, fix config
* add env var for garage size
* use better compression level
* simpler garage config
* add migration support
* feat: use seaweedfs as nodestore backend (#3842)
* feat: seaweedfs as s3 nodestore backend
* fix: 'server' was missing for seaweed
* feat: remove minimum volume free space
* feat: specify hostname on ip
* fix: grpc port on seaweed should be `-{service}.port.grpc` instead of `-{service}.grpcPort`
* fix: wrong access key & secret key; use localhost for internal comms
* fix: create index directory
* test: add sentry-seaweedfs volume into expected volumes
* debug: aaaaaaaaaaaaaaaaaaaaaaarrrrggggggghhhhhhhhhhhhhhh
* test: correct ordering for expected volumes
* chore: seaweedfs healthcheck to multiple urls
See https://stackoverflow.com/a/14578575/3153224
* chore: add swap for arm64 runners
* ci: debug memory issues for arm64 runners
* ci: turn off swapfile first
Turns out the arm64 runners already have 3GB of swap
* feat: nodestore config update behind a prompt/flag
* feat: set s3 lifecycle policy
* fix: seaweed is a busybox
* fix: try xml policy
* fix: go back to simplified json
* Revert "fix: go back to simplified json"
This reverts commit 2f1575dfe33db6f781b09d09b01f5382716b8826.
* chore: reword debug lifecycle policy
* fix: don't pollute APPLY_AUTOMATIC_CONFIG_UPDATES variable
---------
Co-authored-by: Reinaldy Rafli <github@reinaldyrafli.com>
feat(tasks) Remove taskworker option override and add worker healthcheck
Along with getsentry/sentry#99374 and this change taskworkers will be
enabled by default in self-hosted. I've left the celery workers active
to smooth over any tasks that are in-flight during an upgrade.
Add a worker healtcheck as we have one now.
Refs STREAM-450
* Add patch for pgbouncer
* pgcat over pgbouncer
* Add patch for .env file
* Apply patches and add initial pgcat tolm file
* feat: hardcode pgcat image
* Fixes from review
* Align usernames defaults
* Remove postgres from default depends_on; Covered by pgcat by extension
* Set user and password - pgcat maybe doesnt support host auth trust
* Pool name maybe has to match, for some reason
* Use healthcheck from pgcat PR
* Reduce pool size, leave some for healthchecks and other clients running
* Start pgcat for bash scripts with postgres
* Update docker-compose.yml
* Use pgbouncer
* Revert to TRUST method
* feat: run snuba-items consumer
* feat: remove process-spans as it turns out relay does not publish to that topic yet
* feat: try re-adding process-spans
* feat: add snuba subscriptions-scheduler-executor for eap_items
* feat: run process-segment sentry consumer
---------
Co-authored-by: Burak Yigit Kaya <byk@sentry.io>
* Revert "fix(vroom): Explicitly set PROFILES_DIR for upcoming change (#3759)"
This reverts commit e07445d6be41793165316a3e077ebec343740530.
It also very importantly changes where we mount the profiles volume which fixes the issue. Our theory is as follows:
1. Vroom Dockerfile had a line doing `mkdirp /var/lib/sentry-profiles` at image build time. This makes the directory owned by `root`
2. When we mount over that directory, and change permissions we can store the permissions changes _in_ the directory but not the directory itself
3. So when we start the vroom image with the new mount, the contents are owned by `vroom` but the main directory is still owned by `root`. This is also why [this approach](a23a4e3952) worked as the entrypoint script did this at the start of every container instance.
---------
Co-authored-by: Burak Yigit Kaya <byk@sentry.io>
PROFILES_DIR was defaulting to `/var/lib/sentry-profiles` which requires root access. When Vroom image decided to go with non-root default user, this started causing permission issues. Now the image is being refactored and it will not use `/var/lib/sentry-profiles` as the default path so we need to explicitly pass it.
`geoipupdate` is not used by any other service nor it is needed for any
service to run. Moreover, it is a one-shot command, causing
`docker compose up --wait` to fail when it exits with a non-zero status.
This happens when one has not yet set up their credentials and they may
choose to never do this.
This PR removes `geoipupdate` from the `docker-compose.yml` file and
moves the command directly into the geoip related script. One may run
this whenever they want to update their GeoIP database.
This PR needs an accompanying docs change.
As Sentry continues to evolve, effective resource management becomes crucial for maintaining performance and stability. This update includes configurations that will help optimize Redis's memory usage, ensuring that the system runs efficiently under varying loads.
**Key Changes:**
- **Added `maxmemory` Directive**: Configured Redis to limit its memory usage to a specified size. This prevents excessive memory consumption and helps maintain system stability.
- **Set `maxmemory-policy` to `allkeys-lru`**: This policy allows Redis to evict the least recently used keys when it reaches the memory limit, ensuring that frequently accessed data remains available while older, less-used data is removed.
Things weren't as smooth as I thought it would be.
### Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
