261 Commits

Author SHA1 Message Date
Burak Yigit Kaya
acbaec64c0
build(python): Make default config py3 compatible (#692) 2020-10-03 02:44:24 +03:00
Burak Yigit Kaya
e333dbd56a
fix(install): Only use .env for fallbacks
This fixes a serious bug in install.sh where it ignored externaly set env variable values such as `SENTRY_IMAGE` in favor of the ones defined in `.env`, essentially making all our e2e tests usless.
2020-10-03 01:30:56 +03:00
Pataar
de2b610fc3
State that config files need to be updated as well (#690)
Our relay connection wasn't behaving correctly because of some missing config in our `sentry.conf.py`. Also, we didn't have the performance module enabled because of a missing feature flag.

That's why an entry in the documentation regarding this would be nice.
2020-10-02 19:40:46 +03:00
Burak Yigit Kaya
dcc7fa4e6f
meta(issues: Improve and clarify install log request 2020-09-28 23:08:41 +03:00
Anton Ovchinnikov
0a4b2ad033
fix(install): Check that all ingest topics exist (#686)
Probably this is not needed anymore after https://github.com/getsentry/sentry/pull/20984 is merged, but just in case
2020-09-25 20:54:52 +03:00
Burak Yigit Kaya
3408e3db99
fix(redis): Increase file descriptors to 10032 (#681)
@xbenjii reported the following error on #629:
>You requested maxclients of 10000 requiring at least 10032 max file descriptors.

Increasing this limit by default makes sense to make Redis more available to heaveier loads.
2020-09-23 00:17:40 +03:00
Burak Yigit Kaya
74464695ec
fix(docker): Lower docker version to 19.03.6 (#682)
Fixes #689.
2020-09-23 00:17:21 +03:00
Burak Yigit Kaya
ce8538e83f
fix(ingest-consumer): Create the missing Kafka topics (#685)
Fixes #683, supercedes #684.
2020-09-23 00:04:42 +03:00
getsentry-bot
21d1b6d42e license: Update BSL change date 2020-09-15 21:21:14 +00:00
Burak Yigit Kaya
49dbeabb2b
ci(gcb): Drop Docker version to 19.03.8 as that's the latest on GCB 2020-09-15 20:19:58 +03:00
Burak Yigit Kaya
f8a2e48ed6
build(docker): Bump min Docker & Compose versions (#676)
We need `docker-compose ps -a` for CI so we were already using 1.24.1, this aligns the rest with that.

For Docker, there are a bunch of network-related fixes in 19.03.12 and prior (DNS fallback and IPv6 advertising) that we'd like to have to see if they are going to fix some reported connectivity issues w/ onpremise.
2020-09-15 19:40:42 +03:00
Bruno Bronosky
5631d45115
fix(install): Use proper bash testing (#673)
Fixes https://github.com/getsentry/onpremise/issues/672

I split this PR up into 4 commits. The first one is the bare minimum for the issue. The rest are just consistency corrections that we neckbeards at irc://chat.freenode.net/%23bash would always make.
2020-09-14 23:19:20 +03:00
Burak Yigit Kaya
9a18a4a366
fix(kafka): Reduce Kafka resource usage (#674)
Fixes #502 and applies the suggestions from there:

- Number of partitons=1 (from 40)
- Log retention to 1 day (from 7 days)

These settings should be more suited towards the scale this repo is intended for.

NOTE: The partition count change will only affect new installs unless `sentry-kafka` and related volumes are cleaned.
2020-09-14 23:03:44 +03:00
Burak Yigit Kaya
d6247e2374
feat(env): Add SENTRY_BIND var (#512)
Closes #279, supercedes #306. This is much simpler and safer now that we have `nginx` in front of everything.

Thanks a lot @larsnystrom!
2020-09-14 22:41:53 +03:00
Burak Yigit Kaya
313fef026a
fix(symbolicator): Use the latest Symbolicator version for nightlies (#670) 2020-09-11 13:29:42 +03:00
Burak Yigit Kaya
9b3aee1b1f
ref(self-hosted): More on-premise -> self-hosted (#669) 2020-09-09 22:30:27 +03:00
Burak Yigit Kaya
23251e79b9
ci(gha): Fix intermittent test failures (#664)
GitHub team pointed us to docker/compose#3586 as the likely root cause and some digging around revealed got us to https://git.io/JUn7p as a potential fix, which we are trying here.
2020-09-08 22:03:04 +03:00
Lyn Nagara
9b9066fc02
feat(snuba): Run new migration system (#663)
This is for the onpremise release on Sept 15th.

The new migration system has a migration to handle recreating the
transaction table if the old one is present, we no longer need to do
this in `install.sh`.
2020-09-08 21:17:55 +03:00
Burak Yigit Kaya
d83432996d
feat(clickhouse): Reduce max memory usage to 30% of RAM (#662)
Closes #616, supersedes #651

Adds an option to reduce max memory usage of Clickhouse server. Sets it to 30% of all available RAM as the default.

Co-authored-by: Renaud Chaput <renchap@gmail.com>
2020-09-08 21:06:09 +03:00
Burak Yigit Kaya
1ba417663a
fix(release): Bump the license date on master w/ calver releases (#658) 2020-09-08 18:52:46 +03:00
Tomasz Kontusz
e293a0fc97
fix(install): Don't use getopt for options parsing (#660)
The code assumed GNU getopt, which is not the default on many platforms, notably OSX.

Fixes #656, and replaces #659.
2020-09-07 11:04:43 +03:00
Burak Yigit Kaya
98b011ae88
fix[readme): Don't recommend nginx in docker-compose (#657)
Fixes an issue where we recommend adding TLS terminating Nginx into the docker-compose file. This is no longer recommended as we already have an nginx instance there for routing purposes.

Fixes #653.
2020-09-04 22:49:56 +03:00
Burak Yigit Kaya
59db1f2957
ref(symbolicator): Tag the latest symbolicator release for releases (#654) 2020-09-03 21:52:26 +03:00
Burak Yigit Kaya
5bcc795964
meta(readme): Point documentation URL to the new develop site
Follow up to getsentry/develop#123
2020-09-01 23:53:34 +03:00
Burak Yigit Kaya
d55e7dbae3
meta(readme): Update build badge from TravisCI to GHA
Follow up to #648
2020-09-01 23:51:49 +03:00
Tomasz Kontusz
1a9b45fb9f
EditorConfig and indentation fixes (#650)
EditorConfig based on conversation in https://github.com/getsentry/onpremise/pull/615#discussion_r477338006, and with the indentation fixes.

This PR will probably conflict with everything, but at least the code is a little bit more consistent :-D
2020-08-31 14:40:52 +03:00
Burak Yigit Kaya
61bc028a17
ci(craft): Remove the delay between prepare and publish
This should be obsolete by getsentry/craft#117.
2020-08-27 21:45:54 +03:00
Burak Yigit Kaya
cb8bb3b2e7
ci: Disable beacon on CI e2e tests (#649)
This also brings in common pre-test commands into test.sh file.
2020-08-27 20:27:55 +03:00
Kevin Waddle
175ad09057
install: Add flag to skip create user prompt during install (#646)
Fixes #611.
2020-08-27 18:32:20 +03:00
Burak Yigit Kaya
a5593e99ae
ci: Remove TravisCI (#648)
Since #634 and #639 we are using GitHub Actions for pushes and PRs and don't need Travis CI. Thanks for all the 🐟
2020-08-27 18:26:39 +03:00
Tomasz Kontusz
627c366f27
install: Keep relay available while upgrading (#615)
This continues on the ideas from #607. By "downtime" here I mean "not accepting events" - web, smtp and background processes are out of scope.

This PR adds a `--minimize-downtime` option to install.sh. This options changes the behaviour of the script by:
1. keeping nginx and relay running until the very end,
2. disabling cleanup on exit and failure,
3. explicitly reloading nginx configuration,
4. and starting the whole cluster at the end.

The results are promising: no downtime if relay version doesn't change, and only a second when it does. So far this was only tested with a curl loop, so I'm still not sure if Relay flushes the events to Sentry before getting recreated by `$dc up`.
2020-08-27 18:02:33 +03:00
Burak Yigit Kaya
704e4c3b5b
meta(issues): Add issue templates (#645)
https://app.asana.com/0/1169344595888357/1190341230301209/f

Co-authored-by: Fiona <61481573+PeloWriter@users.noreply.github.com>
2020-08-25 22:10:29 +03:00
MK
6a40e33073
Use the custom Nginx access log format (#642)
If we are going to define a custom log format, we should probably be using that format.

Fixes #635
2020-08-24 08:56:06 +03:00
Chad Whitacre
e8d882ca05
ci(gha): Fine-tune GHA event bindings (#639) 2020-08-23 21:34:01 +03:00
Burak Yigit Kaya
a7b2ddbf2e
upgrade(clickhouse): Use the Clickhouse version we use in prod (#630)
Co-authored-by: josh <josh@jrl.ninja>
2020-08-21 22:34:51 +03:00
Chad Whitacre
4b8d424477
ci(github): Reimplement CI in GitHub Actions (#634)
Part of #627, sets us up for expanding the OS matrix we test against.
2020-08-20 22:05:19 +03:00
Burak Yigit Kaya
fb125a1e4c
fix(install): TSDB migration should not create invalid config file (#631)
Fixes #624
2020-08-17 18:31:15 +03:00
Chad Whitacre
67f70915b5
fix(env): Read and set .env in install.sh, portably (#626)
h/t https://unix.stackexchange.com/a/79077

Fixes #622, cf. #600 

cc: @NullIsNot0
2020-08-17 16:16:06 +03:00
Burak Yigit Kaya
918cee5749
ci(release): Fix cancellation URL 2020-08-15 17:16:24 +03:00
Burak Yigit Kaya
7bf8355815
ci(release): Add note when killswitch is activated 2020-08-15 17:11:46 +03:00
Burak Yigit Kaya
1410d00296
ci(release): Add the killswitch option to workflow UI 2020-08-15 16:45:45 +03:00
Burak Yigit Kaya
cf350bd700
"ci(release): Add killswitch via issues w/ release-blocker label
Implements https://app.asana.com/0/1169344595888357/1146357826982899/f which would cancel the workflow (stop the release) when the repo has open issues with the label 'release-blocker'".
2020-08-15 16:40:37 +03:00
buffcode
9c8e2a29eb
typo: Example configuration comment (#620)
Fixes a small typo in a example configuration comment
2020-08-14 21:59:36 +03:00
Burak Yigit Kaya
5d00d613fa
ci(release): Move to workflow_dispatch for UI-triggered releases 2020-08-12 13:05:14 +03:00
Tomasz Kontusz
35ad7dc477
feat: Only stop the cluster after building images (#614)
This reduces downtime for users with custom Dockerfiles. Fixes #607.
2020-08-03 23:32:56 +03:00
Burak Yigit Kaya
4dbfcbcebe
ci(release): Fix quotes for auto version setting 2020-07-18 01:26:47 +03:00
Burak Yigit Kaya
17d97e13c0
ci(release): Smarter and safer auto version setting
If the release action is run without an explicit version in the same calendar month more than once, all of them will try to release the same version where the patch version is set to 0. This is never the intended action: if we are making a new release in the same month where an old one exists, it is 100% a patch release.

This PR automatically implements patch version increment based on existing versions.
2020-07-17 22:34:11 +03:00
Burak Yigit Kaya
094d8e38a3
fix(install): Read and set .env in install.sh (#600)
Fixes #597.
2020-07-17 15:29:55 +03:00
Burak Yigit Kaya
86864d0edf
feat(compose): Add ability to use custom images for any Sentry service (#602)
This change allows one to override any Sentry service image, mostly for testing purposes. It also removes the SENTRY_VERSION variable as docker-compose makes it very hard to cascade default values for these. Next step is to have integration tests in getsentry/snuba and getsentry/relay (and possibly for getsentry/symbolicator) for getsentry/onpremise using this PR.

Also related: #596.
2020-07-17 15:02:45 +03:00
Burak Yigit Kaya
e82506f5b7
fix(snuba): Add consumer group to transactions consumer (#599)
Without the consumer-group option, transactions consumer and events consumer will compete for messages on the same topic and usually events win, which is the first one. This may cause some data loss for performance and make it seem not work.

Should address https://forum.sentry.io/t/perfomance-tracing-for-sentry-itself/10405/5?u=byk
2020-07-17 06:50:36 +03:00