986 Commits

Author SHA1 Message Date
Steven Achilles
b9f8719919 Fix nginx configuration 2025-02-22 20:09:00 +01:00
Kazunori Kimura
fc0012a696 Change nginx log format name from main
it reports error as it is duplicated
2025-02-21 11:37:04 +09:00
Kazunori Kimura
9dff930f5f Change path to install nginx configuration
Directories such as /etc/nginx/sites-{enabled, available and similar}
are not available for nginx package comes from official nginx repository.

Ubuntu source may create this directory
refer1 : ubuntu distributes nginx with a file "nginx-common.dirs"
which lists these directories
https://git.launchpad.net/ubuntu/+source/nginx/tree/debian/nginx-common.dirs?h=ubuntu/jammy-updates

By default, nginx recognizes configuration files in /etc/nginx/conf.d/
Just change the destination path to install nginx configuration files
2025-02-21 11:37:03 +09:00
Kazunori Kimura
049df9a120 Just use result of $(command -v pg_isready)
instead of `find /usr/lib/postgresql -name pg_isready | sort | tail -n1`
for db readiness query command
To solve following error:

    find /var/lib/postgresql : No such file or directory

I'm sure the directory is exists so I'm not sure why the error reported
2025-01-20 19:21:09 +01:00
Kazunori Kimura
d7c3997285 remove libyaml-dev from BUILD_DEPENDENCIES
Because this is required on runtime
and BUILD_DEPENDENCIES will be removed at the end of installation
Installation of `libyaml-dev` is written in Dockerfile
so additional installation is not reeded

if not, following error reported on runtime

> LoadError: libyaml-0.so.2: cannot open shared object file: No such or directory - /usr/local/lib/ruby/3.2.0/x86_64-linux/psych.so

This is because Ruby 3.2.0 (psych 5.0.0) or later
does not contain libyaml any more
https://github.com/ruby/psych/pull/541
2025-01-20 19:20:04 +01:00
Kazunori Kimura
3cde3caf98 update patch 0003-fix_preinstall.mjs-to-avoid-removing-node_modules-dir.patch
File updated
No changes to the scripts we want to modify

- Simplify fileURLToPath usage in scripts
  https://gitlab.com/gitlab-org/gitlab/-/merge_requests/173402
  especially in commit 462273d2ba
  first contained in : v17.7.0-ee
2025-01-19 10:18:58 +01:00
Joerg Stoever
e5dc2dd5b6 healthcheck: Use IPv4 loopback address instead of localhost
Access to health check resources such as /-/liveness
is restricted to IPs specified in gitlab.monitoring.ip_whitelist
(`GITLAB_MONITORING_IP_WHITELIST`).
The name `localhost` is solved to IPv6 loopback address (::1)
that is not listed in the whitelist by default.

Possible alternate designs:
- Add IPv6 loopback to whitelist
- Disable IPv6 for gitlab container by specifying `net.ipv6.conf.all.disable_ipv6=1`
  in docker-compose.yml for example
  See https://github.com/sameersbn/docker-gitlab/issues/2766#issuecomment-2098030791
2024-09-21 02:12:02 +09:00
Kazunori Kimura
ba429ab996 Unset default value for GITLAB_MONITORING_IP_WHITELIST
On upstream, expected default value is `127.0.0.1/8`
and it is already listed in corresponding configuration.
`GITLAB_MONITORING_IP_WHITELIST` is used to allow monitoring from hosts other than loopback (localhost).

So just unset default value for it.
If the value is not set, the line specifying this "additional" IP range will be removed.
2024-09-20 23:14:38 +09:00
Joerg Stoever
f9d2c98995 health check: remove curl's --location option
When the healthcheck feature introduced, the script were generated on build time
and url was fixed to "http://localhost/-/liveness".
See pull request #2102

nginx is configured to redirect all http traffic to https when `GITLAB_HTTPS` is enabled.
(see https://github.com/sameersbn/docker-gitlab/blob/ac9e1fe/assets/runtime/config/nginx/gitlab-ssl#L41-L54),
`--location` option is set to follow the redirection.
See pull request #2165

Health check script generation has been ported to the runtime,
allowing us to dynamically generate health check URLs
while referencing configuration parameters.
See #2338

If configured correctly, the redirect will not occur and the option can be removed.

Original removal suggestion by @Gaibhne , additional (historical) research by @kkimurak.
Co-authored-by: Kazunori Kimura <kkimura@ims.ac.jp>
2024-09-20 23:14:38 +09:00
Joerg Stoever
38be108bc0 health check: use https only when https enabled with non-self signed cert 2024-09-20 23:14:38 +09:00
Kazunori Kimura
b3b09c619c fix indent in script configure_feature_flags.rb 2024-08-14 00:51:49 +09:00
Kazunori Kimura
dabfd42eac fix script to configure feature flags that cannot enable flags
When generating the list of flags that would actually be enabled,
the value specified in the option was ANDed with the list of flags that actually existed.
At this time, I mistakenly performed a logical AND with the list
of "values ​​to invalidate" specified in the option.
This mistake caused a problem in which some flags could not be enabled.
This commit fixes this to AND with the list of values ​​you want to enable.
2024-08-14 00:50:24 +09:00
Steven Achilles
c98e4def27
Merge pull request #2953 from molnarpe/omniauth-oidc
Add support for Omniauth OpenID Connect login.
2024-07-12 05:08:03 +02:00
Sebastian Toll
0431ba5ace Create external-diffs directory if missing, closes #2965 2024-07-10 21:35:43 +02:00
molnarpe
b35d4f2aca
Merge branch 'sameersbn:master' into omniauth-oidc 2024-07-06 06:01:21 +02:00
Kazunori Kimura
3c0be758cb Add ENV RUBYGEMS_VERSION and set to latest 3.5.x
that is GitLab v17.0 recommendation

MR that updates docs/install/installation.md:
- "Bump version of rubygems in installation instructions"
  https://gitlab.com/gitlab-org/gitlab/-/merge_requests/149976

There is no entry in .tools-version
2024-06-21 23:30:34 +09:00
molnarpe
de8d0d52a9
Update env-defaults
Value for client_auth_method is 'query' in the example, but default value is 'basic' as described in [GitLab OmniAuth OIDC documentation, step 4.](https://docs.gitlab.com/ee/administration/auth/oidc.html).
2024-06-19 15:25:46 +02:00
Molnár Péter
f30613b192 Add support for Omniauth OpenID Connect login. 2024-06-18 21:37:37 +02:00
Yannis Mazzer
5204971ce0 #2420 Handle encrypted_settings_key_base variable to allow restoring backups from gitlab instances not running from this image and using encrypted settings feature. 2024-05-24 20:34:18 +02:00
Filipe Roque
c9973b4ba6 Fix recompile spelling 2024-04-30 09:26:00 +01:00
Kazunori Kimura
351711c4cc Simplify message on feature flag target specified as reviewed
like below:

````
Configuring gitlab::feature_flags...
- Launching rails runner to set feature flags. This will take some time....
- Specified feature flags  to be disabled:
--- "auto_devops_banner_disabled"
--- "invalid_flag_name"
- Specified feature flags  to be enabled
--- "git_push_create_all_pipelines"
--- "another_invalid_flag_name"
- Following flags are probably invalid and have been ignored
--- invalid_flag_name
--- another_invalid_flag_name
- Apply user defined feature flags:
--- auto_devops_banner_disabled : off
--- git_push_create_all_pipelines : on
````

- Do not `puts` parsed options (formatted by ruby's `.to_s`),
  `puts` each specified target with three hyphens instead
- `puts` each ignored target on each line with three hyphens instead of single line csv
2024-03-24 19:17:25 +09:00
Kazunori Kimura
05794a2b47 Allow setting feature flags via configuration parameter
- It requires database is set up because
  feature flags are stored to DB (table `application_settings`)
- Add configuration parameter GITLAB_FEATURE_FLAGS_ENABLE_TARGETS
  and GITLAB_FEATURE_FLAGS_DISABLE_TARGETS
- Add ruby script to configure feature flags from command line
  and invoke runtime (from configure_gitlab())
2024-03-24 18:43:51 +09:00
Kazunori Kimura
846a051bbe Allow additional configurations after DB migration
Such as feature flags (which is stored to database, application_settings table)

At this commit, it does nothing (no operation `:`)
2024-03-24 18:43:51 +09:00
Kazunori Kimura
9f5961939d Allow patching Ruby, add patch for lib/securernadom.rb
Backport: fix behavior of `SecureRandom.gen_random_openssl(n)`
commit:
64e503eb62

This has been merged into the Ruby 3.3 release, but Ruby (at least) 3.0 and later are affected by the issues fixed by this commit.
2024-02-13 04:27:50 +09:00
Kazunori Kimura
e636b969a9 Revise build patch structure
to allow patching each middleware or library

Just move patches to assets/build/patches/gitlabhq
Folder name are matched to assets/runtime/config
2024-02-13 04:27:50 +09:00
Steven Achilles
74f8514385
Merge pull request #2901 from danyill/issue-2790-secure-ci-folder
Create ci_secure_files directory if missing, closes #2790
2024-02-12 19:55:10 +01:00
Daniel Mulholland
3f2e8c2ce4
Create ci_secure_files directory if missing, closes #2790 2024-02-11 10:17:36 +13:00
Kazunori Kimura
1c286f72d1 Remove trailing whitespace in patch for assets:compile
to suppress warning on build like below:

    Applying patch /etc/docker-gitlab/build/patches/0004-fix-raketask-gitlab-assets-compile.patch for gitlab-foss...
    <stdin>:15: trailing whitespace.
           removal_targets += Dir.children(Tasks::Gitlab::Assets::PUBLIC_ASSETS_DIR).map {|child| File.join(Tasks::Gitlab::Assets::PUBLIC_ASSETS_DIR, child)}
    warning: 1 line adds whitespace errors.
2024-01-30 06:39:52 +00:00
Kazunori Kimura
9345235f27 Replace removed function in patch for assets::compile
`Dir.exists?(file_name) -> bool` have been deprecated since Ruby 2.1
and removed on Ruby 3.2.
See https://bugs.ruby-lang.org/issues/17391

Replacement is `Dir.exist?(file_name) -> bool`.
2024-01-30 06:39:52 +00:00
Steven Achilles
5b092729fa
Merge pull request #2884 from kkimurak/issue/2866-fix-assets_webpack_manifest.json-not-found
Add build patch to prevent rake task assets:compile to remove assets dir
2024-01-19 22:53:40 +01:00
Kazunori Kimura
3885a455d1 Add build patch to prevent rake task assets:compile to remove assets dir
GitLab does not launch after second run if relative url is used.
This is caused by following upstream change to remove assets directory on assets compile.
See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103715

This is introduced on v15.6.0
````sh
$ git -C ../gitlab.git/ tag --contains e46d92c0 | sort --version-sort | head -n 1
v15.6.0-ee
````

1. `sameersbn/gitlab` create symbolic link /home/git/gitlab/public/assets/
  to point /home/git/data/tmp/assets if relative url is used.
  This is to store assets in the docker volume to avoid unnecessary recompilations.
  These assets are removed and recompiled only when the gitlab version or relative url root is changed.
2. By the change provided by gitlab.com/gitlab-org/gitlab!103715,
  rake task `gitlab:assets:compile` became to remove assets directory directly (by `FileUtils.rm_rf()`).
  It does not remove compiled assets itself, but remove symlink /home/git/gitlab/public/assets .
  Then it compiles assets as usual, but they will be stored in newly-created normal directory /home/git/gitlab/public/assets/
3. On container down, whole container statement (except volumes) will be reset.
  These compiled assets will be removed as well because they are not in docker volume.
4. As we store version info and relative url root path to /home/git/data/tmp/,
  we cannot recognize we have to recompile assets (that have been removed by mistake)

To avoid the issue, this commit add a build time patch to change the behavior of rake task `gitlab:assets:compile`
to empty assets instead of removing assets directory itself.
2024-01-18 18:10:26 +09:00
Kazunori Kimura
d16119b54b Remove ajv as an explicit dependency
There was an issue that `ajv` is not installed by mistake.
This issue have been fixed on upstream, a few years ago.

Now, we can revert the change.
This commit partially reverts 985d57afb9673b2f5acb1f12cbc13f230f6ec074
See sameersbn/gitlab#1358

It seems that fix MR on upstream is https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14543
but we cannot check diff because source / target branch removed by mistake

Anyway, no need to add ajv as an explicit dependency with (at least) gitlab v11 or later
2024-01-18 15:42:17 +09:00
Steven Achilles
af17c908cf
Merge pull request #2847 from kkimurak/add-patch-for-preinstall.mjs-relative_url
Add build patch to avoid removing node_modules
2023-11-19 13:29:05 +01:00
massej
e574613c9e
Update assets/runtime/config/gitlabhq/gitlab.yml
Co-authored-by: Kazunori Kimura <33391846+kkimurak@users.noreply.github.com>
2023-10-24 08:23:11 -04:00
massej
0001125859
Update env-defaults 2023-10-24 08:22:08 -04:00
massej
d11ae7b617
Added missing configuration variable OAUTH_ALLOW_BYPASS_TWO_FACTOR. 2023-10-23 22:17:20 -04:00
Steven Achilles
daae8b70d3
Merge pull request #2841 from massej/patch-1
Added - OAuth2 Generic - missing label and scope variables
2023-10-23 22:45:00 +02:00
Kazunori Kimura
da0d78bf5b Add build patch to avoid removing node_modules
note: Only affected if using the relative_url feature

`gitlab/script/frontent/preinstall.mjs` is introduced since v16.4.0.
This script is executed on container startup, if using the relative url.
This script removes `NODE_MODULES` (/home/git/gitlab/node_modules) when
"the folder seems to end up being a corrupted somehow"
See more detail:
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/130938

On sameersbn/gitlab, the folder node_modules is declared as volume.
The volume is always busy so that cannot be removed in the container.
You can see following error reported on container startup
(sameersbn/gitlab:16.4.0 or later).

````
yarn install v1.22.19
$ node ./scripts/frontend/preinstall.mjs
[WARNING] package.json changed significantly. Removing node_modules to be sure there are no problems. node:internal/process/esm_loader:97
    internalBinding('errors').triggerUncaughtException(
                              ^

[Error: EBUSY: resource busy or locked, rmdir '/home/git/gitlab/node_modules'] {
  errno: -16,
  code: 'EBUSY',
  syscall: 'rmdir',
  path: '/home/git/gitlab/node_modules'
}

Node.js v18.17.1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command. -----
````

This PR add a build time patch to change the behavior of
script/frontend/preinstall.mjs to not to remove node_modules directly,
but empty it instead.
2023-10-21 01:24:39 +09:00
Kazunori Kimura
a2dd4e5ed8 Accept multiple hosts for RACK_ATTACK_WHITELIST
see sameersbn/docker-gitlab#2828

The current setup also accepts multiple hosts,
but the syntax is a bit strange.
The leading/trailing double quotes are embedded
in the configuration file itself,
so users should expect double quotes around the string they set.
In other words, when setting two hosts 0.0.0.0 and 1.1.1.1,
you will set the strings 0.0.0.0","1.1.1.1 in the
environment variables. This is not intuitive.

This commit removes double quote around corresponding config
and set backward compatibility fallback process
to surround whole with [], each host with double quote.
Also, validation script (written in ruby) will be executed during configuration.

Example docker-compose.yml
````yaml
services:
  gitlab:
    image: sameersbn/gitlab:latest
    environment:
    - RACK_ATTACK_WHITELIST='["127.0.0.1","0.0.0.0"]'
````

Co-authored-by: Mikhail Khadarenka <chodorenko@mail.ru>
2023-10-20 22:24:22 +09:00
massej
a903d98290
Update functions 2023-10-01 13:45:02 -04:00
massej
af693c5ecc
Update env-defaults 2023-10-01 13:44:28 -04:00
massej
c8bc43b914
Added missing label and scope variable field. 2023-10-01 13:40:45 -04:00
Steven Achilles
2fb3680976 Remove temporary patch for lfs object URL (see c51770292e) 2023-09-02 08:28:56 +02:00
Steven Achilles
c51770292e Add temporary patch for lfs object URL 2023-08-23 18:57:51 +02:00
Steven Achilles
fbb9a7422b Update puma.rb 2023-06-24 08:40:57 +02:00
Steven Achilles
a63405d433 Add pool option to database.yml 2023-06-18 09:42:35 +02:00
Kazunori Kimura
7140f0342b Fix unused client removal on restarted container
Handle the case where the target does not exist
in the process of deleting unused clients.
Such a situation will not occur in newly launched containers,
but will occur if they are restarted.

During container restarts, container status are preserved.
If the unused database client was deleted in the last run,
grep will not match anything.
It returns non-zero code and the container stops there
because entrypoint sets option `-e`
(exit immediately on non-zero exit code excluding some special cases)

This commit make the uninstall process to handle the case
UNUSED_DB_CLIENTS is empty.
2023-06-02 22:12:29 +09:00
Steven Achilles
230284ebcf Do not copy ruby by installing gitaly 2023-05-25 18:14:23 +02:00
Steven Achilles
5fc732eb3c Add new configuration options for database.yml 2023-05-25 06:44:20 +02:00
Kazunori Kimura
598a666693 Add -y option on uninstalling unused db client
Co-authored-by: Steven Achilles <steven.achilles@googlemail.com>
Co-authored-by: @f-io
2023-05-25 12:44:32 +09:00