Author Topic: All BASIC Sandbox  (Read 4176 times)

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #60 on: December 15, 2018, 09:59:08 PM »
Thanks for the link.

Quote
Prepare for the Let's Encrypt Web Root Domain Verification

In order to receive an SSL certificate from the Let's Encrypt certificate authority, we must prove that we own the domain that the certificate will be provided for. There are multiple methods of proving domain ownership, each of which require root or administrator access to the server.

GitLab contains an internally managed Nginx web server for serving the application itself. This makes the installation rather self-contained, but it does add an additional layer of complexity when attempting to modify the web server itself.

Since the embedded Nginx is currently being utilized to serve GitLab itself, the best domain validation method is the web root method. Certbot will use the existing web server to serve a known file from the server on port 80. This proves to the certificate authority that the person requesting the certificate has administrative control over the web server, which effectively proves ownership over the server and domain.

To set up web root domain validation for GitLab, our first step will be to create a dummy document root:

    sudo mkdir -p /var/www/letsencrypt

This will be unused by normal Nginx operations, but will be used by Certbot for domain verification.

Next, we need to adjust GitLab's Nginx configuration to use this directory. Open up the main GitLab configuration file by typing:

    sudo nano /etc/gitlab/gitlab.rb

Inside, we need to add a line that will inject a custom directive into GitLab's Nginx configuration file. It's probably best to scroll down to the GitLab Nginx section of the file, but the line can be placed anywhere.

Paste in the following line:
/etc/gitlab/gitlab.rb

. . .
nginx['custom_gitlab_server_config'] = "location ^~ /.well-known { root /var/www/letsencrypt; }"
. . .

The Let's Encrypt web root verification method places a file within a .well-known directory in a document root so that the certificate authority can validate it. This line tells Nginx to serve requests for /.well-known from the web root we created a moment ago.

When you are finished, save and close the file.

Next, apply the changes to GitLab's Nginx configuration by reconfiguring the application again:

    sudo gitlab-ctl reconfigure

The server should now be set up to successfully validate your domain.

Since I already have an installed LE cert for sandbox.allbasic.info via the Plesk extension, shouldn't I be able to point nginx to the Plesk generated cert? This way Plesk auto renews the cert (like all the other sites I host) and this problem goes away forever.

My curiosity still haunts me how Gitlab created its initial cert?
« Last Edit: December 15, 2018, 10:13:04 PM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #61 on: December 16, 2018, 05:13:35 PM »
This is where Plesk stores the well-known hash for sandbox.allbasic.info.

Should I be able to point the Gitlab nginx well-know to this path?

[root@ip-172-30-0-53 acme-challenge]# ls -l
total 4
-rw-r--r--. 1 allbasic psacln 87 Dec  8 00:19 61Zs0ZbyRgTanhyS9WLODW36i5zsEOHjQbE9YGBQUMk
[root@ip-172-30-0-53 acme-challenge]# pwd
var/www/vhosts/allbasic.info/sandbox.allbasic.info/.well-known/acme-challenge
[root@ip-172-30-0-53 acme-challenge]#

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #62 on: December 16, 2018, 05:39:28 PM »
I added the following to the /etc/gitlab/gitlab.rb file.

nginx['custom_gitlab_server_config'] = "location ^~ /.well-known { root /var/www/vhosts/allbasic.info/sandbox.allbasic.info/.well-known/acme-challenge; }"

I then reconfigured Gitlab. No change if I eliminate the :8181 port reference and let the proxy reverse handle the redirect.

I then enabled Let's Encrypt in the gitlab.rb file and got the following error on reconfigure.


Running handlers:
There was an error running gitlab-ctl reconfigure:

letsencrypt_certificate[sandbox.allbasic.info] (letsencrypt::http_authorization line 3) had an error: RuntimeError: acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 20) had an error: RuntimeError: [sandbox.allbasic.info] Validation failed for domain sandbox.allbasic.info


Here is the gitlab.rb Template file fo reference.

Docs on Gitlab - Let's Encrypt Auto-Renewal.

Here is the certificate.rb file.
Code: Ruby
  1. property :cn, String, name_property: true
  2. property :fullchain, String, required: true
  3. property :key, String, required: true
  4. property :owner, [String, nil], default: lazy { node['letsencrypt']['owner'] }
  5. property :chain, [String, nil], default: lazy { node['letsencrypt']['chain'] }
  6. property :wwwroot, String, default: lazy { node['letsencrypt']['wwwroot'] }
  7. property :alt_names, Array, default: lazy { node['letsencrypt']['alt_names'] }
  8. property :key_size, [Integer, nil], default: lazy { node['letsencrypt']['key_size'] }
  9. property :crt, [String, nil], default: lazy { node['letsencrypt']['crt'] }
  10. property :group, [String, nil], default: lazy { node['letsencrypt']['group'] }
  11.  
  12. action :create do
  13.   # Attempt to fetch a certificate from Let's Encrypt staging instance
  14.   # If that succeeds, then fetch a certificate from production
  15.   # This helps protect users from hitting Let's Encrypt rate limits if
  16.   # they provide invalid data
  17.   helper = LetsEncryptHelper.new(node)
  18.   contact_info = helper.contact
  19.  
  20.   acme_certificate 'staging' do
  21.     alt_names new_resource.alt_names unless new_resource.alt_names.empty?
  22.     key_size new_resource.key_size unless new_resource.key_size.nil?
  23.     group new_resource.group unless new_resource.group.nil?
  24.     owner new_resource.owner unless new_resource.owner.nil?
  25.     chain "#{new_resource.chain}-staging" unless new_resource.chain.nil?
  26.     crt "#{new_resource.crt}-staging" unless new_resource.crt.nil?
  27.     contact contact_info
  28.     fullchain "#{new_resource.fullchain}-staging"
  29.     cn new_resource.cn
  30.     key "#{new_resource.key}-staging"
  31.     endpoint 'https://acme-staging.api.letsencrypt.org/'
  32.     wwwroot new_resource.wwwroot
  33.     sensitive true
  34.   end
  35.  
  36.   ruby_block 'reset private key' do
  37.     block do
  38.       node.normal['acme']['private_key'] = nil
  39.     end
  40.   end
  41.  
  42.   acme_certificate 'production' do
  43.     alt_names new_resource.alt_names unless new_resource.alt_names.empty?
  44.     key_size new_resource.key_size unless new_resource.key_size.nil?
  45.     group new_resource.group unless new_resource.group.nil?
  46.     owner new_resource.owner unless new_resource.owner.nil?
  47.     chain new_resource.chain unless new_resource.chain.nil?
  48.     crt new_resource.crt unless new_resource.crt.nil?
  49.     contact contact_info
  50.     fullchain new_resource.fullchain
  51.     cn new_resource.cn
  52.     key new_resource.key
  53.     wwwroot new_resource.wwwroot
  54.     notifies :run, 'execute[reload nginx]'
  55.     sensitive true
  56.   end
  57. end
  58.  


[root@ip-172-30-0-53 resources]# find / -name *.crt
/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
/etc/pki/ca-trust/source/ca-bundle.legacy.crt
/etc/pki/tls/certs/ca-bundle.crt
/etc/pki/tls/certs/ca-bundle.trust.crt
/etc/pki/tls/certs/localhost.crt
/etc/gitlab/ssl/sandbox.allbasic.info.crt
/root/gitlab-cleanse-2018-09-29T19:18/ssl/oxygenbasic.org.crt
/root/gitlab-cleanse-2018-09-29T19:42/ssl/oxygenbasic.org.crt
/var/opt/gitlab/postgresql/data/server.crt
/var/www/vhosts/northwestliving.info/httpdocs/blog/wp-includes/certificates/ca-bundle.crt
/var/www/vhosts/clouds-r.us/httpdocs/home/wp-includes/certificates/ca-bundle.crt
/usr/share/pki/ca-trust-legacy/ca-bundle.legacy.default.crt
/usr/share/pki/ca-trust-legacy/ca-bundle.legacy.disable.crt
/usr/local/psa/var/apspackages/adilUsnhf.zipcddac49b-7807-d14e-9058/cache/htdocs/wp-includes/certificates/ca-bundle.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/chef-13.6.4/spec/data/trusted_certs/example.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/chef-13.6.4/spec/data/trusted_certs/example_no_cn.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/eventmachine-1.0.9.1/tests/client.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/excon-0.62.0/tests/data/127.0.0.1.cert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/excon-0.62.0/tests/data/excon.cert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/bogushost.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/ca.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/selfsigned.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/server.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/trusted-ca.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/trusted-cert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/untrusted-ca.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/untrusted-cert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rest-client-2.0.2/spec/integration/capath_digicert/digicert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rest-client-2.0.2/spec/integration/capath_verisign/verisign.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rest-client-2.0.2/spec/integration/certs/digicert.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rest-client-2.0.2/spec/integration/certs/verisign.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/ruby-saml-1.7.2/test/certificates/ruby-saml-2.crt
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/ruby-saml-1.7.2/test/certificates/ruby-saml.crt
[root@ip-172-30-0-53 resources]#

[root@ip-172-30-0-53 resources]# find / -name *.key
find: /proc/21624: No such file or directory
/etc/pki/tls/private/localhost.key
/etc/named.iscdlv.key
/etc/named.root.key
/etc/trusted-key.key
/etc/gitlab/ssl/sandbox.allbasic.info.key
/root/gitlab-cleanse-2018-09-29T19:18/ssl/oxygenbasic.org.key
/root/gitlab-cleanse-2018-09-29T19:42/ssl/oxygenbasic.org.key
/var/opt/gitlab/postgresql/data/server.key
/var/named/chroot/var/run/named/session.key
/usr/share/doc/openssh-7.4p1/PROTOCOL.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/public.es512.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/private-ne.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/public.es384.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/private.es384.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/private.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/public.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/private.es512.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/private.es256.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/public.es256.key
/usr/local/psa/admin/plib/modules/letsencrypt/vendor/namshi/jose/tests/public-ne.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/chef-13.6.4/spec/data/ssl/chef-rspec.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/eventmachine-1.0.9.1/tests/client.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/excon-0.62.0/tests/data/127.0.0.1.cert.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/excon-0.62.0/tests/data/excon.cert.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/ca.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httparty-0.13.7/spec/fixtures/ssl/generated/server.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.3/test/client-pass.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.3/test/client.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.3/test/server.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/trusted-ca.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/trusted-cert.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/untrusted-ca.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/redis-3.3.5/test/support/ssl/untrusted-cert.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/ruby-saml-1.7.2/test/certificates/ruby-saml.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.15.0-x86_64-linux/src/ruby/spec/testdata/client.key
/opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/grpc-1.15.0-x86_64-linux/src/ruby/spec/testdata/server1.key
[root@ip-172-30-0-53 resources]#




Not sure where to go from here.

« Last Edit: December 17, 2018, 01:58:41 AM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #63 on: December 17, 2018, 01:46:23 PM »
I updated the sandbox Gitlab Prometheus integration as I was getting a warning when I would run the Gitlab reconfigure that the current version was being depreciated in version 12.

Quote
Prometheus is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. GitLab provides out of the box monitoring with Prometheus, providing easy access to high quality time-series monitoring of GitLab services.


Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #64 on: December 20, 2018, 04:44:30 PM »
I think the link you sent me only solves 1/2 the problem. I need to get the Gitlab cert links pointing tp the Plesk Let's Encrypt active certs paths and it might solve the problem.

I'll let you know.

Code: Ruby
  1.     "nginx": {
  2.       "listen_port": 8181,
  3.       "custom_gitlab_server_config": "location ^~ /.well-known { root var/www/vhosts/allbasic.info/sandbox.allbasic.info/.well-known/acme-challenge; }",
  4.       "ssl_certificate": "/etc/gitlab/ssl/sandbox.allbasic.info.crt",
  5.       "ssl_certificate_key": "/etc/gitlab/ssl/sandbox.allbasic.info.key",
  6.  

I'm not sure wihere the ssl_certificate links should point to.


Code: Bash
  1. [root@ip-172-30-0-53 gitlab]# find / -name 'sandbox.allbasic.info*'
  2. /etc/httpd/conf/plesk.conf.d/vhosts/sandbox.allbasic.info.conf
  3. /etc/nginx/plesk.conf.d/vhosts/sandbox.allbasic.info.conf
  4. /etc/gitlab/ssl/sandbox.allbasic.info.crt
  5. /etc/gitlab/ssl/sandbox.allbasic.info.crt_bk
  6. /etc/gitlab/ssl/sandbox.allbasic.info.key
  7. /etc/gitlab/ssl/sandbox.allbasic.info.key_bk
  8. /etc/gitlab/ssl/sandbox.allbasic.info.key-staging
  9. /etc/gitlab/ssl/sandbox.allbasic.info.key-staging_bk
  10. /var/lib/psa/dumps/domains/allbasic.info/sites/sandbox.allbasic.info
  11. /var/www/vhosts/system/sandbox.allbasic.info
  12. /var/www/vhosts/allbasic.info/logs/sandbox.allbasic.info
  13. /var/www/vhosts/allbasic.info/sandbox.allbasic.info
  14. /usr/local/psa/etc/logrotate.d/sandbox.allbasic.info
  15. /usr/local/psa/var/modules/letsencrypt/etc/archive/sandbox.allbasic.info
  16. /usr/local/psa/var/modules/letsencrypt/etc/live/sandbox.allbasic.info
  17. /opt/plesk/php/7.2/etc/php-fpm.d/sandbox.allbasic.info.conf
  18. [root@ip-172-30-0-53 gitlab]#
  19.  
« Last Edit: December 20, 2018, 04:56:02 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: All BASIC Sandbox
« Reply #65 on: December 20, 2018, 06:56:43 PM »
Try looking in: /usr/local/psa/var/modules/letsencrypt/etc/live/sandbox.allbasic.info

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #66 on: December 20, 2018, 08:07:13 PM »
Code: Bash
  1. [root@ip-172-30-0-53 sandbox.allbasic.info]# ls -la
  2. total 8
  3. drwx------.  2 psaadm psaadm   93 Dec 17 01:49 .
  4. drwx------. 15 psaadm psaadm 4096 Dec 10 15:51 ..
  5. lrwxrwxrwx.  1 psaadm psaadm   46 Dec 17 01:49 cert.pem -> ../../archive/sandbox.allbasic.info/cert10.pem
  6. lrwxrwxrwx.  1 psaadm psaadm   47 Dec 17 01:49 chain.pem -> ../../archive/sandbox.allbasic.info/chain10.pem
  7. lrwxrwxrwx.  1 psaadm psaadm   51 Dec 17 01:49 fullchain.pem -> ../../archive/sandbox.allbasic.info/fullchain10.pem
  8. lrwxrwxrwx.  1 psaadm psaadm   49 Dec 17 01:49 privkey.pem -> ../../archive/sandbox.allbasic.info/privkey10.pem
  9. -rw-r--r--.  1 psaadm psaadm  544 Nov  5 14:48 README
  10. [root@ip-172-30-0-53 sandbox.allbasic.info]# pwd
  11. /usr/local/psa/var/modules/letsencrypt/etc/live/sandbox.allbasic.info
  12. [root@ip-172-30-0-53 sandbox.allbasic.info]#
  13.  
  14. [root@ip-172-30-0-53 sandbox.allbasic.info]# cat README
  15. This directory contains your keys and certificates.
  16.  
  17. `privkey.pem`  : the private key for your certificate.
  18. `fullchain.pem`: the certificate file used in most server software.
  19. `chain.pem`    : used for OCSP stapling in Nginx >=1.3.7.
  20. `cert.pem`     : will break many server configurations, and should not be used
  21.                  without reading further documentation (see link below).
  22.  
  23. We recommend not moving these files. For more information, see the Certbot
  24. User Guide at https://certbot.eff.org/docs/using.html#where-are-my-certificates .
  25. [root@ip-172-30-0-53 sandbox.allbasic.info]#
  26.  
  27.  
« Last Edit: December 20, 2018, 08:11:45 PM by John »

Offline AIR

  • BASIC Developer
  • Posts: 684
Re: All BASIC Sandbox
« Reply #67 on: December 20, 2018, 10:02:19 PM »
I think you need to do this in your gitlab.rb file (disable the built in LE support first, then use the path you listed before):

« Last Edit: December 20, 2018, 10:26:57 PM by AIR »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #68 on: December 20, 2018, 11:26:09 PM »
We're back!

Gitlab hung with this parameter enabled.

Code: Text
  1. nginx['redirect_http_to_https'] = true
  2.  

Thanks for all your help with this. It's been a long road. On a positive note, Plesk will automatically keep the cert. renewed like it does for the other sites I host.

« Last Edit: December 21, 2018, 12:42:30 AM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #69 on: December 20, 2018, 11:51:33 PM »
There is still a problem if I use the proxy reverse as the commit data isn't loading. With the :8181 reference in the URL, everything seems fine.

I may try the PHP redirect I use for sites with no home page to resolve :8181 being in the URL.
« Last Edit: December 21, 2018, 01:53:14 AM by John »

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #70 on: December 21, 2018, 03:08:38 AM »
Quote
I may try the PHP redirect I use for sites with no home page to resolve :8181 being in the URL.

That worked.

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #71 on: December 25, 2018, 09:44:27 PM »
AIR,

Would you have time to point me in the right direction with getting syntax highlighting working for the languages being hosted in the sandbox?

Offline John

  • Forum Support / SB Dev
  • Posts: 2741
    • ScriptBasic Open Source Project
Re: All BASIC Sandbox
« Reply #72 on: June 17, 2019, 04:17:14 PM »
The clone URL now includes the :8181 port number.