DoltLab v2.1.4 Installer Update
We've been on a tear as of late trying to make the new user experience of DoltLab simpler and easier.
Tim, our CEO, went through the process of setting up his own DoltLab instance and published two blogs about it; one that explains how to get started running DoltLab, and a second, which explains how to run a DoltLab Enterprise instance.
Just before Tim started running his own DoltLab instance, we announced a much needed improvement to DoltLab: the installer. The installer is a binary shipped with DoltLab that, when run, generates the configuration files and assets DoltLab requires to run. Prior to the installer, we shipped all of these files and assets as static files, which meant DoltLab administrators needed to hand-edit them to make configuration alterations.
Luckily for Tim, his DoltLab deep-dive benefited from having the installer ready for him to use, instead of the static configuration files of older DoltLab versions.
But, after working through the DoltLab setup process himself multiple times, Tim had a great suggestion for making the DoltLab experience, and the installer, more user-friendly. He suggested we ship one static file with DoltLab, a configuration file for using the installer. This would ensure users don't need to recall what arguments they'd previously run with the installer, when they wanted to alter the configuration of their DoltLab instance. I guess going from many static configuration files, to none, to one, is... progress 🤠. He also recommended we follow command-line-tool convention by making the installer's command line flags take priority over the fields set in the configuration file, so we've made that change as well.
Today we're excited to share that as of DoltLab v2.1.4, the installer has become even better! It will now run with the file ./installer_config.yaml, which we ship with DoltLab.
To show off what this looks like, I'll replay some portions of Tim's Getting Started: DoltLab, only this time using the installer's configuration file, instead of its command line arguments. Since nothing has changed regarding provisioning a DoltLab host on AWS EC2, I'll skip that section in this post.
Download DoltLab and its Dependencies
We've provisioned a new host, have opened ports 80, 100, 4321, and 50051, and have connected to it via ssh. Let's go ahead and download DoltLab v2.1.4 and install its dependencies.
ubuntu@ip-10-2-0-65:~$ sudo apt install unzip
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
zip
The following NEW packages will be installed:
unzip
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 175 kB of archives.
After this operation, 384 kB of additional disk space will be used.
Get:1 http://us-west-2.ec2.archive.ubuntu.com/ubuntu noble/main amd64 unzip amd64 6.0-28ubuntu4 [175 kB]
Fetched 175 kB in 0s (7115 kB/s)
Selecting previously unselected package unzip.
(Reading database ... 71839 files and directories currently installed.)
Preparing to unpack .../unzip_6.0-28ubuntu4_amd64.deb ...
Unpacking unzip (6.0-28ubuntu4) ...
Setting up unzip (6.0-28ubuntu4) ...
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...
Scanning linux images...
Running kernel seems to be up-to-date.
No services need to be restarted.
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
ubuntu@ip-10-2-0-65:~$ curl -LO https://doltlab-releases.s3.amazonaws.com/linux/amd64/doltlab-v2.1.4.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 19.1M 100 19.1M 0 0 11.8M 0 0:00:01 0:00:01 --:--:-- 11.8M
ubuntu@ip-10-2-0-65:~$ unzip doltlab-v2.1.4.zip -d doltlab
Archive: doltlab-v2.1.4.zip
inflating: doltlab/smtp_connection_helper
inflating: doltlab/installer
inflating: doltlab/installer_config.yaml
ubuntu@ip-10-2-0-65:~$ cd doltlabGreat! If we list the doltlab directory contents we can see the new configuration file for the installer:
ubuntu@ip-10-2-0-65:~/doltlab$ ls
installer installer_config.yaml smtp_connection_helperBefore we get to the new configuration file though, let's continue on dependency installation. We can use the --ubuntu flag with the installer to generate an install script for us:
ubuntu@ip-10-2-0-65:~/doltlab$ ./installer --ubuntu
2024-05-15T22:12:26.585Z INFO metrics/emitter.go:111 Successfully sent DoltLab usage metrics
2024-05-15T22:12:26.585Z INFO cmd/main.go:484 To install DoltLab's dependencies, use this script {"script": "/home/ubuntu/doltlab/ubuntu_install.sh"}
ubuntu@ip-10-2-0-65:~/doltlab$ ls
installer installer_config.yaml smtp_connection_helper ubuntu_install.shNow I run ./ubuntu_install.sh to complete the dependency installation.
Installer with a configuration file
Before using the installer, I make sure that I can run docker without the sudo command, by running sudo newgrp docker:
ubuntu@ip-10-2-0-65:~/doltlab$ sudo newgrp docker
root@ip-10-2-0-65:/home/ubuntu/doltlab#Now I edit the installer_config.yaml file, which the installer looks to use by default.
Here is what the configuration file contains by default:
version: v2.1.4
# "Host" is the hostname or IP address of the host running DoltLab. Required.
host: ""
# "Services" contains config for specific DoltLab services.
# The first time DoltLab runs, it uses the passwords defined in `doltlabdb.admin_password`
# and `doltlabdb.dolthubapi_password` to initialize DoltLab's application database.
services:
doltlabdb:
admin_password: "DoltLab1234"
dolthubapi_password: "DoltLab1234"
# "Default User" configures the default user account created by DoltLab the first time it starts up. This account is able to use
# all of DoltLab's features, without connecting DoltLab to an SMTP server.
default_user:
name: "admin"
password: "DoltLab1234"
email: "admin@localhost"
## "SMTP" is used to configure a connection to an existing SMTP server. Optional.
## For more information on connecting your instance to an SMTP server,
## see https://docs.doltlab.com/guides/administrator#connect-smtp-server.
# smtp:
# auth_method: "plain"
# host: ""
# port: 0
# no_reply_email: ""As you can see, it contains some default values for passwords that we'll want to change, and it contains some helpful comments about each section of the file.
Just to get DoltLab running quickly, though, I only need to supply a value host.
# "Host" is the hostname or IP address of the host running DoltLab. Required.
host: "52.40.213.187"I save my changes, and run the installer.
root@ip-10-2-0-65:/home/ubuntu/doltlab# ./installer
2024-05-15T22:20:54.801Z INFO metrics/emitter.go:111 Successfully sent DoltLab usage metrics
2024-05-15T22:20:54.801Z INFO cmd/main.go:491 Successfully configured DoltLab {"version": "v2.1.4"}
2024-05-15T22:20:54.802Z INFO cmd/main.go:497 To start DoltLab, use: {"script": "/home/ubuntu/doltlab/start.sh"}
2024-05-15T22:20:54.802Z INFO cmd/main.go:502 To stop DoltLab, use: {"script": "/home/ubuntu/doltlab/stop.sh"}And just like that the installer produces the start.sh script I can use to start my DoltLab instance.
root@ip-10-2-0-65:/home/ubuntu/doltlab# ./start.sh
dbacbb0ea43d9910da88a2b887817c191a79d7d23addaa8a12fc90f88713dc43
[+] Running 40/32
✔ doltlabfileserviceapi Pulled 2.5s
✔ doltlabui Pulled 141.2s
✔ doltlabremoteapi Pulled 14.3s
✔ doltlabgraphql Pulled 133.6s
✔ doltlabdb Pulled 19.6s
✔ doltlabenvoy Pulled 19.5s
✔ doltlabapi Pulled
[+] Running 13/13
✔ Volume "doltlab_doltlabdb-dolt-root" Created 0.0s
✔ Volume "doltlab_doltlabdb-dolt-configs" Created 0.0s
✔ Volume "doltlab_doltlabdb-dolt-backups" Created 0.0s
✔ Volume "doltlab_doltlab-user-uploads" Created 0.1s
✔ Volume "doltlab_doltlab-remote-storage" Created 0.1s
✔ Volume "doltlab_doltlabdb-dolt-data" Created 0.0s
✔ Container doltlab-doltlabenvoy-1 Started 14.4s
✔ Container doltlab-doltlabdb-1 Started 14.3s
✔ Container doltlab-doltlabfileserviceapi-1 Started 0.9s
✔ Container doltlab-doltlabremoteapi-1 Started 1.1s
✔ Container doltlab-doltlabapi-1 Started 1.3s
✔ Container doltlab-doltlabgraphql-1 Started 1.5s
✔ Container doltlab-doltlabui-1 Started If I navigate to http://52.40.213.187, I can see my running DoltLab instance, ready to go.
Now let's say I want to connect my DoltLab instance to Gmail's SMTP server so new users can create accounts on my DoltLab instance. If I wanted to do this with previous version of the installer I could only do so using the following command line flags:
./installer \
--host="52.40.213.187" \
--smtp-host=smtp.gmail.com \
--smtp-port=587 \
--smtp-auth-method=plain \
--smtp-username=dustin@dolthub.com \
--smtp-password="xxx xxx xxx xxx"Notice, too, that I'd also would have had to supply the --host flag again to ensure the installer continued to generate assets with the proper host IP address. In these older installer versions, arguments were always additive, meaning I would need to supply the same ones I'd supplied the last time I ran the installer, in order for it to generate the same output. This was good for generating deterministic DoltLab configuration, but the number of arguments passed to the installer grew unwieldy pretty quickly... which is bad.
This is where installer_config.yaml shines. Now, as of DoltLab v2.1.4, to enable connection to Gmail I can simply edit the installer's configuration file like so:
# "SMTP" is used to configure a connection to an existing SMTP server. Optional.
# For more information on connecting your instance to an SMTP server,
# see https://docs.doltlab.com/guides/administrator#connect-smtp-server.
smtp:
auth_method: "plain"
host: "smtp.gmail.com"
port: 587
no_reply_email: "dustin@dolthub.com"
username: "dustin@dolthub.com"
password: "xxx xxx xxx xxx"I save these edits to ./installer_config.yaml and run ./stop.sh to stop my running instance. Then I rerun the installer:
root@ip-10-2-0-65:/home/ubuntu/doltlab# ./installer
2024-05-15T22:50:26.119Z INFO metrics/emitter.go:111 Successfully sent DoltLab usage metrics
2024-05-15T22:50:26.120Z INFO cmd/main.go:491 Successfully configured DoltLab {"version": "v2.1.4"}
2024-05-15T22:50:26.120Z INFO cmd/main.go:497 To start DoltLab, use: {"script": "/home/ubuntu/doltlab/start.sh"}
2024-05-15T22:50:26.120Z INFO cmd/main.go:502 To stop DoltLab, use: {"script": "/home/ubuntu/doltlab/stop.sh"}And I restart my instance with ./start.sh to see my connection to Gmail was successful, and new accounts can be created on my instance!
root@ip-10-2-0-65:/home/ubuntu/doltlab# ./start.sh
[+] Running 7/7
✔ Container doltlab-doltlabenvoy-1 Started 0.6s
✔ Container doltlab-doltlabdb-1 Started 0.5s
✔ Container doltlab-doltlabfileserviceapi-1 Started 0.8s
✔ Container doltlab-doltlabremoteapi-1 Started 1.1s
✔ Container doltlab-doltlabapi-1 Started 1.3s
✔ Container doltlab-doltlabgraphql-1 Started 1.5s
✔ Container doltlab-doltlabui-1 Started 1.7s
Conclusion
If you haven't given DoltLab a try, now is the best time to do so. We're hard at work making the experience of running a DoltLab instance easy and powerful. Swing by our Discord if you'd like to learn more.
Don't forget to check out each of our cool products below:
- Dolt—it's Git for data.
- DoltHub—it's GitHub for data.
- DoltLab—it's GitLab for data.
- Hosted Dolt—it's RDS for Dolt databases.