Buildkite Test Agent Setup¶
We are using Buildkite for Windows and macOS testing. The build machines and buildkite-agent
must be set up before use.
Windows Test Agent Setup¶
- Create the user “testbot” on the machine. Use the password for
ddevtestbot@gmail.com
, available in LastPass. - In admin PowerShell,
wsl --install
. - In admin PowerShell,
Set-ExecutionPolicy -Scope "CurrentUser" -ExecutionPolicy "RemoteSigned"
. - In admin PowerShell, download and run windows_buildkite_start.ps1 with
curl <url> -O windows_buildkite_start.ps1
. - After restart, in administrative Git Bash window,
Rename-Computer <testbot-win10(home|pro)-<description>-1
and thenexport BUILDKITE_AGENT_TOKEN=<token>
. - Now download and run
windows_buildkite-testmachine_setup.sh
. - Download and run windows_postinstall.sh.
- Launch Docker. It may require you to take further actions.
- Log into Chrome with the user
ddevtestbot@gmail.com
and enable Chrome Remote Desktop. - Enable gd, fileinfo, and curl extensions in
/c/tools/php*/php.ini
. - If a laptop, set the “lid closing” setting to do nothing.
- Set the “Sleep after time” setting in settings to never.
- Install winaero tweaker and “Enable user autologin checkbox”. Set up the machine to automatically log in on boot. Then run netplwiz, provide the password for the main user, uncheck “require a password to log in”.
-
The
buildkite/hooks/environment.bat
file must be updated to contain the Docker pull credentials: -
Set the
buildkite-agent
service to run as the testbot user and use delayed start: Choose “Automatic, delayed start” and on the “Log On” tab in the services widget it must be set up to log in as the testbot user, so it inherits environment variables and home directory (and can access NFS, has testbot Git config, etc). git config --global --add safe.directory '*'
.- Manually run
testbot_maintenance.sh
,curl -sL -O https://raw.githubusercontent.com/ddev/ddev/master/.buildkite/testbot_maintenance.sh && bash testbot_maintenance.sh
. - Run
.buildkite/sanetestbot.sh
to check your work. - Reboot the machine and do a test run. (On Windows, the machine name only takes effect on reboot.)
- Verify that
go
,ddev
,git-bash
are in the path. - In “Advanced Windows Update Settings” enable “Receive updates for other Microsoft products” to make sure you get WSL2 kernel upgrades. Make sure to run Windows Update to get the latest kernel.
Additional Windows Setup for WSL2+Docker Desktop Testing¶
- Do not set up
buildkite-agent
on the Windows side, or disable it. -
Edit Ubuntu’s
/etc/wsl.conf
to contain: -
Update WSL2 to WSL2 Preview from Microsoft Store and
wsl --shutdown
and then restart. wsl --update
- Open WSL2 and check out ddev/ddev.
- As normal user, run
.github/workflows/linux-setup.sh
. -
export PATH=/home/linuxbrew/.linuxbrew/bin:$PATH echo "export PATH=/home/linuxbrew/.linuxbrew/bin:$PATH" >>~/.bashrc
-
As root user, add sudo capability with
echo "ALL ALL=NOPASSWD: ALL" >/etc/sudoers.d/all && chmod 440 /etc/sudoers.d/all
. - Manually run
testbot_maintenance.sh
,curl -sL -O https://raw.githubusercontent.com/ddev/ddev/master/.buildkite/testbot_maintenance.sh && bash testbot_maintenance.sh
. git config --global --add safe.directory '*'
-
Install basics in WSL2:
curl -fsSL https://pkg.ddev.com/apt/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/ddev.gpg > /dev/null echo "deb [signed-by=/etc/apt/keyrings/ddev.gpg] https://pkg.ddev.com/apt/ * *" | sudo tee /etc/apt/sources.list.d/ddev.list >/dev/null # Update package information and install DDEV sudo apt update && sudo apt install -y ddev sudo mkdir -p /usr/sharekeyrings && curl -fsSL https://keys.openpgp.org/vks/v1/by-fingerprint/32A37959C2FA5C3C99EFBC32A79206696452D198 | sudo gpg --dearmor -o /usr/share/keyrings/buildkite-agent-archive-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/buildkite-agent-archive-keyring.gpg] https://apt.buildkite.com/buildkite-agent stable main" | sudo tee /etc/apt/sources.list.d/buildkite-agent.list sudo apt update && sudo apt install -y build-essential buildkite-agent ca-certificates curl ddev gnupg lsb-release make mariadb-client sudo snap install ngrok
-
Configure
buildkite-agent
in WSL2. It needs the same changes as macOS, but tagstags="os=wsl2,architecture=amd64,dockertype=dockerforwindows"
and build-path should be in~/tmp/buildkite-agent
. -
The buildkite/hooks/environment file must be updated to contain the Docker pull credentials:
-
Verify that
buildkite-agent
is running. - In Task Scheduler, create a task that runs on User Logon and runs
C:\Windows\System32\wsl.exe
with arguments-d Ubuntu
. - Add
buildkite-agent
to thedocker
andtestbot
groups in/etc/group
echo "capath=/etc/ssl/certs/" >>~/.curlrc
And then do the same asbuildkite-agent
usersudo chmod -R ug+w /home/linuxbrew
nc.exe -l -p 9003
on Windows to trigger and allow Windows Defender.- Run
ngrok config add-authtoken <token>
with token for free account. - Copy ngrok config into
buildkite-agent
account,sudo cp -r ~/.ngrok2 ~buildkite-agent/ && sudo chown -R buildkite-agent:buildkite--agent ~buildkite-agent/ngrok2
- Add
/home/linuxbrew/.linuxbrew/bin
toPATH
in/etc/environment
. - Copy ngrok config into
buildkite-agent
account,sudo cp -r ~/.ngrok2 ~buildkite-agent/ && sudo chown -R buildkite-agent:buildkite--agent ~buildkite-agent/ngrok2
- Add
buildkite-agent
tosudo
group in/etc/groups
- Give
buildkite-agent
a password withsudo passwd buildkite-agent
- As
buildkite-agent
usermkcert -install
Additional Windows Setup for WSL2+Docker-Inside Testing¶
- Uninstall Docker Desktop.
- Remove all of the entries (especially
host.docker.internal
) that Docker Desktop has added inC:\Windows\system32\drivers\etc\hosts
. -
Install Docker and basics in WSL2:
sudo mkdir -p /etc/apt/keyrings sudo mkdir -p /etc/apt/keyrings && sudo rm -f /etc/apt/keyrings/docker.gpg && curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt update && sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin sudo usermod -aG docker $USER
-
Configure buildkite agent in /etc/buildkite-agent:
- tags=”os=wsl2,architecture=amd64,dockertype=wsl2”
- token=”xxx”
- Create
/etc/buildkite-agent/hooks/environment
and set to executable with contents:
-
Run
.buildkite/sanetestbot.sh
macOS Test Agent Setup (Intel and Apple Silicon)¶
- Create the user “testbot” on the machine. Use the password for
ddevtestbot@gmail.com
, available in LastPass. - Change the name of the machine to something in keeping with current style. Maybe
testbot-macstadium-macos-3
. - Install Homebrew
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
- Install everything you’ll need with
brew install buildkite/buildkite/buildkite-agent bats-core colima composer ddev/ddev/ddev git golang jq mariadb mkcert netcat p7zip && brew install --cask docker iterm2 google-chrome nosleep ngrok
. - Run
ngrok config add-authtoken <token>
with token for free account. - Run
mkcert -install
. - Run Docker manually and go through its configuration routine.
- Run iTerm. On Mojave and higher you may need to allow full disk access permissions.
- Set up
nfsd
by runningmacos_ddev_nfs_setup.sh
. git config --global --add safe.directory '*'
- Edit
/usr/local/etc/buildkite-agent/buildkite-agent.cfg
or/opt/homebrew/etc/buildkite-agent/buildkite-agent.cfg
to add- the agent token
- tags, like
"os=macos,architecture=arm64,osvariant=monterrey,dockertype=dockerformac"
build-path="~/tmp/buildkite-agent/builds"
-
The buildkite/hooks/environment file must be updated to contain the Docker pull credentials:
-
Run
brew services start buildkite-agent
. - Manually run
testbot_maintenance.sh
,curl -sL -O https://raw.githubusercontent.com/ddev/ddev/master/.buildkite/testbot_maintenance.sh && bash testbot_maintenance.sh
. - Enable nosleep using its shortcut in the Mac status bar.
- In nosleep Preferences, enable “Never sleep on AC Adapter”, “Never sleep on Battery”, and “Start nosleep utility on system startup”.
sudo chown testbot /usr/local/bin
- Set up Mac to automatically log in on boot.
- Try checking out ddev/ddev and running
.buildkite/sanetestbot.sh
to check your work. - Log into Chrome with the user
ddevtestbot@gmail.com
and enable Chrome Remote Desktop. - Set the timezone (US MT).
- Start the agent with
brew services start buildkite-agent
.