Skip to content

Development

Jump to bottom
Garth Goodson edited this page Dec 17, 2025 · 3 revisions

Code Structure

Root directories

Directory Description
src/ Source tree by module
include/ Header tree by module
external/ Where vcpkg installs its packages
scripts/ Helper scripts
python/ Python libraries for testing; coordinator for deployment; localcluster controller and utilities
docker/ Docker files, scripts
cmake/ CMake helper functions
local_cluster/ Local cluster setup scripts for running cluster using containers on local machine
debug/ * Conditionally created by CMake for debug builds
release/ * Conditionally created by CMake for production builds

Important files

Filename Description
CMakeLists.txt Top level CMake file for building code. Add new modules/sub-dirs here. Other configuration should be local to each module
vcpkg.json List of C++ vcpkg dependencies; See list of packages
clean.sh / debug.sh / release.sh Helper scripts, wrapper around CMake to clean, build debug or build release
system.json Properties JSON file for specifying hostnames, and other configuration info for modules
python/testing/springtail.py Initialize redis with properties specified in system.json.* file; bring up system, bring down system, run checks on system
python/testing/test_runner.py Run full system tests, uses springtail.py to bring up and down system, runs tests located in test_cases

Modules

Name Description
common Common utilities, springtail_init() should be called in every main; logging, exceptions, concurrent queues, thread pool, etc.
storage Threaded storage layer; B-tree, extent and schema handling and definition
pg_repl Low-level Postgres replication; connecting to primary and decoding replication messages
pg_log_mgr Consumes Postgres replication stream to log, process log, extracts transaction start/commit, inserts commit records into a queue to be consumed by the committer. The committer is responsible for applying log changes to the tables and committing them.
write_cache Cache for holding dirty extents by XID; indexed by Table ID, Extent ID, XID and primary key value
xid_mgr Persistently stores last committed XID
benchmarks Code for benchmarking file system performance (for distributed file systems)
proto gRPC protocol definitions
sys_tbl_mgr System table service & cache; accessed via Thrift; centralizes management of system table metadata
proxy Postgres Proxy that handles read/write splitting sending writes to primary, and reads to FDWs
pg_fdw Postgres frontend query nodes implemented using Postgres Foreign Data Wrapper (FDW) extension; also includes the ddl_daemon for managing DDL changes for the FDW (to create/alter foreign tables)
redis Redis helper classes
admin_http Embedded internal http server
pg_ext Postgres extension related

Compiling the code

Compiling the C++ code is fairly straightforward. All dependencies will be installed via vcpkg; note the first build may take a long time due to downloading and compiling all dependencies.

Make sure all dependencies from Laptop Setup have been installed; otherwise vcpkg deps will not compile

To do a debug build run:

springtail % ./debug.sh  # first time build, setups up springtail/debug
springtail % cd debug; make # compile source; or switch to appropriate module
springtail % ./clean.sh. # does a make clean and removes the springtail/debug dir

Adding a C++ module

  • Create new subdirs under src/ and include/

  • Add module to top level CMakeLists.txt: add_subdirectory(src/module)

  • Within module (/src/module/) create a test subdir and a CMakeLists.txt

  • Add source code and library dependencies to CMakeLists.txt

  • Add test source code (or helper utils) under test/ subdir

    • Add to function springtail_unit_test() to CMakeLists.txt
    • E.g.
    springtail_unit_test(test_replica_shutdown
  SOURCES test/test_replica_shutdown.cc
  INCLUDES ${CMAKE_SOURCE_DIR}/include
  LIBS springtail_proxy GTest::gtest_main
)

Accessing postgres database on you machine

Run the following command: psql "dbname=springtail user=springtail host=/var/run/postgresql password=springtail”

Dev Process

  • Issue Creation:

    • Create a new issue in Linear.
    • Clearly describe the feature's functionality and purpose.
    • Include relevant details, user stories, or acceptance criteria.
    • Assign the issue to yourself if you will be working on it.

  • Branching:

    • Once the issue is created, create a new development branch based on the main branch (git checkout -b branch_name).
      • Use a descriptive branch name that references the issue number (e.g., SPR-123-feature or SPR-123-issue).
    • Push the newly created branch to your remote repository on GitHub (git push -u).
    • If the above push command does not work, do git push --set-upstream origin <branch name>
    • If something is wrong and you need to delete remote branch, do git push -d origin <branch name>
    • To delete local branch, do git branch -d <branch name>
    • To see the list of remote branches and to verify that you have created a remote branch, do git branch -r

  • Development:

    • Implement the feature code within your development branch.
    • Make frequent commits with clear and concise commit messages that describe the specific changes made.
    • The commit message should reference the github issue, e.g.,
      • git commit -m "Fix bug reported ref SPR-123"
    • Use unit tests to ensure the functionality of your code.
    • Run all tests locally prior to submitting a Pull Request.
      • From root of the build tree (e.g., in springtail/debug/) run make test

  • Pull Request Creation:

    • Once development is complete, create a Pull Request (PR) from your development branch to the main branch.
    • Reference the issue number in the PR title or description to link the feature to the initial request.
    • Provide a clear summary of the changes made and the purpose of the feature.
    • Request code reviews from other developers.

  • Review and Merge:

    • Address any feedback or suggestions received during the code review process.
    • Make necessary code changes and push them to your development branch.
    • Rerun all unit and integration tests.
    • Once the code is approved for merging, contact @Garth or @Craig to merge into main.
      • A squash merge option should be used to merge your changes into the main branch. This creates a single commit representing the entire feature development.

  • Bring your branch up to date with main:

    • Perform the following commands (best to have done a git commit locally before merging):

      git pull
git merge origin/main
git push

  • Pull in updates from a branch other than main

    • Perform the following commands:

      git stash # optional - saves uncommited changes
git pull
git merge origin/<branch name>    # will apply the changes from the branch
git push # optional - only if you want to apply the same change to the remote branch
git stash pop # optional - get the uncommited changes back and merge if needed

    • If after merge command you want to revert to the pre-merge state, run git reset --hard ORIG_HEAD

    • To see the diff of the last commit, do git diff HEAD^

    • If you want a remote branch with the name different than your local branch:

      • Create branch: git push --set-upstream origin <local branch>:<remote branch>
      • Push to remote branch: git push origin HEAD:<remote branch>

  • Cleanup remote branch view from git: git remote prune origin

  • Delete local branch: git branch -D <branch name>

  • Restore uncommitted, but not staged file: git restore <filename>

  • Restore uncommitted, but staged file: git restore --staged <filename>

Container Development

This section outlines how to use a Linux container for development and testing. The Linux container contains a full development environment including compiler, linker, as well as PostgreSQL and Redis servers.

Prerequisites

Install Hombrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# install if not already installed
brew install colima docker docker-compose docker-Buildx

Colima (Preferred)

Install colima and docker (this is preferred); make sure you are using macOS Virtualization.Framework and virtiofs with colima.

If not using Colima, if using Docker Desktop see below.

NOTE: Rosetta is a x86 translation layer, if not using Apple Silicon Mx then it may not be necessary.

# NOTE: 12 = GB memory, use less if needed 
#       Also if Apple M4, pass --nested-virtualization to make gdb work
colima start --memory 12 --cpu 5 --vm-type=vz --vz-rosetta

# run colima status and check the output
springtail % colima status
INFO[0000] colima is running using macOS Virtualization.Framework 
INFO[0000] arch: aarch64                                
INFO[0000] runtime: docker                              
INFO[0000] mountType: virtiofs

Build the code

git clone [git@github.com](mailto:git@github.com):Springtail-inc/springtail.git
cd springtail

Docker Desktop

If using Docker desktop you may need to set some options to enable Rosetta and VirtioFS. See: https://www.docker.com/blog/docker-desktop-4-25/ Go to Settings→General and select VirtioFS and Rosetta.

Building the docker image

From within springtail dir, create the docker image and run it.

  • The -p options will map the ports from container to host. Use Postgres (5432), Redis (6379) and SSH (22) (only SSH is required).
  • The -v option will map your local springtail github source into the container (source:target). The source should be your local springtail dir, the target must be /home/dev/springtail. The source is the location at which you checked out and cloned the github source code; replace <springtail_dir> .
  • The --privileged and --cap-add=SYS_PTRACE options may be useful for debugging to attach to a running process
# from springtail root
cd docker
docker-buildx build --progress plain -t springtail:dev -f ./Dockerfile.base .

# note: replace <springtail_dir> with the location of the cloned springtail dir
# ~ is short for /Users/<username>, so use one or the other
docker run -p 2222:22 -p 6666:6379 -v ~/<springtail_dir>:/home/dev/springtail -d --name dev --shm-size=1g -it springtail:dev --privileged --cap-add=SYS_PTRACE

The Dockerfile.base edits the Postgres configuration and creates the script entrypoint.sh as the container entry point. This script ensures the Postgres and Redis services are running, and it creates the Postgres springtail user. The Dockerfile also edits the Postgres configuration as necessary.

Logging in

You should see 3 directories after logging in:

ssh -p 2222 dev@localhost # use password 'dev'
dev@941221cd102e:~$ ls
debug  external  springtail

The springtail directory should be the mounted directory from your local machine containing the code. The debug directory is where the code will be built within the container, and the external directory will host the vcpkg packages and source.

After Mac Reboot

After reboot if your container is no longer accessible, check if colima is running and if not start it:

> colima start --memory 16 --cpu 5 --vm-type=vz --vz-rosetta
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
WARN[0000] Unable to enable Rosetta: Rosetta2 is not installed 
WARN[0000] Run 'softwareupdate --install-rosetta' to install Rosetta2 
INFO[0000] starting ...                                  context=vm
INFO[0011] provisioning ...                              context=docker
INFO[0012] starting ...                                  context=docker
INFO[0012] done

Now verify if the docker container is running:

docker container ls

If it did not find anything, the docker container needs to be restarted. Run the following command:

> docker container ls --all
CONTAINER ID   IMAGE            COMMAND                  CREATED       STATUS                        PORTS                                                                                            NAMES
6bc0f4fe0fb3   springtail:dev   "/usr/local/bin/entr…"   2 weeks ago   Exited (255) 24 minutes ago   5432/tcp, 0.0.0.0:2222->22/tcp, [::]:2222->22/tcp, 0.0.0.0:6666->6379/tcp, [::]:6666->6379/tcp   dev

The first number is the container id and this is what is needed to get container restarted.

docker container start <container id>

Now you can login into your container. If you try to do docker run command instead of docker container start, it will fail as follows:

> docker run -p 2222:22 -p 6666:6379 -v ~/springtail:/home/dev/springtail -d --name dev -it springtail:dev --priviliged --cap-add=SYS_PTRACE
docker: Error response from daemon: Conflict. The container name "/dev" is already in use by container "6bc0f4fe0fb32c62945a411c67b4ddbaf522c6f4fcedbdc4a232d8ac0c9cee63". You have to remove (or rename) that container to be able to reuse that name.

Running out of space

If you have trouble with disk space or RAM or CPU, you can restart colima:

# e.g. 300G disk, 20G RAM, and 6 CPUs
colima start --disk 300 --memory 20 --vz-rosetta --cpu 6 --vm-type=vz

You would need to restart the docker container as specified above.

Building

From within the dev container; this will create symlinks to ../debug and ../external and will start the build. First it will download vcpkg and build it, then it will download the dependency packages and build them, lastly it will build the Springtail code.

# first time; go grab a coffee it may be a while...
cd springtail
./debug.sh

# subsequent times
cd debug
make

Troubleshooting the build

  • Check for the existence of the following two files, and make sure they are symlinks:
  • debug -> /home/dev/debug/
  • external -> /home/dev/external/
# make sure you are in the springtail dir
cd /home/dev/springtail

# look for debug and external
ls -l
...
lrwxr-xr-x  1 dev dev     16 Oct  4 23:29 debug -> /home/dev/debug/
...
lrwxr-xr-x  1 dev dev     19 Oct  4 23:29 external -> /home/dev/external/

# if they don't exist:
ln -s /home/dev/debug
ln -s /home/dev/external
  • Sometimes running ./debug.sh results in errors while building the external dependencies in vcpkg. If this happens, just try rerunning ./debug.sh and see if you can make progress (i.e., check if it is failing on a different dependency, if so keep going).

VS-Code Extensions

Since you have mounted the springtail source dir using the -v option when running the docker container, the code it is building is the same code on your local machine. Any edits to the code on your local machine will be reflected in the container. So after you make a code change, just rebuild from the container ( cd debug; make )

Dev containers

If you are using VSCode or Cursor (based on VSCode), and want more functionality to just work, it is best to have VSCode connect to the docker image and edit there. You can do this by installing the “Dev Containers” extension. And then access the Command Palette (cmd+shift+P) and run “Dev Containers: Attach to Running Container…”. You can then select your container. It will first connect as “root”, which is not what you want, so then run “Dev Containers: Open Container Configuration File” and set the contents to the following:

{
	"remoteUser": "dev",
	"workspaceFolder": "/home/dev/springtail",
}

You can then reconnect with “Dev Containers: Attach to Running Container…”. Note that in this environment, you can also install the “CMake” extension and run things like “CMake: Build” and get good language server integration.

Clangd

Additionally, in VSCode, I’d recommend installing the clangd extension, and also run sudo apt install clangd inside of your container. After installing the VSCode extension, you should get an option to disable “Microsoft Intellisense” (as both extensions provide the same functionality). The clangd extension should give much more accurate/faster support for “Find all references”.

Clone this wiki locally