Integration Testing¶
See also
Manual credential testing โ hands-on walkthrough of the OS-keychain storage mode against a real workstation, for scenarios that are awkward to mock.
GTB includes integration tests that exercise real external services โ GitHub APIs, git operations over the network, and multi-service lifecycle coordination. These tests are excluded from the default test suite and must be explicitly enabled via environment variables.
Quick Start¶
# 1. Copy the example env file
cp .env.example .env
# 2. Fill in your credentials
# At minimum: GITHUB_TOKEN with `repo` scope
# 3. Run all integration tests
just test-integration
# 4. Run only VCS integration tests
INT_TEST_VCS=1 go test ./pkg/vcs/... -v
# 5. Generate coverage including integration tests
just coverage-full
Gating Mechanism¶
Integration tests are gated at runtime using testutil.SkipIfNotIntegration from internal/testutil/. This approach was chosen over //go:build tags for:
- Compile-time safety โ integration tests are always compiled, so breakages are caught by
go buildandgo veteven when not running them. - Discoverability โ tests appear in IDE test explorers and
go test -listoutput. - Granular control โ targeted
INT_TEST_*variables allow running specific test groups without all-or-nothing gating.
Environment Variables¶
| Variable | Effect |
|---|---|
INT_TEST=1 |
Enables all integration tests |
INT_TEST_VCS=1 |
Enables only tests tagged "vcs" |
INT_TEST_CONFIG=1 |
Enables only tests tagged "config" |
INT_TEST_CONTROLS=1 |
Enables only tests tagged "controls" |
INT_TEST_GENERATOR=1 |
Enables only tests tagged "generator" |
INT_TEST_SETUP=1 |
Enables only tests tagged "setup" |
INT_TEST_CMD=1 |
Enables only tests tagged "cmd" |
INT_TEST_ERRORHANDLING=1 |
Enables only tests tagged "errorhandling" |
INT_TEST_CHAT=1 |
Enables only tests tagged "chat" |
INT_TEST_BITBUCKET=1 |
Enables only tests tagged "bitbucket" |
INT_TEST_GITEA=1 |
Enables only tests tagged "gitea" |
INT_TEST_E2E=1 |
Enables all E2E BDD tests (Godog) |
INT_TEST_E2E_SMOKE=1 |
Enables only @smoke-tagged E2E scenarios |
INT_TEST_E2E_CONTROLS=1 |
Enables only @controls-tagged E2E scenarios |
INT_TEST_E2E_CLI=1 |
Enables only @cli-tagged E2E scenarios |
INT_TEST_E2E_CHAT=1 |
Enables only @chat-tagged E2E scenarios |
When neither INT_TEST nor the relevant INT_TEST_<TAG> is set, the test is skipped with a message explaining how to enable it:
Usage in Test Files¶
package mypackage_test
import (
"testing"
"gitlab.com/phpboyscout/go-tool-base/internal/testutil"
)
func TestSomethingIntegration(t *testing.T) {
testutil.SkipIfNotIntegration(t, "mytag")
// ... test code that talks to external services
}
Integration tests must live in dedicated *_integration_test.go files to keep them clearly separated from unit tests.
Credential Variables¶
| Variable | Required | Description |
|---|---|---|
GITHUB_TOKEN |
Yes (VCS tests) | GitHub personal access token with repo scope. Used by VCS tests to interact with the GitHub API (PR management, label operations). |
GITHUB_KEY |
No | Path to an SSH private key for git-over-SSH tests (clone, push). If unset, SSH-based tests are skipped. |
BITBUCKET_USERNAME |
Yes (Bitbucket tests) | Bitbucket username for Downloads API integration tests. |
BITBUCKET_APP_PASSWORD |
Yes (Bitbucket tests) | Bitbucket app password with read access to the test repository's Downloads. |
BITBUCKET_TEST_WORKSPACE |
Yes (Bitbucket tests) | Workspace slug for the test repository. |
BITBUCKET_TEST_REPO |
Yes (Bitbucket tests) | Repository slug for the test repository. |
GITEA_TOKEN |
Yes (Gitea tests) | Personal access token for the Gitea/Forgejo instance under test. |
GITEA_HOST |
Yes (Gitea tests) | Base URL of the Gitea/Forgejo instance (e.g. https://git.example.com). |
GITEA_TEST_OWNER |
Yes (Gitea tests) | Org or username that owns the test repository. |
GITEA_TEST_REPO |
Yes (Gitea tests) | Repository slug for the test repository. |
The .env file is loaded automatically by just via dotenv-load. You can also export these variables directly in your shell.
Never commit .env
The .env file is git-ignored. Use .env.example as the template โ it contains no secrets.
Test Inventory¶
pkg/controls/ โ Service Lifecycle¶
| File | Tests | Dependencies |
|---|---|---|
integration_test.go |
HTTP and gRPC servers on separate ports | Local network (localhost) |
shutdown_test.go |
Graceful shutdown via signals, context cancellation, and timeout | Local network, OS signals |
server_integration_test.go |
Health endpoints, middleware bypass, custom health checks, gRPC probes, interceptors, graceful shutdown, app handlers | Local network |
These tests require no external credentials โ only local network access.
pkg/config/ โ Configuration Merging¶
| File | Tests | Dependencies |
|---|---|---|
integration_test.go |
Multi-source merge, env var overrides, deep nesting, embedded config, dotenv loading, schema validation | Filesystem |
pkg/errorhandling/ โ Error Propagation¶
| File | Tests | Dependencies |
|---|---|---|
propagation_integration_test.go |
Cross-package error chains, hints, stacktraces, special errors, help config | None |
pkg/cmd/root/ โ Feature Flags¶
| File | Tests | Dependencies |
|---|---|---|
integration_test.go |
Command registration based on feature flags, tool metadata propagation | None |
pkg/setup/ โ Init Flow¶
| File | Tests | Dependencies |
|---|---|---|
init_integration_test.go |
Directory creation, config merge/clean, gitignore, initialisers, API key warnings | Filesystem (in-memory) |
pkg/vcs/repo/ โ Git Operations¶
| File | Tests | Dependencies |
|---|---|---|
repo_integration_test.go |
Branch creation, push, clone, checkout, file operations, in-memory repo | Network access to GitHub, GITHUB_TOKEN, optionally GITHUB_KEY |
pkg/vcs/github/ โ GitHub API¶
| File | Tests | Dependencies |
|---|---|---|
client_integration_test.go |
PR lookup by branch, label management | Network access to GitHub API, GITHUB_TOKEN |
pkg/vcs/bitbucket/ โ Bitbucket Downloads¶
Not yet implemented
Integration tests for the Bitbucket release provider are planned but not yet implemented. The env vars and gating are documented below for when they are added.
| File | Tests | Dependencies |
|---|---|---|
release_integration_test.go |
Latest release detection, asset listing via real Downloads API | Network access to Bitbucket, BITBUCKET_USERNAME, BITBUCKET_APP_PASSWORD, BITBUCKET_TEST_WORKSPACE, BITBUCKET_TEST_REPO |
Gate with testutil.SkipIfNotIntegration(t, "bitbucket") and enable with INT_TEST_BITBUCKET=1.
pkg/vcs/gitea/ โ Gitea / Forgejo¶
Not yet implemented
Integration tests for the Gitea release provider are planned but not yet implemented. The env vars and gating are documented below for when they are added.
| File | Tests | Dependencies |
|---|---|---|
release_integration_test.go |
Latest/tagged/listed releases, asset download via real Gitea REST API | Network access to a Gitea/Forgejo instance, GITEA_TOKEN, GITEA_HOST, GITEA_TEST_OWNER, GITEA_TEST_REPO |
Gate with testutil.SkipIfNotIntegration(t, "gitea") and enable with INT_TEST_GITEA=1.
pkg/chat/ โ AI Chat Providers¶
| File | Tests | Dependencies |
|---|---|---|
parallel_integration_test.go |
Parallel tool execution across providers, concurrency limits | AI provider API keys (ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY) |
streaming_integration_test.go |
Streaming chat responses, event callback sequencing | AI provider API keys |
Gate with testutil.SkipIfNotIntegration(t, "chat") and enable with INT_TEST_CHAT=1.
internal/generator/ โ Code Generation Pipeline¶
| File | Tests | Dependencies |
|---|---|---|
pipeline_integration_test.go |
Full lifecycle, deep hierarchy, manifest consistency, protection, command options, dry-run, manifest recovery, feature flags | Filesystem (in-memory) |
test/e2e/ โ E2E BDD Tests (Godog)¶
E2E tests use Godog (Cucumber for Go) to express multi-step behavioural scenarios in Gherkin feature files. Feature files live in features/, step definitions in test/e2e/steps/.
| Feature File | Scenarios | Dependencies |
|---|---|---|
features/controls/lifecycle.feature |
State machine transitions, status messages, context cancellation, concurrent stop idempotency, start errors | None (in-process) |
features/controls/graceful_shutdown.feature |
SIGINT with HTTP+gRPC, in-flight request draining, early signal during startup | Local network (localhost) |
features/controls/health_monitoring.feature |
Health check types (readiness/liveness/both), status mapping, registration rules, async caching, health-triggered restarts | None (in-process) |
features/cli/help.feature |
Root help lists commands, unknown command error | Binary compilation |
features/cli/version.feature |
Text output, JSON output, help flag | Binary compilation |
features/cli/doctor.feature |
Text diagnostic output, JSON structured report | Binary compilation |
features/cli/update.feature |
Help/usage, semver validation, error paths | Binary compilation |
features/cli/init.feature |
Non-interactive init, config merge, clean reset, JSON output | Binary compilation, filesystem |
features/cli/config.feature |
Get/set/list/validate, sensitive masking, JSON output | Binary compilation, filesystem |
features/cli/telemetry.feature |
Enable/disable/status/reset, consent withdrawal, machine ID | Binary compilation, filesystem |
features/chat/persistence.feature |
Save/load/list/delete snapshots, encryption, provider mismatch, tool exclusion | None (in-process) |
These tests require no external credentials. Run via just test-e2e or filter with INT_TEST_E2E_CONTROLS=1 or INT_TEST_E2E_CLI=1.
See docs/development/specs/2026-03-28-godog-bdd-strategy.md for the full BDD strategy and phased rollout plan.
Just Recipes¶
| Recipe | Command | Description |
|---|---|---|
just test-integration |
INT_TEST=1 go test ./... -v |
Run all integration tests |
just coverage-full |
INT_TEST=1 go test ./... -coverprofile=... |
Generate HTML coverage report including integration tests |
just test |
go test ./... -v -cover |
Unit tests only (default) |
just test-e2e |
INT_TEST_E2E=1 go test ./test/e2e/... -v -timeout 5m |
E2E BDD tests via Godog |
just test-e2e-smoke |
INT_TEST_E2E=1 INT_TEST_E2E_SMOKE=1 go test ./test/e2e/... -v -timeout 2m |
E2E smoke tests only (fast) |
just ci |
tidy, generate, test, test-race, lint |
CI suite โ unit tests only |
CI Configuration¶
In GitHub Actions, integration tests run as a separate job with secrets injected:
- name: Integration Tests
env:
INT_TEST: "1"
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: go test ./... -v
The GITHUB_TOKEN provided by GitHub Actions has repo scope by default for the current repository.
Writing New Integration Tests¶
When adding integration tests:
- Use the shared helper โ call
testutil.SkipIfNotIntegration(t, "tag")at the top of every integration test function, choosing an appropriate tag for the test group. - Place in dedicated files โ integration tests must live in
*_integration_test.gofiles, separate from unit tests. - Document dependencies in this guide's test inventory.
- Use
t.Cleanupfor teardown (remove branches, labels, temp files). - Don't hardcode credentials โ always read from environment variables.
- Keep tests idempotent โ they should be safe to re-run without manual cleanup.