Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .gitbook/assets/image (2) (2).png
Binary file not shown.
Binary file removed .gitbook/assets/image (2).png
Binary file not shown.
Binary file removed .gitbook/assets/image (3).png
Binary file not shown.
56 changes: 56 additions & 0 deletions .github/workflows/pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Simple workflow for deploying static content to GitHub Pages
name: Deploy static content to Pages

on:
# Runs on pushes targeting selected branches
push:
branches: ["master"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true

jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout master
uses: actions/checkout@v3
with:
ref: master
path: master
- name: Checkout
uses: actions/checkout@v3
with:
ref: v0.5.3
path: v0.5.3
- run: |
touch .nojekyll
echo -e '<html><head><meta http-equiv="refresh" content="0; url=./master" /></head></html>' > index.html
echo -e "* master\n * <a href=\"../master\">**master**</a>\n * <a href=\"../v0.5.3\">v0.5.3</a>\n$(cat master/NAV.md)" > master/NAV.md
echo -e "* v0.5.3\n * <a href=\"../master\">master</a>\n * <a href=\"../v0.5.3\">**v0.5.3**</a>\n$(cat v0.5.3/NAV.md)" > v0.5.3/NAV.md
cp master/assets/favicon.ico .
- name: Setup Pages
uses: actions/configure-pages@v2
- name: Upload artifact
uses: actions/upload-pages-artifact@v1
with:
# Upload entire repository
path: '.'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1
Empty file added .nojekyll
Empty file.
5 changes: 5 additions & 0 deletions NAV.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
* SDKs
* [Go](https://github.com/testground/sdk-go)
* [Rust](https://github.com/testground/sdk-rust)
* [JS](https://github.com/testground/sdk-js)
* [Nim](https://github.com/status-im/testground-nim-sdk)
33 changes: 14 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,53 @@
# What is Testground?

![](.gitbook/assets/image%20%283%29%20%281%29.png)
![](assets/hero.png)

{% hint style="info" %}
You are reading the Testground documentation for the [master](https://github.com/testground/testground) branch.

The Testground team maintains documentation for the `master` branch and for the latest stable release.
{% endhint %}
> You are reading the Testground documentation for the [master](https://github.com/testground/testground) branch.
>
> The Testground team maintains documentation for the `master` branch and for the latest stable release.

## Overview

Testground is a platform for testing, benchmarking, and simulating distributed and peer-to-peer systems at scale. It's designed to be multi-lingual and runtime-agnostic, scaling gracefully from 2 to 10k instances, only when needed.

The Testground project was started at Protocol Labs because we couldn't find a platform that would allow us to reliably and reproducibly test and measure how changes to the IPFS and libp2p codebases would impact the performance and health of large networks \(as well as individual nodes\), so we decided to invent it.

![](.gitbook/assets/testground-demo.gif)
![](assets/demo.gif)

## How does it work?

### 1. **You develop distributed test plans as if you were writing unit tests against local APIs.**
### 1. You develop distributed test plans as if you were writing unit tests against local APIs

* No puppeteering necessary.
* No need to package and ship the system as a separate daemon with an external API in order to puppeteer it.
* No need to expose every internal setting over an external API, just for the sake of testing.

### **2. Your test plan calls out to the coordination API to...**
### 2. Your test plan calls out to the coordination API to

* Communicate out-of-band information \(such as endpoint addresses, peer ids, etc.\).
* Leverage synchronization and ordering primitives such as signals and barriers to model a distributed state machine.
* Programmatically apply network traffic shaping policies, which you can alter during the execution of a test to simulate various network conditions.

### **3. There is no special "conductor" node telling instances what to do when...**
### 3. There is no special "conductor" node telling instances what to do when

* The choreography and sequencing emerges from within the test plan itself.

### **4. You decide what versions of the upstream software you want to exercise your test against.**
### 4. You decide what versions of the upstream software you want to exercise your test against

* Benchmark, simulate, experiment, run attacks, etc. against versions v1.1 and v1.2 of the components under test in order to compare results, or test compatibility.
* Assemble hybrid test runs mixing various versions of the dependency graph.

### **5. Inside your test plan...**
### 5. Inside your test plan

* You record observations, metrics, success/failure statuses.
* You emit structured or unstructured assets you want collected, such as event logs, dumps, snapshots, binary files, etc.

### **6. Via a TOML-based** _**composition**_ **file, you instruct Testground to...**
### 6. Via a TOML-based _composition_ file, you instruct Testground to

* Assemble a test run comprising groups of 2, 200, or 10000 test instances, each with different test parameters, or built against different dependency sets.
* Schedule them for run locally \(executable or Docker\), or in a cluster \(Kubernetes\).

### **7. You collect the outputs of the test plan for analysis...**
### 7. You collect the outputs of the test plan for analysis

* With a single command...
* Using data processing scripts and platforms \(such as the upcoming Jupyter notebooks integration\) to draw conclusions.
Expand All @@ -66,7 +64,7 @@ The Testground project was started at Protocol Labs because we couldn't find a p
* Comparative testing 🌖
* Backwards/forward-compatibility testing 🌖
* Interoperability testing 🌑
* Continuous integration 🌑
* Continuous integration 🌑
* Stakeholder/acceptance testing 🌑

### 📄 Simple, normalized, formal runtime environment for tests
Expand Down Expand Up @@ -121,7 +119,4 @@ Emit and collect/export/download test outputs \(logs, assets, event trails, run

## Documentation issues

{% hint style="warning" %}
**This docs site is work-in-progress!** You're bound to find dragons 🐉 in some sections, so please bear with us! If something looks wrong, please [open a docs issue](https://github.com/testground/testground/issues/new?assignees=&labels=docs&template=DOCS.md&title=docs%20site:%20%3Cdescribe%20the%20problem%3E) on our main repo.
{% endhint %}

!> **This docs site is work-in-progress!** You're bound to find dragons 🐉 in some sections, so please bear with us! If something looks wrong, please [open a docs issue](https://github.com/testground/testground/issues/new?assignees=&labels=docs&template=DOCS.md&title=docs%20site:%20%3Cdescribe%20the%20problem%3E) on our main repo.
94 changes: 44 additions & 50 deletions SUMMARY.md
Original file line number Diff line number Diff line change
@@ -1,50 +1,44 @@
# Table of contents

* [What is Testground?](README.md)
* [Concepts and architecture](concepts-and-architecture/README.md)
* [Test plans and test cases](concepts-and-architecture/test-structure.md)
* [Daemon and client](concepts-and-architecture/daemon-and-client.md)
* [Synchronization service](concepts-and-architecture/sync-service.md)
* [Networking](concepts-and-architecture/networking.md)
* [Sidecar](concepts-and-architecture/sidecar.md)
* [Builders](concepts-and-architecture/builders-1.md)
* [Runners](concepts-and-architecture/runners.md)
* [Runtime environment \(runenv\)](concepts-and-architecture/runtime.md)
* [Client-Server communication](concepts-and-architecture/client-server-communication.md)
* [Getting started](getting-started.md)
* [Writing test plans](writing-test-plans/README.md)
* [Quick start](writing-test-plans/quickstart.md)
* [Understanding the test plan manifest](writing-test-plans/test-plan-manifest.md)
* [Parameters and test cases](writing-test-plans/paramaters-and-testcases.md)
* [Keeping instances in sync](writing-test-plans/synchronization.md)
* [Communication between instances](writing-test-plans/communication-between-instances.md)
* [Observability, assets and metrics](writing-test-plans/observability-assets-and-metrics.md)
* [Managing test plans](managing-test-plans.md)
* [Running test plans](running-test-plans.md)
* [Traffic shaping](traffic-shaping.md)
* [Analyzing test run results](analyzing-the-results/README.md)
* [Capturing profiles](analyzing-the-results/capturing-profiles.md)
* [Debugging test plans](debugging-test-plans.md)
* [Docker Settings](docker-settings.md)
* [Featured projects](featured-projects/README.md)

## Runner library

* [local:exec](runner-library/local-exec.md)
* [local:docker](runner-library/local-docker/README.md)
* [System overview](runner-library/local-docker/system-overview.md)
* [Runner flags](runner-library/local-docker/runner-flags.md)
* [Troubleshooting](runner-library/local-docker/troubleshooting.md)
* [cluster:k8s](runner-library/cluster-k8s/README.md)
* [System overview](runner-library/cluster-k8s/system-overview.md)
* [How to create a Kubernetes cluster for Testground](runner-library/cluster-k8s/how-to-create-a-kubernetes-cluster-for-testground.md)
* [Monitoring and Observability](runner-library/cluster-k8s/monitoring.md)
* [Understanding Testground performance on Kubernetes](runner-library/cluster-k8s/understanding-testground-performance-on-kubernetes.md)
* [Troubleshooting](runner-library/cluster-k8s/troubleshooting.md)

## BUILDER LIBRARY

* [docker:go](builder-library/docker-go.md)
* [exec:go](builder-library/exec-go.md)
* [docker:generic](builder-library/docker-generic.md)

* Table of contents
* [What is Testground?](README.md)
* [Concepts and architecture](concepts-and-architecture/README.md)
* [Test plans and test cases](concepts-and-architecture/test-structure.md)
* [Daemon and client](concepts-and-architecture/daemon-and-client.md)
* [Synchronization service](concepts-and-architecture/sync-service.md)
* [Networking](concepts-and-architecture/networking.md)
* [Sidecar](concepts-and-architecture/sidecar.md)
* [Builders](concepts-and-architecture/builders.md)
* [Runners](concepts-and-architecture/runners.md)
* [Runtime environment \(runenv\)](concepts-and-architecture/runtime.md)
* [Client-Server communication](concepts-and-architecture/client-server-communication.md)
* [Getting started](getting-started.md)
* [Writing test plans](writing-test-plans/README.md)
* [Quick start](writing-test-plans/quickstart.md)
* [Understanding the test plan manifest](writing-test-plans/test-plan-manifest.md)
* [Parameters and test cases](writing-test-plans/paramaters-and-testcases.md)
* [Keeping instances in sync](writing-test-plans/synchronization.md)
* [Communication between instances](writing-test-plans/communication-between-instances.md)
* [Observability, assets and metrics](writing-test-plans/observability-assets-and-metrics.md)
* [Managing test plans](managing-test-plans.md)
* [Running test plans](running-test-plans.md)
* [Traffic shaping](traffic-shaping.md)
* [Analyzing test run results](analyzing-the-results/README.md)
* [Capturing profiles](analyzing-the-results/capturing-profiles.md)
* [Debugging test plans](debugging-test-plans.md)
* [Docker Settings](docker-settings.md)
* [Featured projects](featured-projects/README.md)
* Runner library
* [local:exec](runner-library/local-exec.md)
* [local:docker](runner-library/local-docker/README.md)
* [System overview](runner-library/local-docker/system-overview.md)
* [Runner flags](runner-library/local-docker/runner-flags.md)
* [Troubleshooting](runner-library/local-docker/troubleshooting.md)
* [cluster:k8s](runner-library/cluster-k8s/README.md)
* [System overview](runner-library/cluster-k8s/system-overview.md)
* [How to create a Kubernetes cluster for Testground](runner-library/cluster-k8s/how-to-create-a-kubernetes-cluster-for-testground.md)
* [Monitoring and Observability](runner-library/cluster-k8s/monitoring.md)
* [Understanding Testground performance on Kubernetes](runner-library/cluster-k8s/understanding-testground-performance-on-kubernetes.md)
* [Troubleshooting](runner-library/cluster-k8s/troubleshooting.md)
* Builder Library
* [docker:go](builder-library/docker-go.md)
* [exec:go](builder-library/exec-go.md)
* [docker:generic](builder-library/docker-generic.md)
Binary file modified analyzing-the-results/capturing-profiles.md
Binary file not shown.
Binary file added assets/apple-touch-icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
Binary file added assets/favicon.ico
Binary file not shown.
File renamed without changes
Binary file added assets/icon-192.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/icon-512.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions assets/icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading