Advanced Test Configuration

Test generation can be configured in a number of ways to either run against a real API or mock server, or to pull values from the environment or skip generation of tests for specific operations.

Configuring the API to Test Against

By default, tests will be generated to run against Speakeasy’s mock server (URL of http://localhost:18080) which will validate the SDKs are functioning correctly but not guaranteeing the correctness of the API.

The generator can be configured to run all tests against another URL or just individual tests. This is done through the use x-speakeasy-test-server extensions in the .speakeasy/tests.arazzo.yaml file.

If the extension is found at the top level of the Arazzo file then all workflows/tests will be configured to run againsts the specified server URL. If the extension is found within a workflow then only that workflow will be configured to run against the specified server URL.

The server URL can either be a static URL or a x-env: EXAMPLE_ENV_VAR value that will be pull the value from the environment variable EXAMPLE_ENV_VAR where the name of the environment variable can be any specified name.

arazzo: 1.0.0
# ...
x-speakeasy-test-server: https://api.example.com # If specified at the top level of the Arazzo file, all workflows will be configured to run against the specified server URL
workflows:
  - workflowId: some-test
    x-speakeasy-test-server: x-env: TEST_SERVER_URL # If specified within a workflow, only that workflow will be configured to run against the specified server URL. This will override any top level configuration.
    # ...

A default value can be provided in the x-env directive if the environment variable is not set. This can be useful for local development or non-production environments.

x-speakeasy-test-server: x-env: TEST_SERVER_URL; http://localhost:18080 # Run against the local mock server if the environment variable is not set

If all tests are configured to run against other server URLs, mock server generation can be disabled within the .speakeasy/gen.yaml file.

# ...
generation:
  # ...
  mockServer:
    disabled: true # Setting this to true will disable mock server generation

Configuring security credentials for SDK test instances

When running tests against a real API, the SDK may need to be configured with security credentials to authenticate with the API. This can be done by adding the x-speakeasy-test-security extension to the document, a workflow or a individual step.

The x-speakeasy-test-security extension allows static values or values pulled from the environment to be used when instantiating an SDK instance and making requests to the API.

arazzo: 1.0.0
# ...
x-speakeasy-test-security: # Defined at the top level of the Arazzo file, all workflows will be configured to use the specified security credentials
  value:
    # The keys in the value map are the names of the security schemes defined in the OpenAPI document.
    # For simple schemes, the values is for example the API key value required.
    # For schemes like basic auth or OAuth2 that require multiple values, the value is a map of the required values.
    apiKey: x-env: TEST_API_KEY # Values can be pulled from the environment
    basicAuth:
      username: "test-user" # Or defined as static values
      password: x-env: TEST_PASSWORD
workflows:
  - workflowId: some-test
    x-speakeasy-test-security: # Security can be defined/overridden for a specific workflow
      value:
        apiKey: "test-key"
    # ...
    steps:
      - stepId: step1
        x-speakeasy-test-security: # Or security can be defined/overridden for a specific step
          value:
            authToken: x-env: TEST_AUTH_TOKEN
        # ...
      - stepId: step2
        # ...

Disable auto generation of tests for specific operations

When generateNewTests is enabled in the .speakeasy/gen.yaml file, all new operations found on the next generation after they are added will automatically have workflows created for then in the .speakeasy/tests.arazzo.yaml file and therefore tests will be generated for them.

To disable auto generation of tests for specific operations, the x-speakeasy-test extension can be added to the operation in the OpenAPI document.

openapi: 3.1.0
# ...
paths:
  /example1:
    get:
      x-speakeasy-test: false # Disables auto generation of tests for this operation
      # ...
# ...

alternatively if a workflow/test already exists that references the operation in the .speakeasy/tests.arazzo.yaml file, then no new workflow will be created for the operation.

Grouping tests

By default, all tests will be generated into a single file related to the main SDK class for example sdk.test.ts or test_sdk.py. This can be configured by adding a x-speakeasy-test-group extension to the workflow which will determine which tests will be grouped together in seperate test files.

arazzo: 1.0.0
# ...
workflows:
  - workflowId: some-test
    x-speakeasy-test-group: user # All tests in the user group will end up in the `user.test.ts`/`test_user.py`/`user_test.go` files
    # ...

Generate tests only for specific targets

By default, all tests will be generated for all supported targets. But using the x-speakeasy-test-targets extension, it is possible to generate tests only for specific targets. This may be useful if tests are either not needed for certain languages or they may be failing in a certain language.

arazzo: 1.0.0
# ...
workflows:
  - workflowId: some-test
    x-speakeasy-test-targets: # Only generate tests for the typescript target
      - typescript
    # ...