Why does artillery have to be in a separate repository

[chinwahc]

Hi,
So I'm looking at artillery for load testing of our graphql api, reading the docs and in this section it says:

Generally we do not recommend installing Artillery as a dev dependency inside application codebases (i.e. installing it with npm install -D). The recommended best practice is to maintain a separate repository/project with performance tests and install Artillery as a dependency in that project.

I was wondering what the reasoning behind this is. and what are the risks of adding it in the same repo as the api?

[hassy]

Hi @chinwahc 👋 The recommendation is based on experience developing and managing large performance test suites that exercise multiple services or different parts of a large system. If you're only going to load test a single API, keeping load test definitions in the same repo as the service's codebase will work fine. If your test suite grows (and they usually tend to), and other services add load testing for example, or if multiple teams start wanting to run load tests - it will become easier to manage everything in one separate place.

We'll add more information to the relevant part of our Best Practices guide (and link to it from the installation guide too)

[hassy]

With the approach of keeping it in a separate repo are there any issues around running it in ci/cd pipelines as part of a workflow?

It depends on the setup & tooling you use, but for example with GitHub Actions, you could set a schedule to trigger load tests from that centralized repo, or trigger workflows that run load tests from other GitHub Actions workflows using the workflow dispatch API. That would give you the flexibility to run load tests on certain commits, on deployments to certain environments and so on.

[chinwahc]

Makes sense, we use circleci so shouldn't be a problem, thanks for sharing your knowledge, much appreciated 🙏🏿

[nikolajovancevic]

Hey @hassy tnx for the explanation!

I'm kinda jumping into this convo, but I wanna check if I understood this correctly.

It is recommended to keep Artillery in the separate away from the App codebase itself, but it's fine to keep Artillery inside the automation tests repository with other UI/API automation scripts?

EDIT:

Soo I've found this digging through the docs - Reusing Typescript Playwright code as Artillery code

From which I assume it's perfectly fine to have Artillery along side other automation scripts.

[hassy]

@nikolajovancevic yes that's exactly it. What tends to work best at scale is a single monorepo containing test / automation code for multiple apps & services. "Tends to" is an important disclaimer of course :) there are always exceptions, but it's a good default practice that works well in our experience

[nikolajovancevic]

@hassy ack, tnx for clarification! 🙌🏻