Question 1

Is QA Guardian a platform or a service?

Accepted Answer

We're a platform-enabled service. The QA Guardian platform is used by our team of in-house QA engineers to build, run, and maintain end-to-end tests for our customers. The platform uses Microsoft's Playwright framework and a specially-optimized Kubernetes back-end that lets us run millions of tests in parallel in headful browsers, each running inside a separate Docker container.

Question 2

What do you need from my team to get started?

Accepted Answer

We need access to your staging or QA environment where we can run tests. We'll also need details about your application, user workflows, and testing priorities. Our team will work with you to create a comprehensive test plan.

Question 3

Can you integrate tests into my CI/CD pipeline?

Accepted Answer

Yes. QA Guardian integrates with most CI/CD platforms including GitHub Actions, GitLab CI, CircleCI, Jenkins, and more. You can also install our GitHub app to get test results inline on pull requests.

Question 4

Can I run tests on every commit?

Accepted Answer

Yes. Our cloud infrastructure is designed for high volume. We encourage you to integrate Guardian into your CI/CD pipeline and run the full flow on every Pull Request at no extra cost.

Question 5

How does QA Guardian communicate with my team?

Accepted Answer

QA Guardian sets up a shared Slack for communication. We use the channel to share information about the test flow and bug reports, and you can use it to ask questions, request new coverage, or just say hello. We recommend you add your development team to this channel to maximize visibility into any issues we find.

Question 6

How does QA Guardian report bugs?

Accepted Answer

We report any bugs we find in the shared Slack, your issue tracker, and through the QA Guardian platform.

Question 7

What are QA Guardian's hours of operation?

Accepted Answer

Our QA engineers are available during regular business hours.

Question 8

How do I request test coverage for new features?

Accepted Answer

Anyone on your team can request coverage in a few ways: (1) Send a message in Slack with context and short videos. (2) Make a request through the QA Guardian platform. (3) Add a QA Guardian label to a ticket in your issue tracker. We close the loop with you in Slack after completion.

Question 9

What if my UI changes - do you update the tests?

Accepted Answer

Yes. Your product is always changing, and we're always updating your tests along with it. No matter how big the change or how often you ship. Even if you do a total front-end refactor, we'll rebuild all your tests from the ground up — no charge.

Question 10

How do you handle flaky tests?

Accepted Answer

We always re-run a test at least three times to rule out temporary network or environment issues. Flakes never get reported as bugs or slow you down.

Question 11

What happens if a test flakes?

Accepted Answer

We fix it. If the UI changed, we update the selector. If it's a network issue, we adjust the retry logic. You don't pay extra for maintenance.

Question 12

Can I test multiple user roles in one Flow?

Accepted Answer

Yes. We call this Flow Bundling. For instance, we can test Admin, Manager, and User logins in a single script using Playwright's browser.newContext() for cookie isolation.

Question 13

Why don't you charge per test?

Accepted Answer

Charging per test encourages volume over value ("Test Bloat"). We charge per Flow (User Journey) to align our incentives with your application's actual functionality.

Question 14

What is a "Flow"?

Accepted Answer

A Flow is a single automated user journey (e.g., "User Logs In" or "User Purchases Item"). We set reasonable limits (25 Interactions, 10 assertions, 3 minutes) to ensure tests remain stable and performant.

Question 15

What if my test flow exceeds the limits?

Accepted Answer

It counts as multiple Flows. To maintain speed and reliability, we keep each Flow to 25 Interactions, 10 assertions, and 3 minutes.

Question 16

Can I upgrade or switch between plans?

Accepted Answer

Yes. You can move from Core to Advanced (or vice versa) with prorated billing. Enterprise is custom.

Question 17

Do I really own the code?

Accepted Answer

Yes. We write standard Playwright tests and you have full access. There is no proprietary vendor language or black box execution. You can export your tests and run them locally anytime.

Question 18

Why not just hire a contractor hourly?

Accepted Answer

Contractors bill for time, so they aren't incentivized to fix flaky tests quickly. With Guardian, maintenance is included in the flat monthly rate. We take the risk on flaky tests, ensuring your flow stays stable.

Question 19

Do you offer annual contracts?

Accepted Answer

Yes, all of our plans are annual contracts. We offer multi-year contracts at a discounted rate.

Question 20

What if I am dissatisfied with the service?

Accepted Answer

We offer a 90-day paid pilot. If you are not satisfied with the service, we will release you from the contract, no questions asked.

Question 21

What environments do you test against?

Accepted Answer

We typically test against your staging or QA environment. We can also support multiple environments depending on your plan. We do not run tests directly against production by default, though it can be configured for specific monitoring flows.

Question 22

How do you handle test credentials securely?

Accepted Answer

Test credentials are stored encrypted in our secrets manager and are never logged, shared, or exposed in test output. They are injected into the test environment at runtime and scoped per tenant. Our engineers cannot view the raw values.

Question 23

Do you test mobile browsers or responsive layouts?

Accepted Answer

We support responsive viewport testing using Playwright's built-in device emulation (e.g., iPhone 14, Pixel 7, iPad). For native mobile app testing (iOS/Android), our platform focuses on web applications running in a browser.

Question 24

What does onboarding look like and how long does it take?

Accepted Answer

Onboarding typically takes 1-2 weeks. Week 1: environment access, shared Slack, and journey mapping. Week 2: first batch of flows built and validated in CI/CD. Most teams reach 80% coverage of critical paths within 30 days.

Question 25

Which CI/CD systems do you support?

Accepted Answer

We support GitHub Actions, GitLab CI, CircleCI, Jenkins, Bitbucket Pipelines, and any system that can run an npm script or webhook. We provide an npx CLI and a GitHub App for inline PR status checks. Setup typically takes under 30 minutes.

Question 26

How long does it take to run my test flow?

Accepted Answer

QA Guardian provides infrastructure to run your tests in 100% parallel. This means you get can get test results for all test flows in minutes.

Question 27

How fast do tests run?

Accepted Answer

Fast! Playwright is incredibly efficient and we run each test flow in an isolated environment, and 100% concurrently. Most tests complete in less than a minute or two.

Question 28

What do I do when someone new joins my team?

Accepted Answer

When someone new joins your team, we recommend adding them to the Slack channel. If it's helpful, your QA Lead or Customer Success Manager can also schedule a meeting to get them up to speed on how QA Guardian works with your team and the current testing priorities.

Question 29

Can I request coverage proactively?

Accepted Answer

Absolutely! We're always watching for new features that need test coverage. You can tag QA Guardian in your issue tracker and we'll prepare new coverage — even if it's still in design. We can also receive short videos or screenshots and will code up the tests for you.

Question 30

What if I have very simple tests (e.g., 3 different logins)?

Accepted Answer

We encourage "Flow Bundling." Because our limits are generous, we can combine multiple simple tests into one sequence at no extra cost. For example, testing Admin Login, Supervisor Login, and User Login fit comfortably within single flow limits. That counts as just 1 Flow.

Question 31

How can I meet with QA Guardian?

Accepted Answer

Existing customers should reach out in their Slack channel and your Customer Success Manager will schedule some time. If you're interested in joining QA Guardian, book a demo to discuss your testing needs.

Frequently Asked Questions

Platform & Service

Communication & Support

Test Coverage & Maintenance

Team Management

Pricing & Plans

Still have questions?