In Progress
Global playwright.config.ts support (viewport, setup, teardown, storageState ...)
To truly unite E2E testing and monitoring, Checkly CLI has to support the global configuration defined in playwright.config.ts, so that it can be used in both scenarios. playwright.config.ts is very useful to handle stuff like: Common viewport for tests Common setup, teardown for tests Common storageState for tests Turn screenshot, trace, video on/off And all the other useful playwright.config features that make playwright awesome
Younes Jallouli Over 1 year ago
Browser Checks
In Progress
Global playwright.config.ts support (viewport, setup, teardown, storageState ...)
To truly unite E2E testing and monitoring, Checkly CLI has to support the global configuration defined in playwright.config.ts, so that it can be used in both scenarios. playwright.config.ts is very useful to handle stuff like: Common viewport for tests Common setup, teardown for tests Common storageState for tests Turn screenshot, trace, video on/off And all the other useful playwright.config features that make playwright awesome
Younes Jallouli Over 1 year ago
Browser Checks
Dark. Mode.
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. My eyes at night. It hurz Describe the solution you'd like Dark mode. Describe alternatives you've considered Wearing sunglasses looking at Checkly, at night. Additional context 🌚 original GH issue link https://github.com/checkly/public-roadmap/issues/124
Tim Nolet Over 1 year ago
General
Dark. Mode.
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. My eyes at night. It hurz Describe the solution you'd like Dark mode. Describe alternatives you've considered Wearing sunglasses looking at Checkly, at night. Additional context 🌚 original GH issue link https://github.com/checkly/public-roadmap/issues/124
Tim Nolet Over 1 year ago
General
Next
Subscribe to status updates
Is your feature request related to a problem? Please describe. I want to allow users to subscribe to status updates from our service. I want to subscribe to Checkly's status so I know when the service is down or degraded, because: This might impact how our customers see the status or our service; We want to know that our services might not be monitored for the duration of the inciden. Describe the solution you'd like An option on Checkly's status page allowing users to subscribe to updates. Ideally, they would be able to choose between multiple ways to get notifications, like RSS/Atom feeds, email, Slack, maybe even webhook - see example from Github's status page: Which is similar to Sentry's - it seems they use Atlassian Status Page. I see #297 asks for a specific type of update, but I'd like to have more options than just email. original GH issue link https://github.com/checkly/public-roadmap/issues/304
Tim Nolet Over 1 year ago
Status Pages
Next
Subscribe to status updates
Is your feature request related to a problem? Please describe. I want to allow users to subscribe to status updates from our service. I want to subscribe to Checkly's status so I know when the service is down or degraded, because: This might impact how our customers see the status or our service; We want to know that our services might not be monitored for the duration of the inciden. Describe the solution you'd like An option on Checkly's status page allowing users to subscribe to updates. Ideally, they would be able to choose between multiple ways to get notifications, like RSS/Atom feeds, email, Slack, maybe even webhook - see example from Github's status page: Which is similar to Sentry's - it seems they use Atlassian Status Page. I see #297 asks for a specific type of update, but I'd like to have more options than just email. original GH issue link https://github.com/checkly/public-roadmap/issues/304
Tim Nolet Over 1 year ago
Status Pages
Completed
Multi-step API check
Describe the solution you'd like For a CRUD API set, it makes sense to test these in a multi-step sequence. For example, to do a POST to create an object, assert that it was created in the response, then POST to delete the object and assert the response. Describe alternatives you've considered Could potentially do this in setup/teardown scripts, although then it is less clear in the monitoring. original GH issue link https://github.com/checkly/public-roadmap/issues/46
Tim Nolet Over 1 year ago
API Checks
Completed
Multi-step API check
Describe the solution you'd like For a CRUD API set, it makes sense to test these in a multi-step sequence. For example, to do a POST to create an object, assert that it was created in the response, then POST to delete the object and assert the response. Describe alternatives you've considered Could potentially do this in setup/teardown scripts, although then it is less clear in the monitoring. original GH issue link https://github.com/checkly/public-roadmap/issues/46
Tim Nolet Over 1 year ago
API Checks
Completed
Static IPs
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. I would like to browser-checks for our internal tools (Blocked IPs, Some in private network). Which is not open public? I don't have much control on web server level to these tools. Currently, there is no way to have checks to these. Documentation suggesting whitelisting AWS-IPs doesn't make sense, this basically means opening to internet. Describe the solution you'd like Static IP sets. I don't know how your infrastructure looks like, if it's possible to run check runners behind few NATs. We can have IPs sets which can be whitelisted in our LB/firewalls. Any ways these IPs can be changed if your infra needs some update, So an API endpoint which list current IPs help more, to automate IP whitelisting. If NAT is not a possible solution, having an open AWS SNS, Even-Bridge some sort where I can subscribe in our AWS accounts to load new IPs to SGs using lambda. Having private-link or VPC peer to our VPC. This can help to have checks on private network itself. original GH issue link https://github.com/checkly/public-roadmap/issues/154
Tim Nolet Over 1 year ago
General
Completed
Static IPs
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. I would like to browser-checks for our internal tools (Blocked IPs, Some in private network). Which is not open public? I don't have much control on web server level to these tools. Currently, there is no way to have checks to these. Documentation suggesting whitelisting AWS-IPs doesn't make sense, this basically means opening to internet. Describe the solution you'd like Static IP sets. I don't know how your infrastructure looks like, if it's possible to run check runners behind few NATs. We can have IPs sets which can be whitelisted in our LB/firewalls. Any ways these IPs can be changed if your infra needs some update, So an API endpoint which list current IPs help more, to automate IP whitelisting. If NAT is not a possible solution, having an open AWS SNS, Even-Bridge some sort where I can subscribe in our AWS accounts to load new IPs to SGs using lambda. Having private-link or VPC peer to our VPC. This can help to have checks on private network itself. original GH issue link https://github.com/checkly/public-roadmap/issues/154
Tim Nolet Over 1 year ago
General
Customisation of NPM Dependencies
Summary: Allow users to customise or override the NPM dependencies within the Checkly runtime environment. Description: Currently, users are restricted to a set list of packages and versions in the Checkly runtime, which may not cover all the specific use cases or compatibility issues they encounter. As a Checkly user, I need to be able to dynamically use packages in my own runtime. I process dependency updates in projects regularly to use the latest versions for new features and security updates.
Alex Noyes About 1 year ago
Runtimes
Customisation of NPM Dependencies
Summary: Allow users to customise or override the NPM dependencies within the Checkly runtime environment. Description: Currently, users are restricted to a set list of packages and versions in the Checkly runtime, which may not cover all the specific use cases or compatibility issues they encounter. As a Checkly user, I need to be able to dynamically use packages in my own runtime. I process dependency updates in projects regularly to use the latest versions for new features and security updates.
Alex Noyes About 1 year ago
Runtimes
Planned
DNS checks
I want to check the core metrics and correctness of my DNS and DNS records. A DNS check should be able to: monitor the response time of a DNS query. assert the corrects values and existence of specific DNS records.
Tim Nolet Over 1 year ago
Check Types
Planned
DNS checks
I want to check the core metrics and correctness of my DNS and DNS records. A DNS check should be able to: monitor the response time of a DNS query. assert the corrects values and existence of specific DNS records.
Tim Nolet Over 1 year ago
Check Types
In Progress
add ability for tcp and icmp checks.
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We do http monitoring but we have some api's that are not http but are at tcp level. Having monitoring for tcp connections basic check on URL and any Port is great. Can integrate tls as well. Describe the solution you'd like as part of addition of api checks, we can add tcp as an option in the drop down. Describe alternatives you've considered using datadog right now . Additional context Add any other context or screenshots about the feature request here. original GH issue link https://github.com/checkly/public-roadmap/issues/96
Tim Nolet Over 1 year ago
Check Types
In Progress
add ability for tcp and icmp checks.
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We do http monitoring but we have some api's that are not http but are at tcp level. Having monitoring for tcp connections basic check on URL and any Port is great. Can integrate tls as well. Describe the solution you'd like as part of addition of api checks, we can add tcp as an option in the drop down. Describe alternatives you've considered using datadog right now . Additional context Add any other context or screenshots about the feature request here. original GH issue link https://github.com/checkly/public-roadmap/issues/96
Tim Nolet Over 1 year ago
Check Types
In Progress
Store tokens and other credentials across check runs
Any check that requires auth / credentials that expire needs to fetch these credentials on every check run. While this is the safest way to make sure checks are authenticated, it creates friction: - extra runtime in a browser check or setup script. - extra load on any authentication endpoint. - waste of general cloud resource. A user should be able to store tokens securely in a check run and fetch or revalidate the credentials when or before they expire.
Tim Nolet Over 1 year ago
General
In Progress
Store tokens and other credentials across check runs
Any check that requires auth / credentials that expire needs to fetch these credentials on every check run. While this is the safest way to make sure checks are authenticated, it creates friction: - extra runtime in a browser check or setup script. - extra load on any authentication endpoint. - waste of general cloud resource. A user should be able to store tokens securely in a check run and fetch or revalidate the credentials when or before they expire.
Tim Nolet Over 1 year ago
General
Scheduled checks at specific times
It would be great to be able to run checks at specific times (like a cron job schedule), as we have an API that publishes certain customer-critical data daily at specific times. So instead of every X minutes which can be any time depending on when the check is first created, we would be able to run reliably at specific times.
Yawar Amin Over 1 year ago
Scheduling
Scheduled checks at specific times
It would be great to be able to run checks at specific times (like a cron job schedule), as we have an API that publishes certain customer-critical data daily at specific times. So instead of every X minutes which can be any time depending on when the check is first created, we would be able to run reliably at specific times.
Yawar Amin Over 1 year ago
Scheduling
Completed
Smarter retries strategies: same region and backoff
The current retry strategy for all checks is fairly limited. We want to expand the possibilities for retrying with two options: - same region: right now, retries are always scheduled on a different region than the original attempt. This is not ideal for geo-fenced checks or region-aware websites rendering different content for different regions. - backoff strategies: we want to enable multiple retries according to a strategy the user can define: linear or exponential so the target system has more time to recover from an initial failure.
Tim Nolet Over 1 year ago
Scheduling
Completed
Smarter retries strategies: same region and backoff
The current retry strategy for all checks is fairly limited. We want to expand the possibilities for retrying with two options: - same region: right now, retries are always scheduled on a different region than the original attempt. This is not ideal for geo-fenced checks or region-aware websites rendering different content for different regions. - backoff strategies: we want to enable multiple retries according to a strategy the user can define: linear or exponential so the target system has more time to recover from an initial failure.
Tim Nolet Over 1 year ago
Scheduling
Increase Response time limit for checks
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We have a time out of apis defined at our load balancer level as 60 sec, checkly times out the request at 30 sec max with which we are not able to identify if the system is not responding or checkly servers not able to communicate with our servers. Describe the solution you'd like Providing an option of custom timeout (maximum configurable to 100 sec) Describe alternatives you've considered NA for now Additional context NA original GH issue link https://github.com/checkly/public-roadmap/issues/207
Tim Nolet Over 1 year ago
Scheduling
Increase Response time limit for checks
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We have a time out of apis defined at our load balancer level as 60 sec, checkly times out the request at 30 sec max with which we are not able to identify if the system is not responding or checkly servers not able to communicate with our servers. Describe the solution you'd like Providing an option of custom timeout (maximum configurable to 100 sec) Describe alternatives you've considered NA for now Additional context NA original GH issue link https://github.com/checkly/public-roadmap/issues/207
Tim Nolet Over 1 year ago
Scheduling
Completed
Support for `toHaveScreenshot` / visual regression testing
Is your feature request related to a problem? Please describe. I have a particular use case for toHaveScreenshot in that my Checkly tests goes through our UI to generate a PDF export of a particular page that you're on. I'd love to be able to compare a screenshot of the PDF generated to a snapshot, as it's impossible to run typical assertions on the information shown in PDFs. Describe the solution you'd like I understand that Checkly agents run in an isolated environment for each test. However, toHaveScreenshot could be supported in a way similar to Buildkite in how it allows you to manage artifacts between builds and build steps, it would be cool if the Checkly agent could detect snapshots generated from toHaveScreenshot, store them somewhere, and then pull them down for the next test to verify against. This could be automatic, or something that the test does, wherer you test can import some kind of Checkly npm package that exports some functions to do these particular tasks. Describe alternatives you've considered What I'm trying to achieve here I can write this code myself. I.e i can store the screenshot snapshots in S3 at the end of my test and write them to disk when the test starts again but that's not ideal at scale if I need to do this in multiple tests. There are visual comparison libraries out there but they aren't in the Checkly runtimes. Adding some kind of visual comparison npm package to the runtimes would suffice too!!! original GH issue link https://github.com/checkly/public-roadmap/issues/282
Tim Nolet Over 1 year ago
Browser Checks
Completed
Support for `toHaveScreenshot` / visual regression testing
Is your feature request related to a problem? Please describe. I have a particular use case for toHaveScreenshot in that my Checkly tests goes through our UI to generate a PDF export of a particular page that you're on. I'd love to be able to compare a screenshot of the PDF generated to a snapshot, as it's impossible to run typical assertions on the information shown in PDFs. Describe the solution you'd like I understand that Checkly agents run in an isolated environment for each test. However, toHaveScreenshot could be supported in a way similar to Buildkite in how it allows you to manage artifacts between builds and build steps, it would be cool if the Checkly agent could detect snapshots generated from toHaveScreenshot, store them somewhere, and then pull them down for the next test to verify against. This could be automatic, or something that the test does, wherer you test can import some kind of Checkly npm package that exports some functions to do these particular tasks. Describe alternatives you've considered What I'm trying to achieve here I can write this code myself. I.e i can store the screenshot snapshots in S3 at the end of my test and write them to disk when the test starts again but that's not ideal at scale if I need to do this in multiple tests. There are visual comparison libraries out there but they aren't in the Checkly runtimes. Adding some kind of visual comparison npm package to the runtimes would suffice too!!! original GH issue link https://github.com/checkly/public-roadmap/issues/282
Tim Nolet Over 1 year ago
Browser Checks
Next
Show planned maintenance windows in dashboard
Is your feature request related to a problem? Please describe. we're very happy with the new dashboards, which have become integral to our communication (https://status.qfield.cloud/). There is, unfortunately, a big shortcoming for us. A planned maintenance window is only visible once it becomes active. To really become fully operational as a status page, these should be visible ahead of time. Describe the solution you'd like show planned maintenance as non-active planned incidents Describe alternatives you've considered A minimal solution could be to show planned maintenance as in the admin dashboard Additional context potentially part of #242 and #240 and #65 original GH issue link https://github.com/checkly/public-roadmap/issues/302
Tim Nolet Over 1 year ago
Status Pages
Next
Show planned maintenance windows in dashboard
Is your feature request related to a problem? Please describe. we're very happy with the new dashboards, which have become integral to our communication (https://status.qfield.cloud/). There is, unfortunately, a big shortcoming for us. A planned maintenance window is only visible once it becomes active. To really become fully operational as a status page, these should be visible ahead of time. Describe the solution you'd like show planned maintenance as non-active planned incidents Describe alternatives you've considered A minimal solution could be to show planned maintenance as in the admin dashboard Additional context potentially part of #242 and #240 and #65 original GH issue link https://github.com/checkly/public-roadmap/issues/302
Tim Nolet Over 1 year ago
Status Pages
Planned
Automated Account Creation
HI Checkly Team, We have lots of Checkly accounts created under our org and are looking to get more in the future. We have a tooling suite for creating checks on Checkly and its pretty much streamlined except for the account creation part. We are looking to streamline account creations as well, as the current process is to email our Checkly representative and that has a turn-around time of ~24hours. We are looking to have this automated via API, preferably get it to be a Terraform resource. We are currently using terraform to create Checkly resources, but whatever way would be faster to get this to become a reality would be great
Giancarlo Over 1 year ago
Auth / Access / Roles
Planned
Automated Account Creation
HI Checkly Team, We have lots of Checkly accounts created under our org and are looking to get more in the future. We have a tooling suite for creating checks on Checkly and its pretty much streamlined except for the account creation part. We are looking to streamline account creations as well, as the current process is to email our Checkly representative and that has a turn-around time of ~24hours. We are looking to have this automated via API, preferably get it to be a Terraform resource. We are currently using terraform to create Checkly resources, but whatever way would be faster to get this to become a reality would be great
Giancarlo Over 1 year ago
Auth / Access / Roles
Mark/Delete failures as false-positive
We use Checkly statistics to monitor our uptime %. Sometimes we have failures that are false positives (since there was a bug in the test or specific to the test environment, and not in production, or just things didn't really break). We don't want to count those failures as downtime, hopefully with adding a note to why. Thanks!
Amir Jaron About 1 year ago
Insights
Mark/Delete failures as false-positive
We use Checkly statistics to monitor our uptime %. Sometimes we have failures that are false positives (since there was a bug in the test or specific to the test environment, and not in production, or just things didn't really break). We don't want to count those failures as downtime, hopefully with adding a note to why. Thanks!
Amir Jaron About 1 year ago
Insights
Support for more flexible thresholds in Alerts escalations
Currently, time-based escalation cannot be set to less than 5 minutes. In most cases, this threshold is pretty high for sensitive API checks. I'd highly appreciate it if you could make it more flexible. time_based_escalation { minutes_failing_threshold = 5 // (TIME_BASED escalation only) after how many minutes spent in failing state to send an alert () } As of now, it can be set to 5, 10, 15, 30 minutes The requested thresholds are, 1, 2, 3, 4, 5 minutes Reference: https://www.checklyhq.com/docs/terraform-provider/alerting/#alert-settings
Mehmet Sever 7 months ago
Alerting
Support for more flexible thresholds in Alerts escalations
Currently, time-based escalation cannot be set to less than 5 minutes. In most cases, this threshold is pretty high for sensitive API checks. I'd highly appreciate it if you could make it more flexible. time_based_escalation { minutes_failing_threshold = 5 // (TIME_BASED escalation only) after how many minutes spent in failing state to send an alert () } As of now, it can be set to 5, 10, 15, 30 minutes The requested thresholds are, 1, 2, 3, 4, 5 minutes Reference: https://www.checklyhq.com/docs/terraform-provider/alerting/#alert-settings
Mehmet Sever 7 months ago
Alerting
Access to fake media devices
We have a use case where we're interested in using Checkly to monitor a WebRTC application. For this application, cameras and microphones are a critical part of the experience. Fortunately, chromium has fake media devices which can be used in Playwright. It would be great to be able to access these from within Checkly tests.
Brian Hill 9 months ago
Browser Checks
Access to fake media devices
We have a use case where we're interested in using Checkly to monitor a WebRTC application. For this application, cameras and microphones are a critical part of the experience. Fortunately, chromium has fake media devices which can be used in Playwright. It would be great to be able to access these from within Checkly tests.
Brian Hill 9 months ago
Browser Checks
Access to the complete Lighthouse metrics
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We are not able to access Lighthouse report details from the API when executing browser checks. Describe the solution you'd like We would like to be able to access Lighthouse report details so we can fail tests when performance degrades. Describe alternatives you've considered Some data is available thru the performance.getEntriesByName object, we are using this as a workaround. original GH issue link https://github.com/checkly/public-roadmap/issues/79
Tim Nolet Over 1 year ago
Check Types
Access to the complete Lighthouse metrics
💡 For general support requests and bug reports, please go to checklyhq.com/support Is your feature request related to a problem? Please describe. We are not able to access Lighthouse report details from the API when executing browser checks. Describe the solution you'd like We would like to be able to access Lighthouse report details so we can fail tests when performance degrades. Describe alternatives you've considered Some data is available thru the performance.getEntriesByName object, we are using this as a workaround. original GH issue link https://github.com/checkly/public-roadmap/issues/79
Tim Nolet Over 1 year ago
Check Types
Next
Uptime % history on dashboards
Right now you can see up to 30 days back. It will be nice to have a 30,90,180,365 days option as we have a guaranteed SLA calculated per year.
Amir Jaron About 1 year ago
Status Pages
Next
Uptime % history on dashboards
Right now you can see up to 30 days back. It will be nice to have a 30,90,180,365 days option as we have a guaranteed SLA calculated per year.
Amir Jaron About 1 year ago
Status Pages