Dashboard status should show when a canary server is down

So apparently the tests-passed canary is broken since this commit but all plugins show “compatible”.

There is no sign on the status page that this build broke down for the past 7 days.

Also, when looking at the plugin statuses, three plugin statuses were apparently updated 6 minutes after the build failed and set to compatible? (locations, cw and events)

      "status": {
        "name": "discourse-ratings",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:30",
        "last_status_at": "2023-05-10 10:04:58",
        "test_status": "passing"
      "name": "discourse-mentionables",
      "status": {
        "name": "discourse-mentionables",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:29",
        "last_status_at": "2023-05-10 10:04:58",
        "test_status": "passing"
      "name": "discourse-locations",
      "status": {
        "name": "discourse-locations",
        "status": "compatible",
        "status_changed_at": "2023-05-10 09:42:49",
        "last_status_at": "2023-05-16 22:34:33",
        "test_status": "passing"
      "name": "discourse-multilingual",
      "status": {
        "name": "discourse-multilingual",
        "status": "compatible",
        "status_changed_at": "2023-03-10 00:07:33",
        "last_status_at": "2023-05-10 10:04:58",
        "test_status": "passing"
      "name": "discourse-verifiable-credentials",
      "status": {
        "name": "discourse-verifiable-credentials",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:29",
        "last_status_at": "2023-05-10 10:04:58"
      "name": "discourse-layouts",
      "status": {
        "name": "discourse-layouts",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:29",
        "last_status_at": "2023-05-10 10:04:58",
        "test_status": "passing"
      "name": "discourse-legal-tools",
      "status": {
        "name": "discourse-legal-tools",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:30",
        "last_status_at": "2023-05-10 10:04:58"
      "name": "discourse-wikimedia-auth",
      "status": {
        "name": "discourse-wikimedia-auth",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:29",
        "last_status_at": "2023-05-10 10:04:58",
        "test_status": "passing"
      "name": "discourse-custom-wizard",
      "status": {
        "name": "discourse-custom-wizard",
        "status": "compatible",
        "status_changed_at": "2023-05-02 00:20:23",
        "last_status_at": "2023-05-16 22:34:36",
        "test_status": "passing"
      "name": "discourse-tickets",
      "status": {
        "name": "discourse-tickets",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:29",
        "last_status_at": "2023-05-10 10:04:58"
      "name": "discourse-journal",
      "status": {
        "name": "discourse-journal",
        "status": "compatible",
        "status_changed_at": "2023-03-08 09:00:30",
        "last_status_at": "2023-05-10 10:04:58"
      "name": "discourse-events",
      "status": {
        "name": "discourse-events",
        "status": "compatible",
        "status_changed_at": "2023-05-10 09:42:49",
        "last_status_at": "2023-05-16 22:34:37",
        "test_status": "passing"

Because those plugins statuses were determined before the breaking plugin broke the build. The status post request is sent immediately, not at the end of the entire build cycle.

I don’t think that if one plugin breaks a canary server all plugins should flip to “incompatible”. But perhaps we should have a status indicator for the canary server itself?

So this is a new feature (a needed one I think), not an issue. The PMS is working as expected.

I don’t understand that. I said “after”, you’re saying “before”.

The build failed at 2023-05-16T22:26:43 and the last message before the process exited was 16 May 2023 22:28:29.

The timestamps of the three statuses are 2023-05-16 22:34:3X

I believe that the accuracy of the PMS is a fundamental requirement and not just an enhancement, so I suggest that we address this issue with the priority typically reserved for bug fixes.

We can delve further into the why on the timing front, but fundamentally this is not an issue as much as it is a request for an explanation. Those plugins are in fact compatible. The statuses are correct.

I agree it’s important, but let’s solve one problem at a time. The first thing I need to do is get the tests-passed server back online. We can add a status indicator for the canary servers once we’ve done that.