Improve automated plugin error messages

angus · March 10, 2023, 9:49am

The PMS will generate an automated ticket whenever a plugin status moves from “Compatible” to “Tests Failing” (if CI on the main branch of the plugin is failing) or “Incompatible” (a canary server plugin guard picked up an issue with the plugin). These automated messages could be improved in a few ways:

The title of the topic.
The content of the topic.
The error details when the error is failing tests (i.e. ingesting the error from CI).

Context

The error topics are generated by the Notifier class. In particular see

github.com

paviliondev/discourse-plugin-manager/blob/main/lib/plugin_manager/notifier.rb#L122


      
              Time: #{log.updated_at}
              Message: #{log.message}
              Discourse Branch: #{log.discourse_branch}
              Discourse SHA: #{log.discourse_sha}
              Plugin Branch: #{log.branch}
              Plugin SHA: #{log.sha}
              #{post_markdown_details(log)}
            EOF
          end
          
          def self.post_markdown(type, log, plugin_name)
            body = I18n.t("plugin_manager.notifier.#{type.to_s}.body", plugin_name: plugin_name)
          
            if type === :fixed
              body
            else
              <<~EOF
                #{body}
          
                Time: #{log.updated_at}
                Message: ``#{log.message}``

github.com

paviliondev/discourse-plugin-manager/blob/main/lib/plugin_manager/notifier.rb#L148


      
                ### Plugin
                Branch: #{log.branch}
                SHA: #{log.sha}
          
                ### Details
                #{post_markdown_details(log)}
              EOF
            end
          end
          
          def self.post_markdown_details(log)
            result = ""
            result += "Test url: #{log.test_url}\n" if log.test_url
          
            if log.backtrace
              result += <<~EOF
                Backtrace:
                #{log.backtrace}
              EOF
            end
          end

See also

cc @richard

angus · March 10, 2023, 10:05am

@richard On this topic, I’m not sure I agree with unlisting these topics. The are meant to be a signal, unlisting them dilutes that signal. I appreciate that they have not been a useful signal in the past, but I’m not sure that means they can’t be a useful signal with some of the issues addressed.

richard · March 10, 2023, 10:25am

These topics are severely cluttering the support categories and at this moment they do not contain useful information*, which basically means they’re only cluttering the support channels (at this moment!)

People have complained about this in the past and I’m myself a bit unhappy with this (apparently unfinished) functionality running in a public category on a production instance as well, since it makes me miss actual bug reports. I have asked a few times whether these messages could be moved elsewhere but that has not been addressed yet. Which I understand - we’re all busy - but in the meanwhile I’d rather not have this interfere with our day to day operations any more than needed.

*) except for admins. And unlisting them does not hide them for us admins, so I thought that would be acceptable for everyone.

angus · March 10, 2023, 11:51am

I think you might be overstating the problem a bit. We fixed a specific issue with them. We need to see the effect of that fix. We won’t if they’re unlisted.

richard · March 15, 2023, 9:07am

The automated messages are coming in again. Is there anything I can do to help getting this fixed?

https://coop.pavilion.tech/tag/automated

angus · March 15, 2023, 9:35am

The cause is what I’ve described above. It’s because CI is failing on the main branch of the Custom Wizard plugin. When someone improves the CI <> error report integration it’ll address the issue.

The CI <> error reporting concerns the interaction between these three classes

github.com

paviliondev/discourse-plugin-manager/blob/main/lib/plugin_manager/test_manager.rb

# frozen_string_literal: true
class ::PluginManager::TestManager
  attr_reader :host,
              :domain,
              :branch,
              :discourse_branch

  def initialize(host_name, branch, discourse_branch)
    @host = PluginManager::TestHost.get(host_name)
    return unless @host

    @branch = branch
    @discourse_branch = discourse_branch
    @domain = host.domain
  end

  def self.status
    @status ||= ::Enum.new(
      passing: 0,
      failing: 1

This file has been truncated. show original

github.com

paviliondev/discourse-plugin-manager/blob/main/lib/plugin_manager/notifier.rb

# frozen_string_literal: true
class PluginManager::Notifier

  attr_reader :plugin
  attr_accessor :log

  def initialize(plugin_name)
    @plugin = ::PluginManager::Plugin.get(plugin_name)
  end

  def send(type, log)
    @log = log
    @type = type
    send_post if send_post?
    send_email if send_email?
  end

  def send_email
    Jobs.enqueue(:send_plugin_notification,
      plugin: @plugin.name,

This file has been truncated. show original

github.com

paviliondev/discourse-plugin-manager/blob/main/lib/plugin_manager/plugin/status.rb#L55


      
          end
          
          
def self.required_git_attrs
            [:branch, :sha, :discourse_branch, :discourse_sha]
          end
          
          
def self.has_required_git_attrs?(git)
            required_git_attrs.all? { |attr| git[attr].present? }
          end
          
          
def self.update(name, git = {}, attrs = {})
            result = OpenStruct.new(success: false, errors: [])
            current_status = get(name, git[:branch], git[:discourse_branch])
          
          
  if attrs[:status].present? && attrs[:test_status].nil?
              if !has_required_git_attrs?(git)
                result.errors << "Does not have required git attributes"
                return result
              end
          
          
    if current_status.present? && !attrs[:skip_git_check]

I don’t have the time to fix it right now.

edit I’ve scheduled in some time this evening to take another look at the CI <> notifier issue

angus · March 15, 2023, 4:29pm

@richard I’ve raised a PR for this. The PR notes and code should be relatively self explanatory. Let me know what you think

FYI the “Discourse Plugin” workflow is a new standardised workflow for plugins which CDCK have published. I’ve been migrating our plugins to it, e.g.

github.com

paviliondev/discourse-custom-wizard/blob/59f00d62783d294e82ee6b1e380deabf082481de/.github/workflows/discourse-plugin.yml

name: Discourse Plugin

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  ci:
    uses: discourse/.github/.github/workflows/discourse-plugin.yml@v1

angus · April 28, 2023, 2:16pm

This is now deployed.