Allow manually-created issues

[jameseastwood]

Current automation breaks if any issue in the repo doesn't have a corresponding email in the inbox (I think that's what's causing it to break; haven't tested it with an issue created from an email that has been deleted). It would sometimes be useful to create an issue a different way, such as directly on github, but this usage is not possible for now.

[jameseastwood]

The same behavior causes another annoying issue: Adding a noisy sender (such as accounts.google.com) to the ignore list causes those emails not to be recognized as associated with their existing issues, breaking the automation.

By the time a sender has created enough spurious issues to merit adding them to the ignore list, it's a bit of a slog to delete all of these issues (since github web interface doesn't support batch delete)

[dwhswenson]

The inbox only loads recent emails (set by the workflows.emails-to-issues.recent parameter), so it shouldn't be a problem to delete older emails. However, we do load all the GitHub issues. The logic is that, since the flow is email -> issue, we need to know all the issues, and only the recent emails.

I thought that we had enabled support for non-ticket emails. The issues are loaded here:

ticgithub/ticgithub/tasks/emails_to_issues.py

Line 157 in 342b000

issues = self.bot.get_all_email_ticket_issues()

and that function should warn, but allow, non-ticket issues (and if it errors, it should error with a clear NonTicketIssueError):

ticgithub/ticgithub/bot.py

Lines 78 to 88 in 342b000

	def get_all_email_ticket_issues(self, nonticket="warn"):
	for iss in self.get_all_issues():
	try:
	iss.unique_id
	except NonTicketIssueError as e:
	if nonticket == "warn":
	warnings.warn(str(e))
	elif nonticket == "error":
	raise
	else:
	yield iss

My best guess is that your current error is because you're hitting an edge case I hadn't seen before, and the code to convert underlying errors to NonTicketIssueError was missing it. I think that #59 might fix.

---

Workaround if there's still a problem with non-ticket issues

I think non-ticket issues should work (although maybe they broke one of the other automations?). But if not, the workaround is to preface the issue body with:

Date: '1970-01-01 00:00:00+00:00'
ticket_id: 'any-nonsense-here'
---

I believe those are the only fields that this looks for.

[jameseastwood]

emails-to-issues fails, with the following trace. I am no longer certain what is causing this behavior, since I have deleted all of the issues that corresponded to tickets from now-ignored senders.

`Run python -m ticgithub.tasks.emails_to_issues --loglevel INFO
INFO:name:Running <main.EmailsToIssues object at 0x7f5cce153070> with config {'filters': [<function message_from_bot..bot_filter_inner at 0x7f5cce17bb50>, <function message_from_team_not_to..team_filter_inner at 0x7f5cce178790>, <function message_from_specified..message_from_specified_inner at 0x7f5cce17bbe0>], 'recent': datetime.timedelta(days=2), 'reply_template': <string.Template object at 0x7f5cce1533d0>}
Traceback (most recent call last):

File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,

File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/tasks/emails_to_issues.py", line 178, in
EmailsToIssues.run_cli()

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/tasks/task.py", line 54, in run_cli
task(opts.dry)

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/tasks/task.py", line 97, in call
self._run(cfg, dry)

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/tasks/emails_to_issues.py", line 159, in _run
id_to_issue = {iss.unique_id: iss for iss in issues}

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/tasks/emails_to_issues.py", line 159, in
id_to_issue = {iss.unique_id: iss for iss in issues}

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/bot.py", line 81, in get_all_email_ticket_issues
iss.unique_id

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/issues.py", line 53, in unique_id
return self._unique_id_from_body(self._issue.body)

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/issues.py", line 39, in _unique_id_from_body
return Issue._get_frontmatter(body)['ticket_id']

File "/home/runner/.local/lib/python3.10/site-packages/ticgithub/issues.py", line 29, in _get_frontmatter
for frontmatter in yaml.load_all(body, Loader=yaml.FullLoader):

File "/usr/lib/python3/dist-packages/yaml/init.py", line 127, in load_all
loader = Loader(stream)

File "/usr/lib/python3/dist-packages/yaml/loader.py", line 24, in init
Reader.init(self, stream)

File "/usr/lib/python3/dist-packages/yaml/reader.py", line 85, in init
self.determine_encoding()

File "/usr/lib/python3/dist-packages/yaml/reader.py", line 124, in determine_encoding
self.update_raw()

File "/usr/lib/python3/dist-packages/yaml/reader.py", line 178, in update_raw
data = self.stream.read(size)

AttributeError: 'NoneType' object has no attribute 'read'

Error: Process completed with exit code 1.`

[dwhswenson]

@jameseastwood : Can you check if the changes in #59 work for you?

Install instructions below not actually tested, but should work. Check that you're getting a dev version of ticgithub during the installation when the action runs.

The most "proper" way to check this is to rebuild ticgithub using a different installl approach. Create a config file with:

# in file config_pr59.yaml
ticgithub:
  install: python -m pip install git+https://github.com/omsf-eco-infra/ticgithub.git@fix-issue-body-is-none
  suffix: ""
  force-dry: false
  install-path: ./.github/workflows/

builders:
  - config-name: emails-to-issues
    run-command: python -m ticgithub.tasks.emails_to_issues
    template: scheduled_workflow.yml
    build-params:
      cron: CRON
  - config-name: assignment-to-gmail
    run-command: python -m ticgithub.tasks.assignment_to_gmail
    template: assignment-to-gmail.yml
  - config-name: unassigned-reminder
    run-command: python -m ticgithub.tasks.unassigned_reminder
    template: scheduled_workflow.yml
    build-params:
      cron: CRON
  - config-name: unclosed-reminder
    run-command: python -m ticgithub.tasks.unclosed_reminder
    template: scheduled_workflow.yml
    build-params:
      cron: CRON

Then run python -m ticgithub.build -c config_pr59.yaml in your issue tracker repo and commit/push the changes (should see that the only change is in the install lines).

Alternative quick-and-dirty solution: just change the install step in .github/workflows/emails-to-issues.yaml from python -m pip install ticgithub to python -m pip install git+https://github.com/omsf-eco-infra/ticgithub.git@fix-issue-body-is-none.