Ticket migrate

[danuker]

This is the ticket to migrate tickets.

Fixes https://github.com/chevah/server/issues/1271 and https://github.com/chevah/server/issues/1267 .

GitHub REST Issue creation docs: https://docs.github.com/en/rest/reference/issues#create-an-issue

What is not ideal:

• Tables or links to wiki do not work (not sure if we have any tables in a ticket).
• Trac DB columns not used:
  • version
  • milestone
• Lots of Trac DB tables not used (should keep the wiki DB backup around).
  • but a quick look by me did not show something important missing; more like Trac config values and history (and the wiki is already migrated).
• Attachments are not handled. We have just 7. Are they need to for Twisted? Not in this PR. Created Handle attachments if needed by Twisted Trac #14
• Have not tried the interaction between the automated comments and the GitHub bot changing ticket statuses/labels. Should I censor command keywords from the comments? Mentioned in Story: Perform Twisted ticket migration #16
• Add "incremental migration" to remember and skip tickets already posted. This allows safe re-runs, which will not double-post tickets. This should help us do a migration PR -> sftpplus.com tickets first. Mentioned in Story: Perform Twisted ticket migration #16
• TODO (just noticed): if repo used is fallback (server), apply component name as a tag
• Assign tickets to a separate project for each Trac milestone.
  • Close all created projects by default. If projects need to be open, they should be opened manually.
• Convert links from the Trac ID to the GitHub ID.
• Include tickets_created.tsv in requestNextNumber.
• Add a comment created later than the issue's PR, that mentions the PR, to see if it links it to the issue.
• for chevah/server, instead of T2618 use trac-2618, to have links; we might keep Trac around read-only for a while.

Formatting comparison

Trac URL	GitHub URL	tested part
https://trac.chevah.com/ticket/98	https://github.com/chevah/trac-migration-staging/issues/14	preformatted text
https://trac.chevah.com/ticket/637	https://github.com/chevah/trac-migration-staging/issues/15	embedded {{{#!rst
https://trac.chevah.com/ticket/822	https://github.com/chevah/trac-migration-staging/issues/16	horizontal lines
https://trac.chevah.com/ticket/3928	https://github.com/chevah/trac-migration-staging/issues/25	comments
https://trac.chevah.com/ticket/5519	https://github.com/chevah/trac-migration-staging/issues/32	closed status and label; attachment is not added; PR link is added; project is added

How to try and test the changes

reviewers: @adiroiban

Test with DRY_RUN=True: generate and check proposed trac -> GH URL mappings on the real repos.
I could not do this because I don't have access to the salt repo, nor the current Trac dump.
If you want, test with one or more custom Trac ticket ID conditions (preferably ones with matching GitHub ID) in the select_tickets method, and DRY_RUN=False.

[adiroiban]

it's ok not to use RST for GitHub issues comments.

GitHub Issues only supports "GitHub Flavored Markdown"... No need to try to convert to RST
It's better to try to convert to markdown.

I don't know if you saw this project https://github.com/tracboat/tracboat

https://github.com/tracboat/tracboat/blob/master/src/tracboat/trac2down.py

[adiroiban]

Another nice to have feature is trying to preserve the ID of trac tickets, at least for the main chevah/server repo

For example, in chevah/server we now have about 1860 PRs already created... so IDs up to 1860 are in conflict.
But at least for Trac tickets from ID 1861 and above, we can migrate them to GitHub and keep the same ID.

This will automatically enable a lot of links in the closed PR https://github.com/chevah/server/pulls?q=is%3Apr+is%3Aclosed

[danuker]

@adiroiban This is ready for review. Please look at the What is not ideal part especially, I am uncertain whether to handle attachments.

needs-review

[danuker]

FALLBACK_REPOSITORY = None
or
FALLBACK_REPOSITORY = SkipMigration()

We can more easily exclude these repositories from the select_tickets function. I will add a feature to skip already-submitted tickets seen in tickets_created.tsv.

[danuker]

I believe this is nearing completion.

Any partial run (with conditions like component being pr) will save the tickets and skip them on later runs.

Org-wide projects are created and closed (and also saved to avoid creating duplicates).

There is no sleep until the rate-limit is (nearly) hit, so most requests will happen as fast as possible in one thread.

Would appreciate another review.

needs-review