CRAVEX GitHub workflow integration

[tdruez]

A few notes

• The interaction goes one way only, pushing the data from DejaCode to GitHub (any changes on the GitHub side is not pushed to DejaCode)

Changes

• Add a new issue_tracker_id field on the RequestTemplate model. It should be enough to provide the type and the location of the external tracker, for example: https://github.com/org/repo (platform = github, repo = org/repo)
• Add a new model to support integrations: ExternalIssueLink to keep the link between a DejaCode Request and the external Issue on a tracker (GitHub, Jira, ...). This model is also used to provide links to the external issue in the DejaCode UI.
• New DEJACODE_GITHUB_INTEGRATION_TOKEN setting to provide authentication credentials

How to use

• Set the DEJACODE_GITHUB_INTEGRATION_TOKEN in your settings: provide a "Fine-Grained Token" that has read and write permissions on your GitHub repo.
• Create a new RequestTemplate in your Dataspace and use your repo URL as the Issue Tracker ID, e.g.: https://github.com/org/repo_name
• From now on, any new DejaCode Request using this template will be pushed to GitHub provided repo. Edits such as field updates and closing the Request are also propagated to the GitHub issue (comments are not supported yet)

[DennisClark]

@tdruez we're off to a great start! The GH link on the DejaCode side is perfect. Great to see the priority of the request showing up as a label in GH -- very nice touch.

A few comments/suggestions:

I submitted a request from staging nexB signed on as dmclark, but the GH issue says it was opened by tdruez. If possible, it would be better to say it was opened by DennisClark (my GH name).

Instead of "Product Context - None" it would be better not to show that field at all when it is null.

I assigned the request in DejaCode to tomd, and it shows that in the GH description, but it would be better if it could also set the actual GH assignment to tdruez

[DennisClark]

Thinking about the user experience on the GH side now: Unless that user is also a DejaCode user who can go back and look at the product in question, there is not much in the way of information. It might make sense for the originator of the DejaCode Request to also generate an SBOM (or some kind of report, perhaps formatted as a spreadsheet) to be applied as an attachment to the Request and passed along to GH. Something to think about.

[tdruez]

I submitted a request from staging nexB signed on as dmclark, but the GH issue says it
was opened by tdruez. If possible, it would be better to say it was opened by DennisClark (my GH name).

In the GitHub API, we cannot submit an Issue on behalf of someone else.
The problem is that each DejaCode user would need to create a personal token, with the right permissions setup, and add it to their DejaCode account. That’s not very practical, and it could raise some security concerns.

In this case, the issue shows tdruez because the token used was from my account for demo purposes.
A better approach might be to create a dedicated GitHub user for the integration, like dejacode-integration, and use a single token from that account. We should mention this in the integration docs.

The actual Request "Submitted by" info is still available in the GH Issue body, along with other details.

I assigned the request in DejaCode to tomd, and it shows that in the GH description,
but it would be better if it could also set the actual GH assignment to tdruez

This assumes the integration manager has somehow provided a mapping between a DejaCode user and a GitHub user.
In this example, the username are non consistent across the platforms, and it's likely the norm rather than an exception.
That said, we should be able to map by default when the username is the same on both sides.
As an alternative, the email may be available from GitHub API to make another attempt at an automatic mapping.
I'll look more into this.

[DennisClark]

Hi @tdruez thanks for the comments. I like this: "A better approach might be to create a dedicated GitHub user for the integration, like dejacode-integration, and use a single token from that account. We should mention this in the integration docs."

[tdruez]

Changes:

• Move the GitHub token storage from settings to Dataspace model, as a new github_token configuration field.
• Reorganize the DataspaceConfiguration fieldsets in the Dataspace admin form. Separation of "AboutCode Integrations" and new/future Integrations.
• Refactor the github integration module into a GitHubIntegration class. This will serve as a base for future Integrations.
• Add support for posting comments from the DejaCode Request to GitHub Issue.
• Add DejaCode URL in the GitHub Issue body.
• The undefined fields, such as "Product context", are not part of the Issue body anymore. Addressing:

  Instead of "Product Context - None" it would be better not to show that field at all when it is null.

• Add new documentation available at https://dejacode.readthedocs.io/en/349-cravex-github-integration/integrations-github.html

[DennisClark]

@tdruez Testing in staging nexB has gone very smoothly. I updated a request template, created a new request, which was promptly sent to GitHub with the expected data. The integration of an additional comment on the DejaCode Request works just fine as well. The link back to DejaCode from the generated GH issue works really well (takes me to the sign-in form in case I'm not already in), and addresses my concerns about "well now that I see this issue in GH, what do I do about it?".

no problems found. I think this feature is probably ready for deployment.

[DennisClark]

@tdruez Actually I found one little problem just now. In DejaCode, if I am the original requester of an open request that I am looking at, there is a button enabled with the label "Close this Request" and if I click it I get a dialog requiring a "Reason" and after I enter that and complete the closing process my "Reason" ends up as a Comment on the Request.

So when I did that with my test request it closed the issue in GitHub but it did not create the "Reason" Comment in GitHub.

[tdruez]

Changes

• Closing a Request on the DejaCode side now posts a comment including the reason in the GitHub Issue.
• Add tests suite for the GitHub integration and Model methods related to integrations
• The code was cleaned up and refined, and migration files consolidated.