[Bug][GitLab] Missing the notes of Merge Request

[usharerose]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

Some merge requests themselves have been collected, but their notes (comments) are missing.

It seems that only opened Merge Request would be the input of mr_notes_collector (reference), which means the following MRs' note data would not be collected

• the history MRs (closed, merged) of one GitLab project
• one MR's complete lifecycle (created, commented, and merged) fell within the gap between DevLake pipelines

What do you expect to happen

The notes have been collected

How to reproduce

Collect any GitLab project with merged or closed MRs, those MRs' related notes would be missing.

Anything else

No response

Version

v1.0

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

The issue you're encountering is due to the current implementation of the GetMergeRequestsIterator function, which only considers open merge requests for note collection. This is intended to speed up the process by focusing on active merge requests ^[1].

To address this and include notes from closed or merged merge requests, you would need to modify the GetMergeRequestsIterator function. Specifically, you can adjust the SQL WHERE condition to include "closed" and "merged" states in addition to "opened". Here's a snippet of how you can modify the code:

func GetMergeRequestsIterator(taskCtx plugin.SubTaskContext, apiCollector *api.StatefulApiCollector) (*api.DalCursorIterator, errors.Error) {
	db := taskCtx.GetDal()
	data := taskCtx.GetData().(*GitlabTaskData)
	clauses := []dal.Clause{
		dal.Select("gmr.gitlab_id, gmr.iid"),
		dal.From("_tool_gitlab_merge_requests gmr"),
		// collect notes and commits for opened, closed, and merged merge requests
		dal.Where(
			`gmr.project_id = ? and gmr.connection_id = ? AND state IN (?, ?, ?)`,
			data.Options.ProjectId, data.Options.ConnectionId, "opened", "closed", "merged",
		),
	}
	if apiCollector != nil {
		if apiCollector.GetSince() != nil {
			clauses = append(clauses, dal.Where("gitlab_updated_at > ?", *apiCollector.GetSince()))
		}
	}
	// construct the input iterator
	cursor, err := db.Cursor(clauses...)
	if err != nil {
		return nil, err
	}

	return api.NewDalCursorIterator(db, cursor, reflect.TypeOf(GitlabInput{}))
}

This change will allow the collection of notes from all states of merge requests, ensuring that historical and lifecycle-complete notes are included in the data collection process ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}