Skip to content

Conversation

sowen1023
Copy link
Contributor

What does this PR do

feat: neo4j connector

Standards checklist

  • ✅ The PR title is descriptive
  • ✅ The commit messages are semantic
  • ✅ Necessary tests are added
  • ✅ Updated the release notes
  • ✅ Necessary documents have been added if this is a new feature

@sowen1023
Copy link
Contributor Author

sowen1023 commented Oct 9, 2025

neo4j.datasource.config.mp4

@sowen1023
Copy link
Contributor Author

sowen1023 commented Oct 9, 2025

Doc example

{
    "_index": "coco_document-v2",
    "_type": "_doc",
    "_id": "99786427481574a4b5e5d4640daf5493",
    "_score": 0,
    "_source": {
        "id": "99786427481574a4b5e5d4640daf5493",
        "payload": {
            "coco_property": "2025-10-09T04:28:44.918Z",
            "n": {
                "category": "Legal",
                "disabled": false,
                "document_id": "doc-AA27",
                "lang": "en",
                "opened_date": "2025-09-20",
                "size": 180000,
                "summary": "Updated policy regarding the retention and deletion of client data.",
                "title": "Data Retention Policy v2",
                "type": "PDF",
                "updated": "2025-10-09T04:28:44.918Z"
            },
            "n.disabled": false,
            "n.element_id": "4:1093e8af-4022-4813-9534-8c7121ac10d4:557",
            "n.labels": [
                "Document"
            ],
            "n.opened_date": "2025-09-20",
            "n.updated": "2025-10-09T04:28:44.918Z"
        },
        "source": {
            "type": "connector",
            "name": "Neo4j Docs",
            "id": "d3d34pdafbilvmlmnlsg"
        },
        "type": "PDF",
        "category": "Legal",
        "title": "Data Retention Policy v2",
        "summary": "Updated policy regarding the retention and deletion of client data.",
        "lang": "en",
        "size": 180000,
        "last_updated_by": {}
    }
}

Tie *StoredCursorValue `json:"tie,omitempty"`
}

type SyncState struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed there’s a sync state management feature — that’s great to have. I’d like to extend this concept to the generic connector management infrastructure, so other connectors can also benefit from it.

I’m working on integrating existing connectors into the processor layer and leveraging the generic pipeline framework to unify data processing capabilities — stay tuned!

sowen1023 and others added 2 commits October 21, 2025 22:50
- Resolved conflicts in web frontend files
- Merged connector types from both branches
- Added Neo4j, Feishu, and Lark to supported types
- All connector refactoring work preserved from both branches

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@sowen1023
Copy link
Contributor Author

sowen1023 commented Oct 21, 2025

  • Make the Neo4j connector pipeline-based.
  • For RDBMS connectors, use the ConnectorProcessorBase.Collect function instead of directly pushing documents to the processing queues.

"default" : "/assets/icons/connector/neo4j/icon.png"
}
},
"builtin": true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processor is missing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

coco.yml Outdated
interval: 30s
queue:
name: indexing_documents
neo4j:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not necessary needed anymore.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Enable the connector in `coco.yml` so coco-server can schedule sync jobs:

```yaml
connector:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs need to update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


| **Field** | **Type** | **Description** |
|--------------|-----------|------------------------------------------------------------------------------------------|
| `enabled` | `boolean` | Turns the Neo4j connector on or off. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this config section is not needed any more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

ormCtx := orm.NewContextWithParent(ctx)
state := &SyncState{}
state.SetID(makeSyncStateID(connectorID, datasourceID))
exists, err := orm.GetV2(ormCtx, state)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to add ctx.DirectAccess() for load and save function, as they are background jobs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@RainyNight9 RainyNight9 merged commit ef3dd2a into infinilabs:main Oct 23, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants