Refactor Neo4j ingestion through configurable pipelines

[royisme]

Summary

• refactor the Neo4j knowledge service to use reusable LlamaIndex ingestion pipelines with connectors, transformations, and writers
• add a configurable ingestion_pipelines setting plus default pipeline bundle helpers and reusable components
• document how to customize ingestion pipelines through configuration and cover the new helpers with unit tests (skipped when llama_index cannot be imported)

Testing

• PYTHONPATH=src pytest tests/services/test_pipeline_components.py

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull Request Overview

This PR refactors Neo4j document ingestion to use configurable LlamaIndex ingestion pipelines, replacing inline document processing with a modular architecture featuring connectors, transformations, and writers.

Key Changes:

• Introduced reusable pipeline components (ManualDocumentConnector, SimpleFileConnector, SimpleDirectoryConnector, Neo4jKnowledgeGraphWriter, etc.)
• Added configuration-driven pipeline management with default bundles and user override support via ingestion_pipelines setting
• Refactored add_document, add_file, and add_directory to execute through a unified _run_ingestion_pipeline method

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/services/test_pipeline_components.py	Unit tests for pipeline bundle execution and configuration merging
src/codebase_rag/services/knowledge/pipeline_components.py	Core pipeline components including connectors, transformations, writers, and configuration builders
src/codebase_rag/services/knowledge/neo4j_knowledge_service.py	Refactored service to use pipeline bundles instead of direct document insertion
src/codebase_rag/config/settings.py	Added ingestion_pipelines configuration field
docs/getting-started/configuration.md	Documentation for customizing ingestion pipelines

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.