feat: Implement framework for Client-Mode RAG #7924

wangqingkeyi · 2025-05-22T02:49:37Z

This commit introduces the foundational framework for supporting Retrieval Augmented Generation (RAG) in client mode (desktop application).

Key changes include:

Local RAG Service (LocalRagSrv.ts):
- I added a new service in the Electron main process (apps/desktop/src/main/services/LocalRagSrv.ts).
- This service initializes and manages a local pglite vector database for storing text chunks and embeddings.
- It handles file processing (text extraction using @lobechat/file-loaders, chunking).
- Currently, it uses placeholder embeddings for development and testing of the pipeline. Actual semantic search quality will be minimal until real embedding models are integrated.
- It provides methods for adding files to your local knowledge base, performing semantic searches (with placeholder embeddings), listing indexed documents, and deleting documents.
Modified RemoteServerSyncCtr.ts:
- The proxyTRPCRequest method in this controller now conditionally routes RAG-related tRPC calls (specifically chunk.semanticSearch) to the LocalRagSrv.ts if local RAG is enabled via settings.
- Other tRPC calls or when local RAG is disabled continue to be proxied to the remote server.
UI Settings for Local RAG:
- I added settings in apps/desktop/src/main/const/store.ts and related types for managing knowledge base settings.
- I created IPC handlers in SystemCtr.ts for getting and updating these settings from the renderer.
- I implemented a new UI page (src/app/settings_features_KnowledgeBase/index.tsx) allowing you to:
  - Enable or disable local RAG functionality (useLocalKnowledgeBase setting).
  - Add files to the local knowledge base (triggering processing by LocalRagSrv.ts).
  - View a list of currently indexed documents (with filename hints and chunk counts).
  - Delete documents from the local knowledge base.
State Management (Zustand):
- I extended the global Zustand store in the renderer to manage local RAG settings and the list of indexed documents.
Error Handling and Feedback:
- I improved error handling in the main process services and IPC controllers related to local RAG.
- I enhanced UI feedback in the settings page for loading states, successes, and errors during file management operations.
Documentation Content:
- I generated markdown content outlining the new feature, how to use it, and its current limitations (placeholder embeddings).

This framework lays the groundwork for a fully client-side RAG solution. Future work will focus on integrating real embedding model providers (e.g., OpenAI API, local Ollama models) to replace the current placeholder mechanism.

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

This commit introduces the foundational framework for supporting Retrieval Augmented Generation (RAG) in client mode (desktop application). Key changes include: 1. **Local RAG Service (`LocalRagSrv.ts`):** * I added a new service in the Electron main process (`apps/desktop/src/main/services/LocalRagSrv.ts`). * This service initializes and manages a local `pglite` vector database for storing text chunks and embeddings. * It handles file processing (text extraction using `@lobechat/file-loaders`, chunking). * Currently, it uses **placeholder embeddings** for development and testing of the pipeline. Actual semantic search quality will be minimal until real embedding models are integrated. * It provides methods for adding files to your local knowledge base, performing semantic searches (with placeholder embeddings), listing indexed documents, and deleting documents. 2. **Modified `RemoteServerSyncCtr.ts`:** * The `proxyTRPCRequest` method in this controller now conditionally routes RAG-related tRPC calls (specifically `chunk.semanticSearch`) to the `LocalRagSrv.ts` if local RAG is enabled via settings. * Other tRPC calls or when local RAG is disabled continue to be proxied to the remote server. 3. **UI Settings for Local RAG:** * I added settings in `apps/desktop/src/main/const/store.ts` and related types for managing knowledge base settings. * I created IPC handlers in `SystemCtr.ts` for getting and updating these settings from the renderer. * I implemented a new UI page (`src/app/settings_features_KnowledgeBase/index.tsx`) allowing you to: * Enable or disable local RAG functionality (`useLocalKnowledgeBase` setting). * Add files to the local knowledge base (triggering processing by `LocalRagSrv.ts`). * View a list of currently indexed documents (with filename hints and chunk counts). * Delete documents from the local knowledge base. 4. **State Management (Zustand):** * I extended the global Zustand store in the renderer to manage local RAG settings and the list of indexed documents. 5. **Error Handling and Feedback:** * I improved error handling in the main process services and IPC controllers related to local RAG. * I enhanced UI feedback in the settings page for loading states, successes, and errors during file management operations. 6. **Documentation Content:** * I generated markdown content outlining the new feature, how to use it, and its current limitations (placeholder embeddings). This framework lays the groundwork for a fully client-side RAG solution. Future work will focus on integrating real embedding model providers (e.g., OpenAI API, local Ollama models) to replace the current placeholder mechanism.

vercel · 2025-05-22T02:49:40Z

@google-labs-jules[bot] is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2025-05-22T02:49:48Z

👍 @wangqingkeyi

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

gru-agent · 2025-05-22T02:49:56Z

TestGru Assignment

Summary

Link	CommitId	Status	Reason
Detail	`c2de6cf`	✅ Finished

Files

File	Pull Request
src/store/global/slices/settings/selectors/knowledgeBase.ts	❌ Failed (I failed to setup the environment.)
src/store/global/slices/settings/initialState.ts	❌ Failed (I failed to setup the environment.)
src/store/global/store.ts	❌ Failed (I failed to setup the environment.)
src/store/global/slices/settings/index.ts	❌ Failed (I failed to setup the environment.)
src/store/global/slices/settings/actions/knowledgeBase.ts	❌ Failed (I failed to setup the environment.)
src/store/global/initialState.ts	❌ Failed (I failed to setup the environment.)

Tip

You can @gru-agent and leave your feedback. TestGru will make adjustments based on your input

greptile-apps

PR Summary

This PR implements a foundational framework for client-mode RAG (Retrieval Augmented Generation) in the desktop application, enabling local vector database storage and document processing capabilities.

Introduces LocalRagSrv service in apps/desktop/src/main/services/LocalRagSrv.ts managing a local PGlite vector database with placeholder embeddings for initial testing
Adds knowledge base settings UI in src/app/settings_features_KnowledgeBase/index.tsx for enabling local RAG and managing indexed documents
Modifies RemoteServerSyncCtr to conditionally route RAG-related tRPC calls between local and remote handlers based on settings
Extends Zustand store with new slices and actions for managing knowledge base settings and document operations
Implements comprehensive error handling and logging throughout the main process services and IPC communication

_{💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!}

_{16 file(s) reviewed, 29 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

greptile-apps · 2025-05-22T02:50:19Z

CLIENT_MODE_RAG_DOCS.md

+*   **Reporting Issues:** Please report any bugs or unexpected behavior through our official issue tracker (e.g., GitHub Issues for the project).
+*   **Feedback:** Share your feedback and ideas in our community forum or discussion channels.
+
+Your input is crucial for making this feature robust and user-friendly.Okay, I have created the markdown documentation for the Client-Mode RAG feature. It covers the introduction, enabling the feature, managing files, and importantly, the current limitations regarding placeholder embeddings and future enhancements.


syntax: Remove this line as it appears to be a development note that was accidentally included in the final documentation

Suggested change

Your input is crucial for making this feature robust and user-friendly.Okay, I have created the markdown documentation for the Client-Mode RAG feature. It covers the introduction, enabling the feature, managing files, and importantly, the current limitations regarding placeholder embeddings and future enhancements.

Your input is crucial for making this feature robust and user-friendly.

greptile-apps · 2025-05-22T02:50:20Z

CLIENT_MODE_RAG_DOCS.md

+**Key Benefits:**
+
+*   **Privacy:** Your data remains on your local machine, offering enhanced privacy as files are not uploaded to a remote server for processing in this mode.
+*   **Offline Access (to indexed data):** Once files are processed and indexed into your local knowledge base, you can perform searches and retrieve information even when you are offline. (Note: Initial processing or future updates requiring new model data might need an internet connection).


style: Clarify what specific features or operations might require an internet connection

greptile-apps · 2025-05-22T02:50:26Z

apps/desktop/src/main/controllers/RemoteServerSyncCtr.ts

+    // --- Local RAG Routing Logic ---
+    // Hypothetical setting check for local RAG mode.
+    // Replace with actual global store or setting manager access.
+    const knowledgeBaseSettings = this.app.storeManager.get('knowledgeBaseSettings');
+    const shouldUseLocalRag = knowledgeBaseSettings?.useLocalKnowledgeBase === true;


style: Consider using a type-safe settings accessor instead of direct storeManager.get() to ensure type safety and prevent runtime errors

greptile-apps · 2025-05-22T02:50:27Z

apps/desktop/src/main/controllers/SystemCtr.ts

+        return { success: false, message: `Local RAG service initialization failed: ${serviceError.message}` };
+      }
+    }
+     if (!this.localRagService) {


syntax: Indentation is inconsistent with surrounding code (extra space)

Suggested change

if (!this.localRagService) {

if (!this.localRagService) {

greptile-apps · 2025-05-22T02:50:27Z

apps/desktop/src/main/controllers/RemoteServerSyncCtr.ts

+          // Extract parameters for semanticSearch.
+          // Based on `src/app/api/rest/rag/route.ts` and `src/services/rag/chunk/index.ts`
+          // the body for `chunk.semanticSearch` might look like: { json: { query: string, fileIds?: string[], limit?: number } }
+          // Or it might be flatter if the tRPC client sends it directly.
+          // We assume `args.body.json` is the already parsed JSON payload or needs parsing from string.
+          // For this example, let's assume the structure is { query: string, limit?: number }
+          // This might need adjustment based on actual client-side tRPC request structure.


logic: The request body structure assumption needs validation - could cause runtime errors if tRPC client sends data in a different format

greptile-apps · 2025-05-22T02:52:16Z

src/types/user/settings.ts

+   *
+   * @default false
+   */
+  useLocalKnowledgeBase?: boolean;


style: Consider making useLocalKnowledgeBase non-optional since it has a default value. This would simplify type checking.

greptile-apps · 2025-05-22T02:52:17Z

src/types/user/settings.ts

+export interface SettingsState {
+  // ... other settings fields
+  knowledgeBaseSettings?: KnowledgeBaseSettings;
+}


style: SettingsState appears to duplicate KnowledgeBaseSettings structure. Consider using composition: extends Partial<UserSettings>

greptile-apps · 2025-05-22T02:53:01Z

vector-db-setup.js

+console.log(`Database will be stored at: ${dbFilePath}`);
+// Initialize PGlite, persisting to the specified file path
+// The `dataDir` option tells PGlite where to store its data.
+const db = new PGlite(dbFilePath); // Using the file path directly persists data


logic: db instance is created at module scope but used across async functions. Should be initialized after ensuring directory exists and handled more carefully for connection lifecycle management

greptile-apps · 2025-05-22T02:53:01Z

vector-db-setup.js

+            `SELECT id, chunk_text, document_id, metadata, embedding <-> $1 AS distance
+             FROM text_chunks
+             ORDER BY embedding <-> $1
+             LIMIT $2;`,
+            [queryEmbeddingString, k]


style: Duplicate embedding <-> $1 in ORDER BY and SELECT may impact performance. Consider using a CTE or subquery to compute distance once

greptile-apps · 2025-05-22T02:53:02Z

vector-db-setup.js

+module.exports = {
+    initializeDatabase,
+    addRecord,
+    findSimilarRecords,
+    deleteRecordsByDocumentId,
+    // Export `db` instance if needed directly by other modules, though encapsulating is better.
+    // dbInstance: db
+};


style: Module exports a singleton db instance indirectly through functions. Consider exporting a class or factory function instead for better instance management

Suggested change

module.exports = {

initializeDatabase,

addRecord,

findSimilarRecords,

deleteRecordsByDocumentId,

// Export `db` instance if needed directly by other modules, though encapsulating is better.

// dbInstance: db

};

class VectorDatabase {

constructor(dbPath) {

this.db = new PGlite(dbPath);

}

async initialize() {

return initializeDatabase(this.db);

}

async addRecord(id, chunkText, embedding, documentId, metadata) {

return addRecord(this.db, id, chunkText, embedding, documentId, metadata);

}

async findSimilarRecords(queryEmbedding, k) {

return findSimilarRecords(this.db, queryEmbedding, k);

}

async deleteRecordsByDocumentId(documentId) {

return deleteRecordsByDocumentId(this.db, documentId);

}

async close() {

return this.db.close();

}

}

module.exports = VectorDatabase;

arvinxx · 2025-05-27T10:35:56Z

vector-db-setup.js

actually we already have the pglite in the desktop mode, so there is no need to init another pglite db?

arvinxx · 2025-05-27T10:36:41Z

apps/desktop/src/main/controllers/RemoteServerSyncCtr.ts

@@ -7,6 +7,7 @@ import http, { IncomingMessage, OutgoingHttpHeaders } from 'node:http';
 import https from 'node:https';
 import { URL } from 'node:url';

+import { LocalRagSrv } from '@/main/services/LocalRagSrv'; // Added for Local RAG


@/services/LocalRagSrv

arvinxx · 2025-05-27T10:42:11Z

apps/desktop/src/main/controllers/RemoteServerSyncCtr.ts

+      logger.info(`${logPrefix} Local RAG mode is active.`);
+
+      // Route: chunk.semanticSearch
+      if (procedurePath === 'chunk.semanticSearch' && args.method === 'POST') {


I think is very fragile to route the semanticSearch or file chunking with this mode. A better way to handle it is in the request endpoint: https://github.com/lobehub/lobe-chat/blob/main/src/services/rag.ts#L21-L23

it will like this: https://github.com/lobehub/lobe-chat/blob/main/src/services/upload.ts#L40-L43

arvinxx · 2025-05-27T10:44:02Z

apps/desktop/src/types/settings.ts

it seems no need to have other user settings?

arvinxx · 2025-05-27T10:44:29Z

apps/desktop/src/types/store.ts

What's the file usage?

arvinxx · 2025-05-27T10:44:53Z

src/store/global/initialState.ts

revert this update

arvinxx · 2025-05-27T10:46:16Z

src/store/global/slices/settings/actions/knowledgeBase.ts

not good to get config from this settings. please consider use useElectronStore. we can get settings state from useInitElectronAppState https://github.com/lobehub/lobe-chat/blob/main/src/store/electron/actions/app.ts#L36-L58

arvinxx · 2025-05-27T14:17:10Z

src/app/settings_features_KnowledgeBase/index.tsx

it's not accessable as the right page.

arvinxx · 2025-05-27T14:19:23Z

Thanks for the contribution. I think the local RAG mode is an interesting idea, but crrently the PR quality is not match our standard. Please read our codebase and update the implement.

If you need any help, just leave a comment or dm me in the discord (id: arvinxu)

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label May 22, 2025

dosubot bot added the 🌠 Feature Request New feature or request | 特性与建议 label May 22, 2025

greptile-apps bot reviewed May 22, 2025

View reviewed changes

arvinxx requested changes May 27, 2025

View reviewed changes

	Your input is crucial for making this feature robust and user-friendly.Okay, I have created the markdown documentation for the Client-Mode RAG feature. It covers the introduction, enabling the feature, managing files, and importantly, the current limitations regarding placeholder embeddings and future enhancements.
	Your input is crucial for making this feature robust and user-friendly.

-module.exports = {
-    initializeDatabase,
-    addRecord,
-    findSimilarRecords,
-    deleteRecordsByDocumentId,
-    // Export `db` instance if needed directly by other modules, though encapsulating is better.
-    // dbInstance: db
-};
+class VectorDatabase {
+    constructor(dbPath) {
+        this.db = new PGlite(dbPath);
+    }
+    async initialize() {
+        return initializeDatabase(this.db);
+    }
+    async addRecord(id, chunkText, embedding, documentId, metadata) {
+        return addRecord(this.db, id, chunkText, embedding, documentId, metadata);
+    }
+    async findSimilarRecords(queryEmbedding, k) {
+        return findSimilarRecords(this.db, queryEmbedding, k);
+    }
+    async deleteRecordsByDocumentId(documentId) {
+        return deleteRecordsByDocumentId(this.db, documentId);
+    }
+    async close() {
+        return this.db.close();
+    }
+}
+module.exports = VectorDatabase;

Uh oh!

feat: Implement framework for Client-Mode RAG #7924

Are you sure you want to change the base?

feat: Implement framework for Client-Mode RAG #7924

Conversation

wangqingkeyi commented May 22, 2025

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

Uh oh!

vercel bot commented May 22, 2025

Uh oh!

lobehubbot commented May 22, 2025

Uh oh!

gru-agent bot commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TestGru Assignment

Summary

Files

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

PR Summary

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot May 22, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arvinxx commented May 27, 2025

Uh oh!

Uh oh!

gru-agent bot commented May 22, 2025 •

edited

Loading