- Introduced a new `doc_id` column in MySQL, Postgres, and SQLite record managers to support document identification.
- Updated the `update` method to handle both string and object formats for keys, allowing for better flexibility in document updates.
- Enhanced `listKeys` method to filter by `doc_id` when provided in options.
- Updated vector store integrations to utilize the new `doc_id` filtering capability
* feat: Require workspace ID for API key operations
- Added validation to ensure `activeWorkspaceId` is present in user requests for all API key operations (get, create, update, import, delete).
- Updated `getWorkspaceSearchOptions` and `getWorkspaceSearchOptionsFromReq` to throw an error if `workspaceId` is not provided.
- Modified service methods to enforce `workspaceId` as a required parameter for database operations related to API keys.
* feat: Enforce workspace ID as a required field across multiple interfaces and services
- Updated various interfaces to make `workspaceId` a mandatory field instead of optional.
- Enhanced assistant and export-import service methods to require `workspaceId` for operations, ensuring proper validation and error handling.
- Modified database entity definitions to reflect the change in `workspaceId` from optional to required.
- Improved error handling in controllers to check for `activeWorkspaceId` before proceeding with requests.
* Require workspace ID in various controllers and services
- Updated controllers for credentials, datasets, document stores, evaluations, evaluators, and variables to enforce the presence of `workspaceId`.
- Enhanced error handling to throw appropriate errors when `workspaceId` is not provided.
- Modified service methods to accept `workspaceId` as a mandatory parameter for operations, ensuring consistent validation across the application.
* Update EvaluatorRunner and index to require workspaceId for evaluator retrieval
- Modified the runAdditionalEvaluators function to accept workspaceId as a parameter.
* lint fixes
* Enhancement/Integrate workspaceId in chatflow and flow-config services
- Updated chatflow and flow-config controllers to require workspaceId for fetching chatflows.
- Modified service methods to accept workspaceId as a parameter, ensuring proper context for chatflow retrieval.
* lint fix
* get rid of redundant isApiKeyValidated
* refactor: update permission checks for chatflows and agentflows routes
- Enhanced permission checks in chatflows routes to include agentflows permissions for create, read, update, and delete operations.
- Updated navigation paths in authentication views to redirect to the home page instead of chatflows after successful login or registration.
* fix(DefaultRedirect.jsx): add redirect unauthenticated users to login
* fix(RequireAuth.jsx): check permissions for routes without display property
* fix(DefaultRedirect.jsx): WorkspaceSwitcher api spam
* fix(routes/chatflows/index.ts): use checkAnyPermission for chatflow/has-changed/:id/:lastUpdatedDateTime
* fix(routes/chatflows/index.ts): use checkAnyPermission for delete request chatflow/:id
* fix(controllers/text-to-speech/index.ts): add workspace ID validation in generateTextToSpeech
* fix(controllers/internal-predictions/index.ts): add chatflow retrieval and validation using workspaceId
* feat(services\credentials\index.ts): add filter by workspaceId for getCredentialById
* chore(routes/chat-messages/index.ts): unused chat-messages route
* feat(services/chatflows/index.ts): add filter by workspaceId for deleteChatflow
* feat(services/marketplaces/index.ts): add filter by workspaceId for deleteCustomTemplate
* feat(tools): add filter by workspaceId for read, update, and delete
---------
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
Co-authored-by: Yau <33013947+chungyau97@users.noreply.github.com>
Co-authored-by: chungyau97 <chungyau97@gmail.com>
* fix upser api for file loader
* - Introduced loaderName property in IDocumentStoreUpsertData interface.
- Updated upsertDocStore function to utilize loaderName if provided.
- Enhanced DocStoreAPIDialog to allow loaderName customization in API requests.
- Modified DocumentStoreDetail to display file names when available, improving source formatting logic.
* common pagination component
* Pagination for Doc Store Dashboard
* Pagination for Executions Dashboard
* Pagination Support for Tables
* lint fixes
* update view message dialog UI
* initial loading was ignoring the pagination counts
* 1) default page size change
2) ensure page limits are passed on load
3) co-pilot review comments (n+1 query)
4)
* 1) default page size change
2) ensure page limits are passed on load
3) co-pilot review comments (n+1 query)
4) refresh lists after insert/delete.
* Enhancement: Improve handling of empty responses in DocumentStore and API key services
- Added check for empty entities in DocumentStoreDTO.fromEntities to return an empty array.
- Updated condition in getAllDocumentStores to handle total count correctly, allowing for zero total.
- Refined logic in getAllApiKeys to check for empty keys and ensure correct API key retrieval.
- Adjusted UI components to safely handle potential undefined apiKeys array.
* Refresh API key list on pagination change
* Enhancement: Update pagination and filter handling across components
- Increased default items per page in AgentExecutions from 10 to 12.
- Improved JSON parsing for chat type and feedback type filters in ViewMessagesDialog.
- Enhanced execution filtering logic in AgentExecutions to ensure proper pagination and state management.
- Refactored filter section in AgentExecutions for better readability and functionality.
- Updated refresh logic in Agentflows to use the correct agentflow version.
* add workspaceId to removeAllChatMessages
* Refactor chat message retrieval logic for improved efficiency and maintainability
- Introduced a new `handleFeedbackQuery` function to streamline feedback-related queries.
- Enhanced pagination handling for session-based queries in `getMessagesWithFeedback`.
- Updated `ViewMessagesDialog` to sort messages in descending order by default.
- Simplified image rendering logic in `DocumentStoreTable` for better readability.
* - Update `validateChatflowAPIKey` and `validateAPIKey` functions to get the correct keys array
- Enhanced error handling in the `sanitizeExecution` function to ensure safe access to nested properties
* Refactor API key validation logic for improved accuracy and error handling
- Consolidated API key validation in `validateAPIKey` to return detailed validation results.
- Updated `validateFlowAPIKey` to streamline flow API key validation.
- Introduced `getApiKeyById` function in the API key service for better key retrieval.
- Removed unused function `getAllChatSessionsFromChatflow` from the chat message API.
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* Enhancement: Update issue templates and add new tools
- Updated bug report template to include a default label of 'bug'.
- Updated feature request template to include a default label of 'enhancement'.
- Added new credential class for Agentflow API.
- Enhanced Agent and HTTP nodes to improve tool management and error handling.
- Added deprecation badges to several agent and chain classes.
- Introduced new tools for handling requests (GET, POST, DELETE, PUT) with improved error handling.
- Added new chatflows and agentflows for various use cases, including document QnA and translation.
- Updated UI components for better handling of agent flows and marketplace interactions.
- Refactored utility functions for improved functionality and clarity.
* Refactor: Remove beta badge and streamline template title assignment
- Removed the 'BETA' badge from the ExtractMetadataRetriever class.
- Simplified the title assignment in the agentflowv2 generator by using a variable instead of inline string manipulation.
* Chatflow config - rate limit and override config were overwriting changes.
* prevent post processing JS function from escaping as JSON...
* non-streaming follow up prompts need to be escaped twice
* prevent post processing JS function from escaping as JSON...
* Adding file mimetypes for full file upload...
* lint..
* fixing the issue with storing only filtered nodes..
* return doc store processing response without await when queue mode and request is from UI
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* adding support for prometheus and grafana
* open telemetry
* lint fixes
* missing counter and telemetry standardization
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* add functionality for full file uploads, add remove messages from view dialog and API
* add attachments swagger
* update question to include uploadedFilesContent
* make config dialog modal lg size
* add export all function
* modify exportAll to reuse existing code from other services
* modify routes of export-import
* add exportAll function into UI
* add errorhandler
* add importAll Function into UI for ChatFlow
* modify importAll Function to import tools
* remove appServer variable
* modify exportAll to exportData for new requirement in backend
* chore modify type camelCase to PascalCase in exportImportService
* add import export for variables, assistants, and checkboxes for UI
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* Document Store - Phase 2
* Adding additional columns for vector store config, document store phase 2
* Adding additional columns for vector store config, document store phase 2
* Document Store - Phase 2 - Upsert and Query
* ux cleanup
* retrieval settings and more ux changes
* adding MMR params to execution
* Making the upsert process async.
* add upsert history changes
* making the searchParams dynamic
* removing unnecessary params
* add ability to delete data from vector store
* update margin for vector store query
* adding option to save config in the retrieval playground
* adding chunk number for query return chunks
* Adding a Document Store node in the VectorStore category
* update doc store status, ui touchup
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* datasource: initial commit
* datasource: datasource details and chunks
* datasource: Document Store Node
* more changes
* Document Store - Base functionality
* Document Store Loader Component
* Document Store Loader Component
* before merging the modularity PR
* after merging the modularity PR
* preview mode
* initial draft PR
* fixes
* minor updates and fixes
* preview with loader and splitter
* preview with credential
* show stored chunks
* preview update...
* edit config
* save, preview and other changes
* save, preview and other changes
* save, process and other changes
* save, process and other changes
* alpha1 - for internal testing
* rerouting urls
* bug fix on new leader create
* pagination support for chunks
* delete document store
* Update pnpm-lock.yaml
* doc store card view
* Update store files to use updated storage functions, Document Store Table View and other changes
* ui changes
* add expanded chunk dialog, improve ui
* change throw Error to InternalError
* Bug Fixes and removal of subFolder, adding of view chunks for store
* lint fixes
* merge changes
* DocumentStoreStatus component
* ui changes for doc store
* add remove metadata key field, add custom document loader
* add chatflows used doc store chips
* add types/interfaces to DocumentStore Services
* document loader list dialog title bar color change
* update interfaces
* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.
* use typeorm order chunkNo, ui changes
---------
Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Henry
Henry Heng
Vinod Kiran
Sunny Modi
Ong Chung Yau
Octavian FlowiseAI
Jared McQueen