Commit Graph

1338 Commits

Author SHA1 Message Date
Henry f73a305584 refactor: Implement SecureZodSchemaParser for safe Zod schema handling and add FilterParser for Supabase filters
* Replaced direct Zod schema evaluation with SecureZodSchemaParser in StructuredOutputParserAdvanced and CustomTool.
* Introduced FilterParser to safely handle Supabase filter strings, preventing arbitrary code execution.
* Added new filterParser.ts file to encapsulate filter parsing logic.
* Updated Supabase vector store to utilize the new FilterParser for RPC filters.
* Created secureZodParser.ts for secure parsing of Zod schemas.
2025-07-18 19:06:48 +01:00
Amr Aly d584c0b700
feat: Add header-based splitting to MarkdownTextSplitter (#4861)
* feat: Add header-based splitting to MarkdownTextSplitter

  - Add dropdown for header level selection (H1-H6)
  - Implement hierarchical splitting (H2 includes H1 headers)
  - Headers preserved with content sections
  - Prioritize semantic boundaries over chunk size

* Update MarkdownTextSplitter.ts

* Update MarkdownTextSplitter.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-18 12:45:41 +01:00
Kang Nahoon 2605a1f74e
fix: normalize Milvus similarity scores for threshold filtering (#4880)
* fix: normalize Milvus similarity scores for threshold filtering

* refactor: refact Milvus similarity score normalization #4879

---------

Co-authored-by: 강나훈 <nahoon.kang@mnc.ai>
2025-07-18 12:35:19 +01:00
Karl Stoney 5e5b2a18e2
Added region support to chatGoogleVertexAi (#4839)
* Added region support to chatGoogleVertexAi

* Added region to the vertex ai embeddings loader too

* Updated the available vertex text embedding models to be valid

* Update ChatGoogleVertexAI.ts

* Update GoogleVertexAIEmbedding.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-18 11:28:23 +01:00
Henry Heng 1bed5a264e
Bugfix/empty vars (#4892)
* update workspaceId to vars

* fix vars and duplicated response
2025-07-17 18:32:09 +01:00
Henry Heng 791c1e3274
Feat/support image for grok 4 (#4869)
support image for grok 4
2025-07-14 15:14:47 +01:00
Rick Ryan aea2801b8c
fix(components): promise.all failed silently when loading MCP tools (#4750)
* fix(components): promise.all failed silently when loading MCP tools

* fix: satisfy linter format

* fix(packages): pnpm lint-fix

* Update core.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-09 10:18:18 +01:00
Rostyslav Borovyk a25c5c4514
Add Oxylabs Document Loader (#4625)
* Add Oxylabs Document Loader

* Update Oxylabs.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-08 16:04:46 +01:00
Yuki 768de6140c
feat: add thinking budget parameter support for ChatGoogleVertexAI (#4557)
* chore: update @langchain/google-vertexai add thinkingBudget parameter support

* Update pnpm-lock.yaml

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-02 20:53:06 +01:00
Abdullatif 0627693133
adding performance config option to the AWS ChatBedrock component (#4777)
* adding performance config option to the AWS ChatBedrock component

* Update AWSChatBedrock.ts

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-07-02 19:15:39 +01:00
Henry Heng bbf6970600
Bugfix/Add proper parsing for tool input args (#4780)
add proper parsing for tool input args
2025-07-02 17:28:10 +01:00
Henry Heng 9b60cf1234
Bugfix/Remove Markdown in Condition (#4761)
Feature/Add remove-markdown dependency and integrate into Condition node for text processing

- Added `remove-markdown` package to `packages/components/package.json`.
- Integrated `removeMarkdown` function in `Condition.ts` to sanitize string inputs by removing markdown formatting.
2025-06-30 12:06:23 +01:00
Henry Heng 4c3b729b79
Chore/Google GenAI (#4742)
* update @langchain/core, custom google genai implementation

* update @langchain/core, custom google genai implementation
2025-06-27 00:44:11 +01:00
Henry Heng 9efb70e04c
Feature/Safety settings to google genai (#4737)
* add safety settings to google genai

* add safety settings to google genai
2025-06-26 12:54:47 +01:00
Henry Heng c78b5326b6
Bugfix/Add orgId for file fetching from messages (#4732)
add orgId for file fetching from messages
2025-06-25 17:28:14 +01:00
Henry Heng 4038eb13fc
Bugfix/GoogleDrive Loader Folder id (#4714)
* fix google drive loader folder id

* lint fix
2025-06-24 09:53:42 +01:00
Henry Heng 2e42dfb635
Bugfix/Gemini Structured Output (#4713)
* fix gemini structured output

* update issues templates
2025-06-23 19:10:41 +01:00
Henry Heng 543800562e
Chore/API for AgentflowV2 (#4696)
* Enhancement: Introduce prepended chat history handling in Agent and LLM nodes.

- Added support for `prependedChatHistory` in both `Agent` and `LLM` classes to allow for initial message context.
- Implemented validation for history schema in execution flow to ensure proper format.
- Refactored utility functions to include JSON sanitization and validation methods for improved data handling.

* update prediction swagger
2025-06-22 13:16:35 +01:00
Henry Heng 035b5555a9
Bugfix/HTTP Node Body Parsing (#4694)
Enhancement: Add JSON sanitization and parsing methods to HTTP agentflow.

- Introduced `sanitizeJsonString` to clean problematic escape sequences and fix common JSON formatting issues.
- Added `parseJsonBody` to attempt parsing JSON directly, with fallback to sanitization if parsing fails, providing clearer error messages for invalid JSON.
- Updated request handling to utilize the new parsing method for JSON and x-www-form-urlencoded body types.
2025-06-20 12:29:55 +01:00
Henry Heng a107aa7a77
Chore/Update issue templates and add new tools (#4687)
* Enhancement: Update issue templates and add new tools

- Updated bug report template to include a default label of 'bug'.
- Updated feature request template to include a default label of 'enhancement'.
- Added new credential class for Agentflow API.
- Enhanced Agent and HTTP nodes to improve tool management and error handling.
- Added deprecation badges to several agent and chain classes.
- Introduced new tools for handling requests (GET, POST, DELETE, PUT) with improved error handling.
- Added new chatflows and agentflows for various use cases, including document QnA and translation.
- Updated UI components for better handling of agent flows and marketplace interactions.
- Refactored utility functions for improved functionality and clarity.

* Refactor: Remove beta badge and streamline template title assignment

- Removed the 'BETA' badge from the ExtractMetadataRetriever class.
- Simplified the title assignment in the agentflowv2 generator by using a variable instead of inline string manipulation.
2025-06-19 18:11:24 +01:00
toi500 3d2c5c90e9
Fix: Correct formatting of Condition Agent prompt (#4626) 2025-06-11 14:07:02 +01:00
toi500 dfb401ad83
feat: Add configurable system prompt to Condition Agent (#4587)
* feat: Add configurable system prompt to Condition Agent

* Update system prompt to HTML for UI readability

* fix: Remove invalid default routing and sync hardcoded role-based examples

* Update ConditionAgent.ts

* Update ConditionAgent.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-06-10 18:38:02 +01:00
Henry Heng 2387a06ce4
Feature/GoogleDocs (#4613)
* add teams, gmail, outlook tools

* update docs link

* update credentials for oauth2

* add jira tool

* add google drive, google calendar, google sheets tools, powerpoint, excel, word doc loader

* update jira logo

* Refactor Gmail and Outlook tools to remove maxOutputLength parameter and enhance request handling. Update response formatting to include parameters in the output. Adjust Google Drive tools to simplify success messages by removing unnecessary parameter details.

* Update pnpm-lock.yaml

* add google docs
2025-06-09 00:30:03 +01:00
Henry Heng 6495c64dac
Feat/add headers to custom mcp (#4581)
* add headers to custom mcp

* Refactor MCP fetch method and update CustomMCP documentation to include variable usage in headers
2025-06-07 16:23:42 +01:00
Henry Heng 02a6753498
Bugfix/Overrideconfig Analytics in AgentflowV2 (#4599)
* fix(utils/buildAgentflow): enhance analytic handler configuration to support overrides

* fix(agentflow): integrate analytic handlers for tool execution tracking
2025-06-07 01:53:31 +01:00
Ahmed Rowaihi 7dfa269502
fix(components/weaviate): fix metadata special chars upsertion failure (#4436)
* Enhancement: Add recursive key normalization for metadata in Weaviate vector store

- Introduced `normalizeKeysRecursively` utility to standardize metadata keys.
- Updated Weaviate vector store to apply normalization on document metadata before processing.

* format(compnonents/utils): format for ci

* Update utils.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-06-07 00:19:02 +01:00
Ademílson Tonato 0c5f7ea003
feat: add search functionality to FireCrawl with customizable parameters (#4535)
* feat: add search functionality to FireCrawl with customizable parameters

* refactor: unify request parameters in FireCrawl to include integration identifier

* Update FireCrawl Document Loader to version 4.0, enhancing parameter labels and adding conditional visibility for URL and crawler options based on selected crawler type.

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2025-06-07 00:06:39 +01:00
Henry Heng 30c4180d97
Feature/Add teams, gmail, outlook tools (#4577)
* add teams, gmail, outlook tools

* update docs link

* update credentials for oauth2

* add jira tool

* add google drive, google calendar, google sheets tools, powerpoint, excel, word doc loader

* update jira logo

* Refactor Gmail and Outlook tools to remove maxOutputLength parameter and enhance request handling. Update response formatting to include parameters in the output. Adjust Google Drive tools to simplify success messages by removing unnecessary parameter details.
2025-06-06 19:52:04 +01:00
Ong Chung Yau 5f7f83a5d2
feature/apiloader-ca-upload (#4533)
feat: add CA(.pem, .crt) upload in API Loader
2025-06-05 19:29:52 +01:00
Henry Heng d134b66bd8
Bugfix/tool node selected tool (#4579)
* fix tool node selected tool

* fix tool node config

* update node to display tool node icons
2025-06-04 17:54:36 +01:00
Henry Heng aa0984e802
Bugfix/Tool node selected tool (#4578)
fix tool node selected tool
2025-06-04 13:44:09 +01:00
Henry Heng a88337cc83
Bugfix/config input for multiple same fields (#4548)
* fix config input for multiple same fields

* fix custom tool not selected
2025-05-31 17:03:03 +01:00
Henry Heng 954e6c88f4
Feat/add supergateway MCP (#4303)
* add supergateway MCP

* update supergateway mcp
2025-05-28 17:21:01 +01:00
Saket Aryan 9682a0ccd9
feat:Adds new Param in getChatMessages for better context handling (#4273)
* Adds Current Message Param in getChatMessages to use Mem0 more effectively

* Revert "Adds Current Message Param in getChatMessages to use Mem0 more effectively"

This reverts commit 086be60b0a.

* Used the input param for Getting the Input text inside of Mem0 Node

* Update pnpm-lock.yaml

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-28 16:02:52 +01:00
Henry Heng da8d0f12d6
Bugfix/Disable default user input if memory is disabled for first node (#4530)
disable default user input if memory is disabled for first node
2025-05-28 15:04:45 +01:00
Henry Heng 3d6bf72e73
Feature/Custom MCP vars (#4527)
* add input vars to custom mcp

* add ability to specify vars in custom mcp, fix other ui issues

* update setup org ui
2025-05-28 12:47:53 +01:00
nikitas-novatix 4326cbe6b5
Fix: Patching redis socket crash fix (#4431)
* redis keepalive mechanism for all redis

* removed offline queue commands

* Simplified changes for consistency. Added REDIS_KEEP_ALIVE env variable.

* update redis socket alive env variable

* lint fix

* put pingInterval back again because it's needed

* removed comment

* linting

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-27 19:54:17 +01:00
nikitas-novatix 7e7ff24941
BugFix/Patch: PostGreSQL Record Manager (#4430)
* removed the pgcrypto extension creation (Azure bug)

* changing "now" to "extract"
2025-05-27 19:54:00 +01:00
Henry Heng d06b7d7aef
Bugfix/Google Gemini Image Input (#4515)
always allow image input for all gemini llms
2025-05-27 18:48:43 +01:00
Henry Heng 01dab4365a
Bugfix/AgentflowV2 State (#4512)
* add persistence state, http node variables, custom function flow state

* update marketplace templates
2025-05-27 18:01:39 +01:00
Ademílson Tonato 572fb31a1c
chore: update Firecrawl version and add FirecrawlExtractTool (#4073)
* chore: update Firecrawl version and add FirecrawlExtractTool

* refactor: update outputs format

* chore: update Firecrawl request headers to include X-Origin and X-Origin-Type

* feat: add FireCrawl testing suite for scraping, crawling, and data extraction

- Introduced FireCrawl-TEST.ts to validate FireCrawlLoader functionality.
- Implemented tests for basic scraping, crawling with text splitting, data extraction, and extract status retrieval.
- Enhanced error handling in FireCrawlLoader for better debugging.

* Update pnpm-lock.yaml

* refactor: FireCrawl API integration to improve parameter handling and error logging

* refractor firecrawl

* Update FireCrawl.ts

removed console log

* Update pnpm-lock.yaml

* Update pnpm-lock.yaml

---------

Co-authored-by: Ong Chung Yau <33013947+chungyau97@users.noreply.github.com>
Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-27 14:58:35 +01:00
korade-krushna eca190dca6
[ BUG #4498] Use Correct Key For Basic Auth and Optional Password (#4501)
Use Correct Key For Basic Auth and Optional pass

Co-authored-by: Krishna Korade <kkorde@aistrike.com>
2025-05-27 14:36:20 +01:00
Henry Heng 5a37227d14
Chore/refractor (#4454)
* markdown files and env examples cleanup

* components update

* update jsonlines description

* server refractor

* update telemetry

* add execute custom node

* add ui refractor

* add username and password authenticate

* correctly retrieve past images in agentflowv2

* disable e2e temporarily

* add existing username and password authenticate

* update migration to default workspace

* update todo

* blob storage migrating

* throw error on agent tool call error

* add missing execution import

* add referral

* chore: add error message when importData is undefined

* migrate api keys to db

* fix: data too long for column executionData

* migrate api keys from json to db at init

* add info on account setup

* update docstore missing fields

---------

Co-authored-by: chungyau97 <chungyau97@gmail.com>
2025-05-27 07:29:42 +01:00
Henry Heng 86782e9971
Bugfix/throw error on agent tool call error (#4464)
throw error on agent tool call error
2025-05-20 14:47:21 +08:00
Henry Heng 6cf1c82f04
Bugfix/correctly retrieve past images in agentflowv2 (#4453)
correctly retrieve past images in agentflowv2
2025-05-18 18:16:17 +08:00
k-santos 9170cac58b
bugfix: change field to be optional (#4446)
fix: make customModelName field as optional
2025-05-18 18:16:08 +08:00
Henry Heng 82d16458e4
Bugfix/Update file loader (#4420)
* update file loader

* delete temp for o1

* fix tavily
2025-05-14 10:30:33 +08:00
nikitas-novatix eadf1b11b3
Fix: Add keep-alive options to Redis clients to prevent idle timeouts and socket closing. (#4377)
* redis keepalive mechanism for all redis

* removed offline queue commands

* Simplified changes for consistency. Added REDIS_KEEP_ALIVE env variable.

* update redis socket alive env variable

* lint fix

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-14 10:29:32 +08:00
Henry Heng 98e75ad7d6
Bugfix/ui streaming when model streaming is off (#4424)
fix ui streaming when model streaming is off
2025-05-14 00:07:54 +08:00
Henry Heng 7924fbce0d
Feature/agentflow v2 (#4298)
* agent flow v2

* chat message background

* conditon agent flow

* add sticky note

* update human input dynamic prompt

* add HTTP node

* add default tool icon

* fix export duplicate agentflow v2

* add agentflow v2 marketplaces

* refractor memoization, add iteration nodes

* add agentflow v2 templates

* add agentflow generator

* add migration scripts for mysql, mariadb, posrgres and fix date filters for executions

* update agentflow chat history config

* fix get all flows error after deletion and rename

* add previous nodes from parent node

* update generator prompt

* update run time state when using iteration nodes

* prevent looping connection, prevent duplication of start node, add executeflow node, add nodes agentflow, chat history variable

* update embed

* convert form input to string

* bump openai version

* add react rewards

* add prompt generator to prediction queue

* add array schema to overrideconfig

* UI touchup

* update embedded chat version

* fix node info dialog

* update start node and loop default iteration

* update UI fixes for agentflow v2

* fix async drop down

* add export import to agentflowsv2, executions, fix UI bugs

* add default empty object to flowlisttable

* add ability to share trace link publicly, allow MCP tool use for Agent and Assistant

* add runtime message length to variable, display conditions on UI

* fix array validation

* add ability to add knowledge from vector store and embeddings for agent

* add agent tool require human input

* add ephemeral memory to start node

* update agent flow node to show vs and embeddings icons

* feat: add import chat data functionality for AgentFlowV2

* feat: set chatMessage.executionId to null if not found in import JSON file or database

* fix: MariaDB execution migration script to utf8mb4_unicode_520_ci

---------

Co-authored-by: Ong Chung Yau <33013947+chungyau97@users.noreply.github.com>
Co-authored-by: chungyau97 <chungyau97@gmail.com>
2025-05-10 10:21:26 +08:00