Commit Graph

239 Commits

Author SHA1 Message Date
Henry Heng 346a55b6d8
Chore/remove teradatasql (#5430)
remove teradatasql
2025-11-06 22:21:10 +00:00
Lê Nam Khánh 1fa9303d7c
chore: fix typos in packages/components/nodes/vectorstores/Meilisearch/Meilisearch.ts (#5417)
Fix typos in packages/components/nodes/vectorstores/Meilisearch/Meilisearch.ts
2025-11-05 13:01:00 +00:00
Henry Heng 601de76aea
Bugfix/Json parse metadata filter (#5392)
json parse metadata filter
2025-10-30 13:04:29 +00:00
Håvard Homb 1ae1638ed9
feat: Added acceptVariable to Qdrant Collection name (#5373)
Added acceptVariable to Qdrant Collection name
2025-10-28 13:24:24 +00:00
Shashank Shekhar ac565b8981
Adding Teradata VectorStore support in DocumentStore (#5159)
* Adding Teradata VectorStore support in DocumentStore

* Minor update

* Updated icon

* Updated input fields descriptions

* Added teradatasql dependency in components package.json

* Updated code based on lint-fix command

* Updated pnpm-lock.yaml file

* revert changes on otel compose.yml

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2025-10-28 12:23:59 +00:00
Henry Heng 6885c38d18
Chore/Accept Dynamic Variable From Metadata Filter (#5203)
* add acceptVariable property to vector store metadata filter

* add accepVariadd acceptVariable property to other json fields
2025-09-13 11:28:53 +01:00
anatolii burtsev af1464f7c2
Feat/aws kendra vector search (#5088)
* Add AWS Kendra vector store integration

* Fix import paths in Kendra vector store

* Add AWS Kendra dependencies to package.json

* Update pnpm-lock.yaml with AWS Kendra dependencies

* Fix linter warnings in Kendra vector store

* Fix code formatting in Kendra vector store

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2025-08-20 00:38:59 +08:00
Philip I. Thomas 8562d4a563
Fix Chroma authentication (#4936)
Fix chroma authentication
2025-07-24 20:15:03 +01:00
Henry Heng 96a57a58e7
Refractor/SecureZodSchemaParser (#4898)
* refactor: Implement SecureZodSchemaParser for safe Zod schema handling and add FilterParser for Supabase filters

* Replaced direct Zod schema evaluation with SecureZodSchemaParser in StructuredOutputParserAdvanced and CustomTool.
* Introduced FilterParser to safely handle Supabase filter strings, preventing arbitrary code execution.
* Added new filterParser.ts file to encapsulate filter parsing logic.
* Updated Supabase vector store to utilize the new FilterParser for RPC filters.
* Created secureZodParser.ts for secure parsing of Zod schemas.

* remove console log
2025-07-19 01:11:31 +01:00
Kang Nahoon 2605a1f74e
fix: normalize Milvus similarity scores for threshold filtering (#4880)
* fix: normalize Milvus similarity scores for threshold filtering

* refactor: refact Milvus similarity score normalization #4879

---------

Co-authored-by: 강나훈 <nahoon.kang@mnc.ai>
2025-07-18 12:35:19 +01:00
Ahmed Rowaihi 7dfa269502
fix(components/weaviate): fix metadata special chars upsertion failure (#4436)
* Enhancement: Add recursive key normalization for metadata in Weaviate vector store

- Introduced `normalizeKeysRecursively` utility to standardize metadata keys.
- Updated Weaviate vector store to apply normalization on document metadata before processing.

* format(compnonents/utils): format for ci

* Update utils.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-06-07 00:19:02 +01:00
nikitas-novatix 4326cbe6b5
Fix: Patching redis socket crash fix (#4431)
* redis keepalive mechanism for all redis

* removed offline queue commands

* Simplified changes for consistency. Added REDIS_KEEP_ALIVE env variable.

* update redis socket alive env variable

* lint fix

* put pingInterval back again because it's needed

* removed comment

* linting

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-27 19:54:17 +01:00
Henry Heng 5a37227d14
Chore/refractor (#4454)
* markdown files and env examples cleanup

* components update

* update jsonlines description

* server refractor

* update telemetry

* add execute custom node

* add ui refractor

* add username and password authenticate

* correctly retrieve past images in agentflowv2

* disable e2e temporarily

* add existing username and password authenticate

* update migration to default workspace

* update todo

* blob storage migrating

* throw error on agent tool call error

* add missing execution import

* add referral

* chore: add error message when importData is undefined

* migrate api keys to db

* fix: data too long for column executionData

* migrate api keys from json to db at init

* add info on account setup

* update docstore missing fields

---------

Co-authored-by: chungyau97 <chungyau97@gmail.com>
2025-05-27 07:29:42 +01:00
nikitas-novatix eadf1b11b3
Fix: Add keep-alive options to Redis clients to prevent idle timeouts and socket closing. (#4377)
* redis keepalive mechanism for all redis

* removed offline queue commands

* Simplified changes for consistency. Added REDIS_KEEP_ALIVE env variable.

* update redis socket alive env variable

* lint fix

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2025-05-14 10:29:32 +08:00
Aman Agarwal 3bd2d63a19
[fix 4206]: fix the empty array return for getBaseClasses when using typeorm driver and not the PGVectorStore (#4225)
fix the empty array return for getBaseClasses when using typeorm driver and not the PGVectorStore

Co-authored-by: Aman Agarwal <aman@syook.com>
2025-04-03 01:47:14 +08:00
Gene Ruebsamen 829d2b1597
Weaviate - expose Hybrid Search (#4127)
Hybrid Search for Weaviate
2025-03-13 13:30:24 +00:00
Michael Young 4130156397
feat: Add SSL configuration option for Postgres connections (#4130)
Update README files for Postgres Record Manager and Vector Store to include a new SSL environment variable, allowing optional SSL configuration for database connections
2025-03-13 10:57:38 +00:00
Henry Heng 23ccef165d
Bugfix/Typeorm Postgres Driver Not Found (#4024)
add bugfix for driver not found
2025-02-12 19:19:52 +00:00
Henry Heng a0b4abdd13
Feature/update openai version, add reasoning effort param, add o3 mini (#3973)
* update openai version, add reasoning effort param

* update azure

* add filter for pinecone llamaindex

* update graph cypher qa chain
2025-02-04 08:43:27 +00:00
Henry Heng a2a475ba7a
Feature/Add bullmq redis for message queue processing (#3568)
* add bullmq redis for message queue processing

* Update pnpm-lock.yaml

* update queue manager

* remove singleton patterns, add redis to cache pool

* add bull board ui

* update rate limit handler

* update redis configuration

* Merge add rate limit redis prefix

* update rate limit queue events

* update preview loader to queue

* refractor namings to constants

* update env variable for queue

* update worker shutdown gracefully
2025-01-23 14:08:02 +00:00
itaismith 3fc4e79089
Add tenant and database configs for Chroma Cloud credentials (#3872)
* Add tenant and database configs for Chroma Cloud credentials

* Lint fix
2025-01-20 18:39:00 +00:00
Henry Heng 9a417bdc95
Bugfix/update nodevm sandbox options, sanitize tablename (#3818)
* update nodevm sandbox options, sanitize tablename

* sanitize file name when getFileFromStorage
2025-01-07 15:26:25 +00:00
Nguyễn Đức Hùng b34a82335d
bugfix: Add nil check in escapeAllStrings to prevent TypeError when upsert vectors (#3789) 2025-01-02 16:41:40 +00:00
Henry Heng ddca80d4e0
Bugfix/Prevent open connections on typeorm datasource (#3652)
prevent open connections on typeorm datasource
2024-12-08 21:06:07 +00:00
Eduard-Constantin Ibinceanu ba8a462bf8
[Bugfix] Milvus search error (#3623)
Fix milvus search error
2024-12-03 23:41:39 +00:00
Henry Heng 940c8fd3b0
Chore/LC v0.3 (#3517)
* bump langchain version to 0.3, upgrades on all chat models

* update all docs loader to have documents and text output options

* fix pnpm lock file
2024-11-28 11:06:12 +00:00
Vinod Kiran fe03683f0c
FEATURE:: Support for metrics collection - Prometheus and Open Telemetry (#3420)
* adding support for prometheus and grafana

* open telemetry

* lint fixes

* missing counter and telemetry standardization

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-11-08 00:00:43 +00:00
Henry Heng eeb1d17f50
Chore/MongoDB Connection (#3469)
getting rid of singleton design and properly close connection after every interaction
2024-11-07 19:05:00 +00:00
Jérémy JOURDIN 15d59a9052
[Prostgres Vector Store] Add PGVector Driver option + Fix null character issue w/ TypeORM Driver (#3367)
* Add PGVector Driver option + Fix null character issue w/ TypeORM Driver

* Handle file upload case with PGVector

* Cleanup

* Fix data filtering for chatflow uploaded files

* Add distanceStrategy parameter

* Fix query to improve chatflow uploaded files filtering

* Ensure PGVector release connections

* Await client connected

* Make Postgres credentials optionnal when set on env variables

* Document env variables in nodes directories

* Prevent reuse client

* Fix empty metadataFilter

* Update CONTRIBUTING.md

* Update Postgres.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-11-01 18:13:45 +00:00
Prajwal Pai 884dd8949a
Add couchbase vectore store support (#3093)
* Add couchbase vectore store support

* Uncomment couchbase code

* lint fix

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-10-26 02:36:38 +01:00
Anthony Bryan Gavilan Vinces 235fcfeb40
Enable Filters in MongoDB Vector Search Retriever (#3333)
* added: support to metadata filter using mongodb vector search retriever

* fix: format lint
2024-10-16 16:35:37 +01:00
Ortes d002096759
fix: Fix [object Object] sent to milvus on search (#3336) 2024-10-12 20:36:32 +01:00
Tyler 490855729b
Update placeholder port to match Postgres default (#3307) 2024-10-03 22:50:09 +01:00
Henry Heng 07059182af
Bugfix/implement missing delete method on pgvector (#3180)
implement missing delete method on pgvector
2024-09-12 02:28:45 +01:00
Eyalm321 7a5246d28a
added tls support to milvus (#3068)
* added tls support

* forgot to add tls to init function as well, it works now.

* added partition support for Milvus

* updated to 2.0.6 , pnpm lint fix aswell

* ensure it points to _default partition if none provided

* update milvus versioning

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-30 13:52:32 +01:00
Henry Heng 66acd0c000
Feature/add ability to upload file from chat (#3059)
add ability to upload file from chat
2024-08-25 13:22:48 +01:00
Mohamed Yasser Oaf 0f58d31493
Meilisearch - Added metadata as filterable attributes and polling on enqueued tasks (#3042)
* added polling for enqueued tasks,metadata filterable

* delete index feature with polling on task

* added search filter interface

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-23 16:27:07 +01:00
Mohamed Yasser Oaf d5153c3840
Added meilisearch retriever component (#2824)
* added meilisearch retriever and credentials.ts

* added semantic ratio

* removed a TODO implementatio

* meilisearch component implemented with searching and upsert functionality (#3)

meilisearch retriever component created , searching for an existing index and upserting a new or existing index has been implemented , component utilizes langchain and meilisearch vector search

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/3
Reviewed-by: mohamed1999akram <mohamed1999akram@gmail.com>

* added CI/CD for ntg branch, added proper dockerfile for flowise-ntg (#4)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/4
Reviewed-by: mohammad <mohammad@noreply.git.beyond.cc>

* modified os version , removed linting errors , removed cypress github actions (#5)

added --no-lock-file flag to pass CICD , made the runner run on debian and not ubuntu , removed code that caused warnings to pass linting

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/5
Reviewed-by: omaryassery <omarryassser@gmail.com>

* removed unnecessary QEMU install action (#6)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/6
Reviewed-by: omaryassery <omarryassser@gmail.com>

* removed cypress installation and linting from dockerfile (#7)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/7
Reviewed-by: isameh <isameh@ntgclarity.com>

* dockerfile-ntg-modification (#9)

dockerfile-ntg modified to copy all working directory before calling pnpm install
Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/9
Reviewed-by: isameh <isameh@ntgclarity.com>

* resolved comments, reverted CI/CD

* add test docker build yml back

* moved meilisearch to vector store folder

* Update Meilisearch.ts

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-18 12:23:45 +01:00
apeng-singlestore 36c6c6425c
Updated logo for singlestore (#3018) 2024-08-16 19:46:46 +01:00
Henry Heng 68769f4bbc
Feature/Add content and metadata key for qdrant (#2997)
add content and metadata key for qdrant
2024-08-11 21:00:48 +01:00
Vinod Kiran c0bae635b0
Document Store - Phase 2 (#2912)
* Document Store - Phase 2

* Adding additional columns for vector store config, document store phase 2

* Adding additional columns for vector store config, document store phase 2

* Document Store - Phase 2 - Upsert and Query

* ux cleanup

* retrieval settings and more ux changes

* adding MMR params to execution

* Making the upsert process async.

* add upsert history changes

* making the searchParams dynamic

* removing unnecessary params

* add ability to delete data from vector store

* update margin for vector store query

* adding option to save config in the retrieval playground

* adding chunk number for query return chunks

* Adding a Document Store node in the VectorStore category

* update doc store status, ui touchup

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-08-07 18:59:52 +01:00
Henry Heng ee8f9bb37c
Chore/Move @langchain/core dependency to components (#2945)
* move lc core dependency to components

* anchor @langchain/aws version

* update agentic RAG template
2024-08-06 14:41:21 +01:00
Henry Heng bca4de0c63
Feature/seq agents (#2798)
* update build functions

* sequential agents

* update langchain to 0.2, added sequential agent nodes

* add marketplace templates

* update howto wordings

* Merge branch 'main' into feature/Seq-Agents

# Conflicts:
#	pnpm-lock.yaml

* update deprecated functions and add new sequential nodes

* add marketplace templates

* update marketplace templates, add structured output to llm node

* add multi agents template

* update llm node with bindmodels

* update cypress version

* update templates sticky note wordings

* update tool node to include human in loop action

* update structured outputs error from models

* update cohere package to resolve google genai pipeThrough bug

* update mistral package version, added message reconstruction before invoke seq agent

* add HITL to agent

* update state messages restructuring

* update load and split methods for s3 directory
2024-07-22 17:46:14 +01:00
Neal Beeken 9e88c45051
feat: add driverInfo to mongodb component (#2779)
* feat: add driverInfo to mongodb component

NODE-6240

* chore: add a getVersion utility function
2024-07-15 12:24:00 +01:00
Henry Heng 363d1bfc44
Chore/update deprecating nodes (#2540)
* update deprecating nodes

* add filters use cases to marketplace

* update log level
2024-07-12 18:37:57 +01:00
Pavlo Paliychuk 1015e1193f
chore: Bump zep cloud sdk version and clean up zep cloud vector store node (#2767) 2024-07-12 18:01:00 +01:00
Henry Heng 109b0367cc
Bugfix/Pinecone Text Key (#2708)
* add source tag to pinecone

* update pinecone

* add text key to pinecone

* update pinecone version, and singleton
2024-06-24 02:08:30 +01:00
Henry Heng b662dd79c6
Feature/Add Text Key to Pinecone (#2681)
* add source tag to pinecone

* update pinecone

* add text key to pinecone

* update pinecone version, and singleton
2024-06-19 21:10:09 +01:00
Henry Heng b5ead0745b
Feature/Add filter to Postgres VectorStore (#2674)
* add filter to pgvs

* fix types
2024-06-19 13:44:20 +01:00
Daniel D'Abate a799ac8087
Improve OpenSearchURL credential storing user and password in separate fields from the URL (#2549)
OpenSearch - Improve OpenSearchURL credential storing user and password in separate fields from the URL
2024-06-04 02:31:36 +01:00