Commit Graph

216 Commits

Author SHA1 Message Date
Henry Heng ddca80d4e0
Bugfix/Prevent open connections on typeorm datasource (#3652)
prevent open connections on typeorm datasource
2024-12-08 21:06:07 +00:00
Eduard-Constantin Ibinceanu ba8a462bf8
[Bugfix] Milvus search error (#3623)
Fix milvus search error
2024-12-03 23:41:39 +00:00
Henry Heng 940c8fd3b0
Chore/LC v0.3 (#3517)
* bump langchain version to 0.3, upgrades on all chat models

* update all docs loader to have documents and text output options

* fix pnpm lock file
2024-11-28 11:06:12 +00:00
Vinod Kiran fe03683f0c
FEATURE:: Support for metrics collection - Prometheus and Open Telemetry (#3420)
* adding support for prometheus and grafana

* open telemetry

* lint fixes

* missing counter and telemetry standardization

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-11-08 00:00:43 +00:00
Henry Heng eeb1d17f50
Chore/MongoDB Connection (#3469)
getting rid of singleton design and properly close connection after every interaction
2024-11-07 19:05:00 +00:00
Jérémy JOURDIN 15d59a9052
[Prostgres Vector Store] Add PGVector Driver option + Fix null character issue w/ TypeORM Driver (#3367)
* Add PGVector Driver option + Fix null character issue w/ TypeORM Driver

* Handle file upload case with PGVector

* Cleanup

* Fix data filtering for chatflow uploaded files

* Add distanceStrategy parameter

* Fix query to improve chatflow uploaded files filtering

* Ensure PGVector release connections

* Await client connected

* Make Postgres credentials optionnal when set on env variables

* Document env variables in nodes directories

* Prevent reuse client

* Fix empty metadataFilter

* Update CONTRIBUTING.md

* Update Postgres.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-11-01 18:13:45 +00:00
Prajwal Pai 884dd8949a
Add couchbase vectore store support (#3093)
* Add couchbase vectore store support

* Uncomment couchbase code

* lint fix

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-10-26 02:36:38 +01:00
Anthony Bryan Gavilan Vinces 235fcfeb40
Enable Filters in MongoDB Vector Search Retriever (#3333)
* added: support to metadata filter using mongodb vector search retriever

* fix: format lint
2024-10-16 16:35:37 +01:00
Ortes d002096759
fix: Fix [object Object] sent to milvus on search (#3336) 2024-10-12 20:36:32 +01:00
Tyler 490855729b
Update placeholder port to match Postgres default (#3307) 2024-10-03 22:50:09 +01:00
Henry Heng 07059182af
Bugfix/implement missing delete method on pgvector (#3180)
implement missing delete method on pgvector
2024-09-12 02:28:45 +01:00
Eyalm321 7a5246d28a
added tls support to milvus (#3068)
* added tls support

* forgot to add tls to init function as well, it works now.

* added partition support for Milvus

* updated to 2.0.6 , pnpm lint fix aswell

* ensure it points to _default partition if none provided

* update milvus versioning

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-30 13:52:32 +01:00
Henry Heng 66acd0c000
Feature/add ability to upload file from chat (#3059)
add ability to upload file from chat
2024-08-25 13:22:48 +01:00
Mohamed Yasser Oaf 0f58d31493
Meilisearch - Added metadata as filterable attributes and polling on enqueued tasks (#3042)
* added polling for enqueued tasks,metadata filterable

* delete index feature with polling on task

* added search filter interface

* Update pnpm-lock.yaml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-23 16:27:07 +01:00
Mohamed Yasser Oaf d5153c3840
Added meilisearch retriever component (#2824)
* added meilisearch retriever and credentials.ts

* added semantic ratio

* removed a TODO implementatio

* meilisearch component implemented with searching and upsert functionality (#3)

meilisearch retriever component created , searching for an existing index and upserting a new or existing index has been implemented , component utilizes langchain and meilisearch vector search

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/3
Reviewed-by: mohamed1999akram <mohamed1999akram@gmail.com>

* added CI/CD for ntg branch, added proper dockerfile for flowise-ntg (#4)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/4
Reviewed-by: mohammad <mohammad@noreply.git.beyond.cc>

* modified os version , removed linting errors , removed cypress github actions (#5)

added --no-lock-file flag to pass CICD , made the runner run on debian and not ubuntu , removed code that caused warnings to pass linting

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/5
Reviewed-by: omaryassery <omarryassser@gmail.com>

* removed unnecessary QEMU install action (#6)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/6
Reviewed-by: omaryassery <omarryassser@gmail.com>

* removed cypress installation and linting from dockerfile (#7)

Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/7
Reviewed-by: isameh <isameh@ntgclarity.com>

* dockerfile-ntg-modification (#9)

dockerfile-ntg modified to copy all working directory before calling pnpm install
Reviewed-on: https://git.beyond.cc/ntg/flowise/pulls/9
Reviewed-by: isameh <isameh@ntgclarity.com>

* resolved comments, reverted CI/CD

* add test docker build yml back

* moved meilisearch to vector store folder

* Update Meilisearch.ts

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-08-18 12:23:45 +01:00
apeng-singlestore 36c6c6425c
Updated logo for singlestore (#3018) 2024-08-16 19:46:46 +01:00
Henry Heng 68769f4bbc
Feature/Add content and metadata key for qdrant (#2997)
add content and metadata key for qdrant
2024-08-11 21:00:48 +01:00
Vinod Kiran c0bae635b0
Document Store - Phase 2 (#2912)
* Document Store - Phase 2

* Adding additional columns for vector store config, document store phase 2

* Adding additional columns for vector store config, document store phase 2

* Document Store - Phase 2 - Upsert and Query

* ux cleanup

* retrieval settings and more ux changes

* adding MMR params to execution

* Making the upsert process async.

* add upsert history changes

* making the searchParams dynamic

* removing unnecessary params

* add ability to delete data from vector store

* update margin for vector store query

* adding option to save config in the retrieval playground

* adding chunk number for query return chunks

* Adding a Document Store node in the VectorStore category

* update doc store status, ui touchup

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-08-07 18:59:52 +01:00
Henry Heng ee8f9bb37c
Chore/Move @langchain/core dependency to components (#2945)
* move lc core dependency to components

* anchor @langchain/aws version

* update agentic RAG template
2024-08-06 14:41:21 +01:00
Henry Heng bca4de0c63
Feature/seq agents (#2798)
* update build functions

* sequential agents

* update langchain to 0.2, added sequential agent nodes

* add marketplace templates

* update howto wordings

* Merge branch 'main' into feature/Seq-Agents

# Conflicts:
#	pnpm-lock.yaml

* update deprecated functions and add new sequential nodes

* add marketplace templates

* update marketplace templates, add structured output to llm node

* add multi agents template

* update llm node with bindmodels

* update cypress version

* update templates sticky note wordings

* update tool node to include human in loop action

* update structured outputs error from models

* update cohere package to resolve google genai pipeThrough bug

* update mistral package version, added message reconstruction before invoke seq agent

* add HITL to agent

* update state messages restructuring

* update load and split methods for s3 directory
2024-07-22 17:46:14 +01:00
Neal Beeken 9e88c45051
feat: add driverInfo to mongodb component (#2779)
* feat: add driverInfo to mongodb component

NODE-6240

* chore: add a getVersion utility function
2024-07-15 12:24:00 +01:00
Henry Heng 363d1bfc44
Chore/update deprecating nodes (#2540)
* update deprecating nodes

* add filters use cases to marketplace

* update log level
2024-07-12 18:37:57 +01:00
Pavlo Paliychuk 1015e1193f
chore: Bump zep cloud sdk version and clean up zep cloud vector store node (#2767) 2024-07-12 18:01:00 +01:00
Henry Heng 109b0367cc
Bugfix/Pinecone Text Key (#2708)
* add source tag to pinecone

* update pinecone

* add text key to pinecone

* update pinecone version, and singleton
2024-06-24 02:08:30 +01:00
Henry Heng b662dd79c6
Feature/Add Text Key to Pinecone (#2681)
* add source tag to pinecone

* update pinecone

* add text key to pinecone

* update pinecone version, and singleton
2024-06-19 21:10:09 +01:00
Henry Heng b5ead0745b
Feature/Add filter to Postgres VectorStore (#2674)
* add filter to pgvs

* fix types
2024-06-19 13:44:20 +01:00
Daniel D'Abate a799ac8087
Improve OpenSearchURL credential storing user and password in separate fields from the URL (#2549)
OpenSearch - Improve OpenSearchURL credential storing user and password in separate fields from the URL
2024-06-04 02:31:36 +01:00
Henry Heng 272fd914bd
Bugfix/supabase upsert ids (#2561)
* fix upserting same id with supabase

* remove dedicated addvectors logic for ids

* Update pnpm-lock.yaml

* add fix for null id column
2024-06-03 13:09:59 +01:00
Daniel D'Abate e5f0ca2dd3
Fix OpenSearch vector store upsert (#2545) 2024-06-01 12:36:39 +01:00
Fahreddin Özcan 7706b3484a
Update Upstash Logo for White Background (#2511)
* update: upstash logo for white background

* update: upstash chat memory logo
2024-05-30 22:20:02 +01:00
Henry Heng d734747ec0
Bugfix/Upserting same id with supabase (#2521)
fix upserting same id with supabase
2024-05-30 00:18:51 +01:00
patrickreinan 265de4e97e
Added Opensearch credential (#2458)
* Added Openserch credential

* fix issues detected by linter

* rename to camelcase openSearchUrl

* Update OpenSearch.ts

---------

Co-authored-by: patrick <patrick.alves@br.experian.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-05-23 11:49:15 +01:00
Vinod Kiran ff2381741e
Chore/upgrade llamaindex version (#2440)
* updates to loader to support file upload

* adding a todo

* upgrade llamaindex

* update groq icon

* update azure models

* update llamaindex version

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-22 13:35:08 +01:00
Henry Heng 0365afbeeb
Feature/Add rpc filter to supabase (#2425)
add rpc filter to supabase
2024-05-17 11:40:17 +01:00
Henry Heng 96d4ab66f2
Chore/Temporarily disable couchbase (#2373)
temporarily disable couchbase
2024-05-09 14:14:14 +01:00
Prajwal Pai 09569d0b06
add couchbase vectore store support (#2303)
* add couchbase vectore store support

* Updated on review comments
2024-05-06 11:26:54 +01:00
Henry Heng f9195b6a68
Bugfix/PineconeArgumentError (#2322)
* add source tag to pinecone

* update pinecone
2024-05-05 00:52:04 +01:00
Henry Heng 9971627821
Chore/Pinecone Version Upgrade (#2310)
add source tag to pinecone
2024-05-03 15:46:50 +01:00
Fahreddin Özcan e71266de87
Upstash Vector Integration (#2284)
* add: upstash integration

* update upstash vector credential labels

* add filter to retriever/vectorstore as arg

* require embeddings

* lint-fix
2024-05-02 02:26:38 +01:00
Henry Heng 2b1273ca31
Bugfix/Missing Filter for VectorStore to Document (#2285)
add filter for vector store to document
2024-04-29 22:25:40 +01:00
RABBAH Mahmoud c4eb75ddde
Add batchSize parameter to Qdrant_VectorStores and QdrantUpsert_Vecto… (#2215)
* Add batchSize parameter to Qdrant_VectorStores and QdrantUpsert_VectorStores

* Add optional parameter 'optional' to batchSize in Qdrant_VectorStores and QdrantUpsert_VectorStores also try calling normal mode if it fail failback to batch mode

* Update default values for batchSize in Qdrant_VectorStores and QdrantUpsert_VectorStores

* Update PNPM version to v9.0.2 in Dockerfile

* Update Qdrant.ts

* Update Dockerfile

* Update Qdrant_Upsert.ts

* Update Qdrant.ts

* lint fix Qdrant_Upsert.ts

* increment qdrant node version

* increment qdrant upsert node version

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-04-29 22:20:32 +01:00
Vinod Kiran 7006d64de0
Feature/s3 storage (#2226)
* centralizing file writing....

* allowing s3 as storage option

* allowing s3 as storage option

* update s3 storage

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-23 11:35:38 +01:00
Henry Heng d1c8f7eb96
Bugfix/Supabase Upserting (#2210)
* fix supabase upserting error

* fix supabase upserting error

* update pnpm to 9.0.3
2024-04-19 00:52:28 +01:00
Vinod Kiran 1129782758
Code Cleanup - unnecessary document creation in Pinecone and Astra Vector Stores (#2124)
code cleanup
2024-04-09 15:34:29 +01:00
Vinod Kiran 658fa3984e
Feature/externalize files from chatflow - do not save as base64 (#1976)
* initial commit. Externalizing the file base64 string from flowData

* csv - docloader - Externalizing the file base64 string from flowData

* csv - docloader - Externalizing the file base64 string from flowData

* DocX - docloader - Externalizing the file base64 string from flowData

* Json - docloader - Externalizing the file base64 string from flowData

* Jsonlines - docloader - Externalizing the file base64 string from flowData

* PDF - docloader - Externalizing the file base64 string from flowData

* Vectara - vector store - Externalizing the file base64 string from flowData

* OpenAPIToolkit - tools - Externalizing the file base64 string from flowData

* OpenAPIChain - chain - Externalizing the file base64 string from flowData

* lint fixes

* datasource enabled - initial commit

* CSVAgent - agents - Externalizing the file base64 string from flowData

* Externalizing the file base64 string from flowData

* Externalizing the file base64 string from flowData

* add pnpm-lock.yaml

* update filerepository to add try catch

* Rename FileRepository.ts to fileRepository.ts

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-04-04 21:41:06 +05:30
Henry Heng e422ce287b
Feature/Indexing (#1802)
* indexing

* fix for multiple files upsert

* fix default Postgres port

* fix SQLite node description

* add MySQLRecordManager node

* fix MySQL unique index

* add upsert history

* update jsx ui

* lint-fix

* update dialog details

* update llamaindex pinecone

---------

Co-authored-by: chungyau97 <chungyau97@gmail.com>
2024-04-02 23:47:19 +01:00
Octavian FlowiseAI ea255db15d
Bugfix/closing mongodb connections (#2085)
* mongodb connection issue

* updating and pinning mongodb version

* removing pnpm-lock.yaml

* converting to a singleton

* Update pnpm lock file

* Downgrade mongodb

---------

Co-authored-by: vinodkiran <vinodkiran@usa.net>
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
2024-04-02 14:57:20 +02:00
Henry Heng 70706a7183
Bugfix/Add astradb collection name and namespace (#2071)
add astradb collection name and namespace
2024-03-31 00:07:59 +08:00
paulpaliychuk 6819d5f66b chore: Revert node names changes + update new node names to correct format 2024-02-24 13:47:53 -05:00
paulpaliychuk 7728178253 fix: Linter errors 2024-02-22 09:43:57 -08:00