2b1273ca31
Bugfix/Missing Filter for VectorStore to Document ( #2285 )
...
add filter for vector store to document
2024-04-29 22:25:40 +01:00
7006d64de0
Feature/s3 storage ( #2226 )
...
* centralizing file writing....
* allowing s3 as storage option
* allowing s3 as storage option
* update s3 storage
---------
Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-23 11:35:38 +01:00
4c2ba109fd
Update unstructured document loaders ( #2213 )
...
* Update UnstructuredFile with missing values. Removed deprecated values.
* Update UnstructuredFolder with missing values. Removed deprecated values.
* Added suport for sourceIdKey to unstructured loaders
* Update unstructured hi_res model names
* Update S3File document loader with latest unstructured and model changes
* Update credential method for S3File document loader
* moved pnpm req to engines to avoid minor version changes
* Change unstructured skipInferTableTypes parse to JSON
* Update unstructured with new params. Also fixed list order, missing values, and support for null on multiOptions.
2024-04-21 19:42:28 +01:00
f5be889ea8
Feature: Add pdfUsage parameter setting support to folderFiles ( #2211 )
...
* Add pdfUsage parameter setting support to folderFiles
* Add pdfUsage parameter setting additionalParams: true
2024-04-19 11:43:22 +01:00
658fa3984e
Feature/externalize files from chatflow - do not save as base64 ( #1976 )
...
* initial commit. Externalizing the file base64 string from flowData
* csv - docloader - Externalizing the file base64 string from flowData
* csv - docloader - Externalizing the file base64 string from flowData
* DocX - docloader - Externalizing the file base64 string from flowData
* Json - docloader - Externalizing the file base64 string from flowData
* Jsonlines - docloader - Externalizing the file base64 string from flowData
* PDF - docloader - Externalizing the file base64 string from flowData
* Vectara - vector store - Externalizing the file base64 string from flowData
* OpenAPIToolkit - tools - Externalizing the file base64 string from flowData
* OpenAPIChain - chain - Externalizing the file base64 string from flowData
* lint fixes
* datasource enabled - initial commit
* CSVAgent - agents - Externalizing the file base64 string from flowData
* Externalizing the file base64 string from flowData
* Externalizing the file base64 string from flowData
* add pnpm-lock.yaml
* update filerepository to add try catch
* Rename FileRepository.ts to fileRepository.ts
---------
Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-04-04 21:41:06 +05:30
b9b0c9d227
Bugfix/Web Scraper Limit ( #2083 )
...
fix when limit set to 0, selectedLinks sliced to become empty
2024-04-02 11:14:04 +01:00
4ca82ee733
Extend Confluence Document Loader to Support Server/Data Center with PAT ( #1998 )
...
* Extend Confluence Document Loader to Support Server/Data Center with PAT
- Update authentication to use Personal Access Token (PAT)
- Expand compatibility to include both Confluence Cloud and Server/Data Center
* Update ConfluenceServerDCApi.credential.ts
* use the same confluence loader with different connection logic
* use the same confluence loader with different connection logic
* Apply Prettier formatting
2024-03-25 01:37:55 +08:00
5a45a99620
Merge branch 'main' into chore/Upgrade-LC-version
2024-02-19 17:39:32 +08:00
4e8bf4903d
Merge pull request #1687 from 0xi4o/bug/scrap-limit
...
Fix: relative links method and limit not applying to manage links
2024-02-12 12:35:40 +05:30
5471a4c9aa
Show error when relative links method is not set and allow 0 as limit value
2024-02-12 12:01:19 +05:30
caf54bf31b
add document json output
2024-02-09 16:07:34 +08:00
73112ad122
Merge branch 'main' into chore/Upgrade-LC-version
...
# Conflicts:
# packages/components/package.json
2024-02-07 02:01:32 +08:00
19fb13baf0
fix for linting
2024-02-06 14:36:32 +01:00
2bb2a7588a
add recursive option for folder-loader
2024-02-06 14:25:40 +01:00
c2ae7e138c
Apply limit to selectedLinks even when relative links method is not specified
2024-02-06 14:40:19 +05:30
011a0a75c3
add kotlin files to folder-loader
2024-02-05 17:20:05 +01:00
7881f295ab
update vec 2 doc node
2024-02-01 19:18:14 +00:00
02fe500f21
Merge branch 'main' into chore/Upgrade-LC-version
...
# Conflicts:
# packages/components/nodes/cache/RedisCache/RedisCache.ts
# packages/components/nodes/cache/RedisCache/RedisEmbeddingsCache.ts
# packages/components/nodes/chains/ConversationChain/ConversationChain.ts
# packages/components/nodes/tools/RetrieverTool/RetrieverTool.ts
# packages/components/nodes/vectorstores/Qdrant/Qdrant.ts
# packages/components/nodes/vectorstores/Redis/Redis.ts
# packages/components/nodes/vectorstores/Redis/RedisSearchBase.ts
# packages/components/nodes/vectorstores/Redis/Redis_Existing.ts
# packages/components/nodes/vectorstores/Redis/Redis_Upsert.ts
# packages/components/src/agents.ts
2024-01-31 00:25:37 +00:00
b960f061eb
Clarifying that the Limit value is ignored when Return All is set to true.
2024-01-28 20:42:19 -08:00
66eef84633
Forgot to make maxRecords optional now
2024-01-28 20:42:19 -08:00
37945fc998
The loadAll() function should ignore any maxRecords specified, because the intention is the load *all* of the records. Also, marking both the Return All and Limit params as optional, so as to not confuse the user. Making them both required adds a lot of confusion that doesn't make sense. Ideally, the user either specifies Return All OR specifies the Limit value but not BOTH. It seems there's no way to define "conditional requirements" in Flowise params, so it's better to make both params optional.
2024-01-28 20:42:19 -08:00
dc39d7e2be
So Airtable API expects a maxRecords value to be the total set of records you want across multiple API calls. If the maxRecords is greater than 100, then it will provide pagination hints accordingly.
2024-01-28 20:42:19 -08:00
9b71f683ff
Support pagination even in loadLimit(), so that if a user wants to load more than 100 records but not all of them, they can. Currently, there's a bug where the document loader doesn't work on loading more than 100 records because of internal Airtable API limitations. The Airtable API can only fetch up to 100 records per call - anything more than that requires pagination.
2024-01-28 20:42:19 -08:00
2237b1ab16
Fix worked, removing debug logging, and bumped node version.
2024-01-28 20:42:19 -08:00
3b788e42e1
When you switch from GET to POST, you're supposed to adjust the Airtable URL to use the /listRecords endpoint. I didn't RTFM clearly. This is currently documented here: https://support.airtable.com/docs/enforcement-of-url-length-limit-for-web-api-requests
2024-01-28 20:42:19 -08:00
456dfabc6f
For some reason, Airtable doesn't like the POST operations, so I need to add logging to figure out why this happens
2024-01-28 20:42:18 -08:00
8ae848110e
Clarifying the description for the optional fields param
2024-01-28 20:42:18 -08:00
72ec7878b6
Added more error checking and also fixed yet more build errors
2024-01-28 20:42:18 -08:00
1a7cb5a010
Fixing more build errors
2024-01-28 20:42:18 -08:00
ae64854bae
Fixing a bunch of build errors
2024-01-28 20:42:18 -08:00
71f456af90
Switched to specifying Airtable fields to include rather than exclude - this helps reduce the amount of data fetched by the DocumentLoader when there are massive numbers of fields in an Airtable table.
2024-01-28 20:42:18 -08:00
6f7b7408e1
Merge branch 'FlowiseAI:main' into main
2024-01-28 20:34:00 -08:00
0606d2c6dd
upgrade langchain version 0.1.0
2024-01-26 23:41:55 +00:00
94d8e003e7
Merge branch 'main' of github.com:0xi4o/Flowise into feature/scrapped-links
2024-01-26 03:57:21 +05:30
193e5c4640
Update console statements to use logger
2024-01-22 08:49:04 +05:30
c24708f53b
Set default value for get links limit in web scraper nodes - cheerio, playwright, and puppeteer
2024-01-22 08:42:44 +05:30
bf60a1a2a9
Fix multiple calls to parseInt
2024-01-22 08:30:36 +05:30
b76c3b27a9
Merge branch 'FlowiseAI:main' into main
2024-01-20 00:01:41 -05:00
bfa26a72c4
Use selected links if available when scraping in cheerio, puppeteer, and playwright nodes
2024-01-19 14:25:04 +05:30
f26a99ade2
update figma loader
2024-01-17 22:08:04 +00:00
e3982476b0
Bumping version
2024-01-04 12:07:25 -05:00
53bfd07694
Bumping version to reflect new feature
2024-01-03 21:23:43 -05:00
66701cec8a
Fixing linting issues using 'yarn lint-fix'
2024-01-03 13:35:25 -05:00
c035363d6f
Fixing linting issues using 'yarn lint-fix'
2024-01-03 13:20:39 -05:00
6006157958
Updated Airtable field exclusion support to use field names instead of field ids
2023-12-30 15:17:33 -05:00
3fb8001907
Added support to exclude specific Airtable Field Ids
2023-12-30 14:13:39 -05:00
0885946bae
Merge branch 'FlowiseAI:main' into main
2023-12-30 11:54:33 -05:00
28e32f0ae6
Initial support for Airtable views
2023-12-30 09:46:44 -05:00
fe0c22255b
Bugfix: Upsert successful, but failed to insert documents
2023-12-30 16:17:03 +05:30
1a4ead3544
update S3 loader
2023-12-22 01:58:40 +00:00
Henry Heng
Vinod Kiran
Quinn
louyongjiu
Mr Khachaturov
Henry
Ilango
Ilyes Tascou
Darien Kindlund
Darien Kindlund
Darien Kindlund
vinodkiran