* fix upser api for file loader
* - Introduced loaderName property in IDocumentStoreUpsertData interface.
- Updated upsertDocStore function to utilize loaderName if provided.
- Enhanced DocStoreAPIDialog to allow loaderName customization in API requests.
- Modified DocumentStoreDetail to display file names when available, improving source formatting logic.
* feature/bugfix: added otpional css selector to puppeteer web scraper, fixed error when puppeteerLoader does not work.
* feature: added button to add empty link in web scraper tools
* feature: added custom executable file path as an input to puppeteer to fix issues when puppeteer can not find/launch the browser.
* feature: added new puppeteer features to playwright aswell.
* fixed review comments
refactor: Update code execution sandbox implementation across components
- Replaced NodeVM usage with a new createCodeExecutionSandbox function for improved sandbox management.
- Enhanced JavaScript code execution with executeJavaScriptCode function, allowing for better handling of libraries and output streaming.
- Updated multiple components to utilize the new sandboxing approach, ensuring consistent execution environment.
- Added validation for UUIDs and URLs in various tools to enhance input safety.
- Refactored input handling in CustomFunction and IfElseFunction to streamline variable management.
Enhance file upload capabilities by adding support for additional file types (html, css, js, xml, md, excel, powerpoint) and updating related MIME type mappings. Improve user interface for file type selection in FileUpload component.
* feat: add search functionality to FireCrawl with customizable parameters
* refactor: unify request parameters in FireCrawl to include integration identifier
* Update FireCrawl Document Loader to version 4.0, enhancing parameter labels and adding conditional visibility for URL and crawler options based on selected crawler type.
---------
Co-authored-by: Henry <hzj94@hotmail.com>
* add teams, gmail, outlook tools
* update docs link
* update credentials for oauth2
* add jira tool
* add google drive, google calendar, google sheets tools, powerpoint, excel, word doc loader
* update jira logo
* Refactor Gmail and Outlook tools to remove maxOutputLength parameter and enhance request handling. Update response formatting to include parameters in the output. Adjust Google Drive tools to simplify success messages by removing unnecessary parameter details.
* chore: update Firecrawl version and add FirecrawlExtractTool
* refactor: update outputs format
* chore: update Firecrawl request headers to include X-Origin and X-Origin-Type
* feat: add FireCrawl testing suite for scraping, crawling, and data extraction
- Introduced FireCrawl-TEST.ts to validate FireCrawlLoader functionality.
- Implemented tests for basic scraping, crawling with text splitting, data extraction, and extract status retrieval.
- Enhanced error handling in FireCrawlLoader for better debugging.
* Update pnpm-lock.yaml
* refactor: FireCrawl API integration to improve parameter handling and error logging
* refractor firecrawl
* Update FireCrawl.ts
removed console log
* Update pnpm-lock.yaml
* Update pnpm-lock.yaml
---------
Co-authored-by: Ong Chung Yau <33013947+chungyau97@users.noreply.github.com>
Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
* Make Unstructured API URL optional when environment variable is present
* Fix empty apiUrl option in Unsctructured flowise loader
* Add focumentation for env vars
* add functionality for full file uploads, add remove messages from view dialog and API
* add attachments swagger
* update question to include uploadedFilesContent
* make config dialog modal lg size
* Unstructured Upsert bug
When upserting with the API, the uploaded files are of type pdfFile, txtFile, etc.
but the code reads only fileObject which is the uploaded file using the button
* Update UnstructuredFile.ts
fixed linting error
---------
Co-authored-by: Mohamed Akram <makram@ntgclarity.com>
* add langgraph
* datasource: initial commit
* datasource: datasource details and chunks
* datasource: Document Store Node
* more changes
* Document Store - Base functionality
* Document Store Loader Component
* Document Store Loader Component
* before merging the modularity PR
* after merging the modularity PR
* preview mode
* initial draft PR
* fixes
* minor updates and fixes
* preview with loader and splitter
* preview with credential
* show stored chunks
* preview update...
* edit config
* save, preview and other changes
* save, preview and other changes
* save, process and other changes
* save, process and other changes
* alpha1 - for internal testing
* rerouting urls
* bug fix on new leader create
* pagination support for chunks
* delete document store
* Update pnpm-lock.yaml
* doc store card view
* Update store files to use updated storage functions, Document Store Table View and other changes
* ui changes
* add expanded chunk dialog, improve ui
* change throw Error to InternalError
* Bug Fixes and removal of subFolder, adding of view chunks for store
* lint fixes
* merge changes
* DocumentStoreStatus component
* ui changes for doc store
* add remove metadata key field, add custom document loader
* add chatflows used doc store chips
* add types/interfaces to DocumentStore Services
* document loader list dialog title bar color change
* update interfaces
* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.
* use typeorm order chunkNo, ui changes
* update tabler icons react
* cleanup agents
* add pysandbox tool
* add abort functionality, loading next agent
* add empty view svg
* update chatflow tool with chatId
* rename to agentflows
* update worker for prompt input values
* update dashboard to agentflows, agentcanvas
* fix marketplace use template
* add agentflow templates
* resolve merge conflict
* update baseURL
---------
Co-authored-by: vinodkiran <vinodkiran@usa.net>
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
Rostyslav Borovyk
Henry Heng
Jakub Drobník
Mewyii
Ademílson Tonato
Ong Chung Yau
Shibin Das
Michael Vonrueden
Jérémy JOURDIN
Ajay Satish
ZacharyQin
Ryno
cragwolfe
Rafael Reis
William Espegren
slaplante-raft
Ahmed Osman
Mohamed Akram
Vinod Kiran