Sultan
/
Flowise
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Feature: Option to add apikeys to the DB instead of api.json. (#2783) 

* Feature: Option to add apikeys to the DB instead of api.json.

* add api storage type env variable

* code cleanup and simplification.

* md table fixes

---------

Co-authored-by: Henry <hzj94@hotmail.com>
This commit is contained in:
Vinod Kiran 2024-07-26 22:22:12 +05:30 committed by GitHub
parent e5018d2743
commit 2dd1791ec2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
26 changed files with 647 additions and 38 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
CONTRIBUTING.md
View File

@ -121,7 +121,7 @@ Flowise has 3 different modules in a single mono repository.
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default |
| ---------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| ---------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------ |-------------------------------------|
| PORT | The HTTP port Flowise runs on | Number | 3000 |
| CORS_ORIGINS | The allowed origins for all cross-origin HTTP calls | String | |
| IFRAME_ORIGINS | The allowed origins for iframe src embedding | String | |
@ -133,7 +133,8 @@ Flowise support different environment variables to configure your instance. You
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| LOG_JSON_SPACES | Spaces to beautify JSON logs | | 2 |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| APIKEY_STORAGE_TYPE | To store api keys on a JSON file or database. Default is `json` | Enum String: `json`, `db` | `json` |
| APIKEY_PATH | Location where api keys are saved when `APIKEY_STORAGE_TYPE` is `json` | String | `your-path/Flowise/packages/server` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
@ -146,8 +147,8 @@ Flowise support different environment variables to configure your instance. You
| DATABASE_SSL_KEY_BASE64 | Database SSL client cert in base64 (takes priority over DATABASE_SSL) | Boolean | false |
| DATABASE_SSL | Database connection overssl (When DATABASE_TYPE is postgre) | Boolean | false |
| SECRETKEY_PATH | Location where encryption key (used to encrypt/decrypt credentials) is saved | String | `your-path/Flowise/packages/server` |
| FLOWISE_SECRETKEY_OVERWRITE | Encryption key to be used instead of the key stored in SECRETKEY_PATH | String |
| DISABLE_FLOWISE_TELEMETRY | Turn off telemetry | Boolean |
| FLOWISE_SECRETKEY_OVERWRITE | Encryption key to be used instead of the key stored in SECRETKEY_PATH | String | |
| DISABLE_FLOWISE_TELEMETRY | Turn off telemetry | Boolean | |
| MODEL_LIST_CONFIG_JSON | File path to load list of models from your local config file | String | `/your_model_list_config_file_path` |
| STORAGE_TYPE | Type of storage for uploaded files. default is `local` | Enum String: `s3`, `local` | `local` |
| BLOB_STORAGE_PATH | Local folder path where uploaded files are stored when `STORAGE_TYPE` is `local` | String | `your-home-dir/.flowise/storage` |

4
docker/.env.example
View File

@ -47,4 +47,6 @@ BLOB_STORAGE_PATH=/root/.flowise/storage
# S3_STORAGE_ACCESS_KEY_ID=<your-access-key>
# S3_STORAGE_SECRET_ACCESS_KEY=<your-secret-key>
# S3_STORAGE_REGION=us-west-2
# S3_ENDPOINT_URL=<custom-s3-endpoint-url>
# S3_ENDPOINT_URL=<custom-s3-endpoint-url>
# APIKEY_STORAGE_TYPE=json (json | db)

3
i18n/CONTRIBUTING-ZH.md
View File

@ -128,7 +128,8 @@ Flowise 支持不同的环境变量来配置您的实例。您可以在 `package
| DEBUG | 打印组件的日志 | 布尔值 | |
| LOG_PATH | 存储日志文件的位置 | 字符串 | `your-path/Flowise/logs` |
| LOG_LEVEL | 日志的不同级别 | 枚举字符串: `error`, `info`, `verbose`, `debug` | `info` |
| APIKEY_PATH | 存储 API 密钥的位置 | 字符串 | `your-path/Flowise/packages/server` |
| APIKEY_STORAGE_TYPE | 存储 API 密钥的存储类型 | 枚举字符串: `json`, `db` | `json` |
| APIKEY_PATH | 存储 API 密钥的位置, 当`APIKEY_STORAGE_TYPE`是`json` | 字符串 | `your-path/Flowise/packages/server` |
| TOOL_FUNCTION_BUILTIN_DEP | 用于工具函数的 NodeJS 内置模块 | 字符串 | |
| TOOL_FUNCTION_EXTERNAL_DEP | 用于工具函数的外部模块 | 字符串 | |
| DATABASE_TYPE | 存储 flowise 数据的数据库类型 | 枚举字符串: `sqlite`, `mysql`, `postgres` | `sqlite` |

4
packages/server/.env.example
View File

@ -47,4 +47,6 @@ PORT=3000
# S3_STORAGE_ACCESS_KEY_ID=<your-access-key>
# S3_STORAGE_SECRET_ACCESS_KEY=<your-secret-key>
# S3_STORAGE_REGION=us-west-2
# S3_ENDPOINT_URL=<custom-s3-endpoint-url>
# S3_ENDPOINT_URL=<custom-s3-endpoint-url>
# APIKEY_STORAGE_TYPE=json (json | db)

6
packages/server/src/AppConfig.ts Normal file
View File

@ -0,0 +1,6 @@
export const appConfig = {
apiKeys: {
storageType: process.env.APIKEY_STORAGE_TYPE ? process.env.APIKEY_STORAGE_TYPE.toLowerCase() : 'json'
}
// todo: add more config options here like database, log, storage, credential and allow modification from UI
}

8
packages/server/src/Interface.ts
View File

@ -263,5 +263,13 @@ export interface IUploadFileSizeAndTypes {
maxUploadSize: number
}
export interface IApiKey {
id: string
keyName: string
apiKey: string
apiSecret: string
updatedDate: Date
}
// DocumentStore related
export * from './Interface.DocumentStore'

2
packages/server/src/commands/start.ts
View File

@ -24,6 +24,7 @@ export default class Start extends Command {
IFRAME_ORIGINS: Flags.string(),
DEBUG: Flags.string(),
BLOB_STORAGE_PATH: Flags.string(),
APIKEY_STORAGE_TYPE: Flags.string(),
APIKEY_PATH: Flags.string(),
SECRETKEY_PATH: Flags.string(),
FLOWISE_SECRETKEY_OVERWRITE: Flags.string(),
@ -100,6 +101,7 @@ export default class Start extends Command {
// Authorization
if (flags.FLOWISE_USERNAME) process.env.FLOWISE_USERNAME = flags.FLOWISE_USERNAME
if (flags.FLOWISE_PASSWORD) process.env.FLOWISE_PASSWORD = flags.FLOWISE_PASSWORD
if (flags.APIKEY_STORAGE_TYPE) process.env.APIKEY_STORAGE_TYPE = flags.APIKEY_STORAGE_TYPE
if (flags.APIKEY_PATH) process.env.APIKEY_PATH = flags.APIKEY_PATH
// API Configuration

16
packages/server/src/controllers/apikey/index.ts
View File

@ -41,6 +41,19 @@ const updateApiKey = async (req: Request, res: Response, next: NextFunction) =>
}
}
// Import Keys from JSON file
const importKeys = async (req: Request, res: Response, next: NextFunction) => {
try {
if (typeof req.body === 'undefined' || !req.body.jsonFile) {
throw new InternalFlowiseError(StatusCodes.PRECONDITION_FAILED, `Error: apikeyController.importKeys - body not provided!`)
}
const apiResponse = await apikeyService.importKeys(req.body)
return res.json(apiResponse)
} catch (error) {
next(error)
}
}
// Delete api key
const deleteApiKey = async (req: Request, res: Response, next: NextFunction) => {
try {
@ -72,5 +85,6 @@ export default {
deleteApiKey,
getAllApiKeys,
updateApiKey,
verifyApiKey
verifyApiKey,
importKeys
}

6
packages/server/src/controllers/chatflows/index.ts
View File

@ -1,11 +1,11 @@
import { NextFunction, Request, Response } from 'express'
import { StatusCodes } from 'http-status-codes'
import apiKeyService from '../../services/apikey'
import { ChatFlow } from '../../database/entities/ChatFlow'
import { createRateLimiter } from '../../utils/rateLimit'
import { InternalFlowiseError } from '../../errors/internalFlowiseError'
import { ChatflowType } from '../../Interface'
import chatflowsService from '../../services/chatflows'
import { getApiKey } from '../../utils/apiKey'
import { createRateLimiter } from '../../utils/rateLimit'
const checkIfChatflowIsValidForStreaming = async (req: Request, res: Response, next: NextFunction) => {
try {
@ -67,7 +67,7 @@ const getChatflowByApiKey = async (req: Request, res: Response, next: NextFuncti
`Error: chatflowsRouter.getChatflowByApiKey - apikey not provided!`
)
}
const apikey = await getApiKey(req.params.apikey)
const apikey = await apiKeyService.getApiKey(req.params.apikey)
if (!apikey) {
return res.status(401).send('Unauthorized')
}

21
packages/server/src/database/entities/ApiKey.ts Normal file
View File

@ -0,0 +1,21 @@
import { Column, Entity, PrimaryColumn, UpdateDateColumn } from 'typeorm'
import { IApiKey } from '../../Interface'
@Entity('apikey')
export class ApiKey implements IApiKey {
@PrimaryColumn({ type: 'varchar', length: 20 })
id: string
@Column({ type: 'text' })
apiKey: string
@Column({ type: 'text' })
apiSecret: string
@Column({ type: 'text' })
keyName: string
@Column({ type: 'timestamp' })
@UpdateDateColumn()
updatedDate: Date
}

4
packages/server/src/database/entities/index.ts
View File

@ -9,6 +9,7 @@ import { DocumentStore } from './DocumentStore'
import { DocumentStoreFileChunk } from './DocumentStoreFileChunk'
import { Lead } from './Lead'
import { UpsertHistory } from './UpsertHistory'
import { ApiKey } from './ApiKey'
export const entities = {
ChatFlow,
@ -21,5 +22,6 @@ export const entities = {
DocumentStore,
DocumentStoreFileChunk,
Lead,
UpsertHistory
UpsertHistory,
ApiKey
}

20
packages/server/src/database/migrations/mariadb/1720230151480-AddApiKey.ts Normal file
View File

@ -0,0 +1,20 @@
import { MigrationInterface, QueryRunner } from 'typeorm'
export class AddApiKey1720230151480 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE IF NOT EXISTS \`apikey\` (
\`id\` varchar(36) NOT NULL,
\`apiKey\` varchar(255) NOT NULL,
\`apiSecret\` varchar(255) NOT NULL,
\`keyName\` varchar(255) NOT NULL,
\`updatedDate\` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (\`id\`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;`
)
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE apikey`)
}
}

2
packages/server/src/database/migrations/mariadb/index.ts
View File

@ -20,6 +20,7 @@ import { AddLead1710832127079 } from './1710832127079-AddLead'
import { AddLeadToChatMessage1711538023578 } from './1711538023578-AddLeadToChatMessage'
import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
export const mariadbMigrations = [
@ -45,5 +46,6 @@ export const mariadbMigrations = [
AddLeadToChatMessage1711538023578,
AddAgentReasoningToChatMessage1714679514451,
AddTypeToChatFlow1766759476232,
AddApiKey1720230151480,
AddActionToChatMessage1721078251523
]

20
packages/server/src/database/migrations/mysql/1720230151480-AddApiKey.ts Normal file
View File

@ -0,0 +1,20 @@
import { MigrationInterface, QueryRunner } from 'typeorm'
export class AddApiKey1720230151480 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE IF NOT EXISTS \`apikey\` (
\`id\` varchar(36) NOT NULL,
\`apiKey\` varchar(255) NOT NULL,
\`apiSecret\` varchar(255) NOT NULL,
\`keyName\` varchar(255) NOT NULL,
\`updatedDate\` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (\`id\`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;`
)
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE apikey`)
}
}

2
packages/server/src/database/migrations/mysql/index.ts
View File

@ -20,6 +20,7 @@ import { AddLead1710832127079 } from './1710832127079-AddLead'
import { AddLeadToChatMessage1711538023578 } from './1711538023578-AddLeadToChatMessage'
import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
export const mysqlMigrations = [
@ -45,5 +46,6 @@ export const mysqlMigrations = [
AddLeadToChatMessage1711538023578,
AddAgentReasoningToChatMessage1714679514451,
AddTypeToChatFlow1766759476232,
AddApiKey1720230151480,
AddActionToChatMessage1721078251523
]

20
packages/server/src/database/migrations/postgres/1720230151480-AddApiKey.ts Normal file
View File

@ -0,0 +1,20 @@
import { MigrationInterface, QueryRunner } from 'typeorm'
export class AddApiKey1720230151480 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE IF NOT EXISTS apikey (
id uuid NOT NULL DEFAULT uuid_generate_v4(),
"apiKey" varchar NOT NULL,
"apiSecret" varchar NOT NULL,
"keyName" varchar NOT NULL,
"updatedDate" timestamp NOT NULL DEFAULT now(),
CONSTRAINT "PK_96109043dd704f53-9830ab78f0" PRIMARY KEY (id)
);`
)
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE apikey`)
}
}

2
packages/server/src/database/migrations/postgres/index.ts
View File

@ -21,6 +21,7 @@ import { AddLead1710832137905 } from './1710832137905-AddLead'
import { AddLeadToChatMessage1711538016098 } from './1711538016098-AddLeadToChatMessage'
import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
export const postgresMigrations = [
@ -47,5 +48,6 @@ export const postgresMigrations = [
AddLeadToChatMessage1711538016098,
AddAgentReasoningToChatMessage1714679514451,
AddTypeToChatFlow1766759476232,
AddApiKey1720230151480,
AddActionToChatMessage1721078251523
]

17
packages/server/src/database/migrations/sqlite/1720230151480-AddApiKey.ts Normal file
View File

@ -0,0 +1,17 @@
import { MigrationInterface, QueryRunner } from 'typeorm'
export class AddApiKey1720230151480 implements MigrationInterface {
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`CREATE TABLE IF NOT EXISTS "apikey" ("id" varchar PRIMARY KEY NOT NULL,
"apiKey" varchar NOT NULL,
"apiSecret" varchar NOT NULL,
"keyName" varchar NOT NULL,
"updatedDate" datetime NOT NULL DEFAULT (datetime('now')));`
)
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`DROP TABLE IF EXISTS "apikey";`)
}
}

2
packages/server/src/database/migrations/sqlite/index.ts
View File

@ -21,6 +21,7 @@ import { AddLeadToChatMessage1711537986113 } from './1711537986113-AddLeadToChat
import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
export const sqliteMigrations = [
Init1693835579790,
@ -45,5 +46,6 @@ export const sqliteMigrations = [
AddLeadToChatMessage1711537986113,
AddAgentReasoningToChatMessage1714679514451,
AddTypeToChatFlow1766759476232,
AddApiKey1720230151480,
AddActionToChatMessage1721078251523
]

1
packages/server/src/routes/apikey/index.ts
View File

@ -4,6 +4,7 @@ const router = express.Router()
// CREATE
router.post('/', apikeyController.createApiKey)
router.post('/import', apikeyController.importKeys)
// READ
router.get('/', apikeyController.getAllApiKeys)

226
packages/server/src/services/apikey/index.ts
View File

@ -1,25 +1,94 @@
import { StatusCodes } from 'http-status-codes'
import { addAPIKey, deleteAPIKey, getAPIKeys, updateAPIKey } from '../../utils/apiKey'
import {
addAPIKey as addAPIKey_json,
deleteAPIKey as deleteAPIKey_json,
generateAPIKey,
generateSecretHash,
getApiKey as getApiKey_json,
getAPIKeys as getAPIKeys_json,
updateAPIKey as updateAPIKey_json,
replaceAllAPIKeys as replaceAllAPIKeys_json,
importKeys as importKeys_json
} from '../../utils/apiKey'
import { addChatflowsCount } from '../../utils/addChatflowsCount'
import { getApiKey } from '../../utils/apiKey'
import { InternalFlowiseError } from '../../errors/internalFlowiseError'
import { getErrorMessage } from '../../errors/utils'
import { getRunningExpressApp } from '../../utils/getRunningExpressApp'
import { ApiKey } from '../../database/entities/ApiKey'
import { appConfig } from '../../AppConfig'
import { randomBytes } from 'crypto'
import { Not, IsNull } from 'typeorm'
const _apikeysStoredInJson = (): boolean => {
return appConfig.apiKeys.storageType === 'json'
}
const _apikeysStoredInDb = (): boolean => {
return appConfig.apiKeys.storageType === 'db'
}
const getAllApiKeys = async () => {
try {
const keys = await getAPIKeys()
const dbResponse = await addChatflowsCount(keys)
return dbResponse
if (_apikeysStoredInJson()) {
const keys = await getAPIKeys_json()
return await addChatflowsCount(keys)
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
let keys = await appServer.AppDataSource.getRepository(ApiKey).find()
if (keys.length === 0) {
await createApiKey('DefaultKey')
keys = await appServer.AppDataSource.getRepository(ApiKey).find()
}
return await addChatflowsCount(keys)
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.getAllApiKeys - ${getErrorMessage(error)}`)
}
}
const getApiKey = async (keyName: string) => {
try {
if (_apikeysStoredInJson()) {
return getApiKey_json(keyName)
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
const currentKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
keyName: keyName
})
if (!currentKey) {
return undefined
}
return currentKey
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.createApiKey - ${getErrorMessage(error)}`)
}
}
const createApiKey = async (keyName: string) => {
try {
const keys = await addAPIKey(keyName)
const dbResponse = await addChatflowsCount(keys)
return dbResponse
if (_apikeysStoredInJson()) {
const keys = await addAPIKey_json(keyName)
return await addChatflowsCount(keys)
} else if (_apikeysStoredInDb()) {
const apiKey = generateAPIKey()
const apiSecret = generateSecretHash(apiKey)
const appServer = getRunningExpressApp()
const newKey = new ApiKey()
newKey.id = randomBytes(16).toString('hex')
newKey.apiKey = apiKey
newKey.apiSecret = apiSecret
newKey.keyName = keyName
const key = appServer.AppDataSource.getRepository(ApiKey).create(newKey)
await appServer.AppDataSource.getRepository(ApiKey).save(key)
return getAllApiKeys()
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.createApiKey - ${getErrorMessage(error)}`)
}
@ -28,9 +97,23 @@ const createApiKey = async (keyName: string) => {
// Update api key
const updateApiKey = async (id: string, keyName: string) => {
try {
const keys = await updateAPIKey(id, keyName)
const dbResponse = await addChatflowsCount(keys)
return dbResponse
if (_apikeysStoredInJson()) {
const keys = await updateAPIKey_json(id, keyName)
return await addChatflowsCount(keys)
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
const currentKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
id: id
})
if (!currentKey) {
throw new InternalFlowiseError(StatusCodes.NOT_FOUND, `ApiKey ${currentKey} not found`)
}
currentKey.keyName = keyName
await appServer.AppDataSource.getRepository(ApiKey).save(currentKey)
return getAllApiKeys()
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.updateApiKey - ${getErrorMessage(error)}`)
}
@ -38,22 +121,123 @@ const updateApiKey = async (id: string, keyName: string) => {
const deleteApiKey = async (id: string) => {
try {
const keys = await deleteAPIKey(id)
const dbResponse = await addChatflowsCount(keys)
return dbResponse
if (_apikeysStoredInJson()) {
const keys = await deleteAPIKey_json(id)
return await addChatflowsCount(keys)
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
const dbResponse = await appServer.AppDataSource.getRepository(ApiKey).delete({ id: id })
if (!dbResponse) {
throw new InternalFlowiseError(StatusCodes.NOT_FOUND, `ApiKey ${id} not found`)
}
return getAllApiKeys()
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.deleteApiKey - ${getErrorMessage(error)}`)
}
}
const importKeys = async (body: any) => {
try {
const jsonFile = body.jsonFile
const splitDataURI = jsonFile.split(',')
if (splitDataURI[0] !== 'data:application/json;base64') {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Invalid dataURI`)
}
const bf = Buffer.from(splitDataURI[1] || '', 'base64')
const plain = bf.toString('utf8')
const keys = JSON.parse(plain)
if (_apikeysStoredInJson()) {
if (body.importMode === 'replaceAll') {
await replaceAllAPIKeys_json(keys)
} else {
await importKeys_json(keys, body.importMode)
}
return await addChatflowsCount(keys)
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
const allApiKeys = await appServer.AppDataSource.getRepository(ApiKey).find()
if (body.importMode === 'replaceAll') {
await appServer.AppDataSource.getRepository(ApiKey).delete({
id: Not(IsNull())
})
}
if (body.importMode === 'errorIfExist') {
// if importMode is errorIfExist, check for existing keys and raise error before any modification to the DB
for (const key of keys) {
const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
if (keyNameExists) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Key with name ${key.keyName} already exists`)
}
}
}
// iterate through the keys and add them to the database
for (const key of keys) {
const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
if (keyNameExists) {
const keyIndex = allApiKeys.findIndex((k) => k.keyName === key.keyName)
switch (body.importMode) {
case 'overwriteIfExist': {
const currentKey = allApiKeys[keyIndex]
currentKey.id = key.id
currentKey.apiKey = key.apiKey
currentKey.apiSecret = key.apiSecret
await appServer.AppDataSource.getRepository(ApiKey).save(currentKey)
break
}
case 'ignoreIfExist': {
// ignore this key and continue
continue
}
case 'errorIfExist': {
// should not reach here as we have already checked for existing keys
throw new Error(`Key with name ${key.keyName} already exists`)
}
default: {
throw new Error(`Unknown overwrite option ${body.importMode}`)
}
}
} else {
const newKey = new ApiKey()
newKey.id = key.id
newKey.apiKey = key.apiKey
newKey.apiSecret = key.apiSecret
newKey.keyName = key.keyName
const newKeyEntity = appServer.AppDataSource.getRepository(ApiKey).create(newKey)
await appServer.AppDataSource.getRepository(ApiKey).save(newKeyEntity)
}
}
return getAllApiKeys()
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
} catch (error) {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.importKeys - ${getErrorMessage(error)}`)
}
}
const verifyApiKey = async (paramApiKey: string): Promise<string> => {
try {
const apiKey = await getApiKey(paramApiKey)
if (!apiKey) {
throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
if (_apikeysStoredInJson()) {
const apiKey = await getApiKey_json(paramApiKey)
if (!apiKey) {
throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
}
return 'OK'
} else if (_apikeysStoredInDb()) {
const appServer = getRunningExpressApp()
const apiKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
apiKey: paramApiKey
})
if (!apiKey) {
throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
}
return 'OK'
} else {
throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
}
const dbResponse = 'OK'
return dbResponse
} catch (error) {
if (error instanceof InternalFlowiseError && error.statusCode === StatusCodes.UNAUTHORIZED) {
throw error
@ -71,5 +255,7 @@ export default {
deleteApiKey,
getAllApiKeys,
updateApiKey,
verifyApiKey
verifyApiKey,
getApiKey,
importKeys
}

45
packages/server/src/utils/apiKey.ts
View File

@ -4,6 +4,7 @@ import moment from 'moment'
import fs from 'fs'
import path from 'path'
import logger from './logger'
import { appConfig } from '../AppConfig'
/**
* Returns the api key path
@ -50,6 +51,9 @@ export const compareKeys = (storedKey: string, suppliedKey: string): boolean =>
* @returns {Promise<ICommonObject[]>}
*/
export const getAPIKeys = async (): Promise<ICommonObject[]> => {
if (appConfig.apiKeys.storageType !== 'json') {
return []
}
try {
const content = await fs.promises.readFile(getAPIKeyPath(), 'utf8')
return JSON.parse(content)
@ -94,6 +98,47 @@ export const addAPIKey = async (keyName: string): Promise<ICommonObject[]> => {
return content
}
/**
* import API keys
* @param {[]} keys
* @returns {Promise<ICommonObject[]>}
*/
export const importKeys = async (keys: any[], importMode: string): Promise<ICommonObject[]> => {
const allApiKeys = await getAPIKeys()
// if importMode is errorIfExist, check for existing keys and raise error before any modification to the file
if (importMode === 'errorIfExist') {
for (const key of keys) {
const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
if (keyNameExists) {
throw new Error(`Key with name ${key.keyName} already exists`)
}
}
}
for (const key of keys) {
// Check if keyName already exists, if overwrite is false, raise an error else overwrite the key
const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
if (keyNameExists) {
const keyIndex = allApiKeys.findIndex((k) => k.keyName === key.keyName)
switch (importMode) {
case 'overwriteIfExist':
allApiKeys[keyIndex] = key
continue
case 'ignoreIfExist':
// ignore this key and continue
continue
case 'errorIfExist':
// should not reach here as we have already checked for existing keys
throw new Error(`Key with name ${key.keyName} already exists`)
default:
throw new Error(`Unknown overwrite option ${importMode}`)
}
}
allApiKeys.push(key)
}
await fs.promises.writeFile(getAPIKeyPath(), JSON.stringify(allApiKeys), 'utf8')
return allApiKeys
}
/**
* Get API Key details
* @param {string} apiKey

8
packages/server/src/utils/validateKey.ts
View File

@ -1,7 +1,7 @@
import { Request } from 'express'
import { ChatFlow } from '../database/entities/ChatFlow'
import { getAPIKeys, compareKeys } from './apiKey'
import { compareKeys } from './apiKey'
import apikeyService from '../services/apikey'
/**
* Validate API Key
* @param {Request} req
@ -17,8 +17,8 @@ export const utilValidateKey = async (req: Request, chatflow: ChatFlow) => {
const suppliedKey = authorizationHeader.split(`Bearer `).pop()
if (suppliedKey) {
const keys = await getAPIKeys()
const apiSecret = keys.find((key) => key.id === chatFlowApiKeyId)?.apiSecret
const keys = await apikeyService.getAllApiKeys()
const apiSecret = keys.find((key: any) => key.id === chatFlowApiKeyId)?.apiSecret
if (!compareKeys(apiSecret, suppliedKey)) return false
return true
}

5
packages/ui/src/api/apikey.js
View File

@ -8,9 +8,12 @@ const updateAPI = (id, body) => client.put(`/apikey/${id}`, body)
const deleteAPI = (id) => client.delete(`/apikey/${id}`)
const importAPI = (body) => client.post(`/apikey/import`, body)
export default {
getAllAPIKeys,
createNewAPI,
updateAPI,
deleteAPI
deleteAPI,
importAPI
}

186
packages/ui/src/views/apikey/UploadJSONFileDialog.jsx Normal file
View File

@ -0,0 +1,186 @@
import { createPortal } from 'react-dom'
import PropTypes from 'prop-types'
import { useState, useEffect } from 'react'
import { useDispatch } from 'react-redux'
import { enqueueSnackbar as enqueueSnackbarAction, closeSnackbar as closeSnackbarAction } from '@/store/actions'
// Material
import { Button, Dialog, DialogActions, DialogContent, DialogTitle, Box, Typography, Stack } from '@mui/material'
// Project imports
import { StyledButton } from '@/ui-component/button/StyledButton'
import ConfirmDialog from '@/ui-component/dialog/ConfirmDialog'
import { File } from '@/ui-component/file/File'
// Icons
import { IconFileUpload, IconX } from '@tabler/icons-react'
// API
import apikeyAPI from '@/api/apikey'
// utils
import useNotifier from '@/utils/useNotifier'
// const
import { HIDE_CANVAS_DIALOG, SHOW_CANVAS_DIALOG } from '@/store/actions'
import { Dropdown } from '@/ui-component/dropdown/Dropdown'
const importModes = [
{
label: 'Add & Overwrite',
name: 'overwriteIfExist',
description: 'Add keys and overwrite existing keys with the same name'
},
{
label: 'Add & Ignore',
name: 'ignoreIfExist',
description: 'Add keys and ignore existing keys with the same name'
},
{
label: 'Add & Verify',
name: 'errorIfExist',
description: 'Add Keys and throw error if key with same name exists'
},
{
label: 'Replace All',
name: 'replaceAll',
description: 'Replace all keys with the imported keys'
}
]
const UploadJSONFileDialog = ({ show, dialogProps, onCancel, onConfirm }) => {
const portalElement = document.getElementById('portal')
const dispatch = useDispatch()
// ==============================|| Snackbar ||============================== //
useNotifier()
const enqueueSnackbar = (...args) => dispatch(enqueueSnackbarAction(...args))
const closeSnackbar = (...args) => dispatch(closeSnackbarAction(...args))
const [selectedFile, setSelectedFile] = useState()
const [importMode, setImportMode] = useState('overwrite')
useEffect(() => {
return () => {
setSelectedFile()
}
}, [dialogProps])
useEffect(() => {
if (show) dispatch({ type: SHOW_CANVAS_DIALOG })
else dispatch({ type: HIDE_CANVAS_DIALOG })
return () => dispatch({ type: HIDE_CANVAS_DIALOG })
}, [show, dispatch])
const importKeys = async () => {
try {
const obj = {
importMode: importMode,
jsonFile: selectedFile
}
const createResp = await apikeyAPI.importAPI(obj)
if (createResp.data) {
enqueueSnackbar({
message: 'Imported keys successfully!',
options: {
key: new Date().getTime() + Math.random(),
variant: 'success',
action: (key) => (
<Button style={{ color: 'white' }} onClick={() => closeSnackbar(key)}>
<IconX />
</Button>
)
}
})
onConfirm(createResp.data.id)
}
} catch (error) {
enqueueSnackbar({
message: `Failed to import keys: ${
typeof error.response.data === 'object' ? error.response.data.message : error.response.data
}`,
options: {
key: new Date().getTime() + Math.random(),
variant: 'error',
persist: true,
action: (key) => (
<Button style={{ color: 'white' }} onClick={() => closeSnackbar(key)}>
<IconX />
</Button>
)
}
})
onCancel()
}
}
const component = show ? (
<Dialog
fullWidth
maxWidth='sm'
open={show}
onClose={onCancel}
aria-labelledby='alert-dialog-title'
aria-describedby='alert-dialog-description'
>
<DialogTitle sx={{ fontSize: '1rem' }} id='alert-dialog-title'>
<div style={{ display: 'flex', flexDirection: 'row', alignItems: 'center' }}>
<IconFileUpload style={{ marginRight: '10px' }} />
Import API Keys
</div>
</DialogTitle>
<DialogContent>
<Box sx={{ p: 2 }}>
<Stack sx={{ position: 'relative' }} direction='row'>
<Typography variant='overline'>
Import api.json file
<span style={{ color: 'red' }}>&nbsp;*</span>
</Typography>
</Stack>
<File
disabled={false}
fileType='.json'
onChange={(newValue) => setSelectedFile(newValue)}
value={selectedFile ?? 'Choose a file to upload'}
/>
</Box>
<Box sx={{ p: 2 }}>
<Stack sx={{ position: 'relative' }} direction='row'>
<Typography variant='overline'>
Import Mode
<span style={{ color: 'red' }}>&nbsp;*</span>
</Typography>
</Stack>
<Dropdown
key={importMode}
name={importMode}
options={importModes}
onSelect={(newValue) => setImportMode(newValue)}
value={importMode ?? 'choose an option'}
/>
</Box>
</DialogContent>
<DialogActions>
<Button onClick={() => onCancel()}>{dialogProps.cancelButtonName}</Button>
<StyledButton disabled={!selectedFile} variant='contained' onClick={importKeys}>
{dialogProps.confirmButtonName}
</StyledButton>
</DialogActions>
<ConfirmDialog />
</Dialog>
) : null
return createPortal(component, portalElement)
}
UploadJSONFileDialog.propTypes = {
show: PropTypes.bool,
dialogProps: PropTypes.object,
onCancel: PropTypes.func,
onConfirm: PropTypes.func
}
export default UploadJSONFileDialog

46
packages/ui/src/views/apikey/index.jsx
View File

@ -44,8 +44,20 @@ import useConfirm from '@/hooks/useConfirm'
import useNotifier from '@/utils/useNotifier'
// Icons
import { IconTrash, IconEdit, IconCopy, IconChevronsUp, IconChevronsDown, IconX, IconPlus, IconEye, IconEyeOff } from '@tabler/icons-react'
import {
IconTrash,
IconEdit,
IconCopy,
IconChevronsUp,
IconChevronsDown,
IconX,
IconPlus,
IconEye,
IconEyeOff,
IconFileUpload
} from '@tabler/icons-react'
import APIEmptySVG from '@/assets/images/api_empty.svg'
import UploadJSONFileDialog from '@/views/apikey/UploadJSONFileDialog'
// ==============================|| APIKey ||============================== //
@ -200,6 +212,9 @@ const APIKey = () => {
const [showApiKeys, setShowApiKeys] = useState([])
const openPopOver = Boolean(anchorEl)
const [showUploadDialog, setShowUploadDialog] = useState(false)
const [uploadDialogProps, setUploadDialogProps] = useState({})
const [search, setSearch] = useState('')
const onSearchChange = (event) => {
setSearch(event.target.value)
@ -254,6 +269,17 @@ const APIKey = () => {
setShowDialog(true)
}
const uploadDialog = () => {
const dialogProp = {
type: 'ADD',
cancelButtonName: 'Cancel',
confirmButtonName: 'Upload',
data: {}
}
setUploadDialogProps(dialogProp)
setShowUploadDialog(true)
}
const deleteKey = async (key) => {
const confirmPayload = {
title: `Delete`,
@ -308,6 +334,7 @@ const APIKey = () => {
const onConfirm = () => {
setShowDialog(false)
setShowUploadDialog(false)
getAllAPIKeysApi.request()
}
@ -341,6 +368,15 @@ const APIKey = () => {
) : (
<Stack flexDirection='column' sx={{ gap: 3 }}>
<ViewHeader onSearchChange={onSearchChange} search={true} searchPlaceholder='Search API Keys' title='API Keys'>
<Button
variant='outlined'
sx={{ borderRadius: 2, height: '100%' }}
onClick={uploadDialog}
startIcon={<IconFileUpload />}
id='btn_importApiKeys'
>
Import
</Button>
<StyledButton
variant='contained'
sx={{ borderRadius: 2, height: '100%' }}
@ -468,6 +504,14 @@ const APIKey = () => {
onConfirm={onConfirm}
setError={setError}
></APIKeyDialog>
{showUploadDialog && (
<UploadJSONFileDialog
show={showUploadDialog}
dialogProps={uploadDialogProps}
onCancel={() => setShowUploadDialog(false)}
onConfirm={onConfirm}
></UploadJSONFileDialog>
)}
<ConfirmDialog />
</>
)