Feature: Option to add apikeys to the DB instead of api.json. (#2783)

* Feature: Option to add apikeys to the DB instead of api.json.

* add api storage type env variable

* code cleanup and simplification.

* md table fixes

---------

Co-authored-by: Henry <hzj94@hotmail.com>

CONTRIBUTING.md

@@ -121,7 +121,7 @@ Flowise has 3 different modules in a single mono repository.
 Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
 
 | Variable                     | Description                                                                                     | Type                                             | Default                             |
-| ---------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
+| ---------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------ |-------------------------------------| 
 | PORT                         | The HTTP port Flowise runs on                                                                   | Number                                           | 3000                                |
 | CORS_ORIGINS                 | The allowed origins for all cross-origin HTTP calls                                             | String                                           |                                     |
 | IFRAME_ORIGINS               | The allowed origins for iframe src embedding                                                    | String                                           |                                     |
@@ -133,7 +133,8 @@ Flowise support different environment variables to configure your instance. You
 | LOG_PATH                     | Location where log files are stored                                                             | String                                           | `your-path/Flowise/logs`            |
 | LOG_LEVEL                    | Different levels of logs                                                                        | Enum String: `error`, `info`, `verbose`, `debug` | `info`                              |
 | LOG_JSON_SPACES              | Spaces to beautify JSON logs                                                                    |                                                  | 2                                   |
-| APIKEY_PATH                  | Location where api keys are saved                                                               | String                                           | `your-path/Flowise/packages/server` |
+| APIKEY_STORAGE_TYPE          | To store api keys on a JSON file or database. Default is `json`                                 | Enum String: `json`, `db`                        | `json`                              | 
+| APIKEY_PATH                  | Location where api keys are saved when `APIKEY_STORAGE_TYPE` is `json`                          | String                                           | `your-path/Flowise/packages/server` |
 | TOOL_FUNCTION_BUILTIN_DEP    | NodeJS built-in modules to be used for Tool Function                                            | String                                           |                                     |
 | TOOL_FUNCTION_EXTERNAL_DEP   | External modules to be used for Tool Function                                                   | String                                           |                                     |
 | DATABASE_TYPE                | Type of database to store the flowise data                                                      | Enum String: `sqlite`, `mysql`, `postgres`       | `sqlite`                            |
@@ -146,8 +147,8 @@ Flowise support different environment variables to configure your instance. You
 | DATABASE_SSL_KEY_BASE64      | Database SSL client cert in base64 (takes priority over DATABASE_SSL)                           | Boolean                                          | false                               |
 | DATABASE_SSL                 | Database connection overssl (When DATABASE_TYPE is postgre)                                     | Boolean                                          | false                               |
 | SECRETKEY_PATH               | Location where encryption key (used to encrypt/decrypt credentials) is saved                    | String                                           | `your-path/Flowise/packages/server` |
-| FLOWISE_SECRETKEY_OVERWRITE  | Encryption key to be used instead of the key stored in SECRETKEY_PATH                           | String                                           |
-| DISABLE_FLOWISE_TELEMETRY    | Turn off telemetry                                                                              | Boolean                                          |
+| FLOWISE_SECRETKEY_OVERWRITE  | Encryption key to be used instead of the key stored in SECRETKEY_PATH                           | String                                           |                                     |
+| DISABLE_FLOWISE_TELEMETRY    | Turn off telemetry                                                                              | Boolean                                          |                                     |
 | MODEL_LIST_CONFIG_JSON       | File path to load list of models from your local config file                                    | String                                           | `/your_model_list_config_file_path` |
 | STORAGE_TYPE                 | Type of storage for uploaded files. default is `local`                                          | Enum String: `s3`, `local`                       | `local`                             |
 | BLOB_STORAGE_PATH            | Local folder path where uploaded files are stored when `STORAGE_TYPE` is `local`                | String                                           | `your-home-dir/.flowise/storage`    |

docker/.env.example

@@ -48,3 +48,5 @@ BLOB_STORAGE_PATH=/root/.flowise/storage
 # S3_STORAGE_SECRET_ACCESS_KEY=<your-secret-key>
 # S3_STORAGE_REGION=us-west-2
 # S3_ENDPOINT_URL=<custom-s3-endpoint-url>
+
+# APIKEY_STORAGE_TYPE=json (json | db)

i18n/CONTRIBUTING-ZH.md

@@ -128,7 +128,8 @@ Flowise 支持不同的环境变量来配置您的实例。您可以在 `package
 | DEBUG                        | 打印组件的日志                                                       | 布尔值                                          |                                     |
 | LOG_PATH                     | 存储日志文件的位置                                                   | 字符串                                          | `your-path/Flowise/logs`            |
 | LOG_LEVEL                    | 日志的不同级别                                                       | 枚举字符串: `error`, `info`, `verbose`, `debug` | `info`                              |
-| APIKEY_PATH                  | 存储 API 密钥的位置                                                  | 字符串                                          | `your-path/Flowise/packages/server` |
+| APIKEY_STORAGE_TYPE          | 存储 API 密钥的存储类型                                              | 枚举字符串: `json`, `db`                        | `json`                              |
+| APIKEY_PATH                  | 存储 API 密钥的位置, 当`APIKEY_STORAGE_TYPE`是`json`                 | 字符串                                          | `your-path/Flowise/packages/server` |
 | TOOL_FUNCTION_BUILTIN_DEP    | 用于工具函数的 NodeJS 内置模块                                       | 字符串                                          |                                     |
 | TOOL_FUNCTION_EXTERNAL_DEP   | 用于工具函数的外部模块                                               | 字符串                                          |                                     |
 | DATABASE_TYPE                | 存储 flowise 数据的数据库类型                                        | 枚举字符串: `sqlite`, `mysql`, `postgres`       | `sqlite`                            |

packages/server/.env.example

@@ -48,3 +48,5 @@ PORT=3000
 # S3_STORAGE_SECRET_ACCESS_KEY=<your-secret-key>
 # S3_STORAGE_REGION=us-west-2
 # S3_ENDPOINT_URL=<custom-s3-endpoint-url>
+
+# APIKEY_STORAGE_TYPE=json (json | db)

packages/server/src/AppConfig.ts

@@ -0,0 +1,6 @@
+export const appConfig = {
+    apiKeys: {
+        storageType: process.env.APIKEY_STORAGE_TYPE ? process.env.APIKEY_STORAGE_TYPE.toLowerCase() : 'json'
+    }
+    // todo: add more config options here like database, log, storage, credential and allow modification from UI
+}

packages/server/src/Interface.ts

@@ -263,5 +263,13 @@ export interface IUploadFileSizeAndTypes {
     maxUploadSize: number
 }
 
+export interface IApiKey {
+    id: string
+    keyName: string
+    apiKey: string
+    apiSecret: string
+    updatedDate: Date
+}
+
 // DocumentStore related
 export * from './Interface.DocumentStore'

packages/server/src/commands/start.ts

@@ -24,6 +24,7 @@ export default class Start extends Command {
         IFRAME_ORIGINS: Flags.string(),
         DEBUG: Flags.string(),
         BLOB_STORAGE_PATH: Flags.string(),
+        APIKEY_STORAGE_TYPE: Flags.string(),
         APIKEY_PATH: Flags.string(),
         SECRETKEY_PATH: Flags.string(),
         FLOWISE_SECRETKEY_OVERWRITE: Flags.string(),
@@ -100,6 +101,7 @@ export default class Start extends Command {
         // Authorization
         if (flags.FLOWISE_USERNAME) process.env.FLOWISE_USERNAME = flags.FLOWISE_USERNAME
         if (flags.FLOWISE_PASSWORD) process.env.FLOWISE_PASSWORD = flags.FLOWISE_PASSWORD
+        if (flags.APIKEY_STORAGE_TYPE) process.env.APIKEY_STORAGE_TYPE = flags.APIKEY_STORAGE_TYPE
         if (flags.APIKEY_PATH) process.env.APIKEY_PATH = flags.APIKEY_PATH
 
         // API Configuration

packages/server/src/controllers/apikey/index.ts

@@ -41,6 +41,19 @@ const updateApiKey = async (req: Request, res: Response, next: NextFunction) =>
     }
 }
 
+// Import Keys from JSON file
+const importKeys = async (req: Request, res: Response, next: NextFunction) => {
+    try {
+        if (typeof req.body === 'undefined' || !req.body.jsonFile) {
+            throw new InternalFlowiseError(StatusCodes.PRECONDITION_FAILED, `Error: apikeyController.importKeys - body not provided!`)
+        }
+        const apiResponse = await apikeyService.importKeys(req.body)
+        return res.json(apiResponse)
+    } catch (error) {
+        next(error)
+    }
+}
+
 // Delete api key
 const deleteApiKey = async (req: Request, res: Response, next: NextFunction) => {
     try {
@@ -72,5 +85,6 @@ export default {
     deleteApiKey,
     getAllApiKeys,
     updateApiKey,
-    verifyApiKey
+    verifyApiKey,
+    importKeys
 }

packages/server/src/controllers/chatflows/index.ts

@@ -1,11 +1,11 @@
 import { NextFunction, Request, Response } from 'express'
 import { StatusCodes } from 'http-status-codes'
+import apiKeyService from '../../services/apikey'
 import { ChatFlow } from '../../database/entities/ChatFlow'
+import { createRateLimiter } from '../../utils/rateLimit'
 import { InternalFlowiseError } from '../../errors/internalFlowiseError'
 import { ChatflowType } from '../../Interface'
 import chatflowsService from '../../services/chatflows'
-import { getApiKey } from '../../utils/apiKey'
-import { createRateLimiter } from '../../utils/rateLimit'
 
 const checkIfChatflowIsValidForStreaming = async (req: Request, res: Response, next: NextFunction) => {
     try {
@@ -67,7 +67,7 @@ const getChatflowByApiKey = async (req: Request, res: Response, next: NextFuncti
                 `Error: chatflowsRouter.getChatflowByApiKey - apikey not provided!`
             )
         }
-        const apikey = await getApiKey(req.params.apikey)
+        const apikey = await apiKeyService.getApiKey(req.params.apikey)
         if (!apikey) {
             return res.status(401).send('Unauthorized')
         }

packages/server/src/database/entities/ApiKey.ts

@@ -0,0 +1,21 @@
+import { Column, Entity, PrimaryColumn, UpdateDateColumn } from 'typeorm'
+import { IApiKey } from '../../Interface'
+
+@Entity('apikey')
+export class ApiKey implements IApiKey {
+    @PrimaryColumn({ type: 'varchar', length: 20 })
+    id: string
+
+    @Column({ type: 'text' })
+    apiKey: string
+
+    @Column({ type: 'text' })
+    apiSecret: string
+
+    @Column({ type: 'text' })
+    keyName: string
+
+    @Column({ type: 'timestamp' })
+    @UpdateDateColumn()
+    updatedDate: Date
+}

packages/server/src/database/entities/index.ts

@@ -9,6 +9,7 @@ import { DocumentStore } from './DocumentStore'
 import { DocumentStoreFileChunk } from './DocumentStoreFileChunk'
 import { Lead } from './Lead'
 import { UpsertHistory } from './UpsertHistory'
+import { ApiKey } from './ApiKey'
 
 export const entities = {
     ChatFlow,
@@ -21,5 +22,6 @@ export const entities = {
     DocumentStore,
     DocumentStoreFileChunk,
     Lead,
-    UpsertHistory
+    UpsertHistory,
+    ApiKey
 }

packages/server/src/database/migrations/mariadb/1720230151480-AddApiKey.ts

@@ -0,0 +1,20 @@
+import { MigrationInterface, QueryRunner } from 'typeorm'
+
+export class AddApiKey1720230151480 implements MigrationInterface {
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(
+            `CREATE TABLE IF NOT EXISTS \`apikey\` (
+                \`id\` varchar(36) NOT NULL,
+                \`apiKey\` varchar(255) NOT NULL,
+                \`apiSecret\` varchar(255) NOT NULL,
+                \`keyName\` varchar(255) NOT NULL,
+                \`updatedDate\` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
+                PRIMARY KEY (\`id\`)
+              ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci;`
+        )
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`DROP TABLE apikey`)
+    }
+}

packages/server/src/database/migrations/mariadb/index.ts

@@ -20,6 +20,7 @@ import { AddLead1710832127079 } from './1710832127079-AddLead'
 import { AddLeadToChatMessage1711538023578 } from './1711538023578-AddLeadToChatMessage'
 import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
 import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
+import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
 import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
 
 export const mariadbMigrations = [
@@ -45,5 +46,6 @@ export const mariadbMigrations = [
     AddLeadToChatMessage1711538023578,
     AddAgentReasoningToChatMessage1714679514451,
     AddTypeToChatFlow1766759476232,
+    AddApiKey1720230151480,
     AddActionToChatMessage1721078251523
 ]

packages/server/src/database/migrations/mysql/1720230151480-AddApiKey.ts

@@ -0,0 +1,20 @@
+import { MigrationInterface, QueryRunner } from 'typeorm'
+
+export class AddApiKey1720230151480 implements MigrationInterface {
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(
+            `CREATE TABLE IF NOT EXISTS \`apikey\` (
+                \`id\` varchar(36) NOT NULL,
+                \`apiKey\` varchar(255) NOT NULL,
+                \`apiSecret\` varchar(255) NOT NULL,
+                \`keyName\` varchar(255) NOT NULL,
+                \`updatedDate\` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
+                PRIMARY KEY (\`id\`)
+              ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;`
+        )
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`DROP TABLE apikey`)
+    }
+}

packages/server/src/database/migrations/mysql/index.ts

@@ -20,6 +20,7 @@ import { AddLead1710832127079 } from './1710832127079-AddLead'
 import { AddLeadToChatMessage1711538023578 } from './1711538023578-AddLeadToChatMessage'
 import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
 import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
+import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
 import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
 
 export const mysqlMigrations = [
@@ -45,5 +46,6 @@ export const mysqlMigrations = [
     AddLeadToChatMessage1711538023578,
     AddAgentReasoningToChatMessage1714679514451,
     AddTypeToChatFlow1766759476232,
+    AddApiKey1720230151480,
     AddActionToChatMessage1721078251523
 ]

packages/server/src/database/migrations/postgres/1720230151480-AddApiKey.ts

@@ -0,0 +1,20 @@
+import { MigrationInterface, QueryRunner } from 'typeorm'
+
+export class AddApiKey1720230151480 implements MigrationInterface {
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(
+            `CREATE TABLE IF NOT EXISTS apikey (
+                id uuid NOT NULL DEFAULT uuid_generate_v4(),
+                "apiKey" varchar NOT NULL,
+                "apiSecret" varchar NOT NULL,
+                "keyName" varchar NOT NULL,
+                "updatedDate" timestamp NOT NULL DEFAULT now(),
+                CONSTRAINT "PK_96109043dd704f53-9830ab78f0" PRIMARY KEY (id)
+            );`
+        )
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`DROP TABLE apikey`)
+    }
+}

packages/server/src/database/migrations/postgres/index.ts

@@ -21,6 +21,7 @@ import { AddLead1710832137905 } from './1710832137905-AddLead'
 import { AddLeadToChatMessage1711538016098 } from './1711538016098-AddLeadToChatMessage'
 import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
 import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
+import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
 import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
 
 export const postgresMigrations = [
@@ -47,5 +48,6 @@ export const postgresMigrations = [
     AddLeadToChatMessage1711538016098,
     AddAgentReasoningToChatMessage1714679514451,
     AddTypeToChatFlow1766759476232,
+    AddApiKey1720230151480,
     AddActionToChatMessage1721078251523
 ]

packages/server/src/database/migrations/sqlite/1720230151480-AddApiKey.ts

@@ -0,0 +1,17 @@
+import { MigrationInterface, QueryRunner } from 'typeorm'
+
+export class AddApiKey1720230151480 implements MigrationInterface {
+    public async up(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(
+            `CREATE TABLE IF NOT EXISTS "apikey" ("id" varchar PRIMARY KEY NOT NULL, 
+                "apiKey" varchar NOT NULL, 
+                "apiSecret" varchar NOT NULL, 
+                "keyName" varchar NOT NULL, 
+                "updatedDate" datetime NOT NULL DEFAULT (datetime('now')));`
+        )
+    }
+
+    public async down(queryRunner: QueryRunner): Promise<void> {
+        await queryRunner.query(`DROP TABLE IF EXISTS "apikey";`)
+    }
+}

packages/server/src/database/migrations/sqlite/index.ts

@@ -21,6 +21,7 @@ import { AddLeadToChatMessage1711537986113 } from './1711537986113-AddLeadToChat
 import { AddAgentReasoningToChatMessage1714679514451 } from './1714679514451-AddAgentReasoningToChatMessage'
 import { AddTypeToChatFlow1766759476232 } from './1766759476232-AddTypeToChatFlow'
 import { AddActionToChatMessage1721078251523 } from './1721078251523-AddActionToChatMessage'
+import { AddApiKey1720230151480 } from './1720230151480-AddApiKey'
 
 export const sqliteMigrations = [
     Init1693835579790,
@@ -45,5 +46,6 @@ export const sqliteMigrations = [
     AddLeadToChatMessage1711537986113,
     AddAgentReasoningToChatMessage1714679514451,
     AddTypeToChatFlow1766759476232,
+    AddApiKey1720230151480,
     AddActionToChatMessage1721078251523
 ]

packages/server/src/routes/apikey/index.ts

@@ -4,6 +4,7 @@ const router = express.Router()
 
 // CREATE
 router.post('/', apikeyController.createApiKey)
+router.post('/import', apikeyController.importKeys)
 
 // READ
 router.get('/', apikeyController.getAllApiKeys)

packages/server/src/services/apikey/index.ts

@@ -1,25 +1,94 @@
 import { StatusCodes } from 'http-status-codes'
-import { addAPIKey, deleteAPIKey, getAPIKeys, updateAPIKey } from '../../utils/apiKey'
+import {
+    addAPIKey as addAPIKey_json,
+    deleteAPIKey as deleteAPIKey_json,
+    generateAPIKey,
+    generateSecretHash,
+    getApiKey as getApiKey_json,
+    getAPIKeys as getAPIKeys_json,
+    updateAPIKey as updateAPIKey_json,
+    replaceAllAPIKeys as replaceAllAPIKeys_json,
+    importKeys as importKeys_json
+} from '../../utils/apiKey'
 import { addChatflowsCount } from '../../utils/addChatflowsCount'
-import { getApiKey } from '../../utils/apiKey'
 import { InternalFlowiseError } from '../../errors/internalFlowiseError'
 import { getErrorMessage } from '../../errors/utils'
+import { getRunningExpressApp } from '../../utils/getRunningExpressApp'
+import { ApiKey } from '../../database/entities/ApiKey'
+import { appConfig } from '../../AppConfig'
+import { randomBytes } from 'crypto'
+import { Not, IsNull } from 'typeorm'
+
+const _apikeysStoredInJson = (): boolean => {
+    return appConfig.apiKeys.storageType === 'json'
+}
+
+const _apikeysStoredInDb = (): boolean => {
+    return appConfig.apiKeys.storageType === 'db'
+}
 
 const getAllApiKeys = async () => {
     try {
-        const keys = await getAPIKeys()
-        const dbResponse = await addChatflowsCount(keys)
-        return dbResponse
+        if (_apikeysStoredInJson()) {
+            const keys = await getAPIKeys_json()
+            return await addChatflowsCount(keys)
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            let keys = await appServer.AppDataSource.getRepository(ApiKey).find()
+            if (keys.length === 0) {
+                await createApiKey('DefaultKey')
+                keys = await appServer.AppDataSource.getRepository(ApiKey).find()
+            }
+            return await addChatflowsCount(keys)
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
     } catch (error) {
         throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.getAllApiKeys - ${getErrorMessage(error)}`)
     }
 }
 
+const getApiKey = async (keyName: string) => {
+    try {
+        if (_apikeysStoredInJson()) {
+            return getApiKey_json(keyName)
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            const currentKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
+                keyName: keyName
+            })
+            if (!currentKey) {
+                return undefined
+            }
+            return currentKey
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
+    } catch (error) {
+        throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.createApiKey - ${getErrorMessage(error)}`)
+    }
+}
+
 const createApiKey = async (keyName: string) => {
     try {
-        const keys = await addAPIKey(keyName)
-        const dbResponse = await addChatflowsCount(keys)
-        return dbResponse
+        if (_apikeysStoredInJson()) {
+            const keys = await addAPIKey_json(keyName)
+            return await addChatflowsCount(keys)
+        } else if (_apikeysStoredInDb()) {
+            const apiKey = generateAPIKey()
+            const apiSecret = generateSecretHash(apiKey)
+            const appServer = getRunningExpressApp()
+            const newKey = new ApiKey()
+            newKey.id = randomBytes(16).toString('hex')
+            newKey.apiKey = apiKey
+            newKey.apiSecret = apiSecret
+            newKey.keyName = keyName
+            const key = appServer.AppDataSource.getRepository(ApiKey).create(newKey)
+            await appServer.AppDataSource.getRepository(ApiKey).save(key)
+            return getAllApiKeys()
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
     } catch (error) {
         throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.createApiKey - ${getErrorMessage(error)}`)
     }
@@ -28,9 +97,23 @@ const createApiKey = async (keyName: string) => {
 // Update api key
 const updateApiKey = async (id: string, keyName: string) => {
     try {
-        const keys = await updateAPIKey(id, keyName)
-        const dbResponse = await addChatflowsCount(keys)
-        return dbResponse
+        if (_apikeysStoredInJson()) {
+            const keys = await updateAPIKey_json(id, keyName)
+            return await addChatflowsCount(keys)
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            const currentKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
+                id: id
+            })
+            if (!currentKey) {
+                throw new InternalFlowiseError(StatusCodes.NOT_FOUND, `ApiKey ${currentKey} not found`)
+            }
+            currentKey.keyName = keyName
+            await appServer.AppDataSource.getRepository(ApiKey).save(currentKey)
+            return getAllApiKeys()
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
     } catch (error) {
         throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.updateApiKey - ${getErrorMessage(error)}`)
     }
@@ -38,22 +121,123 @@ const updateApiKey = async (id: string, keyName: string) => {
 
 const deleteApiKey = async (id: string) => {
     try {
-        const keys = await deleteAPIKey(id)
-        const dbResponse = await addChatflowsCount(keys)
-        return dbResponse
+        if (_apikeysStoredInJson()) {
+            const keys = await deleteAPIKey_json(id)
+            return await addChatflowsCount(keys)
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            const dbResponse = await appServer.AppDataSource.getRepository(ApiKey).delete({ id: id })
+            if (!dbResponse) {
+                throw new InternalFlowiseError(StatusCodes.NOT_FOUND, `ApiKey ${id} not found`)
+            }
+            return getAllApiKeys()
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
     } catch (error) {
         throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.deleteApiKey - ${getErrorMessage(error)}`)
     }
 }
 
+const importKeys = async (body: any) => {
+    try {
+        const jsonFile = body.jsonFile
+        const splitDataURI = jsonFile.split(',')
+        if (splitDataURI[0] !== 'data:application/json;base64') {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Invalid dataURI`)
+        }
+        const bf = Buffer.from(splitDataURI[1] || '', 'base64')
+        const plain = bf.toString('utf8')
+        const keys = JSON.parse(plain)
+        if (_apikeysStoredInJson()) {
+            if (body.importMode === 'replaceAll') {
+                await replaceAllAPIKeys_json(keys)
+            } else {
+                await importKeys_json(keys, body.importMode)
+            }
+            return await addChatflowsCount(keys)
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            const allApiKeys = await appServer.AppDataSource.getRepository(ApiKey).find()
+            if (body.importMode === 'replaceAll') {
+                await appServer.AppDataSource.getRepository(ApiKey).delete({
+                    id: Not(IsNull())
+                })
+            }
+            if (body.importMode === 'errorIfExist') {
+                // if importMode is errorIfExist, check for existing keys and raise error before any modification to the DB
+                for (const key of keys) {
+                    const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
+                    if (keyNameExists) {
+                        throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Key with name ${key.keyName} already exists`)
+                    }
+                }
+            }
+            // iterate through the keys and add them to the database
+            for (const key of keys) {
+                const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
+                if (keyNameExists) {
+                    const keyIndex = allApiKeys.findIndex((k) => k.keyName === key.keyName)
+                    switch (body.importMode) {
+                        case 'overwriteIfExist': {
+                            const currentKey = allApiKeys[keyIndex]
+                            currentKey.id = key.id
+                            currentKey.apiKey = key.apiKey
+                            currentKey.apiSecret = key.apiSecret
+                            await appServer.AppDataSource.getRepository(ApiKey).save(currentKey)
+                            break
+                        }
+                        case 'ignoreIfExist': {
+                            // ignore this key and continue
+                            continue
+                        }
+                        case 'errorIfExist': {
+                            // should not reach here as we have already checked for existing keys
+                            throw new Error(`Key with name ${key.keyName} already exists`)
+                        }
+                        default: {
+                            throw new Error(`Unknown overwrite option ${body.importMode}`)
+                        }
+                    }
+                } else {
+                    const newKey = new ApiKey()
+                    newKey.id = key.id
+                    newKey.apiKey = key.apiKey
+                    newKey.apiSecret = key.apiSecret
+                    newKey.keyName = key.keyName
+                    const newKeyEntity = appServer.AppDataSource.getRepository(ApiKey).create(newKey)
+                    await appServer.AppDataSource.getRepository(ApiKey).save(newKeyEntity)
+                }
+            }
+            return getAllApiKeys()
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
+        }
+    } catch (error) {
+        throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `Error: apikeyService.importKeys - ${getErrorMessage(error)}`)
+    }
+}
+
 const verifyApiKey = async (paramApiKey: string): Promise<string> => {
     try {
-        const apiKey = await getApiKey(paramApiKey)
-        if (!apiKey) {
-            throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
+        if (_apikeysStoredInJson()) {
+            const apiKey = await getApiKey_json(paramApiKey)
+            if (!apiKey) {
+                throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
+            }
+            return 'OK'
+        } else if (_apikeysStoredInDb()) {
+            const appServer = getRunningExpressApp()
+            const apiKey = await appServer.AppDataSource.getRepository(ApiKey).findOneBy({
+                apiKey: paramApiKey
+            })
+            if (!apiKey) {
+                throw new InternalFlowiseError(StatusCodes.UNAUTHORIZED, `Unauthorized`)
+            }
+            return 'OK'
+        } else {
+            throw new InternalFlowiseError(StatusCodes.INTERNAL_SERVER_ERROR, `UNKNOWN APIKEY_STORAGE_TYPE`)
         }
-        const dbResponse = 'OK'
-        return dbResponse
     } catch (error) {
         if (error instanceof InternalFlowiseError && error.statusCode === StatusCodes.UNAUTHORIZED) {
             throw error
@@ -71,5 +255,7 @@ export default {
     deleteApiKey,
     getAllApiKeys,
     updateApiKey,
-    verifyApiKey
+    verifyApiKey,
+    getApiKey,
+    importKeys
 }

packages/server/src/utils/apiKey.ts

@@ -4,6 +4,7 @@ import moment from 'moment'
 import fs from 'fs'
 import path from 'path'
 import logger from './logger'
+import { appConfig } from '../AppConfig'
 
 /**
  * Returns the api key path
@@ -50,6 +51,9 @@ export const compareKeys = (storedKey: string, suppliedKey: string): boolean =>
  * @returns {Promise<ICommonObject[]>}
  */
 export const getAPIKeys = async (): Promise<ICommonObject[]> => {
+    if (appConfig.apiKeys.storageType !== 'json') {
+        return []
+    }
     try {
         const content = await fs.promises.readFile(getAPIKeyPath(), 'utf8')
         return JSON.parse(content)
@@ -94,6 +98,47 @@ export const addAPIKey = async (keyName: string): Promise<ICommonObject[]> => {
     return content
 }
 
+/**
+ * import API keys
+ * @param {[]} keys
+ * @returns {Promise<ICommonObject[]>}
+ */
+export const importKeys = async (keys: any[], importMode: string): Promise<ICommonObject[]> => {
+    const allApiKeys = await getAPIKeys()
+    // if importMode is errorIfExist, check for existing keys and raise error before any modification to the file
+    if (importMode === 'errorIfExist') {
+        for (const key of keys) {
+            const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
+            if (keyNameExists) {
+                throw new Error(`Key with name ${key.keyName} already exists`)
+            }
+        }
+    }
+    for (const key of keys) {
+        // Check if keyName already exists, if overwrite is false, raise an error else overwrite the key
+        const keyNameExists = allApiKeys.find((k) => k.keyName === key.keyName)
+        if (keyNameExists) {
+            const keyIndex = allApiKeys.findIndex((k) => k.keyName === key.keyName)
+            switch (importMode) {
+                case 'overwriteIfExist':
+                    allApiKeys[keyIndex] = key
+                    continue
+                case 'ignoreIfExist':
+                    // ignore this key and continue
+                    continue
+                case 'errorIfExist':
+                    // should not reach here as we have already checked for existing keys
+                    throw new Error(`Key with name ${key.keyName} already exists`)
+                default:
+                    throw new Error(`Unknown overwrite option ${importMode}`)
+            }
+        }
+        allApiKeys.push(key)
+    }
+    await fs.promises.writeFile(getAPIKeyPath(), JSON.stringify(allApiKeys), 'utf8')
+    return allApiKeys
+}
+
 /**
  * Get API Key details
  * @param {string} apiKey

packages/server/src/utils/validateKey.ts

@@ -1,7 +1,7 @@
 import { Request } from 'express'
 import { ChatFlow } from '../database/entities/ChatFlow'
-import { getAPIKeys, compareKeys } from './apiKey'
-
+import { compareKeys } from './apiKey'
+import apikeyService from '../services/apikey'
 /**
  * Validate API Key
  * @param {Request} req
@@ -17,8 +17,8 @@ export const utilValidateKey = async (req: Request, chatflow: ChatFlow) => {
 
     const suppliedKey = authorizationHeader.split(`Bearer `).pop()
     if (suppliedKey) {
-        const keys = await getAPIKeys()
-        const apiSecret = keys.find((key) => key.id === chatFlowApiKeyId)?.apiSecret
+        const keys = await apikeyService.getAllApiKeys()
+        const apiSecret = keys.find((key: any) => key.id === chatFlowApiKeyId)?.apiSecret
         if (!compareKeys(apiSecret, suppliedKey)) return false
         return true
     }

packages/ui/src/api/apikey.js

@@ -8,9 +8,12 @@ const updateAPI = (id, body) => client.put(`/apikey/${id}`, body)
 
 const deleteAPI = (id) => client.delete(`/apikey/${id}`)
 
+const importAPI = (body) => client.post(`/apikey/import`, body)
+
 export default {
     getAllAPIKeys,
     createNewAPI,
     updateAPI,
-    deleteAPI
+    deleteAPI,
+    importAPI
 }

packages/ui/src/views/apikey/UploadJSONFileDialog.jsx

@@ -0,0 +1,186 @@
+import { createPortal } from 'react-dom'
+import PropTypes from 'prop-types'
+import { useState, useEffect } from 'react'
+import { useDispatch } from 'react-redux'
+import { enqueueSnackbar as enqueueSnackbarAction, closeSnackbar as closeSnackbarAction } from '@/store/actions'
+
+// Material
+import { Button, Dialog, DialogActions, DialogContent, DialogTitle, Box, Typography, Stack } from '@mui/material'
+
+// Project imports
+import { StyledButton } from '@/ui-component/button/StyledButton'
+import ConfirmDialog from '@/ui-component/dialog/ConfirmDialog'
+import { File } from '@/ui-component/file/File'
+
+// Icons
+import { IconFileUpload, IconX } from '@tabler/icons-react'
+
+// API
+import apikeyAPI from '@/api/apikey'
+
+// utils
+import useNotifier from '@/utils/useNotifier'
+
+// const
+import { HIDE_CANVAS_DIALOG, SHOW_CANVAS_DIALOG } from '@/store/actions'
+import { Dropdown } from '@/ui-component/dropdown/Dropdown'
+
+const importModes = [
+    {
+        label: 'Add & Overwrite',
+        name: 'overwriteIfExist',
+        description: 'Add keys and overwrite existing keys with the same name'
+    },
+    {
+        label: 'Add & Ignore',
+        name: 'ignoreIfExist',
+        description: 'Add keys and ignore existing keys with the same name'
+    },
+    {
+        label: 'Add & Verify',
+        name: 'errorIfExist',
+        description: 'Add Keys and throw error if key with same name exists'
+    },
+    {
+        label: 'Replace All',
+        name: 'replaceAll',
+        description: 'Replace all keys with the imported keys'
+    }
+]
+
+const UploadJSONFileDialog = ({ show, dialogProps, onCancel, onConfirm }) => {
+    const portalElement = document.getElementById('portal')
+
+    const dispatch = useDispatch()
+
+    // ==============================|| Snackbar ||============================== //
+
+    useNotifier()
+
+    const enqueueSnackbar = (...args) => dispatch(enqueueSnackbarAction(...args))
+    const closeSnackbar = (...args) => dispatch(closeSnackbarAction(...args))
+
+    const [selectedFile, setSelectedFile] = useState()
+    const [importMode, setImportMode] = useState('overwrite')
+
+    useEffect(() => {
+        return () => {
+            setSelectedFile()
+        }
+    }, [dialogProps])
+
+    useEffect(() => {
+        if (show) dispatch({ type: SHOW_CANVAS_DIALOG })
+        else dispatch({ type: HIDE_CANVAS_DIALOG })
+        return () => dispatch({ type: HIDE_CANVAS_DIALOG })
+    }, [show, dispatch])
+
+    const importKeys = async () => {
+        try {
+            const obj = {
+                importMode: importMode,
+                jsonFile: selectedFile
+            }
+            const createResp = await apikeyAPI.importAPI(obj)
+            if (createResp.data) {
+                enqueueSnackbar({
+                    message: 'Imported keys successfully!',
+                    options: {
+                        key: new Date().getTime() + Math.random(),
+                        variant: 'success',
+                        action: (key) => (
+                            <Button style={{ color: 'white' }} onClick={() => closeSnackbar(key)}>
+                                <IconX />
+                            </Button>
+                        )
+                    }
+                })
+                onConfirm(createResp.data.id)
+            }
+        } catch (error) {
+            enqueueSnackbar({
+                message: `Failed to import keys: ${
+                    typeof error.response.data === 'object' ? error.response.data.message : error.response.data
+                }`,
+                options: {
+                    key: new Date().getTime() + Math.random(),
+                    variant: 'error',
+                    persist: true,
+                    action: (key) => (
+                        <Button style={{ color: 'white' }} onClick={() => closeSnackbar(key)}>
+                            <IconX />
+                        </Button>
+                    )
+                }
+            })
+            onCancel()
+        }
+    }
+
+    const component = show ? (
+        <Dialog
+            fullWidth
+            maxWidth='sm'
+            open={show}
+            onClose={onCancel}
+            aria-labelledby='alert-dialog-title'
+            aria-describedby='alert-dialog-description'
+        >
+            <DialogTitle sx={{ fontSize: '1rem' }} id='alert-dialog-title'>
+                <div style={{ display: 'flex', flexDirection: 'row', alignItems: 'center' }}>
+                    <IconFileUpload style={{ marginRight: '10px' }} />
+                    Import API Keys
+                </div>
+            </DialogTitle>
+            <DialogContent>
+                <Box sx={{ p: 2 }}>
+                    <Stack sx={{ position: 'relative' }} direction='row'>
+                        <Typography variant='overline'>
+                            Import api.json file
+                            <span style={{ color: 'red' }}>&nbsp;*</span>
+                        </Typography>
+                    </Stack>
+                    <File
+                        disabled={false}
+                        fileType='.json'
+                        onChange={(newValue) => setSelectedFile(newValue)}
+                        value={selectedFile ?? 'Choose a file to upload'}
+                    />
+                </Box>
+                <Box sx={{ p: 2 }}>
+                    <Stack sx={{ position: 'relative' }} direction='row'>
+                        <Typography variant='overline'>
+                            Import Mode
+                            <span style={{ color: 'red' }}>&nbsp;*</span>
+                        </Typography>
+                    </Stack>
+                    <Dropdown
+                        key={importMode}
+                        name={importMode}
+                        options={importModes}
+                        onSelect={(newValue) => setImportMode(newValue)}
+                        value={importMode ?? 'choose an option'}
+                    />
+                </Box>
+            </DialogContent>
+            <DialogActions>
+                <Button onClick={() => onCancel()}>{dialogProps.cancelButtonName}</Button>
+                <StyledButton disabled={!selectedFile} variant='contained' onClick={importKeys}>
+                    {dialogProps.confirmButtonName}
+                </StyledButton>
+            </DialogActions>
+            <ConfirmDialog />
+        </Dialog>
+    ) : null
+
+    return createPortal(component, portalElement)
+}
+
+UploadJSONFileDialog.propTypes = {
+    show: PropTypes.bool,
+    dialogProps: PropTypes.object,
+    onCancel: PropTypes.func,
+    onConfirm: PropTypes.func
+}
+
+export default UploadJSONFileDialog

packages/ui/src/views/apikey/index.jsx

@@ -44,8 +44,20 @@ import useConfirm from '@/hooks/useConfirm'
 import useNotifier from '@/utils/useNotifier'
 
 // Icons
-import { IconTrash, IconEdit, IconCopy, IconChevronsUp, IconChevronsDown, IconX, IconPlus, IconEye, IconEyeOff } from '@tabler/icons-react'
+import {
+    IconTrash,
+    IconEdit,
+    IconCopy,
+    IconChevronsUp,
+    IconChevronsDown,
+    IconX,
+    IconPlus,
+    IconEye,
+    IconEyeOff,
+    IconFileUpload
+} from '@tabler/icons-react'
 import APIEmptySVG from '@/assets/images/api_empty.svg'
+import UploadJSONFileDialog from '@/views/apikey/UploadJSONFileDialog'
 
 // ==============================|| APIKey ||============================== //
 
@@ -200,6 +212,9 @@ const APIKey = () => {
     const [showApiKeys, setShowApiKeys] = useState([])
     const openPopOver = Boolean(anchorEl)
 
+    const [showUploadDialog, setShowUploadDialog] = useState(false)
+    const [uploadDialogProps, setUploadDialogProps] = useState({})
+
     const [search, setSearch] = useState('')
     const onSearchChange = (event) => {
         setSearch(event.target.value)
@@ -254,6 +269,17 @@ const APIKey = () => {
         setShowDialog(true)
     }
 
+    const uploadDialog = () => {
+        const dialogProp = {
+            type: 'ADD',
+            cancelButtonName: 'Cancel',
+            confirmButtonName: 'Upload',
+            data: {}
+        }
+        setUploadDialogProps(dialogProp)
+        setShowUploadDialog(true)
+    }
+
     const deleteKey = async (key) => {
         const confirmPayload = {
             title: `Delete`,
@@ -308,6 +334,7 @@ const APIKey = () => {
 
     const onConfirm = () => {
         setShowDialog(false)
+        setShowUploadDialog(false)
         getAllAPIKeysApi.request()
     }
 
@@ -341,6 +368,15 @@ const APIKey = () => {
                 ) : (
                     <Stack flexDirection='column' sx={{ gap: 3 }}>
                         <ViewHeader onSearchChange={onSearchChange} search={true} searchPlaceholder='Search API Keys' title='API Keys'>
+                            <Button
+                                variant='outlined'
+                                sx={{ borderRadius: 2, height: '100%' }}
+                                onClick={uploadDialog}
+                                startIcon={<IconFileUpload />}
+                                id='btn_importApiKeys'
+                            >
+                                Import
+                            </Button>
                             <StyledButton
                                 variant='contained'
                                 sx={{ borderRadius: 2, height: '100%' }}
@@ -468,6 +504,14 @@ const APIKey = () => {
                 onConfirm={onConfirm}
                 setError={setError}
             ></APIKeyDialog>
+            {showUploadDialog && (
+                <UploadJSONFileDialog
+                    show={showUploadDialog}
+                    dialogProps={uploadDialogProps}
+                    onCancel={() => setShowUploadDialog(false)}
+                    onConfirm={onConfirm}
+                ></UploadJSONFileDialog>
+            )}
             <ConfirmDialog />
         </>
     )