Giter VIP home page Giter VIP logo

macacajs / macaca-datahub Goto Github PK

View Code? Open in Web Editor NEW
475.0 28.0 55.0 15.43 MB

Continuous data provider for development, testing, staging and production. Just enjoy the data out-of-the-box.📦

Home Page: https://macacajs.github.io/macaca-datahub

License: MIT License

JavaScript 89.98% Shell 0.01% Dockerfile 0.10% HTML 0.31% Less 3.80% Makefile 0.13% Python 3.89% Java 1.79%
macaca datahub mock environment

macaca-datahub's People

Contributors

atian25 avatar bernardtolosajr avatar brucejcw avatar chan-chun avatar ed2nd avatar fengmk2 avatar gaius-qi avatar nightink avatar paradite avatar snapre avatar timelorder avatar vagusx avatar xudafeng avatar yesmeck avatar zhangyuheng avatar zhuyali1995 avatar zivyangll avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

macaca-datahub's Issues

[email protected]: DataHub start unsuccessfully: Error: Command failed: sequelize db:migrate

C:\Users\lee>npm i macaca-datahub -g
npm WARN deprecated [email protected]: CircularJSON is in maintenance only, flatted is its successor.
npm WARN deprecated [email protected]: If using 2.x branch, please upgrade to at least 2.1.6 to avoid a serious bug with socket data flow and an import issue introduced in 2.1.0
C:\Users\lee\AppData\Roaming\npm\datahub -> C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub\bin\datahub.js
C:\Users\lee\AppData\Roaming\npm\macaca-datahub -> C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub\bin\datahub.js

C:\Users\lee>datahub server

DataHub start unsuccessfully: Error: Command failed: C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub\node_modules\sequelize-cli\lib\sequelize db:migrate
C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub.sequelizerc:7
'migrations-path': path.join(__dirname, 'database, 'migrations'),
^^^^^^^^^^^^

SyntaxError: missing ) after argument list
at createScript (vm.js:80:10)
at Object.runInThisContext (vm.js:139:10)
at Module._compile (module.js:616:28)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at loadRCFile (C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub\node_modules\sequelize-cli\lib\core\yargs.js:27:78)

代理模式的代理地址期望所有请求共用

现在的代理模式是一对一的,比如说 /api/user 代理到 example.com/api/user,这个是没有问题的。

当我开启了全局模式,想实现代理所有请求,比如 /api/user /api/post /api/search ,分别到 example.com/api/user example.com/api/post example.com/api/search,于是我在其中一个接口上添加了一个 example.com 的代理,并开启全局模式。我发现所有的请求都打到了 example.com 而不是我上面想要的。

我猜测这个全局模式的意思是只是开启了所有的代理,但是每个请求要自己去配置代理的 Endpoint。也就是说代理地址每个请求不共用的。这个在开发的时候其实不太方便,因为后面的时候,几乎所有的接口都是通过代理走的,那么共用一个代理配置其实是比较方便的。我就不需要一个个接口去添加了

上面只是我的理解,以为我也没找到跟代理相关的文档,就使用的结果来说是这个样子的。希望后面能够补齐一下相关的文档。

datahub@2

发布

  • beta datahub-view publish tag
  • use datahub view@2 in config.default.js

数据迁移

  • 自动迁移数据

image
image

DataHub start unsuccessfully

DataHub start unsuccessfully: Error: Command failed: /usr/local/lib/node_modules/macaca-datahub/node_modules/sequelize-cli/lib/sequelize db:migrate
ERROR: Please install sqlite3 package manually

Sequelize CLI [Node: 11.4.0, CLI: 4.1.1, ORM: 4.41.2]

Loaded configuration file "database/config.js".

node 版本 v11.4.0

datahub网站需求

  1. 项目接口很多 (30+),可否二级分类?

  2. 网站部署上去之后谁都能看,谁都能改不安全,能否加权限控制

change sceneName bug(bug)

if i change the currentScene's sceneName, the interface /api/interface get the interface data, and there is a property currentScene remains the older sceneName

datahub支持restful api

现在datahub是以接口名作为key的,但是实际情况下,可能同一个接口名对应不同的http方法,表示不同的含义。比如获取app是get /app,新建app是post /app,更新app是put /app,其实url都一样,只是http方法不同而已

datahub server 启动报ERROR: Error reading "database\config.js"

datahub server

DataHub start unsuccessfully: Error: Command failed: C:\Users\lee\AppData\Roaming\npm\node_modules\macaca-datahub\node_modules\sequelize-cli\lib\sequelize db:migrate
ERROR: Error reading "database\config.js". Error: TypeError: Path must be a string. Received undefined

Sequelize CLI [Node: 8.11.3, CLI: 4.1.1, ORM: 4.42.0]

DataHub 2.x 升级指南

依赖更新

将版本号升级到 2

{
  "macaca-datahub": "2",
  "macaca-wd": "2", // 如果你用到该模块写单元测试,请升级到 2
}
 

不兼容处理

⚠️注意: DataHub 2 的底层服务变更,不再兼容 1.x 的数据格式。

已有的归档数据可以通过如下命令行进行升级:

$ ./node_modules/.bin/datahub-migrate-1-to-2 -s ./data

如果你用到 switchScene,请继续阅读

支持 method 属性,具体见 datahub-nodejs-sdk 文档。

请求字段描述根本无法编辑

编辑接口,接口字段根本无法编辑,双击也实现不了。
image

为何有的字段可以自动添加进去,有的就不行,而且没有文档描述如何添加字段。
image

datahub相关请求及界面路径添加/datahub前缀

  1. 需求
    datahub相关请求,路由地址添加/datahub前缀
  2. 最终实现效果
    项目启动地址为http://localhost:8888
    希望通过在本项目启动地址加上http://localhost:8888/datahub 直接访问配置界面,这样每一个datahub 跟着项目走,而不需要一个单独的地址(端口)去访问datahub配置界面。
    虽然现在通过代理也能实现,因为在datahub-view中的跳转地址是写死的,所以需要代理多个path,占用项目多个path/project,/api,/data,/dashboard等。这样有可能会导致冲突。如果能加上/datahub前缀,这样只需要重写/datahub一个path即可。

接口名称相同报错

我创建了两个项目,恰巧两个项目中的接口名称有重复的情况。第二个项目中新建对应的接口的时候,页面没有任何内容。
打开控制台发现报500的错误。
按理来说两个项目名字本身不相同,已经决定了不存在接口冲突的情况。
image

以下是报错信息:
SequelizeUniqueConstraintError: Validation error
SequelizeUniqueConstraintError: Validation error
at Query.formatError (/home/admin/.nvm/versions/node/v8.9.4/lib/node_modules/macaca-datahub/node_modules/_sequelize@4.37.7@sequelize/lib/dialects/sqlite/query.js:417:16)
at Statement.afterExecute (/home/admin/.nvm/versions/node/v8.9.4/lib/node_modules/macaca-datahub/node_modules/_sequelize@4.37.7@sequelize/lib/dialects/sqlite/query.js:119:32)
at Statement.replacement (/home/admin/.nvm/versions/node/v8.9.4/lib/node_modules/macaca-datahub/node_modules/[email protected]@sqlite3/lib/trace.js:19:31)

支持自定义备份方式,比如 oss/cdn 等配置

当我们将 datahub 部署到 docker 容器后,重启会遇到数据丢失的问题,希望 datahub 可以支持自定义备份方案

  • 每次备份时可以支持备份到 oss/cdn
  • 或者支持自定义备份方法,由用户自定将需要备份的内容进行备份

feat: support for dynamic URL

add capabilities to accept APIs with dynamic get parameter value. example:
/v4/account/pin_recovery/{msisdn}
where msisdn could be any value

datahub server提示js错误

datahub server

/usr/lib/node_modules/macaca-datahub/bin/datahub-server.js:47
(async () => {
^

SyntaxError: Unexpected token (
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:549:28)
at Object.Module._extensions..js (module.js:586:10)
at Module.load (module.js:494:32)
at tryModuleLoad (module.js:453:12)
at Function.Module._load (module.js:445:3)
at Module.runMain (module.js:611:10)
at run (bootstrap_node.js:394:7)
at startup (bootstrap_node.js:160:9)

macaca-wd的switchScene无效

写自动化测试,switchScene切换了dataHub的scene, 但是测试跑起来请求的还是dataHub选择的scene

macaca-datahub server 启动报错

macaca-datahub server
(node:22126) UnhandledPromiseRejectionWarning: ConnectionTimeoutError: Connect timeout for 3000ms, GET http://registry.cnpmjs.org/macaca-datahub/2 -2 (connected: false, keepalive socket: false, socketHandledRequests: 1, socketHandledResponses: 0)
headers: {}
at Timeout._onTimeout (/usr/local/share/.config/yarn/global/node_modules/urllib/lib/urllib.js:845:15)
at ontimeout (timers.js:427:11)
at tryOnTimeout (timers.js:289:5)
at listOnTimeout (timers.js:252:5)
at Timer.processTimers (timers.js:212:10)
(node:22126) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:22126) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

nodejs.SequelizeConnectionError: SQLITE_CANTOPEN: unable to open database file

  • env
 $ docker version                                                                                                                                                                                                                                  [16:51:52]
Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:13:02 2018
 OS/Arch:      darwin/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:22:38 2018
  OS/Arch:      linux/amd64
  Experimental: true

same as in ubuntu server host, just image error

  • docker-compose
version: '3'
services:
  macacaDatahub:
    container_name: "macaca-datahub"
    image: "macacajs/macaca-datahub"
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - "./data/.macaca-datahub/macaca-datahub.data:/root/.macaca-datahub/macaca-datahub.data"

Cuz

DataHub's database to local volume

# same error run as
docker run -it -v ~/.macaca-datahub/macaca-datahub.data:/root/.macaca-datahub/macaca-datahub.data -p 9200:9200 -p 9300:9300 --name macaca-datahub macacajs/macaca-datahub
  • error log
DataHub server start at: http://172.17.0.2:9200

2018-05-25 08:51:51,720 ERROR 28 nodejs.SequelizeConnectionError: SQLITE_CANTOPEN: unable to open database file
    at Database.connections.(anonymous function).lib.Database.err (/usr/local/lib/node_modules/macaca-datahub/node_modules/sequelize/lib/dialects/sqlite/connection-manager.js:66:63)
name: 'SequelizeConnectionError'
parent: { Error: SQLITE_CANTOPEN: unable to open database file errno: 14, code: 'SQLITE_CANTOPEN' }
original: { Error: SQLITE_CANTOPEN: unable to open database file errno: 14, code: 'SQLITE_CANTOPEN' }
pid: 28
hostname: ebf53b157b63

2018-05-25 08:51:51,722 ERROR 28 nodejs.SequelizeConnectionError: SQLITE_CANTOPEN: unable to open database file
    at Database.connections.(anonymous function).lib.Database.err (/usr/local/lib/node_modules/macaca-datahub/node_modules/sequelize/lib/dialects/sqlite/connection-manager.js:66:63)
name: 'SequelizeConnectionError'
parent: { Error: SQLITE_CANTOPEN: unable to open database file errno: 14, code: 'SQLITE_CANTOPEN' }
original: { Error: SQLITE_CANTOPEN: unable to open database file errno: 14, code: 'SQLITE_CANTOPEN' }
pid: 28
hostname: ebf53b157b63

2018-05-25 08:51:51,723 ERROR 28 [agent_worker] start error, exiting with code:1
2018-05-25 08:51:51,724 ERROR 28 [agent_worker] exit with code:1
2018-05-25 08:51:51,738 ERROR 18 nodejs.AgentWorkerDiedError: [master] agent_worker#1:28 died (code: 1, signal: null)
    at Master.onAgentExit (/usr/local/lib/node_modules/macaca-datahub/node_modules/egg-cluster/lib/master.js:317:17)
    at emitOne (events.js:116:13)
    at Master.emit (events.js:211:7)
    at Messenger.sendToMaster (/usr/local/lib/node_modules/macaca-datahub/node_modules/egg-cluster/lib/utils/messenger.js:122:17)
    at Messenger.send (/usr/local/lib/node_modules/macaca-datahub/node_modules/egg-cluster/lib/utils/messenger.js:87:12)
    at ChildProcess.agentWorker.once (/usr/local/lib/node_modules/macaca-datahub/node_modules/egg-cluster/lib/master.js:216:22)
    at Object.onceWrapper (events.js:317:30)
    at emitTwo (events.js:126:13)
    at ChildProcess.emit (events.js:214:7)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
name: 'AgentWorkerDiedError'
pid: 18
hostname: ebf53b157b63

2018-05-25 08:51:51,739 ERROR 18 [master] agent_worker#1:28 start fail, exiting with code:1
2018-05-25 08:51:51,741 ERROR 18 [master] exit with code:1

support preview multi-scene data(optimize)

now the server could only preview currentScene data, and there is an optimization that we can preview scene'data by request /data..?sceneName=default to preview scene default's data

datahub 可以同步 swagger 信息的变更

datahub 可以通过手动或者定时同步某一指定 swagger 信息的变更。

然后之前的可以存档 achieve 到某一个版本中。需要的时候可以切回来。

engines detect not working

node version 8.1.2 also can install macaca-datahub without warning,
and failed to use transform-object-rest-spread feature.
the macaca-datahub configuration is

"engines": {
    "node": ">=8.9.0"
  },

image

Where is macaca datahub store mock data that has been created

I try to run macaca datahub version 2.4.0 with npm run dev and npm run start. I created datahub, data mock json also with some scenes. I try to check my data mock json locally inside my datahub directory but I cannot find it. May I know where is data mock json store on my local?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.