Comments (13)
bkimminich
commented on June 18, 2024
CTFd 2.x exports and imports all tables by default. A possible solution would be to export an "empty" CTFd setup with only the mininum required data (i.e. an admin user, a CTF title etc.) and let the util overwrite the challenges.json, keys.json and hints.json files in the ZIP.
from juice-shop-ctf.
ColdHeat
commented on June 18, 2024
I think the first thing to know is that while generated exports have every table, the importer doesn't require all tables to exist.
You could generate an import with only challenges.json, flags.json, hints.json, files.json and other juice shop relevant tables. After importing the tables that weren't inside of the zip file won't be affected.
This would work well for situations where the user is importing from the command line because they could modify or otherwise run the import.py script without triggering a database wiping. However, this isn't the same for importing via the web UI where deleting the database before import is difficult to avoid at the moment. I think the best thing to do in that scenario is to allow the wipe to happen and let the import import all challenges but require the user to re-create the admin account and others.
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
Ok, this is exactly what my tool generates, a ZIP with only those tables. But upon UI import there is an error being thrown: KeyError("There is no item named 'db/alembic_version.json' in the archive",) - after that happened all subsequent requests for any pages will end in an Internal Server Error. In the server log I saw OperationalError: (sqlite3.OperationalError) no such table: config [SQL: u'SELECT config.id AS config_id, config."key" AS config_key, config.value AS config_value \nFROM config \nWHERE config."key" = ?\n LIMIT ? OFFSET ?'] [parameters: ('ctf_name', 1, 0)] (Background on this error at: http://sqlalche.me/e/e3q8) then.
So, do I have to provide all other .json files as empty to make the new importer happy?
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
OWASP_Juice_Shop.2018-11-29.CTFd.zip
This is a sample export file my tool generates at the moment, which works up to CTFd 1.2.0 just fine.
from juice-shop-ctf.
ColdHeat
commented on June 18, 2024
You need to migrate the tables that are output because there are significant changes in database structure between 1.2.0 and 2.0.0.
You should look at the 1.2.0 -> 2.0.0 migration script: https://github.com/CTFd/CTFd/blob/master/migrations/1_2_0_upgrade_2_0_0.py
And also the new CTFd models:
https://github.com/CTFd/CTFd/blob/master/CTFd/models/__init__.py
Also you do now need to provide db/alembic_version.json but you should be okay to use the alembic_version that's currently in use (8369118943a1) because we should now be able to accept imports from previous version if we do upgrade.
You could try importing your 1.2.0 export into a 1.2.0 CTFd, then using python migrations/1_2_0_upgrade_2_0_0.py, and then exporting to see the difference in what is generated.
from juice-shop-ctf.
mmmod
commented on June 18, 2024
A workaround at this point is to run 1.2.0 of CTFd, import seems to work:
- Checkout 1.2.0 of CTFd from the repo
- Change db image in docker-compose to "mariadb:10.4" (from mariadb:10.2)
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
Official supported CTFd versions are 1.1.x and 1.2.x at the moment, and they should work out of the box. At least with the singular `ctfd/ctfd` Docker image.
Am 11. Januar 2019 22:30:01 MEZ schrieb "Markus Örebrand" <[email protected]>:
…A workaround at this point is to run 1.2.0 of CTFd, import seems to
work:
* Checkout 1.2.0 of CTFd from the repo
* Change db image in docker-compose to "mariadb:10.4" (from
mariadb:10.2)
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
#57 (comment)
from juice-shop-ctf.
ColdHeat
commented on June 18, 2024
@bkimminich I think you might have removed your previous comment but I don't believe getting all of the setup data from the user at creation time is necessary.
I think I would just output a the zip with the json files that you want. The others should be ignored by the importer if they're missing.
Also the import_ctf() function has a parameter to control whether or not we erase. Erasing is safer overall but we could modify the command line importer to make erasing optional if it's helpful.
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
@ColdHeat, thanks, I wrote the now-deleted comment before trying out what you suggested, and that did it. Except for the fact that after import an Internal Server Error pops up and the CTFd server needs a restart. But then all seems okay once you set up your CTF name and admin account again.
I attached a ZIP for CTFd 2.x, so if you can give me any hint what to add in order to prevent this error, I'll happily change my tool a bit.
OWASP_Juice_Shop.2019-01-29.CTFd2.zip
from juice-shop-ctf.
ColdHeat
commented on June 18, 2024
Are you referring to the following?
Traceback (most recent call last):
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask_restplus/api.py", line 583, in error_router
return original_handler(e)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask_restplus/api.py", line 583, in error_router
return original_handler(e)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/kchung/Repositories/CTFd/CTFd/views.py", line 122, in setup
return render_template('setup.html', nonce=session.get('nonce'))
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/Users/kchung/.virtualenvs/ctfd/lib/python2.7/site-packages/flask/templating.py", line 83, in _get_source_fast
return loader.get_source(environment, template)
File "/Users/kchung/Repositories/CTFd/CTFd/__init__.py", line 84, in get_source
template = "/".join([theme, 'templates', template])
TypeError: sequence item 0: expected string, NoneType found
This might be a bug in CTFd ultimately but you can fix it by providing a config.json with ctf_theme set to core.
Importing from the command line worked fine for me.
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
This is my log after the error popup right after imprting the attached ZIP file:
* Loaded module, <module 'CTFd.plugins.challenges' from '/opt/CTFd/CTFd/plugins/challenges/__init__.py'>
* Loaded module, <module 'CTFd.plugins.dynamic_challenges' from '/opt/CTFd/CTFd/plugins/dynamic_challenges/__init__.py'>
* Loaded module, <module 'CTFd.plugins.flags' from '/opt/CTFd/CTFd/plugins/flags/__init__.py'>
Starting CTFd
[2019-01-29 20:15:05 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2019-01-29 20:15:05 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2019-01-29 20:15:05 +0000] [1] [INFO] Using worker: geventwebsocket.gunicorn.workers.GeventWebSocketWorker
[2019-01-29 20:15:05 +0000] [27] [INFO] Booting worker with pid: 27
* Loaded module, <module 'CTFd.plugins.challenges' from '/opt/CTFd/CTFd/plugins/challenges/__init__.pyc'>
* Loaded module, <module 'CTFd.plugins.dynamic_challenges' from '/opt/CTFd/CTFd/plugins/dynamic_challenges/__init__.pyc'>
* Loaded module, <module 'CTFd.plugins.flags' from '/opt/CTFd/CTFd/plugins/flags/__init__.pyc'>
[2019-01-29 20:15:53,957] ERROR in app: Exception on /setup [GET]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/site-packages/flask_restplus/api.py", line 583, in error_router
return original_handler(e)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/opt/CTFd/CTFd/views.py", line 122, in setup
return render_template('setup.html', nonce=session.get('nonce'))
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 83, in _get_source_fast
return loader.get_source(environment, template)
File "/opt/CTFd/CTFd/__init__.py", line 84, in get_source
template = "/".join([theme, 'templates', template])
TypeError: sequence item 0: expected string, NoneType found
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/gevent/pywsgi.py", line 976, in handle_one_response
self.run_application()
File "/usr/local/lib/python2.7/site-packages/geventwebsocket/handler.py", line 87, in run_application
return super(WebSocketHandler, self).run_application()
File "/usr/local/lib/python2.7/site-packages/gevent/pywsgi.py", line 923, in run_application
self.result = self.application(self.environ, self.start_response)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 2309, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/flask_socketio/__init__.py", line 43, in __call__
start_response)
File "/usr/local/lib/python2.7/site-packages/engineio/middleware.py", line 67, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
File "/usr/local/lib/python2.7/site-packages/flask_restplus/api.py", line 583, in error_router
return original_handler(e)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1748, in handle_exception
return self.finalize_request(handler(e), from_error_handler=True)
File "/opt/CTFd/CTFd/errors.py", line 16, in general_error
return render_template('errors/500.html'), 500
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 869, in get_or_select_template
return self.get_template(template_name_or_list, parent, globals)
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 58, in get_source
return self._get_source_fast(environment, template)
File "/usr/local/lib/python2.7/site-packages/flask/templating.py", line 83, in _get_source_fast
return loader.get_source(environment, template)
File "/opt/CTFd/CTFd/__init__.py", line 84, in get_source
template = "/".join([theme, 'templates', template])
TypeError: sequence item 0: expected string, NoneType found
2019-01-29T20:15:54Z {'REMOTE_PORT': '52784', 'HTTP_HOST': '192.168.99.100:32768', 'REMOTE_ADDR': '192.168.99.1', (hidden keys: 25)} failed with TypeError
OWASP_Juice_Shop.2019-01-29.CTFd2.zip
from juice-shop-ctf.
bkimminich
commented on June 18, 2024
The error is not a big deal, though. I wrote a note in the output of my tool:
from juice-shop-ctf.
lock
commented on June 18, 2024
This thread has been automatically locked because it has not had recent activity after it was closed. 🔒 Please open a new issue for regressions or related bugs.
from juice-shop-ctf.
Related Issues (20)
- [🐛] Heroku application returning API error HOT 3
- "Bonus Payload" XSS payload broken from HTML entity encoding [🐛] HOT 7
- Migrate from DH-Autobuild to GH-Workflow
- [🐛] Bug or change the behavior of the function 'fetchCodeSnippets' HOT 2
- [🐛] Bug or Behavior not again inside this extension HOT 4
- Decouple e2e test suite from demo instance [🐛]
- CTFd 3.x generator [🚀] HOT 9
- [🚀] Support multiple flags HOT 3
- [🐛] Unable to import to ctfd HOT 5
- [🚀] Replace current export with CSV generator for CTFd HOT 1
- [🐛] Docker container fails due to permission issue HOT 2
- juice-shop-cff failed in fbctf HOT 3
- Update asciinema video for v6.x HOT 2
- An in-range update of yargs is breaking the build 🚨 HOT 3
- Hints missing when not part of initial API response HOT 2
- Suggestion: Custom flags HOT 16
- RTB import HOT 19
- [🐛] Error when generating hints shows incorrect config option (showChallengeHints instead of showHints) HOT 2
- Juice-shop error while installing in RootTheBox HOT 4
- [🐛] paid insertHints and insertHintUrls not working when set from yaml config HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from juice-shop-ctf.