mattupstate / overholt Goto Github PK

Hi！
I want decorative view function two times with @route decorator like this：

@route(bp,'/')
@route(bp,'/<int:id>')
def index(id):
    .....

bug i get error "View function mapping is overwriting an existing endpoint", is there a way to meet this demand？

I had change the url of connecting mysql,and created a database named overholt ,but no tables in it.
what I want to konw is that how can create the tables???

thank you !
@mattupstate

Hey,
Can you show how to use flask-principal within this, as in where does the Role get managed, and how to use the protected views.
Do you create a new Service object per role that you need? Or just use the Flask-Principal decorators to hide the views?

It would be great if Overholt run under Python 3 and the latest wtforms. Are there plans for this? thanks

Upgrading flask-security beyond version 1.6.5 causes the tests to fail.
More specifically, login/logout doesn't seem to work. In the dashboard tests, the response always contains the full template, even if the user isn't logged in first, or if the user is logged out at the beginning of the test.

To reproduce:

pip install --upgrade Flask-Security==1.6.6
pip install --upgrade Flask-WTF==0.8
nosetests

(Second pip line is necessary else Flask-Security does complain about some missing imports. Not sure why pip pulls the wrong dependencies here.)

Any idea what might have changed between 1.6.5 and 1.6.6 that causes the tests to fail? Or is it really just due to the WTF forms? If yes, I don't understand how this influences _login() of the test module.

Currently, a view function only has one route. @route(bp, '/someurl', methods=['GET']). e.g. But how about if I need more then one url mapping for the same view function?

I'm not sure about this but should the requirements.txt contain == or >=

If == does it install exactly that version of the requirement rather than >= installing the most recent version of the requirement?

I had a look in the Flask-Login source but could not see where this is being written into the cookie but it doesn't seem ideal from a security perspective.

<SecureCookieSession {'_id': '6948290bcaabea200d00961712d41d83', 'user_id': '64', '_fresh': True, '_flashes': [('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].'), ('success', 'Thank you. Confirmation instructions have been sent to [email protected].')], 'csrf_token': 'b345ae0a9bef46fdb0954abc191c71933d802a27'}>
[pid: 5020|app: 0|req: 12/12] 120.148.164.173 () {76 vars in 1822 bytes} [Wed Jul 30 21:37:00 2014] POST /app/api/pages/savepage/128 => generated 16 bytes in 64 msecs (HTTP/1.1 200) 3 headers in 550 bytes (1 switches on core 0)
Loads user from session or remember_me cookie as applicable

in overholt/manage/users.py it should delete a user, not the UserService, users.

users.delete(user)

How is the circular import of celery tasks handled?

When I run the test suite, it hangs and never finishes. I've got all of the dependencies installed, barring this #37. I assume the tests run with nose since it's in the requirements list. When I run nosetests I get 6 dots like it's starting to run tests, then it stops and hangs indefinitely. Not even a timeout. I've also tried running nosetests tests/ to limit it to the tests folder. That produced no change.

In overholt I am trying to send the flask-security emails via Celery. I am following the instructions you have provided here:

pallets-eco/flask-security#194

I cannot find anywhere in Overholt to put the following code:

@security.send_mail_task
def delay_flask_security_mail(msg):
send_flask_mail.delay(msg)

Where should it live?

thanks

I got a quick question for core.Service. So what's the point to wrap sqlalchemy's methods into Service rather than call them directly in each application?

Also I didn't find a way to use some sqlalchemy's methods like (query.filter, concatenating multiple filter_by etc.) without calling directly model.query blabla. It seems that Service doesn't provide all methods of sqlalchemy.

How would you go about running both api and frontend apps in development? Specifically, using manage.py.

Using Python 2.7 after a fresh install I get the following error. I've dug around in the source for hours and traced it to something to do with overholt/tasks.py but I don't know how to fix it properly.

Any ideas? thanks

$ alembic upgrade head
Traceback (most recent call last):
File "/var/www/recruitinginbox.com/venv2.7/bin/alembic", line 8, in
load_entry_point('alembic==0.6.4', 'console_scripts', 'alembic')()
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/config.py", line 298, in main
CommandLine(prog=prog).main(argv=argv)
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/config.py", line 293, in main
self.run_cmd(cfg, options)
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/config.py", line 279, in run_cmd
**dict((k, getattr(options, k)) for k in kwarg)
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/command.py", line 125, in upgrade
script.run_env()
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/script.py", line 203, in run_env
util.load_python_file(self.dir, 'env.py')
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/util.py", line 212, in load_python_file
module = load_module_py(module_id, path)
File "/var/www/recruitinginbox.com/venv2.7/local/lib/python2.7/site-packages/alembic/compat.py", line 58, in load_module_py
mod = imp.load_source(module_id, path, fp)
File "alembic/env.py", line 15, in
app = create_app()
File "/var/www/recruitinginbox.com/overholt/api/init.py", line 23, in create_app
register_security_blueprint=register_security_blueprint)
File "/var/www/recruitinginbox.com/overholt/factory.py", line 44, in create_app
register_blueprints(app, package_name, package_path)
File "/var/www/recruitinginbox.com/overholt/helpers.py", line 26, in register_blueprints
m = importlib.import_module('%s.%s' % (package_name, name))
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
File "/var/www/recruitinginbox.com/overholt/api/stores.py", line 13, in
from ..tasks import send_manager_added_email, send_manager_removed_email
File "/var/www/recruitinginbox.com/overholt/tasks.py", line 12, in
celery = create_celery_app()
File "/var/www/recruitinginbox.com/overholt/factory.py", line 52, in create_celery_app
app = app or create_app('overholt', os.path.dirname(file))
File "/var/www/recruitinginbox.com/overholt/factory.py", line 44, in create_app
register_blueprints(app, package_name, package_path)
File "/var/www/recruitinginbox.com/overholt/helpers.py", line 26, in register_blueprints
m = importlib.import_module('%s.%s' % (package_name, name))
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
ImportError: No module named overholt

Hi. Sorry if I'm doing something daft.

 ⇒  vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

Vagrant:
* Unknown configuration section 'berkshelf'.

I even tried:

⇒  vagrant plugin install berkshelf

I was hoping to follow along to your blog post. There's nothing obvious popping up on a web search. Maybe you've got an idea where I can start to troubleshoot?

Cheers.

overholt/api/init.py line 50

you should be returning f after the wrapper function.

this allows you to have multiple routes associated with a function.

otherwise you get an exception.

It would be really helpful if there was a version of this repository without the JavaScript. Perhaps just a static version to better show how Flask would work with more projects.

I couldn't seem to find any mention of what license you're releasing Overholt under? I assume it's open source, just wondering how much so :)

Hi Matt,

In Overholt, the way I understand it, instances of each service are loaded into the "services.py". To use the services, I then import the instance that I want to use from services.py.

This outcome of this appears to be that I cannot use one service from another I suspect due to circular imports. Is my understanding correct?

If so, and if I cannot import one service from another, what is the correct way to use one service from another? i.e. use the "user" object from "store"?

thanks

Hey Matt,

Having trouble overriding the default registration form for Flask-Security.

Edited these lines in factory.py

security.init_app(app, SQLAlchemyUserDatastore(db, User, Role),
    register_form=ExtendedRegisterForm,
    register_blueprint=register_security_blueprint)

Any thoughts on why the Extended form might not be showing up?

I'm using Flask-Restful for my API but how would I use api.add_resource similar to how you did route?

Also I want to add some wrappers for app and api. Where is the best place to have these wrappers?

This is a little out of the topic maybe but you don't speak about deployment. Do you use a fully-fledged setup.py and install it on the server from sdist or do you just git clone?

A sdist seems nicer but I feel like Flask applications don't really fit in them.

Thoughts?

I am confused how could use the factory celery through the app on the function argument?

It's likely my lack of experience as a Python developer but I really got caught out by the relative imports and spent days trying to diagnose issues that were cleaned up immediately by using explicit absolute imports everywhere and by removing the cases of "import *".

Just some feedback.....

I think you mean to return wrapper here, instead of return f, right?
https://github.com/mattupstate/overholt/blob/master/overholt/api/__init__.py#L50

otherwise the bp.route and other decorators won't apply. maybe i'm misunderstanding!

Matt I love OverHolt and have built a major application with it. But I want to know more and discuss it in more detail.

Is there a "next step"? i.e one of your more sophisticated/larger/more advanced projects?

Also is there a better place to raise these sorts of things than as an issue here?

I tried to launch your example, but found out, that there is no such version for kombu. I tried to remove all version specifications, and found out, that you are using deprecated (and removed in current stable version) factory-boy method set_creation_function(). One more error is in WTForms usage.

Possible fix could be:

diff --git a/overholt/products/forms.py b/overholt/products/forms.py
index 2e7adbf..106d72c 100644
--- a/overholt/products/forms.py
+++ b/overholt/products/forms.py
@@ -6,8 +6,7 @@
     Product forms
 """

-from flask_wtf import Form, TextField, SelectMultipleField, Required, \
-    Optional
+from wtforms import Form, TextField, SelectMultipleField, validators

 from ..services import products

@@ -22,12 +21,12 @@ class ProductFormMixin(object):


 class NewProductForm(ProductFormMixin, Form):
-    name = TextField('Name', validators=[Required()])
+    name = TextField('Name', validators=[validators.Required()])
     categories = SelectMultipleField(
-        'Categories', coerce=int, validators=[Required()])
+        'Categories', coerce=int, validators=[validators.Required()])


 class UpdateProductForm(ProductFormMixin, Form):
-    name = TextField('Name', validators=[Optional()])
+    name = TextField('Name', validators=[validators.Optional()])
     categories = SelectMultipleField(
-        'Categories', coerce=int, validators=[Optional()])
+        'Categories', coerce=int, validators=[validators.Optional()])
diff --git a/overholt/stores/forms.py b/overholt/stores/forms.py
index 4edf74c..5899bae 100644
--- a/overholt/stores/forms.py
+++ b/overholt/stores/forms.py
@@ -6,22 +6,22 @@
     Store forms
 """

-from flask_wtf import Form, TextField, Required, Optional
+from wtforms import Form, TextField, validators

 __all__ = ['NewStoreForm', 'UpdateStoreForm']


 class NewStoreForm(Form):
-    name = TextField('Name', validators=[Required()])
-    address = TextField('Address', validators=[Required()])
-    city = TextField('City', validators=[Required()])
-    state = TextField('State', validators=[Required()])
-    zip_code = TextField('Zip Code', validators=[Required()])
+    name = TextField('Name', validators=[validators.Required()])
+    address = TextField('Address', validators=[validators.Required()])
+    city = TextField('City', validators=[validators.Required()])
+    state = TextField('State', validators=[validators.Required()])
+    zip_code = TextField('Zip Code', validators=[validators.Required()])


 class UpdateStoreForm(Form):
-    name = TextField('Name', validators=[Optional()])
-    address = TextField('Address', validators=[Optional()])
-    city = TextField('City', validators=[Optional()])
-    state = TextField('State', validators=[Optional()])
-    zip_code = TextField('Zip Code', validators=[Optional()])
+    name = TextField('Name', validators=[validators.Optional()])
+    address = TextField('Address', validators=[validators.Optional()])
+    city = TextField('City', validators=[validators.Optional()])
+    state = TextField('State', validators=[validators.Optional()])
+    zip_code = TextField('Zip Code', validators=[validators.Optional()])
diff --git a/tests/factories.py b/tests/factories.py
index 9ebdcb9..63539d2 100644
--- a/tests/factories.py
+++ b/tests/factories.py
@@ -15,22 +15,22 @@ from overholt.core import db
 from overholt.models import *


-def create_sqlalchemy_model_function(class_to_create, *args, **kwargs):
-    entity = class_to_create(**kwargs)
-    db.session.add(entity)
-    db.session.commit()
-    return entity
+class MyFactory(Factory):
+    @classmethod
+    def _create(cls, target_class, *args, **kwargs):
+        entity = target_class(**kwargs)
+        db.session.add(entity)
+        db.session.commit()
+        return entity

-Factory.set_creation_function(create_sqlalchemy_model_function)

-
-class RoleFactory(Factory):
+class RoleFactory(MyFactory):
     FACTORY_FOR = Role
     name = 'admin'
     description = 'Administrator'


-class UserFactory(Factory):
+class UserFactory(MyFactory):
     FACTORY_FOR = User
     email = Sequence(lambda n: 'user{0}@overholt.com'.format(n))
     password = LazyAttribute(lambda a: encrypt_password('password'))
@@ -43,7 +43,7 @@ class UserFactory(Factory):
     active = True


-class StoreFactory(Factory):
+class StoreFactory(MyFactory):
     FACTORY_FOR = Store
     name = Sequence(lambda n: 'Store Number {0}'.format(n))
     address = '123 Overholt Alley'
@@ -52,11 +52,11 @@ class StoreFactory(Factory):
     zip_code = '12345'


-class ProductFactory(Factory):
+class ProductFactory(MyFactory):
     FACTORY_FOR = Product
     name = Sequence(lambda n: 'Product Number {0}'.format(n))


-class CategoryFactory(Factory):
+class CategoryFactory(MyFactory):
     FACTORY_FOR = Category
     name = Sequence(lambda n: 'Category {0}'.format(n))

Hey, this is an issue I've been running into myself and trying to research it a bit I ran into your pretty nice base project!

to be able to give functions the annotation @celery.task we need to be able to import the celery = Celery() instance from somewhere.
but to tie it together with all the Flask related stuff we want to be in app context (and I want to use the config from my app for some celery stuff too) so we need an instance of the app.
so you create that instance when creating the Celery instance ( I do the same in my project ).

but this results in doing more create_app calls (or at least 1 more) when importing tasks to queue them from your blueprints.
The thing is .. I'm not sure if it matters ... but it feels wrong ...

Just putting it here to discuss, hope you don't mind

The Vagrant file references a Vagrant box that cannot be downloaded. The url to this file (on Dropbox) is not working.

Is there an alternative box that can be used? I tried http://cloud-images.ubuntu.com/vagrant/quantal/current/quantal-server-cloudimg-amd64-vagrant-disk1.box but it doesn't work out of the box.

HI Matt

I'm pretty stuck working out how to implement a signal from flask-security that tells me when a user has registered and when a user has logged in.

Are you able to suggest how to do this and where you would put such functionality in Overholt?

thanks

Hi, I really like the suggested structure of your flas applications... I'm really new to Flask and to the freedom it provides. If you fin the time, can you answer a question?

I'm creating a multi-tenant application using the PostgreSQL schemas, where I will have a public schema + a schema for each tenant. So, I need my application to be something like this: http://www.application.com/tenant/

I'm pretty sure the answer to achieve that is something like this: http://flask.pocoo.org/docs/patterns/appdispatch/#dispatch-by-path

The problem is that I do not know how to best acommodate this in the overholt architecture. Could you give me some tips?

Thank you very much

When I try to access a model with a bidirectional many to many relationship, I get the error: TypeError: <sqlalchemy.orm.dynamic.AppenderBaseQuery object at 0x2a01b50> is not JSON serializable The error makes sense, as rv = {..., 'users': <sqlalchemy.orm.dynamic.AppenderBaseQuery object at 0x2758b50>}.

How would I go about executing that query so it becomes serialized?

• Do I create a __json_modifiers__ for 'users' and run a lambda to convert it to a list/json?
• Am I missing something in sqlalchemy that can resolve this?

projects_users = db.Table('projects_users',
    db.Column('id', db.Integer, primary_key=True),
    db.Column('user_id', db.Integer, db.ForeignKey('users.id')),
    db.Column('project_id', db.Integer, db.ForeignKey('projects.id'))
)

class UserJsonSerializer(JsonSerializer):
    pass

class User(UserJsonSerializer, UserMixin, db.Model):
    __tablename__ = 'users'

    id = db.Column(db.Integer, primary_key=True)
    projects = db.relationship('Project', secondary=projects_users,
        backref=db.backref('users', lazy='dynamic'))

class ProjectJsonSerializer(JsonSerializer):
    # __json_hidden__ = ['users']
    #  __json_modifiers__  = {
    #    'users': lambda x, self: x.all()
    # }
    pass

class Project(ProjectJsonSerializer, db.Model):
    __tablename__ = 'projects'

    id = db.Column(db.Integer(), primary_key=True)
    name = db.Column(db.String(255))

Thanks so much for your flask skeleton project. It has cleared up so many questions I had about how to scale large projects.

Hi,
Thanks very much for taking the time to write this code and share its analysis!
I've not used Vagrant before, and was wondering if you might be able to shed some light on its use within overholt.

From what I can tell after reading the docs and the Vagrantfile, it looks like the virtual machine only gets used for MySQL/Redis.io backend storage - with the actual flask development server running on the local machine.. Is that correct?

I suppose I'm just confused what the exact role vagrant plays in the development stack here, and why more of the web server aspects aren't part of the virtual machine?

Best,

I'm wondering what are the drawbacks of using mixins vs using objects. I actually implemented services by using a mixin instead of a separate object on its own.

This helps me in a two ways:

1. I don't have to define model, since I'm using classmethods to obtain the model I'm running the query on.
2. I don't have to create or instantiate objects to use getters like all, find, etc.

class ServiceMixin(object):

    def save(self):
        db.session.add(self)
        db.session.commit()
        return self

    @classmethod
    def all(cls):
        return cls.query.all()

    @classmethod
    def get(cls, _id):
        return cls.query.get(_id)

    @classmethod
    def get_all(cls, id_cl="id", *ids):
        id_column = cls.get(id_cl)
        return cls.query.filter(id_column.in_(ids)).all()

    @classmethod
    def find(cls, **kwargs):
        return cls.query.filter_by(**kwargs)

    @classmethod
    def first(cls, **kwargs):
        return cls.find(**kwargs).first()

    @classmethod
    def get_or_404(cls, id):
        return cls.query.get_or_404(id)

    @classmethod
    def new(cls, **kwargs):
        return cls(**cls._preprocess_params(kwargs))

    @staticmethod
    def _preprocess_params(kwargs):
        kwargs.pop('csrf_token', None)
        return kwargs

    @classmethod
    def create(cls, **kwargs):
        return cls.save(cls.new(**kwargs))

    def update(self, **kwargs):
        for k, v in self._preprocess_params(kwargs).items():
            setattr(self, k, v)
        self.save()
        return self

    def delete(self):
        db.session.delete(self)
        db.session.commit()

To use it, simply inherit it when creating your models:

class Product(db.Model, ServiceMixin):
    id = db.Column(db.Integer(), primary_key=True)
    name = db.Column(db.String(250))
    ...

Product.all() # Gets all of the products
Product.find(name="tacos") # Finds all products named "tacos"
p = Product(name="spegahetti")
p.save() # Saves the newly created Product

Anyway, this project template is great stuff! Thanks for letting us in on this.

Hi

I noticed there will be an error raised when visit the API without authentication.

"""werkzeug.routing.BuildError: ('security.login', {}, None)"""

If this works as design, what would be the right way to avoid the error?

Thanks