Skip to content

update docs to use security policy#3557

Merged
mmerickel merged 46 commits intoPylons:masterfrom
mmerickel:security-docs
Jan 16, 2020
Merged

update docs to use security policy#3557
mmerickel merged 46 commits intoPylons:masterfrom
mmerickel:security-docs

Conversation

@mmerickel
Copy link
Copy Markdown
Member

@mmerickel mmerickel commented Dec 30, 2019

  • rename identify to authenticated_identity on ISecurityPolicy
  • quick_tutorial
  • sync cookiecutter to wiki tutorial
  • use security policy in wiki tutorial
  • sync cookiecutter to wiki2
  • use security policy in wiki2 tutorial
  • revise security chapter - this is at least partially complete
  • sync cookiecutter to starter project

rendered: https://mmerickel-pyramid-fork.readthedocs.io/en/security-docs/

fixes #3548

@mmerickel mmerickel added this to the 2.0 milestone Dec 30, 2019
Copy link
Copy Markdown
Member

@stevepiercy stevepiercy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested one wording change.

Otherwise LGTM so far.

I can probably run through the cookiecutter synch on Tuesday so that this PR can be finished.

@mmerickel
Copy link
Copy Markdown
Member Author

Please do sync the cookiecutters. Waiting on that to finish this PR.

@stevepiercy
Copy link
Copy Markdown
Member

Is your open PR finished? Pylons/pyramid-cookiecutter-starter#81

@mmerickel
Copy link
Copy Markdown
Member Author

yes

@mmerickel
Copy link
Copy Markdown
Member Author

Not sure why you approved this - it isn't done yet.

@stevepiercy
Copy link
Copy Markdown
Member

I approved work thusfar. I can do another one later, too. It sounds like you prefer one final review going forward, which I will do.

@mmerickel
Copy link
Copy Markdown
Member Author

It’s fine I guess. The PR is in draft mode meaning I don’t really care if anyone reviews it yet.

@mmerickel
Copy link
Copy Markdown
Member Author

I can sync the cookiecutters if you don’t have time. Just gotta let me know.

@stevepiercy
Copy link
Copy Markdown
Member

I'm working on it.

@mmerickel
Copy link
Copy Markdown
Member Author

Ok!

@stevepiercy
Copy link
Copy Markdown
Member

This PR depends on the completion of #3556

@mmerickel
Copy link
Copy Markdown
Member Author

mmerickel commented Jan 7, 2020

Ok, I've significantly updated the wiki2 tutorial.

  1. Made significant changes to the starter cookiecutter to add a real test harness.
  2. Synced wiki2 with the starter cookiecutter.
  3. Added CSRF protection to the app in the view chapter.
  4. Added a MySecurityPolicy in authentication and authorization chapters.
  5. Significant revamping of the test chapter, including rewriting everything to use pytest, with useful transactional fixtures.
  6. New release of pyramid_tm (2.4) to support the new fixtures and to stop using request.unauthenticated_userid.
  7. New release of pyramid_debugtoolbar (4.5.2) to stop using request.unauthenticated_userid.

I still need to sync the cookiecutter with wiki tutorial (I won't be nearly as detailed in my changes there unfortunately), and to review the security chapter documentation. There may also be some missing cookiecutter udpates in the starter project chapter.

@merwok
Copy link
Copy Markdown
Contributor

merwok commented Jan 7, 2020

Lots of good stuff here. Thank you for this work!

@mmerickel
Copy link
Copy Markdown
Member Author

You need to install pyramid master into the virtualenv you create for the tutorial step. After that, you need to post an actual error message.

@stevepiercy
Copy link
Copy Markdown
Member

You need to install pyramid master into the virtualenv you create for the tutorial step. After that, you need to post an actual error message.

$VENV/bin/pip install git+https://github.com/Pylons/pyramid@master

Ensured that the source files in docs/tutorials/wiki2/src/tests match those in my project.

$VENV/bin/pytest -q

# Just the last test failure:

__________________________________________________________________________________________________________________________________________ test_editors_member_user_can_view ___________________________________________________________________________________________________________________________________________

request = <Request at 0x108d35240 GET http://example.com/FrontPage>, exc = UndefinedError("'pyramid.request.Request object' has no attribute 'user'")

    def _error_handler(request, exc):
        # NOTE: we do not need to delete exc_info because this function
        # should never be in the call stack of the exception
        exc_info = sys.exc_info()
    
        try:
>           response = request.invoke_exception_view(exc_info)

lib/python3.7/site-packages/pyramid/tweens.py:13: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Request at 0x108d35240 GET http://example.com/FrontPage>, exc_info = (<class 'jinja2.exceptions.UndefinedError'>, UndefinedError("'pyramid.request.Request object' has no attribute 'user'"), <traceback object at 0x10921b8c8>), request = <Request at 0x108d35240 GET http://example.com/FrontPage>
secure = True, reraise = False

    def invoke_exception_view(
        self, exc_info=None, request=None, secure=True, reraise=False
    ):
        """ Executes an exception view related to the request it's called upon.
        The arguments it takes are these:
    
        ``exc_info``
    
            If provided, should be a 3-tuple in the form provided by
            ``sys.exc_info()``.  If not provided,
            ``sys.exc_info()`` will be called to obtain the current
            interpreter exception information.  Default: ``None``.
    
        ``request``
    
            If the request to be used is not the same one as the instance that
            this method is called upon, it may be passed here.  Default:
            ``None``.
    
        ``secure``
    
            If the exception view should not be rendered if the current user
            does not have the appropriate permission, this should be ``True``.
            Default: ``True``.
    
        ``reraise``
    
            A boolean indicating whether the original error should be reraised
            if a :term:`response` object could not be created. If ``False``
            then an :class:`pyramid.httpexceptions.HTTPNotFound`` exception
            will be raised. Default: ``False``.
    
        If a response is generated then ``request.exception`` and
        ``request.exc_info`` will be left at the values used to render the
        response. Otherwise the previous values for ``request.exception`` and
        ``request.exc_info`` will be restored.
    
        .. versionadded:: 1.7
    
        .. versionchanged:: 1.9
           The ``request.exception`` and ``request.exc_info`` properties will
           reflect the exception used to render the response where previously
           they were reset to the values prior to invoking the method.
    
           Also added the ``reraise`` argument.
    
        """
        if request is None:
            request = self
        registry = getattr(request, 'registry', None)
        if registry is None:
            registry = get_current_registry()
    
        if registry is None:
            raise RuntimeError("Unable to retrieve registry")
    
        if exc_info is None:
            exc_info = sys.exc_info()
    
        exc = exc_info[1]
        attrs = request.__dict__
        context_iface = providedBy(exc)
    
        # clear old generated request.response, if any; it may
        # have been mutated by the view, and its state is not
        # sane (e.g. caching headers)
        with hide_attrs(request, 'response', 'exc_info', 'exception'):
            attrs['exception'] = exc
            attrs['exc_info'] = exc_info
            # we use .get instead of .__getitem__ below due to
            # https://github.com/Pylons/pyramid/issues/700
            request_iface = attrs.get('request_iface', IRequest)
    
            manager.push({'request': request, 'registry': registry})
    
            try:
                response = _call_view(
                    registry,
                    request,
                    exc,
                    context_iface,
                    '',
                    view_types=None,
                    view_classifier=IExceptionViewClassifier,
                    secure=secure,
                    request_iface=request_iface.combined,
                )
            except Exception:
                if reraise:
                    reraise_(*exc_info)
                raise
            finally:
                manager.pop()
    
        if response is None:
            if reraise:
                reraise_(*exc_info)
>           raise HTTPNotFound
E           pyramid.httpexceptions.HTTPNotFound: The resource could not be found.

lib/python3.7/site-packages/pyramid/view.py:770: HTTPNotFound

During handling of the above exception, another exception occurred:

testapp = <tests.conftest.TestApp object at 0x108d35438>

    def test_editors_member_user_can_view(testapp):
        testapp.login(editor_login)
>       res = testapp.get('/FrontPage', status=200)

tests/test_functional.py:121: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
lib/python3.7/site-packages/webtest/app.py:335: in get
    expect_errors=expect_errors)
lib/python3.7/site-packages/webtest/app.py:628: in do_request
    res = req.get_response(app, catch_exc_info=True)
lib/python3.7/site-packages/webob/request.py:1310: in send
    application, catch_exc_info=True)
lib/python3.7/site-packages/webob/request.py:1278: in call_application
    app_iter = application(self.environ, start_response)
lib/python3.7/site-packages/webtest/lint.py:201: in lint_app
    iterator = application(environ, start_response_wrapper)
lib/python3.7/site-packages/pyramid/router.py:266: in __call__
    response = self.execution_policy(environ, self)
lib/python3.7/site-packages/pyramid_retry/__init__.py:121: in retry_policy
    response = router.invoke_request(request)
lib/python3.7/site-packages/pyramid/router.py:245: in invoke_request
    response = handle_request(request)
lib/python3.7/site-packages/pyramid_tm/__init__.py:120: in tm_tween
    return handler(request)
lib/python3.7/site-packages/pyramid/tweens.py:43: in excview_tween
    response = _error_handler(request, exc)
lib/python3.7/site-packages/pyramid/tweens.py:17: in _error_handler
    reraise(*exc_info)
lib/python3.7/site-packages/pyramid/util.py:740: in reraise
    raise value
lib/python3.7/site-packages/pyramid/tweens.py:41: in excview_tween
    response = handler(request)
lib/python3.7/site-packages/pyramid/router.py:144: in handle_request
    registry, request, context, context_iface, view_name
lib/python3.7/site-packages/pyramid/view.py:658: in _call_view
    response = view_callable(context, request)
lib/python3.7/site-packages/pyramid/viewderivers.py:321: in secured_view
    return view(context, request)
lib/python3.7/site-packages/pyramid/viewderivers.py:514: in csrf_view
    return view(context, request)
lib/python3.7/site-packages/pyramid/viewderivers.py:453: in rendered_view
    request, result, view_inst, context
lib/python3.7/site-packages/pyramid/renderers.py:443: in render_view
    return self.render_to_response(response, system, request=request)
lib/python3.7/site-packages/pyramid/renderers.py:466: in render_to_response
    result = self.render(value, system_values, request=request)
lib/python3.7/site-packages/pyramid/renderers.py:462: in render
    result = renderer(value, system_values)
lib/python3.7/site-packages/pyramid_jinja2/__init__.py:265: in __call__
    return template.render(system)
lib/python3.7/site-packages/jinja2/asyncsupport.py:76: in render
    return original_render(self, *args, **kwargs)
lib/python3.7/site-packages/jinja2/environment.py:1008: in render
    return self.environment.handle_exception(exc_info, True)
lib/python3.7/site-packages/jinja2/environment.py:780: in handle_exception
    reraise(exc_type, exc_value, tb)
lib/python3.7/site-packages/jinja2/_compat.py:37: in reraise
    raise value.with_traceback(tb)
tutorial/templates/view.jinja2:1: in top-level template code
    {% extends 'layout.jinja2' %}
lib/python3.7/site-packages/jinja2/environment.py:1005: in render
    return concat(self.root_render_func(self.new_context(vars)))
tutorial/templates/view.jinja2:14: in root
    </p>
tutorial/templates/layout.jinja2:31: in root
    <div class="col-md-2">
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyramid_jinja2.Environment object at 0x1088f4898>, obj = Undefined, attribute = 'name'

    def getattr(self, obj, attribute):
        """Get an item or attribute of an object but prefer the attribute.
        Unlike :meth:`getitem` the attribute *must* be a bytestring.
        """
        try:
>           return getattr(obj, attribute)
E           jinja2.exceptions.UndefinedError: 'pyramid.request.Request object' has no attribute 'user'

lib/python3.7/site-packages/jinja2/environment.py:430: UndefinedError
13 failed, 14 passed in 9.52s

@mmerickel
Copy link
Copy Markdown
Member Author

You need to install this branch, not master.

@mmerickel
Copy link
Copy Markdown
Member Author

I do it with pip install -e ~/work/oss/pyramid myself.

Copy link
Copy Markdown
Contributor

@merwok merwok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read quick tuto and narrative docs for security: LGTM!

@stevepiercy
Copy link
Copy Markdown
Member

You need to install this branch, not master.

Yup, did that as well, both now and earlier, unless this is not the correct command to install this branch.

pip install git+https://github.com/mmerickel/pyramid@security-docs

Still got the exact same error message as above.

@stevepiercy
Copy link
Copy Markdown
Member

I tossed out my virtual environment, rebuilt the tutorial from scratch, and now all tests pass.

27 passed in 6.91s

@mmerickel
Copy link
Copy Markdown
Member Author

Thanks @stevepiercy I think this PR is ready to be merged.

@stevepiercy
Copy link
Copy Markdown
Member

Not yet. I still have the last two steps to do in the traversal wiki. Will do late tonight.

- Swap order of editing tutorial/views/default.py so that line numbers in the user's editor align with the rendered docs
- Expand contractions and spell out words
@stevepiercy
Copy link
Copy Markdown
Member

OK, I'm done! Any final revisions?

@mmerickel mmerickel merged commit 9c153e1 into Pylons:master Jan 16, 2020
@mmerickel
Copy link
Copy Markdown
Member Author

Thanks @stevepiercy and @merwok for your review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update authn/authz tutorial

3 participants