[qvarn-devel] Re: Qvarn 0.82 fails to properly initialize resource routes

Mantas sirex at pov.lt
Fri Aug 18 13:31:10 BST 2017


Patch [1] for the reported bug [2].

[1] https://github.com/vaultit/qvarn/pull/1
[2] http://qvarn-devel.qvarnlabs.net/cf7d3977cfaf48e38143e1db65d3f939.html

On 2017-06-15, Saulius Žemaitaitis wrote:
> Hi,
> 
> While upgrading our dev environments to Qvarn 0.82 we ran into a problem
> with the way Qvarn 0.82 initializes routes. The way
> 
> Full description:
> Upon upgrading one of our environments to Qvarn 0.82 we ran a script that
> does GET /persons and then iterates through every item. Initial GET
> /persons succeeded but subsequent GET /persons/<qvarn_resource_id> started
> failing with 404s. Note that we run Qvarn with 10 uwsgi worker processes.
> 
> Upon inspecting the logs we found that Qvarn is doing a lot of SELECTs on
> resource_type table and logs "No spec found of path". Log messages:
> {"_context": "_add_missing_route", "_msg_number": 9, "_process_id": 15079,
> "_thread_id": 140474609706880, "_timestamp": "2017-06-15 12:15:17.639862",
> "msg_text": "Loading spec from database", "msg_type": "debug", "path":
> "/persons/0035-2e003ccf3de917b9e2ff61b1917749bd-0ceab24e"}
> <... lots of selects ...>
> {"_context": "_add_missing_route", "_msg_number": 13, "_process_id": 15079,
> "_thread_id": 140474609706880, "_timestamp": "2017-06-15 12:15:17.842809",
> "msg_text": "No spec found for path", "msg_type": "debug", "path":
> "/persons/0035-2e003ccf3de917b9e2ff61b1917749bd-0ceab24e"}
> 
> Note that the path attribute in log messages is a full path as requested by
> the API client.
> 
> Looking into _get_spec_for_resource_type method on BackendApplication, it
> loads each resource spec from the database and checks spec path against the
> path:
> 
>     def _get_spec_for_resource_type(self, path):
>         qvarn.log.log(
>             'debug', msg_text='Loading spec from database', path=path)
>         rst = qvarn.ResourceTypeStorage()
>         with self._dbconn.transaction() as t:
>             type_names = rst.get_types(t)
>             for type_name in type_names:
>                 spec = rst.get_spec(t, type_name)
>                 if spec[u'path'] == path:
>                     qvarn.log.log(
>                         'debug', msg_text='Found spec', path=path,
> spec=spec)
>                     return spec
>         qvarn.log.log('debug', msg_text='No spec found for path', path=path)
>         return None
> 
> Specs contain only the rousource name portion, eg. /persons and
> _get_spec_for_resource_type checks for equality, thus it fails to find a
> spec for the request unless it's a GET or POST to the /persons.
> 
> Changing comparison to only compare first part of the path fixed the issue:
>     if spec[u'path'].lstrip('/') == path.split('/')[1]:
>         <...>
> 
> I guess YARN tests didn't catch this because they all do GET or POST
> /<resource_type> before proceeding with further testing. We run multiple
> uwsgi processes so we ran into the problem, since only one of the processes
> had the routes set up properly.
> 
> 
> Regards,
> Saulius

-- 
Mantas



More information about the qvarn-devel mailing list