Describe the bug
ipdevpolld fails to run some jobs on some netboxes with a traceback. This affects 1minstats, 5minstats, dns, inventory and topo.
However, not all netboxes are affected and among those affected, some are affected in some of the jobs but not others.
Eg. netbox1 is affected in topo and inventory, netbox2 is affected in topo only, and netbox3 is not affected at all (eg. ipdevpolld is able to run all jobs againts netbox3 without errors).
I have not been able to find where this is coming from. I have included two tracebacks from two different jobs, but they all boil down to this:
builtins.ValueError: A string literal cannot contain NUL (0x00) characters.
Tracebacks
(all netbox names as been replaced with redacted)
Traceback for a topo job
2019-12-11 13:33:47,403 [ERROR jobs.jobhandler] [topo <redacted>] Plugin nav.ipdevpoll.plugins.cdp.CDP('<redacted>') reported an unhandled failure
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 500, in errback
self._startRunCallbacks(fail)
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 567, in _startRunCallbacks
self._runCallbacks()
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1442, in gotResult
_inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
result = result.throwExceptionIntoGenerator(g)
File "/usr/local/lib/python3.7/site-packages/twisted/python/failure.py", line 408, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/plugins/cdp.py", line 66, in handle
yield run_in_thread(self._process_cache)
File "/usr/local/lib/python3.7/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib/python3.7/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/db.py", line 100, in _reset
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/plugins/cdp.py", line 89, in _process_cache
neighbors = [CDPNeighbor(cdp, self.netbox.ip) for cdp in self.cache]
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/plugins/cdp.py", line 89, in <listcomp>
neighbors = [CDPNeighbor(cdp, self.netbox.ip) for cdp in self.cache]
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/neighbor.py", line 117, in __init__
self.identify()
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/neighbor.py", line 120, in identify
self.netbox = self._identify_netbox()
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/plugins/cdp.py", line 155, in _identify_netbox
netbox = self._netbox_from_sysname(self.record.deviceid)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/neighbor.py", line 175, in _netbox_from_sysname
return self._netbox_query(query)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/neighbor.py", line 187, in _netbox_query
netbox = manage.Netbox.objects.values('id', 'sysname').get(query)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 374, in get
num = len(clone)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 232, in __len__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1121, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 106, in __iter__
for row in compiler.results_iter(chunked_fetch=self.chunked_fetch):
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 841, in results_iter
results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
raise original_exception
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
builtins.ValueError: A string literal cannot contain NUL (0x00) characters.
Traceback for a inventory job
2019-12-11 13:33:59,607 [ERROR jobs.jobhandler] [inventory <redacted>] Caught exception during save. Last manager = InterfaceManager(<class 'nav.ipdevpoll.shadows.interface.Interface'>, 'ContainerRepository'(...)). Last model = <class 'nav.ipdevpoll.shadows.interface.Interfac
e'>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/jobs.py", line 442, in _perform_save
manager.save()
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 87, in save
obj.save(self.containers)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 478, in save
self.update(containers)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 508, in update
myself.update(**update)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 650, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
raise original_exception
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
ValueError: A string literal cannot contain NUL (0x00) characters.
2019-12-11 13:33:59,608 [ERROR jobs.jobhandler] [inventory <redacted>] Save stage failed with unhandled error
Traceback (most recent call last):
File "/usr/local/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.7/site-packages/twisted/_threads/_threadworker.py", line 46, in work
task()
File "/usr/local/lib/python3.7/site-packages/twisted/_threads/_team.py", line 190, in doWork
task()
--- <exception caught here> ---
File "/usr/local/lib/python3.7/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/usr/local/lib/python3.7/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/db.py", line 100, in _reset
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/db.py", line 78, in _cleanup
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/jobs.py", line 397, in complete_save_cycle
result = self._perform_save()
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/jobs.py", line 442, in _perform_save
manager.save()
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 87, in save
obj.save(self.containers)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 478, in save
self.update(containers)
File "/usr/local/lib/python3.7/site-packages/nav/ipdevpoll/storage.py", line 508, in update
myself.update(**update)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 650, in update
rows = query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1204, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 899, in execute_sql
raise original_exception
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
builtins.ValueError: A string literal cannot contain NUL (0x00) characters.
Expected behavior
No errors logged
Environment (please complete the following information):
- OS on the NAV server: Ubuntu 16.04 LTS
- NAV version installed: 5.0.1
- Method of installation: source
Describe the bug
ipdevpolld fails to run some jobs on some netboxes with a traceback. This affects 1minstats, 5minstats, dns, inventory and topo.
However, not all netboxes are affected and among those affected, some are affected in some of the jobs but not others.
Eg. netbox1 is affected in topo and inventory, netbox2 is affected in topo only, and netbox3 is not affected at all (eg. ipdevpolld is able to run all jobs againts netbox3 without errors).
I have not been able to find where this is coming from. I have included two tracebacks from two different jobs, but they all boil down to this:
builtins.ValueError: A string literal cannot contain NUL (0x00) characters.
Tracebacks
(all netbox names as been replaced with redacted)
Traceback for a topo job
Traceback for a inventory job
Expected behavior
No errors logged
Environment (please complete the following information):