Project

General

Profile

Bug #27929

Error 22:rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory

Added by Brandon Golway over 2 years ago. Updated over 2 years ago.

Status:
Done
Priority:
Expected
Assignee:
Damian Szidiropulosz
Category:
GUI (new)
Severity:
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

This error appears as soon as I log into the new UI, it started happening yesterday (well for me, I hadn't updated in about a week), I just updated to the latest nightly, 201801230506, and it still shows up. It shows up every time I go to the Dashboard page.

Here's the traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 904, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 872, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 502, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/stats.py", line 111, in get_data
    raise ValueError('rrdtool failed: {}'.format(err.decode()))
ValueError: rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory
Clipboard - February 8 (144 KB) Clipboard - February 8 Peter Southwell, 02/08/2018 11:57 AM
rdderrorlogin.png (60.4 KB) rdderrorlogin.png Rishabh Chauhan, 03/22/2018 10:10 AM
15534

Related issues

Has duplicate FreeNAS - Bug #28000: Error message after logging in.Closed2018-01-26

History

#1 Updated by Dru Lavigne over 2 years ago

  • Status changed from Unscreened to Not Started
  • Assignee changed from Release Council to Erin Clark
  • Target version set to 11.2-BETA1

#2 Updated by Erin Clark over 2 years ago

  • Assignee changed from Erin Clark to Peter Southwell
  • Priority changed from No priority to Expected

I'm not sure if this is something that needs to be handled on the middleware side or the ui side (or maybe both?), could you look into this Damian, perhaps add some error handling to the dashboard if need be.

#3 Updated by Erin Clark over 2 years ago

  • Assignee changed from Peter Southwell to Damian Szidiropulosz

#4 Updated by Brandon Golway over 2 years ago

Just updated to 201801280537 and it still appears, which is to be assumed since it says nothing was done here.

#5 Updated by A B over 2 years ago

Brandon Golway wrote:

Just updated to 201801280537 and it still appears, which is to be assumed since it says nothing was done here.

+1

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 904, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 872, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 502, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/stats.py", line 111, in get_data
    raise ValueError('rrdtool failed: {}'.format(err.decode()))
ValueError: rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory

FreeNAS-11-MASTER-201802010538

#6 Updated by Damian Szidiropulosz over 2 years ago

  • Status changed from Not Started to In Progress

#7 Updated by Damian Szidiropulosz over 2 years ago

  • Status changed from In Progress to Done

#8 Updated by Dru Lavigne over 2 years ago

  • Status changed from Done to Closed
  • Target version changed from 11.2-BETA1 to N/A
  • Reason for Closing set to Cannot Reproduce

#9 Updated by A B over 2 years ago

I'm not sure why this ticket is closed? I think some users have the same issue?

#10 Updated by Dru Lavigne over 2 years ago

This is no longer reproducible on a latest nightly.

#11 Updated by A B over 2 years ago

Dru Lavigne wrote:

This is no longer reproducible on a latest nightly.

I can reproduce it with Version: FreeNAS-11-MASTER-201802070721

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 904, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 872, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 502, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/stats.py", line 111, in get_data
    raise ValueError('rrdtool failed: {}'.format(err.decode()))
ValueError: rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory

#12 Updated by Dru Lavigne over 2 years ago

  • Status changed from Closed to Not Started
  • Target version changed from N/A to 11.2-RC2
  • Reason for Closing deleted (Cannot Reproduce)

Back to the developer then...

#13 Updated by Dru Lavigne over 2 years ago

  • Has duplicate Bug #28000: Error message after logging in. added

#14 Updated by Peter Southwell over 2 years ago

  • Assignee changed from Damian Szidiropulosz to Peter Southwell

Ive seen this before

#15 Updated by Peter Southwell over 2 years ago

#16 Updated by Peter Southwell over 2 years ago

uploaded a screen shot..

but with 2/08.2018

That metric is working just fine.

{"id": "fc0f7af6-5679-c55d-634f-8d6b696701e8", "msg": "result", "result": {"about": "Data for aggregation-cpu-sum/cpu-user,aggregation-cpu-sum/cpu-interrupt,aggregation-cpu-sum/cpu-system,aggregation-cpu-sum/cpu-idle,aggregation-cpu-sum/cpu-nice", "meta": {"start": 1518119350, "end": 1518119950, "step": 10, "legend": ["aggregation-cpu-sum/cpu-user", "aggregation-cpu-sum/cpu-interrupt", "aggregation-cpu-sum/cpu-system", "aggregation-cpu-sum/cpu-idle", "aggregation-cpu-sum/cpu-nice"]}, "data": [[3.22, 0.12, 2.605, 124.44, 0.0], [9.93, 0.07, 4.765, 124.36, 0.0], [9.93, 0.07, 4.765, 124.24, 0.0], [8.08, 0.06, 3.6, 118.74, 0.0], [1.62, 0.14, 1.42, 117.1, 0.0], [1.66, 0.14, 1.42, 124.72, 0.0], [1.56, 0.13, 2.18, 124.02, 0.0], [3.04, 0.13, 2.18, 124.72, 0.0], [3.36, 0.03, 1.6, 122.98, 0.0], [1.84, 0.03, 1.6, 122.5, 0.0], [1.4, 0.0, 0.845, 124.4, 0.0], [1.68, 0.0, 0.845, 125.06, 0.0], [1.84, 0.0, 0.67, 124.48, 0.0], [2.08, 0.0, 0.66, 124.7, 0.0], [2.54, 0.0, 0.56, 124.46, 0.0], [2.54, 0.0, 0.5, 125.3, 0.0], [2.76, 0.0, 0.78, 124.52, 0.0], [1.78, 0.0, 1.795, 124.74, 0.0], [1.82, 0.0, 1.795, 124.84, 0.0], [3.72, 0.0, 3.135, 124.02, 0.0], [4.08, 0.0, 3.135, 121.7, 0.0], [1.66, 0.0, 2.24, 121.44, 0.0], [1.36, 0.0, 0.83, 124.1, 0.0], [1.74, 0.0, 0.83, 125.02, 0.0], [1.58, 0.0, 0.89, 124.68, 0.0], [1.02, 0.0, 0.89, 124.72, 0.0], [0.16, 0.0, 0.56, 125.6, 0.0], [0.48, 0.0, 0.16, 126.86, 0.0], [0.48, 0.0, 0.14, 126.38, 0.0], [0.26, 0.0, 0.2, 126.12, 0.0], [0.36, 0.0, 0.64, 126.24, 0.0], [1.52, 0.0, 0.95, 126.2, 0.0], [1.82, 0.0, 0.95, 124.72, 0.0], [0.16, 0.14, 2.48, 124.76, 0.0], [1.42, 0.14, 2.48, 126.24, 0.0], [2.16, 0.2, 2.2, 123.84, 0.0], [0.46, 0.9, 10.745, 123.3, 0.0], [0.46, 0.9, 10.745, 169.8, 0.0], [0.22, 0.43, 4.725, 159.235, 0.0], [0.36, 0.43, 4.725, 159.235, 0.0], [1.0, 0.18, 0.26, 126.17, 0.0], [1.0, 0.18, 0.32, 125.92, 0.0], [2.91, 0.0, 0.99, 126.16, 0.0], [2.91, 0.0, 0.99, 125.8, 0.0], [2.36, 0.07, 0.64, 125.04, 0.0], [0.34, 0.07, 0.64, 124.7, 0.0], [0.18, 0.06, 0.24, 126.06, 0.0], [0.3, null, 0.7, 126.78, 0.0], [0.46, null, 0.7, 126.78, 0.0], [0.46, null, null, 125.9, null], [0.14, null, null, 125.88, null], [0.2, null, null, 126.04, null], [0.2, null, null, 127.12, null], [0.32, null, null, 127.14, null], [0.42, null, null, 126.46, null], [1.7, null, null, 125.76, null], [7.475, null, null, 124.78, null], [7.475, null, null, 124.56, null], [null, null, null, 121.58, null], [null, null, null, null, null]]}}

And I observe the file exists as well:

/var/db/collectd/rrd/localhost/aggregation-cpu-sum

there is

cpu-idle-rrd
cpu-interrupt-rrd
cpu-nice.rrd
cpu-system.rrd
cpu-uuser.rrd
etc..

So the api is working...
This is a completly fresh install from ISO.

What I will do though.. Is make the sure the website loads.. IF a metric is not present.

#18 Updated by Peter Southwell over 2 years ago

  • % Done changed from 0 to 100

#19 Updated by Peter Southwell over 2 years ago

  • Status changed from Not Started to Blocked

The code is done.. But it's pending a PR Request review. Want to make sure the philosophy of the solution is agreed upon by another :)

#20 Updated by Brandon Golway over 2 years ago

Ok, I haven't checked whether to see if it was merged or not, but I just updated to FreeNAS-11-MASTER-201802100419 and I still see it.

#21 Updated by Brandon Golway over 2 years ago

Is this going to be fixed at some point? It's still present in FreeNAS-11-MASTER-201802170742

#22 Updated by Peter Southwell over 2 years ago

  • Status changed from Blocked to Done

Got the error case working.

Not seeing this error any longer.. Even in fresh iso's. Should be good to go. Wasn't a fix in the UI.

#23 Updated by William Grzybowski over 2 years ago

  • Category changed from Middleware to GUI (new)

#24 Updated by Dru Lavigne over 2 years ago

  • Target version changed from 11.2-RC2 to Master - FreeNAS Nightlies

#25 Updated by Brandon Golway over 2 years ago

I still see it with a fresh install of 11.1-U2

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 908, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 876, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 491, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/stats.py", line 111, in get_data
    raise ValueError('rrdtool failed: {}'.format(err.decode()))
ValueError: rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory

[bran@freenas ~]$ ls /var/db/collectd/rrd/localhost/aggregation-cpu-sum/
percent-idle.rrd        percent-interrupt.rrd   percent-nice.rrd        percent-system.rrd      percent-user.rrd

#26 Updated by Dru Lavigne over 2 years ago

Correct. New UI changes will only appear in the Nightlies until 11.2-BETA1 (that is, they are not pushed to 11.1 updates).

#27 Updated by Brandon Golway over 2 years ago

Just installed FreeNAS-11-MASTER-201802261247 and I'm happy to say this is finally resolved :)

Edit: I take that back. It works initially on the dashboard, but if I go to Reporting the error noted above pops up. When I go back to the dashboard the error pops up again.

This is a completely fresh install, including the hypervisor (before it was ESXi 6.5, now it's KVM/QEMU on Arch Linux), the only similar things are the browser (even the OS I'm using on my laptop is different, initially was Windows 10, now Arch Linux) and my server's hardware.

Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 150, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 904, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 872, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 502, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/asyncio/coroutines.py", line 109, in __next__
    return self.gen.send(None)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/stats.py", line 111, in get_data
    raise ValueError('rrdtool failed: {}'.format(err.decode()))
ValueError: rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//aggregation-cpu-sum/cpu-user.rrd': No such file or directory

#28 Updated by Brandon Golway over 2 years ago

  • % Done changed from 100 to 0

#29 Updated by Brandon Golway over 2 years ago

I still see this in FreeNAS-11-MASTER-201803051006

#30 Updated by Dru Lavigne over 2 years ago

  • Status changed from Done to Broken

Peter: did this ever get merged?

#31 Updated by Peter Southwell over 2 years ago

  • Assignee changed from Peter Southwell to Damian Szidiropulosz

rror 22:rrdtool failed: ERROR: opening '/var/db/collectd/rrd/localhost//disktemp-da0/temperature.rrd': No such file or directory
ValueError

remove_circle_outline More info...

Thatn's actually not an error.. It's just no data found... So basically the message default box needs to be squashed.. But that's deep in the bowels of widget-pool apiService... callWebsocket

It needs an ability to configure an external error handler.. So that the UI using the infastructure can choose how to display the error... In this case.. The chart should just be empty in my opinion because it's true.. I have virtual disks running in a VM.. So.. The disks don't produce temperature data. the widget infastructure needs support for this case... (When the UI wants to custom specify how it deals with the error). If it's burried in the component framework.. It's out of the ability of the uI code to do anything about the matter.

It should support this pradigm:
```
code.subscribe((resp)=>{

console.info("got a response");

},(error) = > {

console.error("Im custom handling the error", error );
});
```

#32 Updated by Dru Lavigne over 2 years ago

  • Target version changed from Master - FreeNAS Nightlies to 11.2-RC2

#33 Updated by Brandon Golway over 2 years ago

Thanks for the update.

#34 Updated by Cosmin BRATU over 2 years ago

In order to get rid of this error I chose do the following:

Dashboard > System > Advanved > Remove check from "*Report CPU usage in percentage*"

Then I reboot FREENAS machine and no more errors.

MoreInfo:
Code must be inspected because if user select above option then in this in location "/var/db/collectd/rrd/localhost//aggregation-cpu-sum/" required files are not named properly to reflect the option.

#35 Updated by Brandon Golway over 2 years ago

Good to know, thanks.

#36 Updated by Rishabh Chauhan over 2 years ago

15534

I think if this is looking for the data which is not present, it must be considered as an error. I don't think it is good the very first thing to see is an error when a system is a setup.
I would suggest maybe put a quick fix (if possible) which does not display the error in this specific case but record it in the logs(for debugging later). if file not present, skip or something...
I am requesting this because it is also affecting the WebUI tests also. Id like to help if you need anything regarding this(ofcourse if I can :P)

#37 Updated by Erin Clark over 2 years ago

  • Target version changed from 11.2-RC2 to 11.2-BETA1

#38 Updated by Brandon Golway over 2 years ago

Rishabh Chauhan wrote:

I think if this is looking for the data which is not present, it must be considered as an error. I don't think it is good the very first thing to see is an error when a system is a setup.
I would suggest maybe put a quick fix (if possible) which does not display the error in this specific case but record it in the logs(for debugging later). if file not present, skip or something...
I am requesting this because it is also affecting the WebUI tests also. Id like to help if you need anything regarding this(ofcourse if I can :P)

The post two above you gets rid of the error, I still have the same sort of pop-ups come up since my drives don't report SMART temps (since they're virtual). Also keep in mind the UI is beta, so things WILL be broken and may take time to fix, no use in wasting time creating a band-aid.

#39 Updated by Damian Szidiropulosz over 2 years ago

  • Status changed from Broken to In Progress

#40 Updated by Damian Szidiropulosz over 2 years ago

PR 605 fixes this. Implemented a stats service that fetches stats on behalf of components. This service makes sure that a source for the desired stat is available before it requests the data from middleware. The dashboard widgets have been adjusted to make use of this service instead of requesting stats on their own.

#41 Updated by Damian Szidiropulosz over 2 years ago

  • Status changed from In Progress to Done

#42 Updated by Dru Lavigne over 2 years ago

  • Target version changed from 11.2-BETA1 to Master - FreeNAS Nightlies
  • Needs Merging changed from Yes to No

Also available in: Atom PDF