Project

General

Profile

Bug #23833

Problem with Firefly plugin

Added by Andrew McCann over 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Critical
Assignee:
William Grzybowski
Category:
Middleware
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

Have been testing some plugins and found a problem with Firefly, the plugin is not working at all, appears on the installed plugins page, the service status is always set OFF, attempting to set it to ON and and a msg "Some error occured" appears.

freenas uwsgi: [plugins.utils:91] Couldn't retrieve http://192.168.1.50/plugins/firefly/5/_s/status: HTTP Error 503: Service Unavailable

This was tested on FreeNAS-11.0-RC (ad90a865b)
Plugin Jails running 11.0-STABLE

Plugins - Firefly.PNG (67.3 KB) Plugins - Firefly.PNG Andrew McCann, 05/07/2017 06:40 PM
11016

Related issues

Is duplicate of FreeNAS - Bug #23832: Problem with Bacula-sd pluginClosed: Duplicate2017-05-08
Is duplicate of FreeNAS - Bug #24060: FreeNAS-11.0-RC2 + Resilio Sync 2.4.4 == unable to startClosed: Duplicate2017-05-18

Associated revisions

Revision 5ce6d44a (diff)
Added by William Grzybowski over 3 years ago

fix(plugins): a bunch of terrible hacks to make 9.X PBI work on 11 Ticket: #23833

Revision 7f2b4b0f (diff)
Added by William Grzybowski over 3 years ago

fix(plugins): remove pdb Ticket: #23833

Revision 1907c55e (diff)
Added by William Grzybowski over 3 years ago

fix(plugins): a bunch of terrible hacks to make 9.X PBI work on 11 Ticket: #23833 (cherry picked from commit 5ce6d44a152e4dc631fb6097cccf75e2058b6529)

Revision 86b028ac (diff)
Added by William Grzybowski over 3 years ago

fix(plugins): remove pdb Ticket: #23833 (cherry picked from commit 7f2b4b0fe29d4a9fb00c568cbb8746002ba5cb9a)

History

#1 Updated by Andrew McCann over 3 years ago

  • File debug-freenas-20170508115537.tgz added

including debug

#2 Updated by Brandon Schneider over 3 years ago

  • Assignee changed from Brandon Schneider to William Grzybowski

#3 Updated by William Grzybowski over 3 years ago

  • Is duplicate of Bug #23832: Problem with Bacula-sd plugin added

#4 Updated by William Grzybowski over 3 years ago

  • Status changed from Unscreened to Screened

#5 Updated by William Grzybowski over 3 years ago

  • Status changed from Screened to 15

Is this a fresh install of 11?

What do you see when you try to access http://192.168.1.50/plugins/firefly/5/_s/status from the browser?

#6 Updated by Andrew McCann over 3 years ago

yes, it was a fresh install from the 11 nightlies, just upgraded to RC. All the plugins shown on the screenshot were fresh installs.

tried link
http://192.168.1.50/plugins/firefly/5/_s/status
[Errno 61] Connection refused

#7 Updated by William Grzybowski over 3 years ago

Can you get me:

ifconfig -a
jexec firefly_1 ifconfig -a
jexec firefly_1 sockstat -4
jexec firefly_1 ps axuww
ping FIREFLY_JAIL_IP

#8 Updated by Andrew McCann over 3 years ago

  • Private changed from No to Yes

here you go!

ifconfig -a
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
ether ac:1f:6b:00:c1:e0
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
ether ac:1f:6b:00:c1:e0
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
ether ac:1f:6b:00:c1:e0
inet 192.168.1.50 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: active
groups: lagg
laggproto lacp lagghash l2,l3,l4
laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:cc:13:4c:9d:00
nd6 options=9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: epair5a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 11 priority 128 path cost 2000
member: epair4a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 10 priority 128 path cost 2000
member: epair3a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000
member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 8 priority 128 path cost 2000
member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 7 priority 128 path cost 2000
member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000
member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 10000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:06:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:07:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:08:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
epair3a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:09:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
epair4a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:0a:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
epair5a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:e0:00:0b:0a
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair

jexec firefly_1 ifconfig -a

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
epair1b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:ff:30:00:0b:0b
inet 192.168.1.64 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair

jexec firefly_1 sockstat -4

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root syslogd 7560 7 udp4 :514 *:

jexec firefly_1 ps axuww
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 7560 0.0 0.0 10472 1804 - SsJ 07:32 0:00.00 /usr/sbin/syslogd -s
root 7745 0.0 0.0 12564 1736 - SsJ 07:32 0:00.00 /usr/sbin/cron -s
root 14988 0.0 0.0 21244 1888 0 R+J 07:37 0:00.00 ps axuww

ping FIREFLY_JAIL_IP

[root@freenas ~]# ping 192.168.1.64
PING 192.168.1.64 (192.168.1.64): 56 data bytes
64 bytes from 192.168.1.64: icmp_seq=0 ttl=64 time=0.039 ms
64 bytes from 192.168.1.64: icmp_seq=1 ttl=64 time=0.074 ms
64 bytes from 192.168.1.64: icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from 192.168.1.64: icmp_seq=3 ttl=64 time=0.061 ms

#9 Updated by William Grzybowski over 3 years ago

Ok, it seems firefly control process is not starting

What happens if you run
service ix-plugins onestart

?

Get me the output of "jexec firefly_1 ps axuww" again after that.
If it remains the same, get me "sh -x /etc/ix.rc.d/ix-plugins start"

#10 Updated by Andrew McCann over 3 years ago

jexec 4 /usr/pbi/plexmediaserver-amd64/control start 192.168.1.60 12346
jexec 5 /usr/pbi/transmission-amd64/control start 192.168.1.61 12347
jexec 1 /usr/pbi/bacula-sd-amd64/control start 192.168.1.62 12348
jexec 6 /usr/pbi/xdm-amd64/control start 192.168.1.63 12349
jexec 2 /usr/pbi/firefly-amd64/control start 192.168.1.64 12350
jexec 3 /usr/pbi/owncloud-amd64/control start 192.168.1.65 12351

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 7560 0.0 0.0 10472 1804 - IsJ 07:32 0:00.01 /usr/sbin/syslogd -s
root 7745 0.0 0.0 12564 1736 - IsJ 07:32 0:00.00 /usr/sbin/cron -s
root 17873 0.0 0.0 21244 1904 0 R+J 07:50 0:00.00 ps axuww

ok, that didn't work. My BSD skills are not great, whats the best way to pipe the results from sh -x /etc/ix.rc.d/ix-plugins start?

#11 Updated by William Grzybowski over 3 years ago

Andrew McCann wrote:

jexec 4 /usr/pbi/plexmediaserver-amd64/control start 192.168.1.60 12346
jexec 5 /usr/pbi/transmission-amd64/control start 192.168.1.61 12347
jexec 1 /usr/pbi/bacula-sd-amd64/control start 192.168.1.62 12348
jexec 6 /usr/pbi/xdm-amd64/control start 192.168.1.63 12349
jexec 2 /usr/pbi/firefly-amd64/control start 192.168.1.64 12350
jexec 3 /usr/pbi/owncloud-amd64/control start 192.168.1.65 12351

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 7560 0.0 0.0 10472 1804 - IsJ 07:32 0:00.01 /usr/sbin/syslogd -s
root 7745 0.0 0.0 12564 1736 - IsJ 07:32 0:00.00 /usr/sbin/cron -s
root 17873 0.0 0.0 21244 1904 0 R+J 07:50 0:00.00 ps axuww

ok, that didn't work. My BSD skills are not great, whats the best way to pipe the results from sh -x /etc/ix.rc.d/ix-plugins start?

You would need to drop into "sh" (not default csh) and use 2>&1 > /tmp/file

But that will not be needed since you provided the control command output.

Do you get any error if you manually try to run:
jexec 2 /usr/pbi/firefly-amd64/control start 192.168.1.64 12350

?

#12 Updated by Andrew McCann over 3 years ago

yes, did get an error

[root@freenas ~]# jexec 2 /usr/pbi/firefly-amd64/control start 192.168.1.64 12350
Shared object "libintl.so.9" not found, required by "python2.7"

#13 Updated by William Grzybowski over 3 years ago

Andrew McCann wrote:

yes, did get an error

[root@freenas ~]# jexec 2 /usr/pbi/firefly-amd64/control start 192.168.1.64 12350
Shared object "libintl.so.9" not found, required by "python2.7"

Did you import a previous pool with an existing jails dataset? That error is a bit suspicious.

#14 Updated by Andrew McCann over 3 years ago

Hi, I re-created my jail dataset to force all jails to use freebsd 11.

#15 Updated by William Grzybowski over 3 years ago

  • Priority changed from No priority to Critical
  • Target version set to 11.0

#16 Updated by Andrew McCann over 3 years ago

I am online for the next 30-40 mins, if you need anymore testing let me know!
Cheers

#17 Updated by William Grzybowski over 3 years ago

  • Status changed from 15 to Investigation

Andrew McCann wrote:

I am online for the next 30-40 mins, if you need anymore testing let me know!
Cheers

Thanks but I am thinking there is a major problem in the plugin template, I am investigating now.

#18 Updated by William Grzybowski over 3 years ago

  • Status changed from Investigation to 19

#19 Updated by William Grzybowski over 3 years ago

  • Status changed from 19 to Needs Developer Review
  • Assignee changed from William Grzybowski to John Hixson

Please review.

#20 Updated by John Hixson over 3 years ago

William Grzybowski wrote:

Please review.

From the surface it looks good. I'm going to test and verify however ;-)

#21 Updated by John Hixson over 3 years ago

  • Status changed from Needs Developer Review to Reviewed
  • Assignee changed from John Hixson to William Grzybowski

well, the plugin installs and the control script starts. When trying to start the plugin I get:

{"message": "Firefly data did not validate, please configure it first.", "error": true}

I have no idea what this means since I am unfamiliar with this plugin. Does this plugin require configuration in the jail before it will work? As for starting the control script and the LD error, that is now resolved.

#22 Updated by William Grzybowski over 3 years ago

John Hixson wrote:

well, the plugin installs and the control script starts. When trying to start the plugin I get:

{"message": "Firefly data did not validate, please configure it first.", "error": true}

I have no idea what this means since I am unfamiliar with this plugin. Does this plugin require configuration in the jail before it will work? As for starting the control script and the LD error, that is now resolved.

Yes, under the tree menu Plugins -> Firefly

#23 Updated by Vaibhav Chauhan over 3 years ago

  • Status changed from Reviewed to Merged

#24 Updated by Vaibhav Chauhan over 3 years ago

  • Target version changed from 11.0 to 11.0-RC3

#25 Updated by William Grzybowski over 3 years ago

  • Is duplicate of Bug #24060: FreeNAS-11.0-RC2 + Resilio Sync 2.4.4 == unable to start added

#26 Updated by Vaibhav Chauhan over 3 years ago

  • Status changed from Merged to Resolved

#27 Updated by Dru Lavigne about 3 years ago

  • File deleted (debug-freenas-20170508115537.tgz)

#28 Updated by Dru Lavigne about 3 years ago

  • Private changed from Yes to No

Also available in: Atom PDF