Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: OpenStack: Operators

Swift configuration

 

 

OpenStack operators RSS feed   Index | Next | Previous | View Threaded


KISIN at il

Jul 24, 2012, 8:28 AM

Post #1 of 4 (864 views)
Permalink
Swift configuration

Hello, Guys.
I've multiple node Openstack env with Essex release.
I'm trying to make it work with Swift ObjectStore.
I've created 4 nodes which are swift-proxy and 3 storage nodes. They are
configured to work with keystone which is installed on different node (that
is also a cloud controller).
My swift-proxy ip is 9.148.27.72 and controller+keystone ip is 9.148.27.76.
All relevant services are up and running (including storage servers).
Trying to test weather the setup works as expected by running the following
command on swift-proxy node:
swift -v -V 2.0 -A http://9.148.27.76:5000/v2.0/ -U admin:admin -K
<my_password> list
I've got an error:
Account GET failed:
http://9.148.27.72:8080/v1/AUTH_0e811b2a0f3e4440a862099e96920bca?format=json
503 Internal Server Error

Looking at the syslog file on swift-proxy node, I saw the messages as
below :
swift-proxy-server 9.148.27.72 9.148.27.72 24/Jul/2012/14/59/28
GET /v1/AUTH_0e811b2a0f3e4440a862099e96920bca%3Fformat%3Djson HTTP/1.0 503
- - 60ad63902be94b7191b44f0efa6e48fc - - -
tx10dc1e59eeb648d4a5dcc5692112bd92
X-Tenant-Name%3A%20admin%0AX-Role%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User-Id%3A%2074888d1bea7346929084be974411ceb1%0AX-Auth-Token%3A%2060ad63902be94b7191b44f0efa6e48fc%0AX-Trans-Id%3A%20tx10dc1e59eeb648d4a5dcc5692112bd92%0AX-Identity-Status%3A%20Confirmed%0AX-Roles%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User%3A%20admin%0AHost%3A%209.148.27.72%3A8080%0AX-Tenant-Id%3A%200e811b2a0f3e4440a862099e96920bca%0AX-Tenant%3A%20admin%0AX-User-Name%3A%20admin%0AContent-Type%3A%20None%0AAccept-Encoding%3A%20identity
0.0023 -
Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
9.148.27.74:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
9.148.27.72)
Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
9.148.27.75:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
9.148.27.72)
Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
9.148.27.73:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
9.148.27.72)
Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Account GET returning
503 for [] (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
9.148.27.72)

Any idea what can be wrong in my configuration scheme ?

Thanks a lot.


Alexandra Kisin
Servers & Network group, IBM R&D Labs in Israel
Unix & Virtualization Team


_______________________________________________
OpenStack-operators mailing list
OpenStack-operators [at] lists
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


andrew.glen-young at canonical

Jul 24, 2012, 10:49 AM

Post #2 of 4 (829 views)
Permalink
Re: Swift configuration [In reply to]

Hello,

On Tue 24-07-2012 18:28 ›, Alexandra Kisin wrote:
>
> Hello, Guys.
> I've multiple node Openstack env with Essex release.
> I'm trying to make it work with Swift ObjectStore.
> I've created 4 nodes which are swift-proxy and 3 storage nodes. They are
> configured to work with keystone which is installed on different node (that
> is also a cloud controller).
> My swift-proxy ip is 9.148.27.72 and controller+keystone ip is 9.148.27.76.
> All relevant services are up and running (including storage servers).
> Trying to test weather the setup works as expected by running the following
> command on swift-proxy node:
> swift -v -V 2.0 -A http://9.148.27.76:5000/v2.0/ -U admin:admin -K
> <my_password> list
> I've got an error:
> Account GET failed:
> http://9.148.27.72:8080/v1/AUTH_0e811b2a0f3e4440a862099e96920bca?format=json
> 503 Internal Server Error
>
> Looking at the syslog file on swift-proxy node, I saw the messages as
> below :
> swift-proxy-server 9.148.27.72 9.148.27.72 24/Jul/2012/14/59/28
> GET /v1/AUTH_0e811b2a0f3e4440a862099e96920bca%3Fformat%3Djson HTTP/1.0 503
> - - 60ad63902be94b7191b44f0efa6e48fc - - -
> tx10dc1e59eeb648d4a5dcc5692112bd92
> X-Tenant-Name%3A%20admin%0AX-Role%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User-Id%3A%2074888d1bea7346929084be974411ceb1%0AX-Auth-Token%3A%2060ad63902be94b7191b44f0efa6e48fc%0AX-Trans-Id%3A%20tx10dc1e59eeb648d4a5dcc5692112bd92%0AX-Identity-Status%3A%20Confirmed%0AX-Roles%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User%3A%20admin%0AHost%3A%209.148.27.72%3A8080%0AX-Tenant-Id%3A%200e811b2a0f3e4440a862099e96920bca%0AX-Tenant%3A%20admin%0AX-User-Name%3A%20admin%0AContent-Type%3A%20None%0AAccept-Encoding%3A%20identity
> 0.0023 -
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.74:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.75:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.73:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Account GET returning
> 503 for [] (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> 9.148.27.72)
>
> Any idea what can be wrong in my configuration scheme ?
>

It looks to me like your proxy-server cannot connect to the
swift-account-server on your storage nodes (9.148.27.74:6002,
9.148.27.75:6002, 9.148.27.73:6002). Is this service running on these
nodes?

You can easily test by using `nc` or `telnet` to connect to the
swift-account-server port from the proxy-server.

Example:

user [at] proxy-serve:~$ nc -vz 9.148.27.74 6002
Connection to 9.148.27.74 6002 port [tcp/x11-2] succeeded!

Hope this helps.

--
Regards.
Andrew Glen-Young

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators [at] lists
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


KISIN at il

Jul 25, 2012, 3:49 AM

Post #3 of 4 (824 views)
Permalink
Re: Swift configuration [In reply to]

Thank you for the tip.
It seems that all services are running on storage nodes.
Running the command that you've advised I've got the following :
root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.73 6002
Connection to 9.148.27.73 6002 port [tcp/x11-2] succeeded!
root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.74 6002
Connection to 9.148.27.74 6002 port [tcp/x11-2] succeeded!
root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.75 6002
Connection to 9.148.27.75 6002 port [tcp/x11-2] succeeded!

So the problem is probably somewhere else.
Any ideas ?

Alexandra Kisin
Servers & Network group, IBM R&D Labs in Israel
Unix & Virtualization Team



From: Andrew Glen-Young <andrew.glen-young [at] canonical>
To: openstack-operators [at] lists
Date: 07/24/2012 08:51 PM
Subject: Re: [Openstack-operators] Swift configuration



Hello,

On Tue 24-07-2012 18:28 ›, Alexandra Kisin wrote:
>
> Hello, Guys.
> I've multiple node Openstack env with Essex release.
> I'm trying to make it work with Swift ObjectStore.
> I've created 4 nodes which are swift-proxy and 3 storage nodes. They are
> configured to work with keystone which is installed on different node
(that
> is also a cloud controller).
> My swift-proxy ip is 9.148.27.72 and controller+keystone ip is
9.148.27.76.
> All relevant services are up and running (including storage servers).
> Trying to test weather the setup works as expected by running the
following
> command on swift-proxy node:
> swift -v -V 2.0 -A http://9.148.27.76:5000/v2.0/ -U admin:admin -K
> <my_password> list
> I've got an error:
> Account GET failed:
>
http://9.148.27.72:8080/v1/AUTH_0e811b2a0f3e4440a862099e96920bca?format=json

> 503 Internal Server Error
>
> Looking at the syslog file on swift-proxy node, I saw the messages as
> below :
> swift-proxy-server 9.148.27.72 9.148.27.72 24/Jul/2012/14/59/28
> GET /v1/AUTH_0e811b2a0f3e4440a862099e96920bca%3Fformat%3Djson HTTP/1.0
503
> - - 60ad63902be94b7191b44f0efa6e48fc - - -
> tx10dc1e59eeb648d4a5dcc5692112bd92
>
X-Tenant-Name%3A%20admin%0AX-Role%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User-Id%3A%2074888d1bea7346929084be974411ceb1%0AX-Auth-Token%3A%2060ad63902be94b7191b44f0efa6e48fc%0AX-Trans-Id%3A%20tx10dc1e59eeb648d4a5dcc5692112bd92%0AX-Identity-Status%3A%20Confirmed%0AX-Roles%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User%3A%20admin%0AHost%3A%209.148.27.72%3A8080%0AX-Tenant-Id%3A%200e811b2a0f3e4440a862099e96920bca%0AX-Tenant%3A%20admin%0AX-User-Name%3A%20admin%0AContent-Type%3A%20None%0AAccept-Encoding%3A%20identity

> 0.0023 -
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.74:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
(client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.75:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
(client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> 9.148.27.73:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
(client_ip:
> 9.148.27.72)
> Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Account GET returning
> 503 for [] (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> 9.148.27.72)
>
> Any idea what can be wrong in my configuration scheme ?
>

It looks to me like your proxy-server cannot connect to the
swift-account-server on your storage nodes (9.148.27.74:6002,
9.148.27.75:6002, 9.148.27.73:6002). Is this service running on these
nodes?

You can easily test by using `nc` or `telnet` to connect to the
swift-account-server port from the proxy-server.

Example:

user [at] proxy-serve:~$ nc -vz 9.148.27.74 6002
Connection to 9.148.27.74 6002 port [tcp/x11-2] succeeded!

Hope this helps.

--
Regards.
Andrew Glen-Young

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators [at] lists
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators [at] lists
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


trey at maldivica

Jul 25, 2012, 7:33 AM

Post #4 of 4 (821 views)
Permalink
Re: Swift configuration [In reply to]

Are the drives in your nodes mounted and healthy? Looking through the doc,
it looks like the 'error limited' message means the node (which I'm pretty
sure means drive) is in an error state and isn't available for taking an
object.

On Wed, Jul 25, 2012 at 6:49 AM, Alexandra Kisin <KISIN [at] il> wrote:

> Thank you for the tip.
> It seems that all services are running on storage nodes.
> Running the command that you've advised I've got the following :
> root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.73 6002
> Connection to 9.148.27.73 6002 port [tcp/x11-2] succeeded!
> root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.74 6002
> Connection to 9.148.27.74 6002 port [tcp/x11-2] succeeded!
> root [at] ensure-sw-prox:/etc/swift# nc -vz 9.148.27.75 6002
> Connection to 9.148.27.75 6002 port [tcp/x11-2] succeeded!
>
> So the problem is probably somewhere else.
> Any ideas ?
>
> Alexandra Kisin
> Servers & Network group, IBM R&D Labs in Israel
> Unix & Virtualization Team
>
>
>
> From: Andrew Glen-Young <andrew.glen-young [at] canonical>
> To: openstack-operators [at] lists
> Date: 07/24/2012 08:51 PM
> Subject: Re: [Openstack-operators] Swift configuration
>
>
>
> Hello,
>
> On Tue 24-07-2012 18:28 ›, Alexandra Kisin wrote:
> >
> > Hello, Guys.
> > I've multiple node Openstack env with Essex release.
> > I'm trying to make it work with Swift ObjectStore.
> > I've created 4 nodes which are swift-proxy and 3 storage nodes. They are
> > configured to work with keystone which is installed on different node
> (that
> > is also a cloud controller).
> > My swift-proxy ip is 9.148.27.72 and controller+keystone ip is
> 9.148.27.76.
> > All relevant services are up and running (including storage servers).
> > Trying to test weather the setup works as expected by running the
> following
> > command on swift-proxy node:
> > swift -v -V 2.0 -A http://9.148.27.76:5000/v2.0/ -U admin:admin -K
> > <my_password> list
> > I've got an error:
> > Account GET failed:
> >
>
> http://9.148.27.72:8080/v1/AUTH_0e811b2a0f3e4440a862099e96920bca?format=json
>
> > 503 Internal Server Error
> >
> > Looking at the syslog file on swift-proxy node, I saw the messages as
> > below :
> > swift-proxy-server 9.148.27.72 9.148.27.72 24/Jul/2012/14/59/28
> > GET /v1/AUTH_0e811b2a0f3e4440a862099e96920bca%3Fformat%3Djson HTTP/1.0
> 503
> > - - 60ad63902be94b7191b44f0efa6e48fc - - -
> > tx10dc1e59eeb648d4a5dcc5692112bd92
> >
>
> X-Tenant-Name%3A%20admin%0AX-Role%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User-Id%3A%2074888d1bea7346929084be974411ceb1%0AX-Auth-Token%3A%2060ad63902be94b7191b44f0efa6e48fc%0AX-Trans-Id%3A%20tx10dc1e59eeb648d4a5dcc5692112bd92%0AX-Identity-Status%3A%20Confirmed%0AX-Roles%3A%20KeystoneAdmin%2Cadmin%2CKeystoneServiceAdmin%0AX-User%3A%20admin%0AHost%3A%209.148.27.72%3A8080%0AX-Tenant-Id%3A%200e811b2a0f3e4440a862099e96920bca%0AX-Tenant%3A%20admin%0AX-User-Name%3A%20admin%0AContent-Type%3A%20None%0AAccept-Encoding%3A%20identity
>
> > 0.0023 -
> > Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> > 9.148.27.74:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
> (client_ip:
> > 9.148.27.72)
> > Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> > 9.148.27.75:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
> (client_ip:
> > 9.148.27.72)
> > Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Node error limited
> > 9.148.27.73:6002 (r0) (txn: tx96afa78117b549ec9be3386f8a266217)
> (client_ip:
> > 9.148.27.72)
> > Jul 24 17:59:36 ensure-sw-proxy swift-proxy-server Account GET returning
> > 503 for [] (txn: tx96afa78117b549ec9be3386f8a266217) (client_ip:
> > 9.148.27.72)
> >
> > Any idea what can be wrong in my configuration scheme ?
> >
>
> It looks to me like your proxy-server cannot connect to the
> swift-account-server on your storage nodes (9.148.27.74:6002,
> 9.148.27.75:6002, 9.148.27.73:6002). Is this service running on these
> nodes?
>
> You can easily test by using `nc` or `telnet` to connect to the
> swift-account-server port from the proxy-server.
>
> Example:
>
> user [at] proxy-serve:~$ nc -vz 9.148.27.74 6002
> Connection to 9.148.27.74 6002 port [tcp/x11-2] succeeded!
>
> Hope this helps.
>
> --
> Regards.
> Andrew Glen-Young
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators [at] lists
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators [at] lists
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>

OpenStack operators RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.