Friday, June 19, 2015

Cobbler login failure for spacewalk proxy

Cobbler login failure for spacewalk proxy



Background

We built out a spacewalk infrasture with one spacewalk_master and several spacewalk_proxy in remote data centers around the world. Once we completed the proxy, spacewalk proxy worked fine but the entire cobbler setup on the proxy did not work.


spacewalk proxy references

These are the document use to build the proxies
https://fedorahosted.org/spacewalk/wiki/HowToInstallProxy
https://www.redhat.com/archives/spacewalk-list/2014-May/msg00088.html

 

Debugging login failure

Once we completed the build out of the proxy, run the comand:

cobbler list

Traceback (most recent call last):
  File "/usr/bin/cobbler", line 35, in <module>
    sys.exit(app.main())
  File "/usr/lib/python2.6/site-packages/cobbler/cli.py", line 511, in main
    rc = cli.run(sys.argv)
  File "/usr/lib/python2.6/site-packages/cobbler/cli.py", line 185, in run
    self.token         = self.remote.login("", self.shared_secret)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1253, in request
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "<class 'cobbler.cexceptions.CX'>:'login failed'">
 
 Debug :
 python -m pdb /usr/bin/cobbler list

> /usr/bin/cobbler(15)<module>()
-> """
(Pdb) n
> /usr/bin/cobbler(17)<module>()
-> import cobbler.cli as app
(Pdb) n
> /usr/bin/cobbler(18)<module>()
-> import sys
(Pdb) n
> /usr/bin/cobbler(20)<module>()
-> PROFILING = False
(Pdb) n
> /usr/bin/cobbler(22)<module>()
-> if PROFILING:
(Pdb) n
> /usr/bin/cobbler(35)<module>()
-> sys.exit(app.main())
(Pdb) s
--Call--
> /usr/lib/python2.6/site-packages/cobbler/cli.py(505)main()
-> def main():
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(509)main()
-> cli = BootCLI()
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(510)main()
-> cli.check_setup()
(Pdb) s
--Call--
> /usr/lib/python2.6/site-packages/cobbler/cli.py(153)check_setup()
-> def check_setup(self):
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(159)check_setup()
-> s = xmlrpclib.Server(self.url_cobbler_xmlrpc)
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(160)check_setup()
-> try:
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(161)check_setup()
-> s.ping()
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(166)check_setup()
-> s = xmlrpclib.Server(self.url_cobbler_api)
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(167)check_setup()
-> try:
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(168)check_setup()
-> s.ping()
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(173)check_setup()
-> if not os.path.exists("/var/lib/cobbler/web.ss"):
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(177)check_setup()
-> if not os.access("/var/lib/cobbler/web.ss", os.R_OK):
(Pdb) n
--Return--
> /usr/lib/python2.6/site-packages/cobbler/cli.py(177)check_setup()->None
-> if not os.access("/var/lib/cobbler/web.ss", os.R_OK):
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(511)main()
-> rc = cli.run(sys.argv)
(Pdb) s
--Call--
> /usr/lib/python2.6/site-packages/cobbler/cli.py(181)run()
-> def run(self, args):
(Pdb) n
> /usr/lib/python2.6/site-packages/cobbler/cli.py(185)run()
-> self.token         = self.remote.login("", self.shared_secret)
(Pdb) p self.shared_secret
'91245c18d9cbc4ba2sdfsdf66c4aac4d9d3fd53eb95b91685f3cd24c985a9e2700b53153d1fb8dfc4931e0cd3e5f57951b3217980f1f4fb9b1e2ccb155e5d3882b163f2b2eca52ea4c60c0403ad215b73fc4666d1db6c3c5a9d5ee5f226db6df600ee35895f4be7d503cf0743efe633016e0aeafc84b816181646dec268176a6a447b4a6363c0b04feda03703066055466c3ee53b3026d427b92344da0108b293b3d033a5c159c56ba1b8672d98a9fc683bb4a2c4a913ad8021490df6ffd3e4a90177f6ed91fe1e7903f87e3291ca0fce6ea4c7beb1677d57a3940150b646f84df7e0fa137283a6fd68ba06cf8d5cf38784207a1fdd29eba499772495e71fdb0625ca56d85db74586f2b4866be699113f66ab2e4b835aadad598997819a64f39d4534c8aa6608d6d8dfa2cbad8d86e2ecdc336d4b7b7566dec4a906a60fd2b8299ec0fc8791c576ee23b7f33916ae86e7c8df7f699b767f5d25b12b5ef7f865260b09fb36dcb39c1120946a4364c5082500918a33a93c5d46913215d4df03f6912bfb13dab0fb35996cee9a699eb1095c89be9e1cd0e237aa7a920bdecd0cb1d55d91db9046b28b755843fa41c30ab3caf5d92a2778642f6ba1700431a13fb53212e2ee8c058a079acad7fe0ddfc8d01e931f5af7c3b528d69d06810e88cf752e949f8e2b8c8dd3b57b3734f767ccc16345c859a5700b1754911b9df08829c9'

Note that the shared secret is read from /var/lib/cobbler/web.ss and that this file changes upon cobblerd restart ( service cobblerd restart)

Furthermore :
/etc/cobbler/modules.conf
[authentication]
  module = authn_spacewalk



/etc/httpd/conf.d/cobbler-proxy.conf
ProxyPass /cobbler_api https://[spacewalk_master]/download//cobbler_api


What this means is that the proxy is authenicating against the spacewalk_master cobbler.

Solution

Each time cobblerd is restarted on the spacewalk_master /var/lib/cobbler/web.ss must be sync'd to the /var/lib/cobbler/web.ss on the spacewalk_proxy.

You can either push from the  spacewalk_master upon restart of cobblerd or pull a cron on the proxy to pull the latest file.

e.g.
crontab -e
# spacewalk_cobbler_sharedsecret_sync on the proxy
* * * * * /usr/bin/rsync -av [spacewalk_master]:/var/lib/cobbler/web.ss /var/lib/cobbler/web.ss