Wednesday 7 April 2010

Hack to get Chromium working with Rocks 5.3

Rocks is "an open-source Linux cluster distribution that enables end users to easily build computational clusters, grid endpoints and visualization tiled-display walls".

Recently I've been using it to get a tiled-display wall, but found that it was not working as expected. Some software that it is supposed to come with SAGE is not there, and Chromium is not working except for the simplest of applications (I only managed to get it working with glxgears). A fix to the Viz Roll is coming, but in the meantime I figured how to get it working by installing my own version of Chromium. Since I've been asked to share this, I put these quick notes here (please let me know if something is not working or something is missing, so I can improve them) in case it can be of interest to anyone, but remember that this is just a quick hack. Hopefully a patched version of the Viz roll will be available soon.

For my setting I have four monitors and two nodes.
I created the file layout.xml:

<wall>
               <col>
                       <display>tile-0-0:0.0</display>
               </col>
               <col>
                       <display>tile-0-0:0.1</display>
               </col>
               <col>
                       <display>tile-0-1:0.0</display>
               </col>
               <col>
                       <display>tile-0-1:0.1</display>
               </col>
</wall>


With this, in simple mode I got glxgears working OK, but when I tried to run anything else (I tried with Stellarium), I get messages like:

CR Warning(tile-0-0:18331): __tcpip_read_exact() error: Bad address
CR Warning(tile-0-0:18331): Bad juju: 10485760 197100 on socket 0xc
CR Warning(tile-0-0:18331): CRServer: Last client disconnected - exiting.

Hence this hack... The Viz roll is supposed to work in mode=simple for Chromium and mode=meta for SAGE. I didn't want to have to change the mode, so I did the following to have Chromium working in mode=meta.

[root@vaiven ~]# rocks remove tile
[root@vaiven ~]# rocks add tile layout layout.xml
[root@vaiven ~]# rocks sync tile mode=meta

I downloaded Google Earth (32 bits application), installed the library dependencies and made sure that it was working OK when Chromium was disabled. The it was a matter of downloading Chromium and compiling both the 32 bits and the 64 bits versions (I put copies in my home directory, at /home/angelv/cr-1.9 and /home/angelv/cr-1.9-32).

Then I created two scripts, to enable and disable my Chromium:

[angelv@vaiven ~]$ cat enable_cr
#!/bin/bash
cd /home/angelv/cr-1.9/lib/Linux
ln -s libcrfaker.so libGL.so.1
ln -s libcrfaker.so libGL.so

cd /home/angelv/cr-1.9-32/lib/Linux
ln -s libcrfaker.so libGL.so.1
ln -s libcrfaker.so libGL.so

[angelv@vaiven ~]$ cat disable_cr
#!/bin/bash
cd /home/angelv/cr-1.9/lib/Linux
rm libGL.so.1
rm libGL.so

cd /home/angelv/cr-1.9-32/lib/Linux
rm libGL.so.1
rm libGL.so
[angelv@vaiven ~]$


and added PATH and LD_LIBRARY_PATH stuff to my .bashrc:

export PATH="/home/angelv/cr-1.9/bin/Linux:$PATH"
export LD_LIBRARY_PATH="/home/angelv/cr-1.9/lib/Linux:/home/angelv/cr-1.9-32/lib/Linux:$LD_LIBRARY_PATH"


I also had to change the .crconfigs file in my home directory, so that it will take my own Chromium configuration file:

[angelv@vaiven ~]$ cat .crconfigs
old-rocks* /opt/rocks/bin/rocks start chromium %p %m
* /home/angelv/first_angel.conf %p
[angelv@vaiven ~]$

And I created the file /home/angelv/first_angel.conf as shown below. Note that this file has a hard-coded configuration for my two compute nodes in meta mode, and that the node.AutoStart stuff has been changed, since for some reason launching the crserver via ssh gave the errors mentioned above. In its place I created a basic start-crserver.sh which is also included below.

[angelv@vaiven ~]$ cat first_angel.conf
import sys
sys.path.append('/home/angelv/cr-1.9/mothership/server')
from mothership import *

TILE_WIDTH = 2560
TILE_HEIGHT = 1024

appnode = CRApplicationNode('vaiven.ll.iac.es')
tilesortspu = SPU('tilesort')
appnode.AddSPU(tilesortspu)
appnode.Conf( 'show_cursor', 1 )
cr = CR()
cr.AddNode(appnode)

renderspu = SPU('render')
renderspu.Conf('fullscreen', 1)
#renderspu.Conf('window_geometry', [0, 0, 512, 512])
renderspu.Conf( 'show_cursor', 1 )


node = CRNetworkNode('tile-0-0.local')
node.AddTile(0, 0, TILE_WIDTH, TILE_HEIGHT)
node.AddSPU(renderspu)
#node.AutoStart( ["/usr/bin/ssh","tile-0-0", "DISPLAY=:0.0 /bin/sh -c
'/home/angelv/cr-1.9/bin/Linux/crserver -mothership vaiven:10000 -port 7000'"] )
node.AutoStart( ["/usr/bin/ssh",'-x',"tile-0-0", "/home/angelv/start-crserver.sh
0.0 crserver vaiven 10000 7000"] )

tilesortspu.AddServer(node, protocol='tcpip', port=7000)

cr.AddNode(node)

renderspu = SPU('render')
renderspu.Conf('fullscreen', 1)
#renderspu.Conf('window_geometry', [0, 0, 512, 512])
renderspu.Conf( 'show_cursor', 1 )

node = CRNetworkNode('tile-0-1.local')
node.AddTile(2560, 0, TILE_WIDTH, TILE_HEIGHT)
node.AddSPU(renderspu)
#node.AutoStart( ["/usr/bin/ssh","tile-0-1", "DISPLAY=:0.0 /bin/sh -c
'/home/angelv/cr-1.9/bin/Linux/crserver -mothership vaiven:10000 -port 7000'"] )
node.AutoStart( ["/usr/bin/ssh",'-x',"tile-0-1", "/home/angelv/start-crserver.sh
0.0 crserver vaiven 10000 7000"] )
#node.AutoStart( ["/usr/bin/ssh",'-x',"tile-0-1", "start-crserver.sh 0.0
/home/angelv/cr-1.9/bin/Linux/crserver vaiven 10000"] )

tilesortspu.AddServer(node, protocol='tcpip', port=7000)

cr.AddNode(node)
demo = sys.argv[1]

appnode.SetApplication(demo)


cr.Go()
[angelv@vaiven ~]$

[angelv@vaiven ~]$ cat start-crserver.sh
#!/bin/bash
export DISPLAY=:$1
$2 -mothership $3:$4 -port $5


[angelv@vaiven ~]$

With these modifications, after running enable_cr, Google Earth (32 bits), atlantis (64 bits) and Stellarium (64 bits) were fine.

These were quick notes that I took while I was doing the modifications, so most probably something is missing. If you find something missing, please let me know. For the time being I'm trying another software stack for the display wall, so I'm not planning on improving this, but I might come back to the Viz Roll in the near future...

16 comments:

Armando Arostegui said...

hi, i try to install a cluster render with chromium and virtual gl, please if you have a step to step manual to install chromium help me
thanks

angelv said...

Hi Armando,

in http://angel-de-vicente.blogspot.com/2010/05/settings-for-tiled-display-wall.html I describe with some more details how to install Chromium, although once you have all the dependencies installed, Chromium is pretty easy to install.

Cheers,

Armando Arostegui said...

thanks for your response, I try install Chromium in Centos 5.5, the dependences are the same that in Ubuntu?? I do the cluster with red hat cluster suite, please I need help, this project is for my thesis

angelv said...

Hi Armando,

in Centos the dependencies are going to be different. Similar but different, so you will have to find out which packages are needed.

Good luck

Armando Arostegui said...

thanks, I compiled chromium and ran the simple example of the tutorial, now I need configure it to for my cluster render!!

angelv said...

Good luck!

Armando Arostegui said...

do you speak spanish?? I need to ask some please

angelv said...

Sí, Armando, hablo español. De todas maneras desde que escribí ésto no he vuelto a probar Rocks 5.3, así que no sé si te podré ayudar...

Armando Arostegui said...

Estoy intentando hacer un cluster de renderización, tengo 4 computadoras, voy a utilizar chromium, virtualgl y turboVNC,
estoy en la instalación y configuración de Chromium,
tengo 4 computadoras, 1 hace de appnode y 3 de server nodes, mi duda es que spus debo utilizar, en los 3 server nodes spu readback y en el app node spu tilesort???
estoy intentando hacer un Sort-First Rendering (Image-Space Decomposition) with Readback, no quiero hacer un display wall, lo que quiero hacer es poder correr aplicaciones 3d muy pesadas en el appnode utilizando las capacidades de render de los server nodes, muchas gracias por tu atención y disculpa la molestia

angelv said...

Hola Armando,

la verdad es que no tengo experiencia con el modo en que tú quieres hacerlo, pero asumo que sí que sería como dices, el tilesort en el app node y los readback en los otros nodos.

¿La documentación de Chromium no tiene ejemplos sobre este modo de usarlo?

Saludos y suerte

Armando Arostegui said...

Hola Angel, encontre un ejemplo en las configuraciones que vienen con Chromium que mas o menos concuerda con lo que te conte que estoy intentando hacer, el ejemplo se llama reassemble.conf, usa 3 pcs, en 1 corre el nodo de aplicación y un nodo reensamble, solo en esta pc tengo monitor, en las otras 2 pcs corren nodos servidores con spu readback y pack, al intentar correr el crserver remotamente mediante ssh me sale el siguiente error:

Start a crappfaker on pc3.cluster
Start a crserver on pc3.cluster
Autostart for node pc1: ['/usr/bin/ssh', '-x', '-l', 'cluster1', '192.168.1.11', 'crserver']
Autostart for node pc2: ['/usr/bin/ssh', '-x', '-l', 'cluster2', '192.168.1.12', 'crserver']
CR Warning(pc2:3866): Couldn't open X display named '(null)'
CR Warning(pc2:3866): Render SPU: Couldn't get a visual, renderspu_SystemInitVisual failed
CR Warning(pc2:3866): Render SPU: Couldn't create a window, renderspuFindVisual returned NULL
CR Error(pc2:3866): Render SPU: Couldn't get a double-buffered, RGB visual with Z!
CR Warning(pc1:3646): Couldn't open X display named '(null)'
CR Warning(pc1:3646): Render SPU: Couldn't get a visual, renderspu_SystemInitVisual failed
CR Warning(pc1:3646): Render SPU: Couldn't create a window, renderspuFindVisual returned NULL
CR Error(pc1:3646): Render SPU: Couldn't get a double-buffered, RGB visual with Z!

configure el autostart para los dos nodos de renderización pero me sale el mismo mensaje, lei en unos foros que tu tenias el mismo problema, como los solucionaste??
seria de mucha ayuda que me lo digas
muchas gracias

angelv said...

Hola Armando,

no sé si es el mismo error, pero es parecido.. Yo tenía mensajes del tipo:

angelv@vaiven:~/cr-1.9/mothership/configs$ CR Warning(vaiven:14640): Render SPU: Display :0.0 doesn't have the necessary visual: RGB, Doublebuffer, Z

Realmente no sé porqué era, pero me pasaba con Ubuntu 10.04. Con Ubuntu 9.10 no tenía ese problema. La configuración de todo ésto está en: http://angel-de-vicente.blogspot.com/2010/05/settings-for-tiled-display-wall.html

Saludos y suerte

Bryant said...

I have been having the same problem as this so I am hoping that this hack works. Quick question: I have done as you said but I keep getting an error saying that the start-crserver.sh file does not exist. I have tried this with and with out "chmod u+x start-crserver.sh" to see if this would potentially help. Do you have any suggestions?

Bryant said...

Never mind, I didn't realize that I was using a different version until after I posted. After playing around with this hack a little bit I was able to get my optiportal working in meta mode. Thank you so much! Next step: getting something besides glxgears and atlantis working on it....

angelv said...

Hi Bryant,

I'm glad it was useful. Good luck with the remaining stuff.

Mr. Srikanth Nagella said...

Hi Armando

I have tried you method but it still complains with this error. I am not using rocks and have compiled chromium as you described. any help please
CR Warning(cn01:4629): EFAULT
CR Warning(cn01:4629): Bad bad bad socket error: Bad address
CR Warning(cn01:4629): Bad juju: 12060672 8880 on socket 0xb