Edison

[cmbant]

Thanks for this handy recipe. But Carver is being retired, do either of you have an update that works on Edison? I tried to follow as-is, but fails for me in Edison shell (after some delay):

Warning: no access to tty (Bad file descriptor).
Thus no job control in this shell.
TUNNEL
Succesfully opened notebook!
Kill this process to end your notebook connection.
channel_new: internal error: channels_alloc 10010 too big.
channel 4: open failed: connect failed: Connection refused

This was on Mac.

I also tried connecting from Windows using latest putty to get any security fixes, but then with Windows browser I also get (quickly) connection refused along with a simultaneous drop in the putty connection.

[marius311]

I don't have any updated version. On Edison its been convenient enough to just run the notebook just on the login node ssh edison -L 8888:localhost:8888 ipython notebook. You could also try ipython.nersc.gov.

[zonca]

I am not using this script anymore, not even on Carver, I have been using a simpler script, which should work on Edison as well.
I get an interactive node, then I have a notebook script:

#!/bin/bash
ssh -N -f -R $NOTEBOOKPORT:localhost:$NOTEBOOKPORT $NOTEBOOKHOST
ipython notebook --port=$NOTEBOOKPORT --no-browser

then define the variables in my bashrc, of course you need a public IP for this to work.
I often run this from work desktop and then forward port to my laptop if I'm working remotely.

[cmbant]

Thanks both, I will use interactive. I think the hint about "public IP" may be a crucial piece of information (this wasn't obvious to me). So for a laptop, connecting via some static server to the cluster, the steps to get something working are something like:

• Make a script "sshipython" on the server, with, e.g.:

  ssh -L localhost:$NOTEBOOKPORT:localhost:$NOTEBOOKPORTuser@edison.nersc.gov ipython notebook --no-browser --port $NOTEBOOKPORT--port-retries=0 || sleep infinity

  (the sleep stuff is supposed to start ipython only if it isn't already running and using the port)

• Every time you want to use ipython notebook, ssh from the laptop to the server a) forwarding localhost:$NOTEBOOKPORT to $NOTEBOOKPORT and b) running "sshipython".on the server

Then view the notebook at localhost:$NOTEBOOKPORT. This assumes you've set up authorized_hosts appropriately on both the server and the cluster, and to itself on the cluster.

However getting additional python modules working is a can of worms that I've not solved yet (not helped by the fact the Edison shell is csh which module python/2.7-anaconda does not support..)

[zonca]

I think your steps look correct. you can use backticks to format code on github, it makes it easier to read.

you can change your login shell on nim.nersc.gov
I installed my own anaconda in my home.

[zonca]

don't make sshipython on the server.

1. ssh from your laptop to Edison
2. you do qsub -I to get an interactive node, then launch my notebook script so you forward the port to your NOTEBOOKHOST (the server). So the notebook script is on Edison.
3. when the job starts from your laptop launch ssh -L to forward a local port from laptop to the server
4. connect to localhost:NOTEBOOKPORT on your laptop

for testing purposes you can login to the server and do wget localhost:NOTEBOOKPORT to check that the port from Edison to the server works.

[zonca]

@marius311 @cmbant

with the retirement of Carver, I had to switch to using notebooks on Edison.
Configuration is quite complex...but I got it working.

See my tutorial:

http://zonca.github.io/2015/09/ipython-jupyter-notebook-nersc-edison.html

if any of you would like to test and send feedback, that would be great.
Still an unsolved issue is that the ccm queue is very slow. I think we should ask NERSC to provide more CCM nodes.