13

I am trying to display TensorBoard from TensorFlow on Docker on Google Cloud.

http://tensorflow.org/how_tos/summaries_and_tensorboard/index.md

tensorboard --logdir ./

I have Apache running on Google Cloud (it may be in my first container "ai-unicorn" Docker made its own container "docker-playground"). I can see the default page from Google Cloud at http://104.197.119.57/ .

I start TensorBoard on Google Cloud like this:

root@6cf64fd299f0:/# tensorboard --logdir ./ Starting TensorBoard on port 6006 (You can navigate to http://localhost:6006)

I tried the Google Cloud SSH option called "Open in browser window on custom port" using port 6006.

It displays: "We are unable to connect to the VM on port 6006."

What is the correct way to view TensorBoard from Google Cloud?

mrry
  • 120,078
  • 23
  • 381
  • 391
technologiclee
  • 198
  • 1
  • 1
  • 10

3 Answers3

25

By default, TensorBoard serves requests on 127.0.0.1, which is only accessible to processes running on the same machine. If you start TensorBoard with --host 0.0.0.0, it will also serve requests on the remote interfaces, so you should be able to connect to it remotely:

$ tensorboard --logdir ./ --host 0.0.0.0

Note that the "Open in browser window on custom port" will not connect you to the TensorBoard server - this option is used to connect to an SSH server on a non-standard port. The Google Cloud Platform docs have information on how to expose ports from your VM. You will need to allow connections on TCP port 6006 for remote access to your VM. You may also need to expose port 6006 from your Docker container, by following the instructions here.

EDIT: Added some step-by-step instructions to help with your Docker configuration. There are several issues here, and it's not possible to tell which one is failing.

  1. Configure port forwarding when you start your Docker container:

    (vm)$ docker run -p 0.0.0.0:7007:6006 -it b.gcr.io/tensorflow/tensorflow
    

    This forwards connections from port 7007 on your VM to 6006 in your Docker container. (Other values are possible.)

  2. Ensure that you can connect to TensorBoard from within the Docker container:

    (container)$ tensorboard --logdir ./ --host 0.0.0.0 --port 6006 &
    (container)$ curl http://localhost:6006/
    

    The second command should print some HTML to the console.

  3. In a shell on the VM, ensure that you can connect to the TensorBoard instance running in the container:

    (vm)$ curl http://localhost:7007/
    

    The command should print the same HTML to the console.

  4. Configure the Google Cloud firewall to allow your local client to connect to port 7007 on your VM.

    (client)$ gcloud compute firewall-rules create tensorboard --allow tcp:7007
    

    You should now be able to connect to TensorBoard in a web browser on your client.

mrry
  • 120,078
  • 23
  • 381
  • 391
  • `gcloud auth login` `sudo gcloud components update` `gcloud compute firewall-rules create firewall_rule --allow tcp:80,tcp:443,tcp:6006` `docker images` `docker run -p 127.0.0.1:$HOSTPORT:$6006 --name ai-unicorn -t XXXXXXXX` `tensorboard --logdir ./ --host 0.0.0.0` External IP 23.251.146.83 Internal IP 10.240.0.3 Firewalls Allow HTTP traffic (not selected) Allow HTTPS traffic (not selected) To begin after logging out of of Google cloud I do this: `docker run b.gcr.io/tensorflow/tensorflow-full` `tensorboard --logdir ./ --host 0.0.0.0` What address to go to to view TensorBoard? – technologiclee Nov 21 '15 at 19:36
  • I tried this http://23.251.146.83:0/ and get "This address uses a network port which is normally used for purposes other than Web browsing. Firefox has canceled the request for your protection." – technologiclee Nov 21 '15 at 20:03
  • Looking at the firewall rules in the Google Developer Console firewall-rule 0.0.0.0/0 tcp:80; tcp:443; tcp:6006 Apply to all targets So I tried https://23.251.146.83/ But it does ping `ping -c 1 23.251.146.83` PING 23.251.146.83 (23.251.146.83) 56(84) bytes of data. 64 bytes from 23.251.146.83: icmp_seq=1 ttl=45 time=189 ms --- 23.251.146.83 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 189.694/189.694/189.694/0.000 ms – technologiclee Nov 22 '15 at 01:41
  • I still have not found how to display Tensorboard from the server. TensorFlow gives the message to open on port 6006 and it pings at the external address. Should it be an http or https address? – technologiclee Nov 23 '15 at 16:28
  • I added some more to the comment - it's hard to tell what your configuration is exactly, but try those steps and let me know where you get stuck. – mrry Dec 02 '15 at 15:54
  • Success! I think adding the rule for port 7007 did it. On step 4 TENSORBOARD should be lowercase `(client)$ gcloud compute firewall-rules create tensorboard --allow tcp:7007` – technologiclee Dec 14 '15 at 23:23
  • Great! I've updated the answer to apply your correction, thanks. – mrry Dec 14 '15 at 23:25
10

You do not have to use Docker to display the TensorBoard. But if you do want to use Docker, just run the TensorBoard inside of your Docker image.

The trick is to allow external access to the default TensorBoard tcp port 6006.

I tried the following working solution to display TensorBoard in my Google Cloud VM.

  1. ensure you pass the gcloud authentication:

    gcloud auth login

  2. allow public access to the tcp port 6006

    gcloud compute firewall-rules create tensorboard-port --allow tcp:6006

  3. Run TensorBoard on your VM

    tensorboard --logdir=workspace/train/

  4. Use external IP address to access the TensorBoard outside your VM:

    Open address http://your_vm_external IP:6006/,

    e.g. http://104.196.140.145:6006/, where 104.196.140.145 is the external IP address of my VM.

Enjoy TensorBoard

user1165814
  • 335
  • 3
  • 5
2

another option is to use ngrok to tunnel. see: Can I use Tensorboard with Google Colab?

$ from jupyter notebook
ps = !ps -ax
is_tensorboard_running = len([f for f in ps if "tensorboard" in f ]) > 0

is_ngrok_running = len([f for f in ps if "ngrok" in f ]) > 0
print("tensorbord={}, ngrok={}".format(is_tensorboard_running, is_ngrok_running))

if not is_ngrok_running:  
#    grok should be installed in /content/ngrok
  get_ipython().system_raw('/content/ngrok http 6006 &')
  is_ngrok_running = True

# get public url for tensorboard
tensorboard_url = !curl -s http://localhost:4040/api/tunnels | python3 -c \
    "import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"    
print("tensorboard url=", tensorboard_url)
michael
  • 3,521
  • 4
  • 39
  • 61