I have already created gcloud dataflow on the Google cloud platform and now I need to do it from the Linux console. The main questions:
- How should it look?
- How can I work with --parameters of gcloud?
Asked
Active
Viewed 65 times
-1
1 Answers
1
Generally (!) everything that you can achieve through Cloud Console, is available to you through the Google Cloud SDK more commonly referred to as gcloud.
gcloud is well-designed, consistent and more powerful than the Console; you have more power and flexibility using gcloud than Console.
I recommend you consult Google's gcloud documentation (link). On the left hand side, you will see an enumeration of all the commands with detailed documentation.
I recommend you consider using the excellent gcloud interactive shell too. This will facilitate your learning of the tool.
One caveat with gcloud is that there are generally-available commands (gcloud [command] ...) and there are alpha and beta commands too. Personally, I find this confusing because you may need to consult three different commands to find exactly what you need.
In your case, gcloud dataflow --help will inform you that you can interact with one 'group' only (jobs), i.e. gcloud dataflow jobs --help. However, if you use the alpha and beta commands, gcloud alpha dataflow --help, you will see that you're able to interact with jobs, logs and metrics. Be mindful of this with all the gcloud commands.
A good starting point for you may be to query (non-destructively) jobs in your project. The documentation shows two options list (link) and describe (link).
Because all gcloud commands are scoped to a Cloud Project, you will need to specify the project. You can do this per command gcloud ... --project=${PROJECT} or you can set the project as a default gcloud config set project ${PROJECT}. I recommend you always specify the project per command.
So, without further ado... this should get you going:
Optionally, use the interactive shell (NB one of the beta commands):
gcloud beta interactive
If not already, login:
gcloud auth login [[YOUR-GOOGLE-ACCOUNT]]
Then something similar to:
gcloud dataflow jobs list --project=[[YOUR-PROJECT]]
Grab one of the job IDs:
gcloud dataflow jobs describe [[YOUR-JOB-ID]] --project=[[YOUR-PROJECT]]
