In python, communicating with stackdriver always returns success, doesn't send anything

Question

For some inexplicable reason, google provides no stackdriver api for appengine, so I'm stuck implementing one. No worries - I thought - I have already worked with the API builder to talk to bigquery, so I built up a client and started trying to send events:

credentials = SignedJwtAssertionCredentials(STACKDRIVER_AUTH_GOOGLE_CLIENT_EMAIL,
                                            STACKDRIVER_AUTH_GOOGLE_PRIVATE_KEY,
                                            scope='https://www.googleapis.com/auth/trace.append')

http = httplib2.Http()
credentials.refresh(http) #Working around an oauth2client bug
credentials = credentials.authorize(http)
service = build('cloudtrace', 'v1', http=http)
batch = service.new_batch_http_request()
batch.add(service.projects().patchTraces(
        body=traces_json,
        projectId=STACKDRIVER_AUTH_GOOGLE_PROJECT_ID))
print batch.execute()

I left out the definition of traces_json because no matter what I send, the service always responds with an error. If traces_json = '{}':

{u'error': {u'code': 400,
            u'errors': [{u'domain': u'global',
                         u'message': u'Invalid value at \'traces\' (type.googleapis.com/google.devtools.cloudtrace.v1.Traces), "{}"',
                         u'reason': u'badRequest'}],
            u'message': u'Invalid value at \'traces\' (type.googleapis.com/google.devtools.cloudtrace.v1.Traces), "{}"',
            u'status': u'INVALID_ARGUMENT'}}

But even if I use a body, crafted from the google documentation, I still get the same error.

I'm running a packet sniffer on the machine where I'm attempting this, and only very rarely see it actually communicating with googleapis.com.

So the question is, really, what am I missing that will get me sending events to stackdriver?

UPDATE

Here's the most recent iteration of what I'd been working with, though using the google doc example verbatim (with the exception of changing the project id) produces the same result.

{
    "traces": [
        {
            "projectId": "projectname",
            "traceId": "1234123412341234aaaabb3412347890",
            "spans": [
                {
                    "kind": "RPC_SERVER",
                    "name": "trace_name",
                    "labels": {"label1": "value1", "label2": "value2"},
                    "spanId": "spanId1",
                    "startTime": "2016-06-01T05:01:23.045123456Z",
                    "endTime": "2016-06-01T05:01:23.945123456Z",
                },
            ],
        },
    ],
}

And the error message that comes with it:

{u'error': {u'code': 400,
            u'errors': [{u'domain': u'global',
                         u'message': u'Invalid value at \'traces\' (type.googleapis.com/google.devtools.cloudtrace.v1.Traces), "MY ENTIRE JSON IS REPEATED HERE"',
                         u'reason': u'badRequest'}],
            u'message': u'Invalid value at \'traces\' (type.googleapis.com/google.devtools.cloudtrace.v1.Traces), "MY ENTIRE JSON IS REPEATED HERE"',
            u'status': u'INVALID_ARGUMENT'}}

SECOND UPDATE

Doing this in the explorer produces approximately the same result. I had to switch to a numeric span_id because, despite the docs' statement that it only has to be a unique string, I get errors about requiring what appears to be a 64-bit integer, any time I provide anything else.

PATCH https://cloudtrace.googleapis.com/v1/projects/[number or name]/traces?key={YOUR_API_KEY}
{
 "traces": [
  {
   "projectId": "[number or name]",
   "traceId": "1234123412341234aaaabb3412347891",
   "spans": [
    {
     "kind": "RPC_SERVER",
     "name": "trace_name",
     "labels": {
      "label1": "value1"
     },
     "startTime": "2016-06-01T05:01:23.045123456Z",
     "endTime": "2016-06-01T05:01:25.045123456Z"
    },
    {
     "spanId": "0"
    }
   ]
  }
 ]
}

Response:

{
 "error": {
  "code": 400,
  "message": "Request contains an invalid argument.",
  "status": "INVALID_ARGUMENT"
 }
}

Answer 1

The issue is in the format of your data. You can not send empty messages either. The best way to explore how to use the API is to go to the StackDriver Trace API explorer, you will find out the exact data structure to send: https://cloud.google.com/trace/api/reference/rest/v1/projects/patchTraces#traces

Pay special attention to the format of traceId. It needs to be 32 character hex string like this: 7d9d1a6e2d1f3f27484992f33d97e5cb

Here is a working python example to show how to use the 3 methods on StackDriver trace on github: https://github.com/qike/cloud-trace-samples-python

Copy paste code below:

def list_traces(stub, project_id):
    """Lists traces in the given project."""
    trace_id = None
    req = trace_pb2.ListTracesRequest(project_id=project_id)
    try:
        resp = stub.ListTraces(req, TIMEOUT)
        for t in resp.traces:
            trace_id = t.trace_id
            print("Trace is: {}".format(t.trace_id))
    except NetworkError, e:
        logging.warning('Failed to list traces: {}'.format(e))
        sys.exit(1)
    return trace_id


def patch_traces(stub, project_id):
    req = trace_pb2.PatchTracesRequest(project_id=project_id)
    trace_id = str(uuid.uuid1()).replace('-', '')
    now = time.time()

    trace = req.traces.traces.add()
    trace.project_id = project_id
    trace.trace_id = trace_id
    span1 = trace.spans.add()
    span1.span_id = 1
    span1.name = "/span1.{}".format(trace_id)
    span1.start_time.seconds = int(now)-10
    span1.end_time.seconds = int(now)

    span2 = trace.spans.add()
    span2.span_id = 2
    span2.name = "/span2"
    span2.start_time.seconds = int(now)-8
    span2.end_time.seconds = int(now)-5

    try:
        resp = stub.PatchTraces(req, TIMEOUT)
        print("Trace added successfully.\n"
              "To view list of traces, go to: "
              "http://console.cloud.google.com/traces/traces?project={}&tr=2\n"
              "To view this trace added, go to: "
              "http://console.cloud.google.com/traces/details/{}?project={}"
              .format(project_id, trace_id, project_id))
    except NetworkError, e:
        logging.warning('Failed to patch traces: {}'.format(e))
        sys.exit(1)


def get_trace(stub, project_id, trace_id):
    req = trace_pb2.GetTraceRequest(project_id=project_id,
                                    trace_id=trace_id)
    try:
        resp = stub.GetTrace(req, TIMEOUT)
        print("Trace retrieved: {}".format(resp))
    except NetworkError, e:
        logging.warning('Failed to get trace: {}'.format(e))
        sys.exit(1)

UPDATED to answer error received from API explorer

Regarding the errors you got from using API explorer, it was due to using 0 as span_id. It should be a 64 bit int other than 0.

I also observed that the span_id you set is in a different span object than the one you intended. Make sure you don't by mistake clicking on a "+" sign to add a new span object.

Below is a successful patch request I sent to my project through API explorer:

{
 "traces": [
  {
   "projectId": "<project ID>",  // I used string ID, not numeric number
   "traceId": "1234123412341234aaaabb3412347891",
   "spans": [
    {
     "spanId": "1",
     "name": "foo",
     "startTime": "2016-06-01T05:01:23.045123456Z",
     "endTime": "2016-06-01T05:01:25.045123456Z"
    }
   ]
  }
 ]
}
Response

200