Go Time shout out

I’ve specifically signed up for this forum to give a shout out to @andrei because of his appearance on the Go Time podcast. He had great answers to all the questions that the Go Time team fired at him and he answered a lot of questions for me personally.

We’re running cockroach internally against which we build the new iteration of our product that will span across the atlantic. And, along with this new iteration, I’m trying to incorporate tracing in all our systems. Having the same goal as cockroach of integrating it in our logger (i’ve built something around gokit’s log.Logger interface), it was great hear that we have a big open source project as an example of how to work with tracing. I was delighted to see that after setting the zipkin setting in the cluster, that it just worked!

While you can expect me to get back here with loads of questions in the future as we keep building, I just have a single question for now: Is there any way to get a tracing context into cockroach to continue the trace from our system?

2 Likes

Also I should mention that I’m giving a talk about opentracing next month and I’d love to incorporate cockroach in the demo instead of just showing some other version of jaegers example app HotRod or the donut salon. So ideas are welcome :wink:

Hi @rio! Thanks for the kind words!

Is there any way to get a tracing context into cockroach to continue the trace from our system?

Not currently, unfortunately. I think the way people do this is by injecting some trace id into SQL queries as comments. This is something that a SQL driver would have to support. For Go drivers, only in recent Go versions did methods in the sql package start taking contexts; I’m not sure if the drivers started supporting those methods in earnest. In any case, I might try to bang out some support in one of the drivers and see if I get it working. I have to remember all the opentracing trace-injecting stuff; from what I remember it was a pretty awkward interface they had for serializing the trace id.

There’s also some server side work to do do create a suitable span. And I think there might be some weirdness where we’re a bit restricted on the server side with the points at which context can be created in the SQL layer, but I was doing something about it in https://github.com/cockroachdb/cockroach/pull/23055. I need to page some stuff back in.

If I get anything working, I’ll ping this thread.

You were also asking for suggestions on showcasing opentracing; I’d say have a 3 node cluster, run a ALTER TABLE ... SPLIT AT... command to distribute the data some, and then run a query over the whole table with a group by and and order by maybe. That should give you some nice spans.

Big +1 to being able to pass in a tracing context. I was curious about doing something similar last year - in this thread Error communicating with TLS secured Zipkin collector

I just remembered this little thread and it seemed appropriate to actually post the video of the talk I gave here. I did it twice, the first time with a bigger audience than in this video but as a first talk I enjoyed doing this and learning about opentracing and about cockroach internally.

I got some of the internals wrong, I know that, so please don’t hate on me :wink: