RATIS-2395. Add Client Span for AyncImpl

[taklwu]

What changes were proposed in this pull request?

After RATIS-2393 / #1341, this change aims to provide the first end-to-end tracing triggered from the any async implementation when sending a rpc request.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/RATIS-2395

How was this patch tested?

unit tests and extend existing tests in ratis-test module with feature enabled.

[szetszwo]

@taklwu , thanks for working on this! Please see the comments inlined and also https://issues.apache.org/jira/secure/attachment/13081378/1385_review.patch

[szetszwo]

We cannot new RaftProperties() here since it will get all the default values.

Ratis is a library and does not manage any conf files. The RaftProperties is passed from the application when building RaftClient/RaftServer

[szetszwo]

If the spanName is missing, should it be a bug? We probably should check null but not setting DEFAULT_OPERATION.

[szetszwo]

When server == null, the client request will be sent to the leader. So, let's use

  private static final String LEADER = "LEADER";

[szetszwo]

Let's change TRACER to AtomicReference. Then, it is enabled if it is not null.

  private static final AtomicReference<Tracer> TRACER = new AtomicReference<>();
  
  public static void setEnabled(RaftProperties properties) {
    setEnabled(TraceConfigKeys.enabled(properties));
  }

  public static void setEnabled(boolean enabled) {
    if (enabled) {
      TRACER.updateAndGet(previous -> previous != null ? previous
          : GlobalOpenTelemetry.getTracer("org.apache.ratis", VersionInfo.getSoftwareInfoVersion()));
    } else {
      TRACER.set(null);
    }
  }

  static boolean isEnabled() {
    return TRACER.get() != null;
  }

[szetszwo]

• Let's check enabled in traceAsyncMethod(..).

@@ -62,6 +78,10 @@ public final class TraceUtils {
    */
   static <T, THROWABLE extends Throwable> CompletableFuture<T> traceAsyncMethod(
       CheckedSupplier<CompletableFuture<T>, THROWABLE> action, Supplier<Span> spanSupplier) throws THROWABLE {
+    if (!isEnabled()) {
+      return action.get();
+    }

[szetszwo]

• We may combine traceAsyncClientSend and traceAsyncImplSend.
• BTW, let move the client related code to a new TraceClient class. We should also move the server related code to a new TraceServer class. Then, TraceUtils will contain only the common methods.

  public static <T, THROWABLE extends Throwable> CompletableFuture<T> asyncSend(
      CheckedSupplier<CompletableFuture<T>, THROWABLE> action,
      RaftClientRequest.Type type, RaftPeerId server) throws THROWABLE {
    return TraceUtils.traceAsyncMethod(action, () -> createClientOperationSpan(type, server, "Async::send"));
  }

[szetszwo]

Let's keep using "raft" prefix.

  public static final AttributeKey<String> PEER_ID = AttributeKey.stringKey("raft.peer.id");
  public static final AttributeKey<String> OPERATION_NAME = AttributeKey.stringKey("raft.operation.name");
  public static final AttributeKey<String> OPERATION_TYPE = AttributeKey.stringKey("raft.operation.type");