hotfix: fix the bug of triggerProcessLoop

[317787106]

What does this PR do?

1. Fix the bug of mismatch trigger in triggerProcessLoop thread if content has the same topic name; check if the data is json before send to server;
2. Fixed the bug that the thread triggerProcessLoop failed to exit when interrupt.
3. Close ExecutorService graceful to avoid missing data; Close mongo connection manually
4. Add column latestSolidifiedBlockNumber for BlockLogTrigger refered in fix: add missing latestSolidifiedBlockNumber field to BlockLogTrigger #46, delete 3 reduant colunm for BlockLogTrigger
5. Use standard Enum EventTopic instead of Constant
6. Delete build directory when use command clean in gradle
7. Add document for how to build indexes for mongo manually in README
8. Rename MessageSenderImpl to KafkaSenderImpl for consistency
9. Fix the bug of no print log in PluginLauncher, using logback.xml instead log4j.properties
10. Maintain same version of Dependencies between different projects.

Why are these changes required?

This PR has been tested by:

• Unit Tests
• Manual Testing

Follow up

Extra details

[waynercheung]

Why is it necessary to verify each time if the data is in JSON format? Could this have an impact on performance?

[317787106]

We need to extract the triggerName from the JSON with minimal CPU overhead.

[317787106]

Using String.contains is logically incorrect as other key may cotains this triggerName also. We should match with it exactly.

[waynercheung]

what about using map instead of for loop?
such as:

private static final Map<Integer, EventTopic> TYPE_MAP = Arrays.stream(values())
      .collect(Collectors.toMap(EventTopic::getType, Function.identity()));
      
  private static final Map<String, EventTopic> NAME_MAP = Arrays.stream(values())
      .collect(Collectors.toMap(EventTopic::getName, Function.identity()));

public static EventTopic getEventTopicByType(int value) {
    return TYPE_MAP.get(value); // O(1) 
  }

  public static EventTopic getEventTopicByName(String topicName) {
    return NAME_MAP.get(topicName); // O(1) 
  }

[317787106]

Actually, it uses setTopic only once when start the plugin, so there is no need to optimize it.

[waynercheung]

should we interrupt the triggerProcessThread which is started in the init?

such as:

  isRunTriggerProcessThread = false;

  if (triggerProcessThread != null) {
    triggerProcessThread.interrupt();
    try {
        triggerProcessThread.join(2000); 
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }
  }

[317787106]

That's alternative way to break the thread. Accept. TriggerProcessThread is Used for transferring data between two queues, no IO operaton with extremely high performance.

[waynercheung]

we should set isRunTriggerProcessThread = false; firstly?

[317787106]

That's alternative way to break the thread. Accept.

[waynercheung]

add try-cath?

if (triggerProcessThread != null) {
    triggerProcessThread.interrupt();
    try {
      triggerProcessThread.join(5000); 
    } catch (InterruptedException e) {
      log.warn("Interrupted while waiting for triggerProcessThread to stop");
      Thread.currentThread().interrupt();
    }
  }

[317787106]

Good suggestion. Tranfering data between two queues is very quick, we use 1000 ms at most.

[waynercheung]

update log to "Mongo service thread pool did not terminate"

[317787106]

Good suggestion. Fix it.

[xxo1shine]

If the triggerQueue contains data, should we wait until the data is written to the database before shutting down the service? Otherwise, the data in the triggerQueue will be lost.

[317787106]

Before close mongo connection, we wait for the tasks to complete using this :

      // Wait a while for existing tasks to terminate
      if (!service.awaitTermination(60, java.util.concurrent.TimeUnit.SECONDS)) {
        service.shutdownNow(); // Cancel currently executing tasks
        // Wait a while for tasks to respond to being cancelled
        if (!service.awaitTermination(60, java.util.concurrent.TimeUnit.SECONDS)) {
          log.warn("Mongo service thread pool did not terminate");
        }
      }

The task is writing data through mongo collection.

[317787106]

triggerProcessThread transfers data from triggerQueue to another queue. It's extremely quick. And 1000 millseconds is enough long:

    if (triggerProcessThread != null) {
      triggerProcessThread.interrupt();
      try {
        triggerProcessThread.join(1000);
      } catch (InterruptedException e) {
        log.warn("Interrupted while waiting for triggerProcessThread to stop");
        Thread.currentThread().interrupt();
      }
    }

[xxo1shine]

Broadcast status may be relatively fast, but synchronizing historical data can take a longer time. Additionally, network jitter can also cause significant network latency.

[317787106]

You can restart the FullNode with updated startSyncBlockNum by query mongo. Even though write all the data timely, you also need check it again.

[317787106]

We use option WriteConcern.JOURNALED to ensure flushing disk before closing mongo connection so that alleviate this problem.

[waynercheung]

LGTM