Eth bridge indexer metrics + coinmarketcap API to fetch a price. by RustNinja · Pull Request #159 · ComposableFi/composable-ibc-private

RustNinja · 2024-05-08T17:57:19Z

Eth bridge indexer metrics + coinmarketcap API to fetch a price.

todo:

upgrade to HOBBYIST version https://coinmarketcap.com/api/pricing/
change the api call from get latest price to get price on date

…packet-indexer fee collector indexer + send packet indexer

introduced DatabaseFeeCollector to save the fee collected/withdrawn

parse and use event of send packet to store into db

…-api Introduce coinmarketcap api to fetch a price and add metrics fee usd.

vmarkushin · 2024-05-08T18:26:35Z

+  input TEXT NOT NULL,
+  max_fee_per_gas TEXT,
+  max_priority_fee_per_gas TEXT,
+  method TEXT NOT NULL,


How do we store multiple method calls?

introduced call_batch_methods column as a vec of method's selectors called by relayer to call_batch solidity fn.

vmarkushin · 2024-05-08T18:48:31Z


+		let fee_collector_address : ethabi::Address = "0x6f11fb4d4178c5d5e3ee4bc7820d8cfed0b59ce7".parse().unwrap();
+		let fee_collector_address_vec = Box::new(vec![fee_collector_address]);
+		let f: &'static mut Vec<ethabi::Address> = Box::leak(fee_collector_address_vec);


Try to avoid using leak (use .clone() for example)

clone does not work here.

vmarkushin · 2024-05-08T19:08:41Z

+
+					for i in transactions{
+						all_txs.insert(i.hash.clone(), i.clone());
+						if i.from_address.eq_ignore_ascii_case("0x6c57e54C379b5716999ae9AAE4917c9B35AC2CB9") && i.to_address.eq_ignore_ascii_case("0xd856f0f9efa054896fe3596e05978bbe686de131"){


Parse the Address from string and compare

vmarkushin · 2024-05-08T19:09:55Z

 		}

+		// relayer_txs.retain(|x| x.from_address.eq_ignore_ascii_case("0x6c57e54C379b5716999ae9AAE4917c9B35AC2CB9"));
+		relayer_txs.sort_by(|a, b| a.block_number.cmp(&b.block_number));


Use .sort_by_key

vmarkushin · 2024-05-08T19:11:11Z

+		relayer_txs.sort_by(|a, b| a.block_number.cmp(&b.block_number));
+		println!("Relayer txs len: {:?}", relayer_txs.len());
+		if relayer_txs.len() > 0 {
+			println!("Relayer txs: {:?}", relayer_txs[0]);


Suggested change

println!("Relayer txs: {:?}", relayer_txs[0]);

println!("Relayer txs: {:?}", relayer_txs);

too much logs. going to remove.

vmarkushin · 2024-05-08T20:22:32Z

+	//  `from_tos` is `block_numbers` shrinked to pairs of `(from_block, to_block)`
+	// e.g. block numbers = [1, 2, 3, 5, 6, 7, 8, 10], from_tos = [(1,3), (5,8), (10, 10)]
+	let from_tos: Vec<(i64, i64)> = block_numbers.iter().fold(vec![], |mut acc, block_number| {
+		if acc.is_empty() {
+			acc.push((*block_number, *block_number));
+			return acc
+		}
+
+		let last_from_to = acc.last_mut().unwrap();
+
+		if last_from_to.1 + 1 == *block_number {
+			last_from_to.1 = *block_number;
+		} else {
+			acc.push((*block_number, *block_number));
+		}
+
+		acc
+	});


Extract this code to a separate function to avoid duplication

…thods parse call batch methods to store into db actions from relayer.

introduce args for metric indexer config instead of hardcoded variables

Add block number to fee collector, fix bug with decoding, extract to fn

rjonczy · 2024-05-10T12:50:55Z

 `TODO: programs and flags.`
+
+5. Usage
+   `DATABASE_URL="pg://postgres:password@localhost/postgres" REDIS_URL="redis://localhost:6379" cargo run --package evm-indexer --bin evm-indexer -- --chain=mainnet --start-block=19789739 --rpcs=https://geth-execution-0.ethereum-mainnet.sre-scratchpad-349209.composablenodes.tech:443 --contract-addresses=0xd856f0f9efa054896fe3596e05978bbe686de131 --ibc-core-address=0xd856f0f9efa054896fe3596e05978bbe686de131 --relayer-address=0x6c57e54C379b5716999ae9AAE4917c9B35AC2CB9 --fee-collector-address=0x6f11fb4d4178c5d5e3ee4bc7820d8cfed0b59ce7 --ics20-transfer-bank-address=0x148acd3cd4d6a17cd2abbecd0745b09b62c64f84 --coinmarketcap-api-key=449a42d7-b66f-4b85-b957-cc6599b8c1e6`


is it a real api-key ?

real key. without any funds. 10k monthly credits calls.
example.
repo is private. you can substitute with your key if you wish

rjonczy · 2024-05-10T12:52:20Z

+	//price-performance-stats is not allowed for free plan 
+    // let mut url = Url::parse("https://pro-api.coinmarketcap.com/v2/cryptocurrency/price-performance-stats/latest").unwrap();
+	//only quotes/latest is allowed for free plan
+    let mut url = Url::parse("https://pro-api.coinmarketcap.com/v2/cryptocurrency/quotes/latest").unwrap();


maybe, this should be configurable? not hardcodeded, in case coinmarketcap changes endpoint

there are so many params for this indexer to set as a dependency.
if endpoint will change then the code probably will not work at all and need to change the logic inside of source code.

RustNinja and others added 20 commits May 6, 2024 22:52

fee collector indexer + send packet indexer

fd8d6cd

Merge pull request #155 from ComposableFi/rustdev/fee-collector-send-…

c19fc7c

…packet-indexer fee collector indexer + send packet indexer

introduced DatabaseFeeCollector to save the fee collected/withdrawn

6128e21

Merge pull request #156 from ComposableFi/rustdev/db-fee-collector

69a4e47

introduced DatabaseFeeCollector to save the fee collected/withdrawn

Decode SendPacketData to use an object to store to db.

61015fe

parse and use event of send packet to store into db

66a385f

Merge pull request #157 from ComposableFi/rustdev/db-send-packet-event

34923a3

parse and use event of send packet to store into db

extract relayer tx that submit calls to the ibc-core contract.

81b8b81

Introduce coinmarketcap api to fetch a price and add metrics fee usd.

85ec404

Merge pull request #158 from ComposableFi/rustdev/coinmarketcap-price…

ae4ec00

…-api Introduce coinmarketcap api to fetch a price and add metrics fee usd.

add migration for indexer db and find the tx timestamp for fee collector

a588239

update migration + feecollectors table

6776314

collect send packet events into vector for storing to db

6498c6c

added store_relayertransactions method to store relayer tx

dc0d3f5

add sendpacketevents into migration

9aad530

added store_feecollectors method

94d54df

use the column count from corresponding database struct

ce1b338

add store_sendpacketevents method

8bca713

Rename to DatabaseRelayerTransaction, update store_data method

abb649e

update sql insert script into sendpacketevents table.

6fc42e8

RustNinja requested a review from vmarkushin May 8, 2024 17:57

RustNinja changed the base branch from main to vmarkushin/test-succint May 8, 2024 17:57

RustNinja changed the title ~~Eth bridge metrics + coinmarketcap API to fetch a price.~~ Eth bridge indexer metrics + coinmarketcap API to fetch a price. May 8, 2024

remove commented not necessary code.

d8b0850

vmarkushin reviewed May 8, 2024

View reviewed changes

RustNinja added 5 commits May 9, 2024 16:10

parse call batch methods to store into db actions from relayer.

403f755

remove commented code.

7d026c1

remove commented code.

a72ad4c

remove "continue" that will push the fee collector event anyway

8f15f86

fix errors when insert to database.

87d08a2

RustNinja and others added 3 commits May 9, 2024 22:33

introduce args for metric indexer config instead of hardcoded varibables

99c6e00

Merge pull request #160 from ComposableFi/rustdev/parse-batch-call-me…

3f618d8

…thods parse call batch methods to store into db actions from relayer.

Merge pull request #161 from ComposableFi/rustdev/metrics-args

0565e9e

introduce args for metric indexer config instead of hardcoded variables

RustNinja requested review from blasrodri and vmarkushin May 9, 2024 21:39

RustNinja and others added 2 commits May 10, 2024 12:08

Add block number to fee collector, fix bug with decoding, extract to fn

079f5c8

Merge pull request #162 from ComposableFi/rustdev/metrics-refactoring

d46272f

Add block number to fee collector, fix bug with decoding, extract to fn

rjonczy reviewed May 10, 2024

View reviewed changes

RustNinja requested a review from rjonczy May 10, 2024 14:20

introduce incremental id field for feecollectors + store usd for fee

a6ad620

	println!("Relayer txs: {:?}", relayer_txs[0]);
	println!("Relayer txs: {:?}", relayer_txs);

Conversation

RustNinja commented May 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RustNinja May 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

RustNinja commented May 8, 2024 •

edited

Loading

RustNinja May 10, 2024 •

edited

Loading