The default CometBFT mempool is incompatible with a lot of Ethereum tooling (Forge, hardhat, other scripting tools). Most of the tools assume async transaction batching, whereas Cosmos mempools work strictly in FIFO ordering.
If transactions come in from batches from Ethereum tooling, they usually arrive out of order, breaking the FIFO assumption of properly ordered nonces.
Ethereum tooling also expects transactions to remain in the mempool even if not immediately executable, like if there are nonce gaps or the declared gas fee is lower than the chain’s feemarket base fee after initial mempool inclusion. The assumption for these cases is that these transactions will soon be executable.
Below is an example of an incompatibility error that @Kevin Kz found during his deployment of a Superchain using a default deployment script:
...
ERROR[06-16|20:46:50.471] unable to publish transaction service=transactor tx=4cd914..1fe292 nonce=39 gasTipCap=1,000,000,000 gasFeeCap=3,000,000,000 gasLimit=5,738,016 err="invalid nonce; got 39, expected 12: invalid sequence: invalid sequence"
WARN [06-16|20:46:50.471] Aborting transaction submission service=transactor tx=4cd914..1fe292 nonce=39 err="failed to get tx into the mempool"
ERROR[06-16|20:46:50.471] transaction failed id=d13290..9dffa8 completed=31 total=33 err="aborted tx send due to critical error: failed to get tx into the mempool"
INFO [06-16|20:46:50.531] Publishing transaction service=transactor tx=279aa7..a0e829 nonce=40 gasTipCap=1,000,000,000 gasFeeCap=3,000,000,000 gasLimit=5,605,232
ERROR[06-16|20:46:50.549] unable to publish transaction service=transactor tx=279aa7..a0e829 nonce=40 gasTipCap=1,000,000,000 gasFeeCap=3,000,000,000 gasLimit=5,605,232 err="invalid nonce; got 40, expected 12: invalid sequence: invalid sequence"
WARN [06-16|20:46:50.549] Aborting transaction submission service=transactor tx=279aa7..a0e829 nonce=40 err="failed to get tx into the mempool"
ERROR[06-16|20:46:50.549] transaction failed id=b320db..a1ce7f completed=32 total=33 err="aborted tx send due to critical error: failed to get tx into the mempool"
INFO [06-16|20:46:50.595] Publishing transaction service=transactor tx=dfa19b..17d2a2 nonce=41 gasTipCap=1,000,000,000 gasFeeCap=3,000,000,000 gasLimit=4,078,230
ERROR[06-16|20:46:50.615] unable to publish transaction service=transactor tx=dfa19b..17d2a2 nonce=41 gasTipCap=1,000,000,000 gasFeeCap=3,000,000,000 gasLimit=4,078,230 err="invalid nonce; got 41, expected 12: invalid sequence: invalid sequence"
WARN [06-16|20:46:50.615] Aborting transaction submission service=transactor tx=dfa19b..17d2a2 nonce=41 err="failed to get tx into the mempool"
ERROR[06-16|20:46:50.615] transaction failed id=9029c0..83411b completed=33 total=33 err="aborted tx send due to critical error: failed to get tx into the mempool"
Application failed: failed to deploy implementations: failed to broadcast: 30 errors occurred:
* aborted tx send due to critical error: failed to get tx into the mempool
* aborted tx send due to critical error: failed to get tx into the mempool
* aborted tx send due to critical error: failed to get tx into the mempool
* aborted tx send due to critical error: failed to get tx into the mempool
...
Note the "invalid nonce; got 39, expected 12: invalid sequence: invalid sequence"
, which denotes a transaction that was batch-sent, but arrived out of order. After this error, the the rest of the script fails.
Gaia has the following transaction flow today, using the default mempool.