Moonworm-crawler with schematic indexing with GCS integration.

[Andrei-Dolgolev]

Language

Crawler was desided write on go.

• More close to native libraries like go-ethereum
• More fast
• Have support of protobuff
• Good in scale
• simple to bring to any system as cli tool

Data

• blockchain_blocks: Raw blocks for the blockchain
• blockchain_transactions: Raw transactions for the blockchain
• blockchain_events: Raw logs for the blockchain
• blockchain_internal_transactions(calls)>: can be part of table 2.(v3.1)
• blockchain_state: all write operations(v3.1)

Node endpoints:

**get_block** - return blocks with transactions
**get_events_log**(from, to) - retrun logs from to block
**trace_block** - return block with transactions trace information( possibly can replace get block )
**debug_trace_transaction** - by providing block and transaction input can restore state call operations.

All crawl happen with batch_size crawler will generate on GCS next paths structure.

blockN-blockN+batch_size/blocks.proto
                        /transactions.proto
                        /events.proto
                        /trace_transactions.proto(suggested by @zomglings add internal calls in transactions file)
                        /debug_trace_transaction.proto

Index files store pace

All routes to files writed to Schematic database.

#1000