Gitcoin Grants Data Portal

Open source, serverless, and local-first data hub for Gitcoin Grants data to improve data access and empower data scientists to conduct research!

Visit the portal website to download weekly updates of Gitcoin data packaged as analysis-ready .parquet files!

Instead of fetching data such as donations, projects, rounds or passport scores from multiple APIs you can now get it all from one place, packaged and cleaned!

The Gitcoin Data Portal repository contains code and artifacts to help process Gitcoin Grants data from the Grants Stack Indexer API.

📦 Data portal Pipeline is an instance of datadex allowing you to:

• Add new data sources to the portal, collaborate on better models (ala Dune) or submit an interesting analysis
• Don't ask, fork it and improve the models, add a new source or update any script
• Declarative stateless transformations tracked in git, executed in GitHub Actions and published to IPFS
• Data, artifacts (like the entire DuckDB database), and models all version controlled.
• Share and explore dashboards and reports with the world!

📦 Key Features

• Code, data and infrastructure are fully open source.
• Uses open standards/formats and modular components (Parquet, IPFS, dbt, Dagster).
• Permissionless!
• Clone and edit things away! You're not blocked by any API rate limits, or closed data like in Dune.
• All other git features like branching, merging are available because all the data is transformed declaratively as code.
• Decentralized!

The project runs on a laptop, a server, a CI runner (that's the way is working right now) or a even decentralized compute network like Bacalhau. Oh, it even works in GitHub Codespaces so you don't even need to setup anything locally!

Data is stored in IPFS. You can run it locally, and it'll generate the same IPFS files if nothing has changed. The more people run it, the more distributed the IPFS files will be!

📦 Extra

• It supports all the cool things data engineers want; typing, tests, materialized views, dev branches
• Uses best practices (declarative transformations) and state of the art tooling (DuckDB + Dagster + DBT + Quatro).
• It has integrated publishing platform to surface Jupyter notebooks with useful analyses.

Made by DavidGasquez, DistributedDoge and always open to new contributors!