Skip to content
This repository has been archived by the owner on Nov 5, 2023. It is now read-only.

jeffbrennan/TexasPandemics

Repository files navigation

Scraping, statistics, and diagnostics of Texas COVID-19 data

Installation

  1. Clone the repository using git clone --depth 1 https://github.com/jeffbrennan/TexasPandemics.git. The --depth 1 flag is optional and will only pull the latest commit to increase install speed.
  2. Install the requirements using pip install -r requirements.txt (preferably in a virtual environment)
  3. Configure the environment variables in .env (see .env.example for an example)
  4. Configure the dagster environment variables see the dagster docs
  5. Launch dagster in the cli using dagster dev in the root directory.
  6. Navigate to localhost:3000 in your browser to view the dagster dashboard.
  7. Create a new run using Materialize All

Future Work

  • Dockerize the project
  • Use a database to store the data instead of csv files
  • Pull in additional sources
  • Add a more robust testing suite & run diagnostics

Current ETL: ETL

About

Scrapes, cleans, and analyzes Texas COVID-19 data

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published