📦 pgoslatara / medium_scraper

A slightly sarcastic, highly over-engineered GitHub-as-a-Data-Platform Proof-of-Concept.

★ 1 stars ⑂ 0 forks 👁 1 watching
medium_scraper
📥 Clone https://github.com/pgoslatara/medium_scraper.git
HTTPS git clone https://github.com/pgoslatara/medium_scraper.git
SSH git clone git@github.com:pgoslatara/medium_scraper.git
CLI gh repo clone pgoslatara/medium_scraper
ZIP Desktop VS Code
Pádraic Slattery Pádraic Slattery Manual hook update (#410) 59c8692 1 days ago 📝 History
📂 main View all commits →
📁 .github
📁 dbt
📁 docs
📁 images
📁 output
📁 src
📁 utils
📄 .gitattributes
📄 .gitignore
📄 .pre-commit-config.yaml
📄 .python-version
📄 makefile
📄 mypy.ini
📄 poetry.lock
📄 pyproject.toml
📄 README.md
📄 README.md

medium_scraper

A slightly sarcastic, highly over-engineered GitHub-as-a-Data-Platform Proof-of-Concept.

Development setup

  • Create virtual env and install required packages:
python3.11 -m virtualenv .venv
    source .venv/bin/activate
    pip install -e .
    pre-commit install

  • Required environment variables:
set -gx DATA_DIR $(pwd)/local_output