3

Scraper design document

j edited this page 2023-05-24 00:30:33 +02:00

Tools:

• https://flask.palletsprojects.com/en/2.3.x/ - Web framework
• https://sqlalchemy.org/ - ORM
• https://alembic.sqlalchemy.org/en/latest/ - Schema migrations

Objects:

• Product
  • Name
  • SKU/PLU/UPC number
  • PriceHistory
• Retailer
  • Name
  • Product(s)
• Price
  • MOQ
  • Price
• ScraperJob
  • ID
  • StartTime (time when scraper was started)
  • EndTime (time when scraper job finished)
  • Parameters (zip codes, retailer...)
  • Statistics
• PriceHistory
  • Location
    • ZipCode
    • CityName
  • PriceAtTime(s)
    • Price
    • ScraperJobID
• User
  • Username
  • Credentials
  • CreationDate
  • ModificationDate