38 lines
893 B
Markdown
38 lines
893 B
Markdown
This branch contains a prototype for a retail price scraper for getweee.com.
|
|
To run it, install:
|
|
|
|
* Python 3.9+,
|
|
* chrome,
|
|
* chromedriver,
|
|
* and the python depedencies:
|
|
|
|
```
|
|
pip install -r /path/to/requirements.txt
|
|
```
|
|
|
|
Once those are installed, install the GCP SDK:
|
|
|
|
https://cloud.google.com/sdk/docs/install
|
|
|
|
You will also have to log into GCP *twice*, once for the gcloud CLI tools,
|
|
and again for the GCP SDK libraries:
|
|
|
|
https://googleapis.dev/python/google-api-core/latest/auth.html
|
|
|
|
Windows hosts need to use WSL as the ansible client. Ubuntu doesn't have a
|
|
ssh-agent manager by default, so it's recommended to install keychain:
|
|
|
|
`apt install keychain`
|
|
`echo 'eval $(keychain -q --eval) >> ~/.bashrc`
|
|
|
|
Allow WSL distributions to use NTFS metadata for storing Linux ACLs:
|
|
|
|
* Add to /etc/wsl.conf:
|
|
```
|
|
[automount]
|
|
options = "metadata"
|
|
```
|
|
|
|
* Restart wsl
|
|
|
|
* chmod 755 the ansible directory |