---
# System prepended metadata

title: '[SEO] Article Forward Links'

---

## Hours
#### David: 3hrs total
- 1hr setting up infrastructure
- 1.5hr paragraph+metadata extraction
- 0.25hr Google Sheet setup
- 0.25hr Project Documentation 
---
## Records

### [David] 2023 June 30th:
    
> I can't figure out how to actually test that. I'm trying to use Vertex AI. But I think that might work if you can figure out how to get it to run in google cloud.

Use [this link to get to the Console UI](https://console.cloud.google.com/vertex-ai/generative/language/create/text). It's a no-code playground UI like OpenAI playground, and you can click on `View Code` button at top-right to retrieve the code.

> I extracted all of the keywords. Here is a link.

It's pretty bad, dude. 8 keywords for an entire article is bad.

> Here is a link to the script I wrote to do so

You use `nltk` & `tfidf` to extract keywords. These are classical methods that people used in the mid 2010s. By late 2010s, people were already moving on to transformer methods such as [keyBERT](https://github.com/MaartenGr/KeyBERT) and [Yake](https://github.com/LIAAD/yake). They produce better resutls but still very poor (I plied my trade as Data Scientists for a while building solutions based on these 2)

> This script will successfully gather the URLs based on the extracted keywords using Bing API. However, I'm having a hard time using the Bing API.

Start [here first](https://pypi.org/project/googlesearch-python). You can [test your coding skill with scraping Bing's search result](https://oxylabs.io/blog/scraping-bing) 

**First thing first**:
Let me know how many hours you have spent on the work above. I'll clock it as R&D. Send me your Paypal ID, I'll send the $20 * hours clocked.

**My Suggestions**:
- Take a look at [this colab](https://colab.research.google.com/drive/19RgiOW-jLTG8361bBF8WyuiZJqhO7-vY?usp=sharing):
    -  Run all cells upto `Select an article` --> select from 0 to 9000 and run the cell
    -  Run the cell `Annotate paragraphs in each article:` will produce  all the paragraphs, each with associated metadata. 
    -  Try a few of them on [OpenAI playground](https://platform.openai.com/playground) or VertexAI playground (link above). 
    -   The idea is to help me fill out the [following Google Sheet link](https://docs.google.com/spreadsheets/d/1sZZ7lYoXoixWNIqyA8NdbZ4ytpwwNShIfvPSbe3SOoU/edit?usp=sharing)
    -   This Google Sheet is how we show progress and get paid.


>