Support for more (Open Access) Paper sites

[BradKML]

Coming from writing a list from the other projects, it would be great if places like OA.mg, InternetArchive, and Unpaywall gets added into the list of places for papers. SmartManoj/Kevin#136 (reply in thread)
P.S. what other forms of optimizations are there (e.g. updated OpenReview) that makes generating new papers better? SmartManoj/Kevin#143 SmartManoj/Kevin#136 (comment)

Currently it is in https://github.com/SakanaAI/AI-Scientist/blob/main/ai_scientist/generate_ideas.py with search_for_papers but since there are no standards API across sites, it might be harder to wrangle this. At the end we are only getting paper title, author(s), venue, year of publication, citation count, and paper abstract. Note: it reads abstract but not necessarily content that enables stronger idea synthesis through deeper content summaries.

A bit of a personal self-note on the progress of things #135

P.S. @MarlNox has done it for https://core.ac.uk and the code can be retrofitted for other sites as well #104 (comment)

[BradKML]

List of issues with problems in SemanticScholar #18 #78 #84 #118 #122 #133

[erkinalp]

don't forget sci-hub.ru

[BradKML]

Please, we need search engines rather than just paper repositories. Also, very likely SakanaAI can't vouch for anything that is not Open Access compliant. SmartManoj/Kevin#136 (reply in thread)

[erkinalp]

In most of the world, AI-generated works get "creator isn't a legal person" treatment hence disqualified from copyright; with the exception of the UK and few ex-British colonies where they instead get work-of-hire treatment.

[BradKML]

I am referring to the acquisition of papers in the same way as the OG founder of Reddit (look it up btw), and how future organizations (especially startups with investors) are not allowed to break compliance and end up going the same direction.