Grab links of list of keywords

From Computational Biophysics and Materials Science Group
Jump to: navigation, search

When you need to find link to a lot of item, like, you want to find links for 100 prof with only their names. The idea is to use Google's "I feel lucky" to quick grab the best match link of particular keyword, and save in a format which we want.

The Chance to find the link depends on "Luck", if not found, the link will be the google search result page.

I think any language can implement this, just pick my favorite to do this demo

Method(Python)

put them into a list, like this: 

list = ['Prof. Mikko Haataja (Princeton University)', 
'Prof. Nikolas Provatas (McGill University)', 
'Prof. Martin Grant (McGill University)', 
'Prof. David Solovatz (University of Pennsylvanian )', 
'Prof. Longqing Chen (Penn State University)', 
'Prof. Lanyuan Lu (Nanyang Tech. University)', 
'Prof. JC Grumber (Galtech)', 
'Prof. Klaus Schulten (UIUC)', 
'Prof. Chris Chipot (UIUC)', 
'Prof. Gregory Voth (University of Chicago)', 
'Prof. Karl Freed (University of Chicago)', 
'Prof. Aron Dinner (University of Chicago)', 
'Prof. Benoit Roux(University of Chicago)', 
'Prof. Xiaojing Yang (University of Chicago)', 
'Prof. Fei Sun (IBP, CAS)', 
'Prof. Yanting Wang (Theor. Phys., CAS)', 
'Prof. Qiang Shi (Chemistry, CAS)']

then, run this: 

import urllib.parse
import requests
for l in list:
	var = requests.get('https://www.google.com/search?hl=en&btnI=1&q=' + urllib.parse.quote_plus(l))
	print('*[' + var.url + ' ' + l + ']')

Then out put is: 

*[https://www.princeton.edu/~mhaataja/ Prof. Mikko Haataja (Princeton University)]
*[http://www.physics.mcgill.ca/~provatas/.index.html Prof. Nikolas Provatas (McGill University)]
*[http://www.physics.mcgill.ca/~grant/ Prof. Martin Grant (McGill University)]
*[https://www.google.com/search?hl=en&btnI=1&q=Prof.+David+Solovatz+%28University+of+Pennsylvanian+%29 Prof. David Solovatz (University of Pennsylvanian )]
*[http://www.ems.psu.edu/~chen/ Prof. Longqing Chen (Penn State University)]
*[http://research.ntu.edu.sg/expertise/academicprofile/pages/StaffProfile.aspx?ST_EMAILID=LYLU Prof. Lanyuan Lu (Nanyang Tech. University)]
*[https://www.google.com/search?hl=en&btnI=1&q=Prof.+JC+Grumber+%28Galtech%29 Prof. JC Grumber (Galtech)]
*[http://www.ks.uiuc.edu/~kschulte/ Prof. Klaus Schulten (UIUC)]
*[http://www.ks.uiuc.edu/~chipot/ Prof. Chris Chipot (UIUC)]
*[http://chemistry.uchicago.edu/faculty/faculty/person/member/gregory-a-voth.html Prof. Gregory Voth (University of Chicago)]
*[http://chemistry.uchicago.edu/faculty/faculty/person/member/karl-f-freed.html Prof. Karl Freed (University of Chicago)]
*[http://chemistry.uchicago.edu/faculty/faculty/person/member/aaron-dinner.html Prof. Aron Dinner (University of Chicago)]
*[http://thallium.bsd.uchicago.edu/RouxLab/ Prof. Benoit Roux(University of Chicago)]
*[https://www.google.com/search?hl=en&btnI=1&q=Prof.+Xiaojing+Yang+%28University+of+Chicago%29 Prof. Xiaojing Yang (University of Chicago)]
*[http://feilab.ibp.ac.cn/ Prof. Fei Sun (IBP, CAS)]
*[https://www.google.com/search?hl=en&btnI=1&q=Prof.+Yanting+Wang+%28Theor.+Phys.%2C+CAS%29 Prof. Yanting Wang (Theor. Phys., CAS)]
*[http://www.polyu.edu.hk/me/Academicstaffs/detail/9 Prof. Qiang Shi (Chemistry, CAS)]

Paste in wiki, it should be:

Retrieved from "https://mini.ourphysics.org/wiki/index.php?title=Grab_links_of_list_of_keywords&oldid=370"