html - Python and Selenium - Scrape data from multiple siblings -

-

okay i'm new python , of course selenium. i'm trying scrape page data , work data in python , have selenium click links , store times etc...

the issue i've come across page isn't formatted way i'd like. instead of having this... title link1 link2 title2 link3 link4/a> have this

<tr>     <td>title<td> </tr> <tr>     <td>         <a href>link1</a>     </td> </tr> <tr>     <td>         <a href>link2</a>     </td> </tr> <tr>     <td>         <a href>link3</a>     </td> </tr>

heres html i'm working - http://pastebin.com/663t7mxc

what i'm trying is, of links categorise them based on title come under. e.g. title link 1 link 2 title 2 link 3 link 4 link 5 title 3 link 6

and on.

since links aren't children of same tag title i'm finding it's impossible me do.

this have far

def test():     print ("testing")     browser = webdriver.chrome()     browser.get("http://urlforpage.com")     meetings = browser.find_elements_by_xpath('/html/body/div[2]/table[2]/tbody/tr/td')     i=0     meet in meetings:         venue = meet.get_attribute("class")         if venue == "bold":             print "venue: " + str(i) + " " + meet.text             i+=1         elif venue == "racing-insert-linked-events nextoff-inner-wrapper nextoff-scrollable-wrapper":             print ("links")             print venue.href   test()

i'm pulling title out based on "bold" class of class, issue is, don't know how pull url , link text links inside other tags.

any appreciated. thanks

trying change little of code possible, you're after?

def test():     print ('testing')     browser = webdriver.chrome()     browser.get('http://urlforpage.com')     meetings = browser.find_elements_by_xpath('/html/body/div[2]/table[2]/tbody/tr/td')     meet in meetings:         if meet.get_attribute('class') == 'bold':             print 'venue: {venue}'.format(venue=meet.text)         else:             try:                 anchor = meet.find_element_by_tag_name('a')                 print 'link: {link}, text: {text}'.format(link = anchor.get_attribute('href'), text = anchor.text)             except nosuchelementexception:                 pass  # worried if neither title (bold) nor contains anchor?   test()

Comments

Post a Comment

Popular posts from this blog

c# - How to get the current UAC mode -

-
can check current uac type set in device before running application through c#. i got know approach check whether current user administrator or not. but, need know uac type set (i.e. default/never notify) internal logic. is possible? please help. this article self-elevation , checking level program runs in. due security reasons i'm afraid won't able retrieve uac level set. http://code.msdn.microsoft.com/windowsdesktop/csuacselfelevation-644673d3
Read more

postgresql - Lazarus + Postgres: incomplete startup packet -

-
i've been building lazarus pascal program using postgresql on back-end. used work fine following lines opening db. dbconn:= tpqconnection.create(nil); dbconn.hostname := 'localhost'; dbconn.databasename:= 'dbhrs'; dbconn.username:='mizk'; dbconn.password:='123'; dbconn.open; if dbconn.connected openhrsdb := true else openhrsdb := false; but moment change localhost server's ip (on lan), program stops. no errors or warnings. have no clue happening. here related details may narrow down problem. the program i'm running workstation on same lan server. both machines run ubuntu 12.04 desktop i can access postgres db on server using pgadmin iii, guess it's not firewall issue. i have allowed postgresql ufw , ufw disabled. strangely, when firewall disabled, can ssh server terminal using workstation. when it's enabled, can't my biggest concern right why pascal program not working when i...
Read more

javascript - Ajax jqXHR.status==0 fix error -

-
$.ajax({ url: urlstring, datatype: "json", type: "get", success: function (data) { alert(data); }, error: function (jqxhr, exception) { if (jqxhr.status === 0) { alert('not connect.\n verify network.'); } else if (jqxhr.status == 404) { alert('requested page not found. [404]'); } else if (jqxhr.status == 500) { alert('internal server error [500].'); } else if (exception === 'parsererror') { alert('requested json parse failed.'); } else if (exception === 'timeout') { alert('time out error.'); } else if (exception === 'abort') { alert('ajax request aborted.'); } else { alert('uncaught error.\n' + jqxhr.responsete...
Read more