Hi everyone, I've created a web scraper that scrapes the information I need from a table on a website. It clicks through 6 pages, but it ends up only including the last page at the end of my loop rather than appending as it goes. I'm new to for loops and Python and hoping for some guidance. Here is my code:
Where should I put the append so it properly runs?
Code:
while True:
driver.implicitly_wait(30)
table = driver.find_element_by_id('preblockBody')
information = []
job_elems = table.find_elements_by_xpath("//*[contains(@class,'pbListingTable')]")
for value in job_elems:
information.append(value.text)
try:
driver.find_element_by_partial_link_text('Next').click()
except:
break
driver.quit()
print(information)
Comment